aboutsummaryrefslogtreecommitdiffstats
path: root/bundle.c
AgeCommit message (Collapse)AuthorFilesLines
2025-07-21Merge branch 'bc/use-sha256-by-default-in-3.0'Junio C Hamano1-2/+2
Prepare to flip the default hash function to SHA-256. * bc/use-sha256-by-default-in-3.0: Enable SHA-256 by default in breaking changes mode help: add a build option for default hash t5300: choose the built-in hash outside of a repo t4042: choose the built-in hash outside of a repo t1007: choose the built-in hash outside of a repo t: default to compile-time default hash if not set setup: use the default algorithm to initialize repo format Use legacy hash for legacy formats builtin: use default hash when outside a repository hash: add a constant for the legacy hash algorithm hash: add a constant for the default hash algorithm
2025-07-01Use legacy hash for legacy formatsbrian m. carlson1-2/+2
We have a large variety of data formats and protocols where no hash algorithm was defined and the default was assumed to always be SHA-1. Instead of explicitly stating SHA-1, let's use the constant to represent the legacy hash algorithm (which is still SHA-1) so that it's clear for documentary purposes that it's a legacy fallback option and not an intentional choice to use SHA-1. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-01odb: rename `repo_read_object_file()`Patrick Steinhardt1-1/+1
Rename `repo_read_object_file()` to `odb_read_object()` to match other functions related to the object database and our modern coding guidelines. Introduce a compatibility wrapper so that any in-flight topics will continue to compile. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-01object-store: rename files to "odb.{c,h}"Patrick Steinhardt1-1/+1
In the preceding commits we have renamed the structures contained in "object-store.h" to `struct object_database` and `struct odb_backend`. As such, the code files "object-store.{c,h}" are confusingly named now. Rename them to "odb.{c,h}" accordingly. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-01object-store: rename `object_directory` to `odb_source`Patrick Steinhardt1-1/+1
The `object_directory` structure is used as an access point for a single object directory like ".git/objects". While the structure isn't yet fully self-contained, the intent is for it to eventually contain all information required to access objects in one specific location. While the name "object directory" is a good fit for now, this will change over time as we continue with the agenda to make pluggable object databases a thing. Eventually, objects may not be accessed via any kind of directory at all anymore, but they could instead be backed by any kind of durable storage mechanism. While it seems quite far-fetched for now, it is thinkable that eventually this might even be some form of a database, for example. As such, the current name of this structure will become worse over time as we evolve into the direction of pluggable ODBs. Immediate next steps will start to carve out proper self-contained object directories, which requires us to pass in these object directories as parameters. Based on our modern naming schema this means that those functions should then be named after their subsystem, which means that we would start to bake the current name into the codebase more and more. Let's preempt this by renaming the structure. There have been a couple alternatives that were discussed: - `odb_backend` was discarded because it led to the association that one object database has a single backend, but the model is that one alternate has one backend. Furthermore, "backend" is more about the actual backing implementation and less about the high-level concept. - `odb_alternate` was discarded because it is a bit of a stretch to also call the main object directory an "alternate". Instead, pick `odb_source` as the new name. It makes it sufficiently clear that there can be multiple sources and does not cause confusion when mixed with the already-existing "alternate" terminology. In the future, this change allows us to easily introduce for example a `odb_files_source` and other format-specific implementations. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-24Merge branch 'ps/object-file-cleanup'Junio C Hamano1-1/+1
Code clean-up. * ps/object-file-cleanup: object-store: merge "object-store-ll.h" and "object-store.h" object-store: remove global array of cached objects object: split out functions relating to object store subsystem object-file: drop `index_blob_stream()` object-file: split up concerns of `HASH_*` flags object-file: split out functions relating to object store subsystem object-file: move `xmmap()` into "wrapper.c" object-file: move `git_open_cloexec()` to "compat/open.c" object-file: move `safe_create_leading_directories()` into "path.c" object-file: move `mkdir_in_gitdir()` into "path.c"
2025-04-15object-store: merge "object-store-ll.h" and "object-store.h"Patrick Steinhardt1-1/+1
The "object-store-ll.h" header has been introduced to keep transitive header dependendcies and compile times at bay. Now that we have created a new "object-store.c" file though we can easily move the last remaining additional bit of "object-store.h", the `odb_path_map`, out of the header. Do so. As the "object-store.h" header is now equivalent to its low-level alternative we drop the latter and inline it into the former. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-08bundle: fix non-linear performance scaling with refsKarthik Nayak1-1/+7
The 'git bundle create' command has non-linear performance with the number of refs in the repository. Benchmarking the command shows that a large portion of the time (~75%) is spent in the `object_array_remove_duplicates()` function. The `object_array_remove_duplicates()` function was added in b2a6d1c686 (bundle: allow the same ref to be given more than once, 2009-01-17) to skip duplicate refs provided by the user from being written to the bundle. Since this is an O(N^2) algorithm, in repos with large number of references, this can take up a large amount of time. Let's instead use a 'strset' to skip duplicates inside `write_bundle_refs()`. This improves the performance by around 6 times when tested against in repository with 100000 refs: Benchmark 1: bundle (refcount = 100000, revision = master) Time (mean ± σ): 14.653 s ± 0.203 s [User: 13.940 s, System: 0.762 s] Range (min … max): 14.237 s … 14.920 s 10 runs Benchmark 2: bundle (refcount = 100000, revision = HEAD) Time (mean ± σ): 2.394 s ± 0.023 s [User: 1.684 s, System: 0.798 s] Range (min … max): 2.364 s … 2.425 s 10 runs Summary bundle (refcount = 100000, revision = HEAD) ran 6.12 ± 0.10 times faster than bundle (refcount = 100000, revision = master) Previously, `object_array_remove_duplicates()` ensured that both the refname and the object it pointed to were checked for duplicates. The new approach, implemented within `write_bundle_refs()`, eliminates duplicate refnames without comparing the objects they reference. This works because, for bundle creation, we only need to prevent duplicate refs from being written to the bundle header. The `revs->pending` array can contain duplicates of multiple types. First, references which resolve to the same refname. For e.g. "git bundle create out.bdl master master" or "git bundle create out.bdl refs/heads/master refs/heads/master" or "git bundle create out.bdl master refs/heads/master". In these scenarios we want to prevent writing "refs/heads/master" twice to the bundle header. Since both the refnames here would point to the same object (unless there is a race), we do not need to check equality of the object. Second, refnames which are duplicates but do not point to the same object. This can happen when we use an exclusion criteria. For e.g. "git bundle create out.bdl master master^!", Here `revs->pending` would contain two elements, both with refname set to "master". However, each of them would be pointing to an INTERESTING and UNINTERESTING object respectively. Since we only write refnames with INTERESTING objects to the bundle header, we perform our duplicate checks only on such objects. Signed-off-by: Karthik Nayak <karthik.188@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-25bundle: avoid closing file descriptor twiceJohannes Schindelin1-1/+3
Already when introduced in c7a8a16239 (Add bundle transport, 2007-09-10), the `bundle` transport had a bug where it would open a file descriptor to the bundle file and then close it _twice_: First, the file descriptor (`data->fd`) is passed to `unbundle()`, which would use it as the `stdin` of the `index-pack` process, which as a consequence would close it via `start_command()`. However, `data->fd` would still hold the numerical value of the file descriptor, and `close_bundle()` would see that and happily close it again. This seems not to have caused too many problems in almost two decades, but I encountered a situation today where it _does_ cause problems: In i686 variants of Git for Windows, it seems that file descriptors are reused quickly after they have been closed. In the particular scenario I faced, `git fetch <bundle> <ref>` gets the same file descriptor value when opening the bundle file and importing its embedded packfile (which implicitly closes the file descriptor) and then when opening a pack file in `fetch_and_consume_refs()` while looking up an object's header. Later on, after the bundle has been imported (and the `close_bundle()` function erroneously closes the file descriptor that has _already_ been closed when using it as `stdin` for `git index-pack`), the same file descriptor value has now been reused via `use_pack()`. Now, when either the recursive fetch (which defaults to "on", unfortunately) or a commit-graph update needs to `mmap()` the packfile, it fails due to a now-invalid file descriptor that _should_ point to the pack file but doesn't anymore. To fix that, let's invalidate `data->fd` after calling `unbundle()`. That way, `close_bundle()` does not close a file descriptor that may have been reused for something different. While at it, document that `unbundle()` closes the file descriptor, and ensure that it also does that when failing to verify the bundle. Luckily, this bug does not affect the bundle URI feature, it only affects the `git fetch <bundle>` code path. Note that this patch does not _completely_ clarifies who is responsible to close that file descriptor, as `run_command()` may fail _without_ closing `cmd->in`. Addressing this issue thoroughly, however, would require a rather thorough re-design of the `start_command()` and `finish_command()` functionality to make it a lot less murky who is responsible for what file descriptors. At least this here patch is relatively easy to reason about, and addresses a hard failure (`fatal: mmap: could not determine filesize`) at the expense of leaking a file descriptor under very rare circumstances in which `git fetch` would error out anyway. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-12-23Merge branch 'ps/build-sign-compare'Junio C Hamano1-0/+1
Start working to make the codebase buildable with -Wsign-compare. * ps/build-sign-compare: t/helper: don't depend on implicit wraparound scalar: address -Wsign-compare warnings builtin/patch-id: fix type of `get_one_patchid()` builtin/blame: fix type of `length` variable when emitting object ID gpg-interface: address -Wsign-comparison warnings daemon: fix type of `max_connections` daemon: fix loops that have mismatching integer types global: trivial conversions to fix `-Wsign-compare` warnings pkt-line: fix -Wsign-compare warning on 32 bit platform csum-file: fix -Wsign-compare warning on 32-bit platform diff.h: fix index used to loop through unsigned integer config.mak.dev: drop `-Wno-sign-compare` global: mark code units that generate warnings with `-Wsign-compare` compat/win32: fix -Wsign-compare warning in "wWinMain()" compat/regex: explicitly ignore "-Wsign-compare" warnings git-compat-util: introduce macros to disable "-Wsign-compare" warnings
2024-12-19Merge branch 'tc/bundle-with-tag-remove-workaround'Junio C Hamano1-30/+0
"git bundle create" with an annotated tag on the positive end of the revision range had a workaround code for older limitation in the revision walker, which has become unnecessary. * tc/bundle-with-tag-remove-workaround: bundle: remove unneeded code
2024-12-12bundle: remove unneeded codeToon Claes1-30/+0
The changes in commit c06793a4ed (allow git-bundle to create bottomless bundle, 2007-08-08) ensure annotated tags are properly preserved when creating a bundle using a revision range operation. At the time the range notation would peel the ends to their corresponding commit, meaning ref v2.0 would point to the v2.0^0 commit. So the above workaround was introduced. This code looks up the ref before it's written to the bundle, and if the ref doesn't point to the object we expect (for tags this would be a tag object), we skip the ref from the bundle. Instead, when the ref is a tag that's the positive end of the range (e.g. v2.0 from the range "v1.0..v2.0"), then that ref is written to the bundle instead. Later, in 895c5ba3c1 (revision: do not peel tags used in range notation, 2013-09-19), the behavior of parsing ranges was changed and the problem was fixed at the cause. But the workaround in bundle.c was not reverted. Now it seems this workaround can cause a race condition. git-bundle(1) uses setup_revisions() to parse the input into `struct rev_info`. Later, in write_bundle_refs(), it uses this info to write refs to the bundle. As mentioned at this point each ref is looked up again and checked whether it points to the object we expect. If not, the ref is not written to the bundle. But, when creating a bundle in a heavy traffic repository (a repo with many references, and frequent ref updates) it's possible a branch ref was updated between setup_revisions() and write_bundle_refs() and thus the extra check causes the ref to be skipped. The workaround was originally added to deal with tags, but the code path also gets hit by non-tag refs, causing this race condition. Because it's no longer needed, remove it and fix the possible race condition. Signed-off-by: Toon Claes <toon@iotcl.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-12-06global: mark code units that generate warnings with `-Wsign-compare`Patrick Steinhardt1-0/+1
Mark code units that generate warnings with `-Wsign-compare`. This allows for a structured approach to get rid of all such warnings over time in a way that can be easily measured. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-11-28bundle: support fsck message configurationJustin Tobler1-6/+7
If the `VERIFY_BUNDLE_FLAG` is set during `unbundle()`, the git-index-pack(1) spawned is configured with the `--fsck-options` flag to perform fsck verification. With this flag enabled, there is not a way to configure fsck message severity though. Extend the `unbundle_opts` type to store fsck message severity configuration and update `unbundle()` to conditionally append it to the `--fsck-objects` flag if provided. This enables `unbundle()` call sites to support optionally setting the severity for specific fsck messages. Signed-off-by: Justin Tobler <jltobler@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-11-28bundle: add bundle verification options typeJustin Tobler1-1/+5
When `unbundle()` is invoked, fsck verification may be configured by passing the `VERIFY_BUNDLE_FSCK` flag. This mechanism allows fsck checks on the bundle to be enabled or disabled entirely. To facilitate more fine-grained fsck configuration, additional context must be provided to `unbundle()`. Introduce the `unbundle_opts` type, which wraps the existing `verify_bundle_flags`, to facilitate future extension of `unbundle()` configuration. Also update `unbundle()` and its call sites to accept this new options type instead of the flags directly. The end behavior is functionally the same, but allows for the set of configurable options to be extended. This is leveraged in a subsequent commit to enable fsck message severity configuration. Signed-off-by: Justin Tobler <jltobler@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-09-03Merge branch 'ps/leakfixes-part-5'Junio C Hamano1-3/+1
Even more leak fixes. * ps/leakfixes-part-5: transport: fix leaking negotiation tips transport: fix leaking arguments when fetching from bundle builtin/fetch: fix leaking transaction with `--atomic` remote: fix leaking peer ref when expanding refmap remote: fix leaks when matching refspecs remote: fix leaking config strings builtin/fetch-pack: fix leaking refs sideband: fix leaks when configuring sideband colors builtin/send-pack: fix leaking refspecs transport: fix leaking OID arrays in git:// transport data t/helper: fix leaking multi-pack-indices in "read-midx" builtin/repack: fix leaks when computing packs to repack midx-write: fix leaking hashfile on error cases builtin/archive: fix leaking `OPT_FILENAME()` value builtin/upload-archive: fix leaking args passed to `write_archive()` builtin/merge-tree: fix leaking `-X` strategy options pretty: fix leaking key/value separator buffer pretty: fix memory leaks when parsing pretty formats convert: fix leaks when resetting attributes mailinfo: fix leaking header data
2024-08-22transport: fix leaking arguments when fetching from bundlePatrick Steinhardt1-3/+1
In `fetch_refs_from_bundle()` we assemble a vector of arguments to pass to `unbundle()`, but never free it. And in theory we wouldn't have to because `unbundle()` already knows to free the vector for us. But it fails to do so when it exits early due to `verify_bundle()` failing. The calling convention that the arguments are freed by the callee and not the caller feels somewhat weird. Refactor the code such that it is instead the responsibility of the caller to free the vector, adapting the only two callsites where we pass extra arguments. This also fixes the memory leak. This memory leak gets hit in t5510, but fixing it isn't sufficient to make the whole test suite pass. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-08-13bundle: default to SHA1 when reading bundle headersPatrick Steinhardt1-1/+6
We hit a segfault when trying to open a bundle via `git bundle list-heads` when running outside of a repository. This is caused by c8aed5e8da (repository: stop setting SHA1 as the default object hash, 2024-05-07), which stopped setting the default object hash so that `the_hash_algo` is a `NULL` pointer when running outside of any repo. This is only a symptom of a deeper issue though. Bundles default to the SHA1 object format unless they advertise an "@object-format=" header. Consequently, it has been wrong in the first place to use the object format used by the current repository when parsing bundles. The consequence is that trying to open a bundle that uses a different object hash than the current repository will fail: $ git bundle list-heads sha1.bundle error: unrecognized header: ee4b540943284700a32591ad09f7e15bdeb2a10c HEAD (45) Fix the bug by defaulting to the SHA1 object hash. We already handle the "@object-format=" header as expected, so we don't need to adapt this part. Helped-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-07-08Merge branch 'xx/bundie-uri-fixes'Junio C Hamano1-0/+3
When bundleURI interface fetches multiple bundles, Git failed to take full advantage of all bundles and ended up slurping duplicated objects. * xx/bundie-uri-fixes: unbundle: extend object verification for fetches fetch-pack: expose fsckObjects configuration logic bundle-uri: verify oid before writing refs
2024-07-08Merge branch 'ps/leakfixes-more'Junio C Hamano1-10/+19
More memory leaks have been plugged. * ps/leakfixes-more: (29 commits) builtin/blame: fix leaking ignore revs files builtin/blame: fix leaking prefixed paths blame: fix leaking data for blame scoreboards line-range: plug leaking find functions merge: fix leaking merge bases builtin/merge: fix leaking `struct cmdnames` in `get_strategy()` sequencer: fix memory leaks in `make_script_with_merges()` builtin/clone: plug leaking HEAD ref in `wanted_peer_refs()` apply: fix leaking string in `match_fragment()` sequencer: fix leaking string buffer in `commit_staged_changes()` commit: fix leaking parents when calling `commit_tree_extended()` config: fix leaking "core.notesref" variable rerere: fix various trivial leaks builtin/stash: fix leak in `show_stash()` revision: free diff options builtin/log: fix leaking commit list in git-cherry(1) merge-recursive: fix memory leak when finalizing merge builtin/merge-recursive: fix leaking object ID bases builtin/difftool: plug memory leaks in `run_dir_diff()` object-name: free leaking object contexts ...
2024-06-20unbundle: extend object verification for fetchesXing Xin1-0/+3
The existing fetch.fsckObjects and transfer.fsckObjects configurations were not fully applied to bundle-involved fetches, including direct bundle fetches and bundle-uri enabled fetches. Furthermore, there was no object verification support for unbundle. This commit extends object verification support in `bundle.c:unbundle` by adding the `VERIFY_BUNDLE_FSCK` option to `verify_bundle_flags`. When this option is enabled, we append the `--fsck-objects` flag to `git-index-pack`. The `VERIFY_BUNDLE_FSCK` option is now used by bundle-involved fetches, where we use `fetch-pack.c:fetch_pack_fsck_objects` to determine whether to enable this option for `bundle.c:unbundle`, specifically in: - `transport.c:fetch_refs_from_bundle` for direct bundle fetches. - `bundle-uri.c:unbundle_from_file` for bundle-uri enabled fetches. This addition ensures a consistent logic for object verification during fetches. Tests have been added to confirm functionality in the scenarios mentioned above. Reviewed-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Xing Xin <xingxin.xx@bytedance.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-14global: introduce `USE_THE_REPOSITORY_VARIABLE` macroPatrick Steinhardt1-0/+2
Use of the `the_repository` variable is deprecated nowadays, and we slowly but steadily convert the codebase to not use it anymore. Instead, callers should be passing down the repository to work on via parameters. It is hard though to prove that a given code unit does not use this variable anymore. The most trivial case, merely demonstrating that there is no direct use of `the_repository`, is already a bit of a pain during code reviews as the reviewer needs to manually verify claims made by the patch author. The bigger problem though is that we have many interfaces that implicitly rely on `the_repository`. Introduce a new `USE_THE_REPOSITORY_VARIABLE` macro that allows code units to opt into usage of `the_repository`. The intent of this macro is to demonstrate that a certain code unit does not use this variable anymore, and to keep it from new dependencies on it in future changes, be it explicit or implicit For now, the macro only guards `the_repository` itself as well as `the_hash_algo`. There are many more known interfaces where we have an implicit dependency on `the_repository`, but those are not guarded at the current point in time. Over time though, we should start to add guards as required (or even better, just remove them). Define the macro as required in our code units. As expected, most of our code still relies on the global variable. Nearly all of our builtins rely on the variable as there is no way yet to pass `the_repository` to their entry point. For now, declare the macro in "biultin.h" to keep the required changes at least a little bit more contained. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-11bundle: plug leaks in `create_bundle()`Patrick Steinhardt1-10/+19
When creating a bundle, we set up a revision walk, but never release data associated with it. Furthermore, we create a mostly-shallow copy of that revision walk where we only adapt its pending objects such that we can reuse the walk. While that copy must not be released, the pending objects array need to be. Plug those memory leaks by releasing the revision walk and the pending objects of the copied revision walk. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-05-07cocci: apply rules to rewrite callers of "refs" interfacesPatrick Steinhardt1-1/+1
Apply the rules that rewrite callers of "refs" interfaces to explicitly pass `struct ref_store`. The resulting patch has been applied with the `--whitespace=fix` option. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-07-25Merge branch 'dk/bundle-i18n-more'Junio C Hamano1-2/+2
Update message mark-up for i18n in "git bundle". * dk/bundle-i18n-more: i18n: mark more bundle.c strings for translation
2023-07-13i18n: mark more bundle.c strings for translationD. Ben Knoble1-2/+2
These two messages were introduced in 8ba221e245 (bundle: output hash information in 'verify', 2022-03-22) and 105c6f14ad (bundle: parse filter capability, 2022-03-09) but never for translation. Signed-off-by: D. Ben Knoble <ben.knoble+github@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-21object-store-ll.h: split this header out of object-store.hElijah Newren1-1/+1
The vast majority of files including object-store.h did not need dir.h nor khash.h. Split the header into two files, and let most just depend upon object-store-ll.h, while letting the two callers that need it depend on the full object-store.h. After this patch: $ git grep -h include..object-store | sort | uniq -c 2 #include "object-store.h" 129 #include "object-store-ll.h" Diff best viewed with `--color-moved`. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-11treewide: remove cache.h inclusion due to object.h changesElijah Newren1-1/+1
Signed-off-by: Elijah Newren <newren@gmail.com> Acked-by: Calvin Wan <calvinwan@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-04Merge branch 'ab/remove-implicit-use-of-the-repository' into ↵Junio C Hamano1-2/+3
en/header-split-cache-h * ab/remove-implicit-use-of-the-repository: libs: use "struct repository *" argument, not "the_repository" post-cocci: adjust comments for recent repo_* migration cocci: apply the "revision.h" part of "the_repository.pending" cocci: apply the "rerere.h" part of "the_repository.pending" cocci: apply the "refs.h" part of "the_repository.pending" cocci: apply the "promisor-remote.h" part of "the_repository.pending" cocci: apply the "packfile.h" part of "the_repository.pending" cocci: apply the "pretty.h" part of "the_repository.pending" cocci: apply the "object-store.h" part of "the_repository.pending" cocci: apply the "diff.h" part of "the_repository.pending" cocci: apply the "commit.h" part of "the_repository.pending" cocci: apply the "commit-reach.h" part of "the_repository.pending" cocci: apply the "cache.h" part of "the_repository.pending" cocci: add missing "the_repository" macros to "pending" cocci: sort "the_repository" rules by header cocci: fix incorrect & verbose "the_repository" rules cocci: remove dead rule from "the_repository.pending.cocci"
2023-03-28cocci: apply the "refs.h" part of "the_repository.pending"Ævar Arnfjörð Bjarmason1-1/+2
Apply the part of "the_repository.pending.cocci" pertaining to "refs.h". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-28cocci: apply the "object-store.h" part of "the_repository.pending"Ævar Arnfjörð Bjarmason1-1/+1
Apply the part of "the_repository.pending.cocci" pertaining to "object-store.h". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-21write-or-die.h: move declarations for write-or-die.c functions from cache.hElijah Newren1-0/+1
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-21environment.h: move declarations for environment.c functions from cache.hElijah Newren1-0/+1
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-21treewide: be explicit about dependence on gettext.hElijah Newren1-0/+1
Dozens of files made use of gettext functions, without explicitly including gettext.h. This made it more difficult to find which files could remove a dependence on cache.h. Make C files explicitly include gettext.h if they are using it. However, while compat/fsmonitor/fsm-ipc-darwin.c should also gain an include of gettext.h, it was left out to avoid conflicting with an in-flight topic. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-23cache.h: remove dependence on hex.h; make other files include it explicitlyElijah Newren1-0/+1
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-22Merge branch 'ab/various-leak-fixes'Junio C Hamano1-2/+4
Leak fixes. * ab/various-leak-fixes: push: free_refs() the "local_refs" in set_refspecs() push: refactor refspec_append_mapped() for subsequent leak-fix receive-pack: release the linked "struct command *" list grep API: plug memory leaks by freeing "header_list" grep.c: refactor free_grep_patterns() builtin/merge.c: free "&buf" on "Your local changes..." error builtin/merge.c: use fixed strings, not "strbuf", fix leak show-branch: free() allocated "head" before return commit-graph: fix a parse_options_concat() leak http-backend.c: fix cmd_main() memory leak, refactor reg{exec,free}() http-backend.c: fix "dir" and "cmd_arg" leaks in cmd_main() worktree: fix a trivial leak in prune_worktrees() repack: fix leaks on error with "goto cleanup" name-rev: don't xstrdup() an already dup'd string various: add missing clear_pathspec(), fix leaks clone: use free() instead of UNLEAK() commit-graph: use free_commit_graph() instead of UNLEAK() bundle.c: don't leak the "args" in the "struct child_process" tests: mark tests as passing with SANITIZE=leak
2023-02-06bundle.c: don't leak the "args" in the "struct child_process"Ævar Arnfjörð Bjarmason1-2/+4
Fix a leak that's been here since 7366096de9d (bundle API: change "flags" to be "extra_index_pack_args", 2021-09-05). If we can't verify the bundle, we didn't call child_process_clear() to clear the "args". But rather than adding an additional child_process_clear() call, let's verify the bundle before we start preparing the process we're going to spawn. If we fail to verify, we don't need to push anything to the child_process "args". Helped-by: Elijah Newren <newren@gmail.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-31bundle: verify using check_connected()Derrick Stolee1-46/+29
When Git verifies a bundle to see if it is safe for unbundling, it first looks to see if the prerequisite commits are in the object store. This is an easy way to "fail fast" but it is not a sufficient check for updating refs that guarantee closure under reachability. There could still be issues if those commits are not reachable from the repository's references. The repository only has guarantees that its object store is closed under reachability for the objects that are reachable from references. Thus, the code in verify_bundle() has previously had the additional check that all prerequisite commits are reachable from repository references. This is done via a revision walk from all references, stopping only if all prerequisite commits are discovered or all commits are walked. This uses a custom walk to verify_bundle(). This check is more strict than what Git applies to fetched pack-files. In the fetch case, Git guarantees that the new references are closed under reachability by walking from the new references until walking commits that are reachable from repository refs. This is done through the well-used check_connected() method. To better align with the restrictions required by 'git fetch', reimplement this check in verify_bundle() to use check_connected(). This also simplifies the code significantly. The previous change added a test that verified the behavior of 'git bundle verify' and 'git bundle unbundle' in this case, and the error messages looked like this: error: Could not read <missing-commit> fatal: Failed to traverse parents of commit <extant-commit> However, by changing the revision walk slightly within check_connected() and using its quiet mode, we can omit those messages. Instead, we get only this message, tailored to describing the current state of the repository: error: some prerequisite commits exist in the object store, but are not connected to the repository's history (Line break added here for the commit message formatting, only.) While this message does not include any object IDs, there is no guarantee that those object IDs would help the user diagnose what is going on, as they could be separated from the prerequisite commits by some distance. At minimum, this situation describes the situation in a more informative way than the previous error messages. Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-12bundle-uri: quiet failed unbundlingsDerrick Stolee1-2/+8
When downloading a list of bundles in "all" mode, Git has no understanding of the dependencies between the bundles. Git attempts to unbundle the bundles in some order, but some may not pass the verify_bundle() step because of missing prerequisites. This is passed as error messages to the user, even when they eventually succeed in later attempts after their dependent bundles are unbundled. Add a new VERIFY_BUNDLE_QUIET flag to verify_bundle() that avoids the error messages from the missing prerequisite commits. The method still returns the number of missing prerequisit commits, allowing callers to unbundle() to notice that the bundle failed to apply. Use this flag in bundle-uri.c and test that the messages go away for 'git clone --bundle-uri' commands. Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-12bundle: add flags to verify_bundle()Derrick Stolee1-4/+5
The verify_bundle() method has a 'verbose' option, but we will want to extend this method to have more granular control over its output. First, replace this 'verbose' option with a new 'flags' option with a single possible value: VERIFY_BUNDLE_VERBOSE. Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-12bundle: properly clear all revision flagsDerrick Stolee1-13/+10
The verify_bundle() method checks two things for a bundle's prerequisites: 1. Are these objects in the object store? 2. Are these objects reachable from our references? In this second question, multiple uses of verify_bundle() in the same process can report an invalid bundle even though it is correct. The reason is due to not clearing all of the commit marks on the commits previously walked. The revision walk machinery was first introduced in-process by fb9a54150d3 (git-bundle: avoid fork() in verify_bundle(), 2007-02-22). This implementation used "-1" as the set of flags to clear. The next meaningful change came in 2b064697a5b (revision traversal: retire BOUNDARY_SHOW, 2007-03-05), which introduced the PREREQ_MARK flag instead of a flag normally controlled by the revision-walk machinery. In 86a0a408b90 (commit: factor out clear_commit_marks_for_object_array, 2011-10-01), the loop over the array of commits was replaced with a new clear_commit_marks_for_object_array(), but simultaneously the "-1" value was replaced with "ALL_REV_FLAGS", which stopped un-setting the PREREQ_MARK flag. This means that if multiple commits were marked by the PREREQ_MARK in a previous run of verify_bundle(), then this loop could terminate early due to 'i' going to zero: while (i && (commit = get_revision(&revs))) if (commit->object.flags & PREREQ_MARK) i--; The flag clearing work was changed again in 63647391e6c (bundle: avoid using the rev_info flag leak_pending, 2017-12-25), but that was only cosmetic and did not change the behavior. It may seem that it would be sufficient to add the PREREQ_MARK flag to the clear_commit_marks() call in its current location. However, we actually need to do it in the "cleanup:" step, since the first loop checking "Are these objects in the object store?" might add the PREREQ_MARK flag to some objects and then terminate without performing a walk due to one missing object. By clearing the flags in all cases, we avoid this issue when running verify_bundle() multiple times in the same process. Moving this loop to the cleanup step alone would cause a segfault when running 'git bundle verify' outside of a repository, but this is because of that error condition using "goto cleanup" when returning is perfectly safe. Nothing has been initialized at that point, so we can return immediately without causing any leaks. This behavior is verified carefully by a test that will be added soon when Git learns to download bundle lists in a 'git clone --bundle-uri' command. Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-07Merge branch 'ab/plug-leak-in-revisions'Junio C Hamano1-4/+8
Plug the memory leaks from the trickiest API of all, the revision walker. * ab/plug-leak-in-revisions: (27 commits) revisions API: add a TODO for diff_free(&revs->diffopt) revisions API: have release_revisions() release "topo_walk_info" revisions API: have release_revisions() release "date_mode" revisions API: call diff_free(&revs->pruning) in revisions_release() revisions API: release "reflog_info" in release revisions() revisions API: clear "boundary_commits" in release_revisions() revisions API: have release_revisions() release "prune_data" revisions API: have release_revisions() release "grep_filter" revisions API: have release_revisions() release "filter" revisions API: have release_revisions() release "cmdline" revisions API: have release_revisions() release "mailmap" revisions API: have release_revisions() release "commits" revisions API users: use release_revisions() for "prune_data" users revisions API users: use release_revisions() with UNLEAK() revisions API users: use release_revisions() in builtin/log.c revisions API users: use release_revisions() in http-push.c revisions API users: add "goto cleanup" for release_revisions() stash: always have the owner of "stash_info" free it revisions API users: use release_revisions() needing REV_INFO_INIT revision.[ch]: document and move code declared around "init" ...
2022-05-16bundle.h: make "fd" version of read_bundle_header() publicÆvar Arnfjörð Bjarmason1-4/+4
Change the parse_bundle_header() function to be non-static, and rename it to parse_bundle_header_fd(). The parse_bundle_header() function is already public, and it's a thin wrapper around this function. This will be used by code that wants to pass a fd to the bundle API. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-13revisions API users: use release_revisions() needing REV_INFO_INITÆvar Arnfjörð Bjarmason1-4/+8
Use release_revisions() to various users of "struct rev_list" which need to have their "struct rev_info" zero-initialized before we can start using it. For the bundle.c code see the early exit case added in 3bbbe467f29 (bundle verify: error out if called without an object database, 2019-05-27). For the relevant bisect.c code see 45b6370812c (bisect: libify `check_good_are_ancestors_of_bad` and its dependents, 2020-02-17). For the submodule.c code see the "goto" on "(!left || !right || !sub)" added in 8e6df65015f (submodule: refactor show_submodule_summary with helper function, 2016-08-31). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-23bundle: output hash information in 'verify'Derrick Stolee1-0/+2
The previous change moved the 'filter' capability to the end of the 'git bundle verify' output. Now, add the 'object-format' capability to the output, when it exists. This change makes 'git bundle verify' output the hash used in all cases, even if the capability is not in the bundle. This means that v2 bundles will always output that they use "sha1". This might look noisy to some users, but it does simplify the implementation and the test strategy for this feature. Since 'verify' ends early when a prerequisite commit is missing, we need to insert this hash message carefully into our expected test output throughout t6020. Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-23bundle: move capabilities to end of 'verify'Derrick Stolee1-5/+4
The 'filter' capability was added in 105c6f14a (bundle: parse filter capability, 2022-03-09), but was added in a strange place in the 'git bundle verify' output. The tests for this show output like the following: The bundle contains these 2 refs: <COMMIT1> <REF1> <COMMIT2> <REF2> The bundle uses this filter: blob:none The bundle records a complete history. This looks very odd if we have a thin bundle that contains boundary commits instead of a complete history: The bundle contains these 2 refs: <COMMIT1> <REF1> <COMMIT2> <REF2> The bundle uses this filter: blob:none The bundle requires these 2 refs: <COMMIT3> <COMMIT4> This separation between tip refs and boundary refs is unfortunate. Move the filter capability output to the end of the output. Update the documentation to match. Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-21Merge branch 'ds/partial-bundles'Junio C Hamano1-12/+62
Bundle file format gets extended to allow a partial bundle, filtered by similar criteria you would give when making a partial/lazy clone. * ds/partial-bundles: clone: fail gracefully when cloning filtered bundle bundle: unbundle promisor packs bundle: create filtered bundles rev-list: move --filter parsing into revision.c bundle: parse filter capability list-objects: handle NULL function pointers MyFirstObjectWalk: update recommended usage list-objects: consolidate traverse_commit_list[_filtered] pack-bitmap: drop filter in prepare_bitmap_walk() pack-objects: use rev.filter when possible revision: put object filter into struct rev_info list-objects-filter-options: create copy helper index-pack: document and test the --promisor option
2022-03-09bundle: unbundle promisor packsDerrick Stolee1-0/+4
In order to have a valid pack-file after unbundling a bundle that has the 'filter' capability, we need to generate a .promisor file. The bundle does not promise _where_ the objects can be found, but we can expect that these bundles will be unbundled in repositories with appropriate promisor remotes that can find those missing objects. Use the 'git index-pack --promisor=<message>' option to create this .promisor file. Add "from-bundle" as the message to help anyone diagnose issues with these promisor packs. Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-09bundle: create filtered bundlesDerrick Stolee1-11/+42
A previous change allowed Git to parse bundles with the 'filter' capability. Now, teach Git to create bundles with this option. Some rearranging of code is required to get the option parsing in the correct spot. There are now two reasons why we might need capabilities (a new hash algorithm or an object filter) so that is pulled out into a place where we can check both at the same time. The --filter option is parsed as part of setup_revisions(), but it expected the --objects flag, too. That flag is somewhat implied by 'git bundle' because it creates a pack-file walking objects, but there is also a walk that walks the revision range expecting only commits. Make this parsing work by setting 'revs.tree_objects' and 'revs.blob_objects' before the call to setup_revisions(). Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-09bundle: parse filter capabilityDerrick Stolee1-1/+14
The v3 bundle format has capabilities, allowing newer versions of Git to create bundles with newer features. Older versions that do not understand these new capabilities will fail with a helpful warning. Create a new capability allowing Git to understand that the contained pack-file is filtered according to some object filter. Typically, this filter will be "blob:none" for a blobless partial clone. This change teaches Git to parse this capability, place its value in the bundle header, and demonstrate this understanding by adding a message to 'git bundle verify'. Since we will use gently_parse_list_objects_filter() outside of list-objects-filter-options.c, make it an external method and move its API documentation to before its declaration. Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-09list-objects: handle NULL function pointersÆvar Arnfjörð Bjarmason1-0/+2
If a caller to traverse_commit_list() specifies the options for the --objects flag but does not specify a show_object function pointer, the result is a segfault. This is currently visible by running 'git bundle create --objects HEAD'. We could fix this problem by supplying a no-op callback in builtin/bundle.c, but that only solves the problem for one builtin, leaving this segfault open for other callers. Replace all callers of the show_commit and show_object function pointers in list-objects.c to call helper functions show_commit() and show_object() which check that the given context has non-NULL functions before passing the necessary data. One extra benefit is that it reduces duplication due to passing ctx->show_data to every caller. Test that this segfault no longer occurs for 'git bundle'. Co-authored-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-07string-list API: change "nr" and "alloc" to "size_t"Ævar Arnfjörð Bjarmason1-4/+4
Change the "nr" and "alloc" members of "struct string_list" to use "size_t" instead of "nr". On some platforms the size of an "unsigned int" will be smaller than a "size_t", e.g. a 32 bit unsigned v.s. 64 bit unsigned. As "struct string_list" is a generic API we use in a lot of places this might cause overflows. As one example: code in "refs.c" keeps track of the number of refs with a "size_t", and auxiliary code in builtin/remote.c in get_ref_states() appends those to a "struct string_list". While we're at it split the "nr" and "alloc" in string-list.h across two lines, which is the case for most such struct member declarations (e.g. in "strbuf.h" and "strvec.h"). Changing e.g. "int i" to "size_t i" in run_and_feed_hook() isn't strictly necessary, and there are a lot more cases where we'll use a local "int", "unsigned int" etc. variable derived from the "nr" in the "struct string_list". But in that case as well as add_wrapped_shortlog_msg() in builtin/shortlog.c we need to adjust the printf format referring to "nr" anyway, so let's also change the other variables referring to it. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-07bundle API: change "flags" to be "extra_index_pack_args"Ævar Arnfjörð Bjarmason1-6/+6
Since the "flags" parameter was added in be042aff24c (Teach progress eye-candy to fetch_refs_from_bundle(), 2011-09-18) there's never been more than the one flag: BUNDLE_VERBOSE. Let's have the only caller who cares about that pass "-v" itself instead through new "extra_index_pack_args" parameter. The flexibility of being able to pass arbitrary arguments to "unbundle" will be used in a subsequent commit. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-06bundle: remove "ref_list" in favor of string-list.c APIÆvar Arnfjörð Bjarmason1-24/+28
Move away from the "struct ref_list" in bundle.c in favor of the almost identical string-list.c API. That API fits this use-case perfectly, but did not exist in its current form when this code was added in 2e0afafebd (Add git-bundle: move objects and references by archive, 2007-02-22), with hindsight we could have used the path-list API, which later got renamed to string-list. See 8fd2cb4069 (Extract helper bits from c-merge-recursive work, 2006-07-25) We need to change "name" to "string" and "oid" to "util" to make this conversion, but other than that the APIs are pretty much identical for what bundle.c made use of. Let's also replace the memset(..,0,...) pattern with a more idiomatic "INIT" macro, and finally add a *_release() function so to free the allocated memory. Before this the add_to_ref_list() would leak memory, now e.g. "bundle list-heads" reports no memory leaks at all under valgrind. In the bundle_header_init() function we're using a clever trick to memcpy() what we'd get from the corresponding BUNDLE_HEADER_INIT. There is a concurrent series to make use of that pattern more generally, see [1]. 1. https://lore.kernel.org/git/cover-0.5-00000000000-20210701T104855Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-06bundle.c: use a temporary variable for OIDs and namesÆvar Arnfjörð Bjarmason1-8/+18
In preparation for moving away from accessing the OID and name via the "oid" and "name" slots in a subsequent commit, change the code that accesses it to use named variables. This makes the subsequent change smaller. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-11bundle: arguments can be read from stdinJiang Xin1-50/+59
In order to create an incremental bundle, we need to pass many arguments to let git-bundle ignore some already packed commits. It will be more convenient to pass args via stdin. But the current implementation does not allow us to do this. This is because args are parsed twice when creating bundle. The first time for parsing args is in `compute_and_write_prerequisites()` by running `git-rev-list` command to write prerequisites in bundle file, and stdin is consumed in this step if "--stdin" option is provided for `git-bundle`. Later nothing can be read from stdin when running `setup_revisions()` in `create_bundle()`. The solution is to parse args once by removing the entire function `compute_and_write_prerequisites()` and then calling function `setup_revisions()`. In order to write prerequisites for bundle, will call `prepare_revision_walk()` and `traverse_commit_list()`. But after calling `prepare_revision_walk()`, the object array `revs.pending` is left empty, and the following steps could not work properly with the empty object array (`revs.pending`). Therefore, make a copy of `revs` to `revs_copy` for later use right after calling `setup_revisions()`. The copy of `revs_copy` is not a deep copy, it shares the same objects with `revs`. The object array of `revs` has been cleared, but objects themselves are still kept. Flags of objects may change after calling `prepare_revision_walk()`, we can use these changed flags without calling the `git rev-list` command and parsing its output like the former implementation. Also add testcases for git bundle in t6020, which read args from stdin. Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-09Merge branch 'jt/interpret-branch-name-fallback'Junio C Hamano1-1/+1
"git status" has trouble showing where it came from by interpreting reflog entries that recordcertain events, e.g. "checkout @{u}", and gives a hard/fatal error. Even though it inherently is impossible to give a correct answer because the reflog entries lose some information (e.g. "@{u}" does not record what branch the user was on hence which branch 'the upstream' needs to be computed, and even if the record were available, the relationship between branches may have changed), at least hide the error to allow "status" show its output. * jt/interpret-branch-name-fallback: wt-status: tolerate dangling marks refs: move dwim_ref() to header file sha1-name: replace unsigned int with option struct
2020-09-02wt-status: tolerate dangling marksJonathan Tan1-1/+1
When a user checks out the upstream branch of HEAD, the upstream branch not being a local branch, and then runs "git status", like this: git clone $URL client cd client git checkout @{u} git status no status is printed, but instead an error message: fatal: HEAD does not point to a branch (This error message when running "git branch" persists even after checking out other things - it only stops after checking out a branch.) This is because "git status" reads the reflog when determining the "HEAD detached" message, and thus attempts to DWIM "@{u}", but that doesn't work because HEAD no longer points to a branch. Therefore, when calculating the status of a worktree, tolerate dangling marks. This is done by adding an additional parameter to dwim_ref() and repo_dwim_ref(). Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-11Merge branch 'bc/sha-256-part-3'Junio C Hamano1-21/+62
The final leg of SHA-256 transition. * bc/sha-256-part-3: (39 commits) t: remove test_oid_init in tests docs: add documentation for extensions.objectFormat ci: run tests with SHA-256 t: make SHA1 prerequisite depend on default hash t: allow testing different hash algorithms via environment t: add test_oid option to select hash algorithm repository: enable SHA-256 support by default setup: add support for reading extensions.objectformat bundle: add new version for use with SHA-256 builtin/verify-pack: implement an --object-format option http-fetch: set up git directory before parsing pack hashes t0410: mark test with SHA1 prerequisite t5308: make test work with SHA-256 t9700: make hash size independent t9500: ensure that algorithm info is preserved in config t9350: make hash size independent t9301: make hash size independent t9300: use $ZERO_OID instead of hard-coded object ID t9300: abstract away SHA-1-specific constants t8011: make hash size independent ...
2020-07-30strvec: rename struct fieldsJeff King1-1/+1
The "argc" and "argv" names made sense when the struct was argv_array, but now they're just confusing. Let's rename them to "nr" (which we use for counts elsewhere) and "v" (which is rather terse, but reads well when combined with typical variable names like "args.v"). Note that we have to update all of the callers immediately. Playing tricks with the preprocessor is hard here, because we wouldn't want to rewrite unrelated tokens. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-30bundle: add new version for use with SHA-256brian m. carlson1-21/+62
Currently we detect the hash algorithm in use by the length of the object ID. This is inelegant and prevents us from using a different hash algorithm that is also 256 bits in length. Since we cannot extend the v2 format in a backward-compatible way, let's add a v3 format, which is identical, except for the addition of capabilities, which are prefixed by an at sign. We add "object-format" as the only capability and reject unknown capabilities, since we do not have a network connection and therefore cannot negotiate with the other side. For compatibility, default to the v2 format for SHA-1 and require v3 for SHA-256. In t5510, always use format v3 so we can be sure we produce consistent results across hash algorithms. Since head -n N lists the top N lines instead of the Nth line, let's run our output through sed to normalize it and compare it against a fixed value, which will make sure we get exactly what we're expecting. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-28strvec: fix indentation in renamed callsJeff King1-5/+5
Code which split an argv_array call across multiple lines, like: argv_array_pushl(&args, "one argument", "another argument", "and more", NULL); was recently mechanically renamed to use strvec, which results in mis-matched indentation like: strvec_pushl(&args, "one argument", "another argument", "and more", NULL); Let's fix these up to align the arguments with the opening paren. I did this manually by sifting through the results of: git jump grep 'strvec_.*,$' and liberally applying my editor's auto-format. Most of the changes are of the form shown above, though I also normalized a few that had originally used a single-tab indentation (rather than our usual style of aligning with the open paren). I also rewrapped a couple of obvious cases (e.g., where previously too-long lines became short enough to fit on one), but I wasn't aggressive about it. In cases broken to three or more lines, the grouping of arguments is sometimes meaningful, and it wasn't worth my time or reviewer time to ponder each case individually. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-28strvec: convert more callers away from argv_array nameJeff King1-6/+6
We eventually want to drop the argv_array name and just use strvec consistently. There's no particular reason we have to do it all at once, or care about interactions between converted and unconverted bits. Because of our preprocessor compat layer, the names are interchangeable to the compiler (so even a definition and declaration using different names is OK). This patch converts remaining files from the first half of the alphabet, to keep the diff to a manageable size. The conversion was done purely mechanically with: git ls-files '*.c' '*.h' | xargs perl -i -pe ' s/ARGV_ARRAY/STRVEC/g; s/argv_array/strvec/g; ' and then selectively staging files with "git add '[abcdefghjkl]*'". We'll deal with any indentation/style fallouts separately. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-28strvec: rename files from argv-array to strvecJeff King1-1/+1
This requires updating #include lines across the code-base, but that's all fairly mechanical, and was done with: git ls-files '*.c' '*.h' | xargs perl -i -pe 's/argv-array.h/strvec.h/' Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-19bundle: detect hash algorithm when reading refsbrian m. carlson1-1/+21
Much like with the dumb HTTP transport, there isn't a way to explicitly specify the hash algorithm when dealing with a bundle, so detect the algorithm based on the length of the object IDs in the prerequisites and ref advertisements. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-11bundle-create: progress output controlRobin H. Johnson1-4/+5
Support the progress output options from pack-objects in git-bundle's create subcommand. Most notably, this provides --quiet as requested on the git mailing list per [1] Reference: https://www.mail-archive.com/git@vger.kernel.org/msg182844.html <robbat2-20190806T191156-796782357Z@orbis-terrarum.net> Signed-off-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-19bundle: switch to use the_hash_algobrian m. carlson1-2/+2
Switch a use of the constant 40 and a use of GIT_SHA1_HEXSZ to use the_hash_algo instead. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-28bundle verify: error out if called without an object databaseJohannes Schindelin1-0/+3
The deal with bundles is: they really are thin packs, with very little sugar on top. So we really need a repository (or more appropriately, an object database) to work with, when asked to verify a bundle. Let's error out with a useful error message if `git bundle verify` is called without such an object database to work with. Reported by Konstantin Ryabitsev. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-24create_bundle(): drop unused "header" parameterJeff King1-2/+2
There's no need to pass a header struct to create_bundle(); it writes the header information directly to a descriptor (and does not report back details to the caller). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-04Merge branch 'nd/the-index'Junio C Hamano1-12/+14
More codepaths become aware of working with in-core repository instance other than the default "the_repository". * nd/the-index: (22 commits) rebase-interactive.c: remove the_repository references rerere.c: remove the_repository references pack-*.c: remove the_repository references pack-check.c: remove the_repository references notes-cache.c: remove the_repository references line-log.c: remove the_repository reference diff-lib.c: remove the_repository references delta-islands.c: remove the_repository references cache-tree.c: remove the_repository references bundle.c: remove the_repository references branch.c: remove the_repository reference bisect.c: remove the_repository reference blame.c: remove implicit dependency the_repository sequencer.c: remove implicit dependency on the_repository sequencer.c: remove implicit dependency on the_index transport.c: remove implicit dependency on the_index notes-merge.c: remove implicit dependency the_repository notes-merge.c: remove implicit dependency on the_index list-objects.c: reduce the_repository references list-objects-filter.c: remove implicit dependency on the_index ...
2018-11-18Merge branch 'jk/close-duped-fd-before-unlock-for-bundle'Junio C Hamano1-21/+18
When "git bundle" aborts due to an empty commit ranges (i.e. resulting in an empty pack), it left a file descriptor to an lockfile open, which resulted in leftover lockfile on Windows where you cannot remove a file with an open file descriptor. This has been corrected. * jk/close-duped-fd-before-unlock-for-bundle: bundle: dup() output descriptor closer to point-of-use
2018-11-17bundle: dup() output descriptor closer to point-of-useJeff King1-21/+18
When writing a bundle to a file, the bundle code actually creates "your.bundle.lock" using our lockfile interface. We feed that output descriptor to a child git-pack-objects via run-command, which has the quirk that it closes the output descriptor in the parent. To avoid confusing the lockfile code (which still thinks the descriptor is valid), we dup() it, and operate on the duplicate. However, this has a confusing side effect: after the dup() but before we call pack-objects, we have _two_ descriptors open to the lockfile. If we call die() during that time, the lockfile code will try to clean up the partially-written file. It knows to close() the file before unlinking, since on some platforms (i.e., Windows) the open file would block the deletion. But it doesn't know about the duplicate descriptor. On Windows, triggering an error at the right part of the code will result in the cleanup failing and the lockfile being left in the filesystem. We can solve this by moving the dup() much closer to start_command(), shrinking the window in which we have the second descriptor open. It's easy to place this in such a way that no die() is possible. We could still die due to a signal in the exact wrong moment, but we already tolerate races there (e.g., a signal could come before we manage to put the file on the cleanup list in the first place). As a bonus, this shields create_bundle() itself from the duplicate-fd trick, and we can simplify its error handling (note that the lock rollback now happens unconditionally, but that's OK; it's a noop if we didn't open the lock in the first place). The included test uses an empty bundle to cause a failure at the right spot in the code, because that's easy to trigger (the other likely errors are write() problems like ENOSPC). Note that it would already pass on non-Windows systems (because they are happy to unlink an already-open file). Based-on-a-patch-by: Gaël Lhez <gael.lhez@gmail.com> Signed-off-by: Jeff King <peff@peff.net> Tested-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-12bundle.c: remove the_repository referencesNguyễn Thái Ngọc Duy1-12/+14
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-19Merge branch 'nd/the-index'Junio C Hamano1-2/+2
Various codepaths in the core-ish part learn to work on an arbitrary in-core index structure, not necessarily the default instance "the_index". * nd/the-index: (23 commits) revision.c: reduce implicit dependency the_repository revision.c: remove implicit dependency on the_index ws.c: remove implicit dependency on the_index tree-diff.c: remove implicit dependency on the_index submodule.c: remove implicit dependency on the_index line-range.c: remove implicit dependency on the_index userdiff.c: remove implicit dependency on the_index rerere.c: remove implicit dependency on the_index sha1-file.c: remove implicit dependency on the_index patch-ids.c: remove implicit dependency on the_index merge.c: remove implicit dependency on the_index merge-blobs.c: remove implicit dependency on the_index ll-merge.c: remove implicit dependency on the_index diff-lib.c: remove implicit dependency on the_index read-cache.c: remove implicit dependency on the_index diff.c: remove implicit dependency on the_index grep.c: remove implicit dependency on the_index diff.c: remove the_index dependency in textconv() functions blame.c: rename "repo" argument to "r" combine-diff.c: remove implicit dependency on the_index ...
2018-09-21revision.c: remove implicit dependency on the_indexNguyễn Thái Ngọc Duy1-2/+2
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-29convert "oidcmp() != 0" to "!oideq()"Jeff King1-1/+1
This is the flip side of the previous two patches: checking for a non-zero oidcmp() can be more strictly expressed as inequality. Like those patches, we write "!= 0" in the coccinelle transformation, which covers by isomorphism the more common: if (oidcmp(E1, E2)) As with the previous two patches, this patch can be achieved almost entirely by running "make coccicheck"; the only differences are manual line-wrap fixes to match the original code. There is one thing to note for anybody replicating this, though: coccinelle 1.0.4 seems to miss the case in builtin/tag.c, even though it's basically the same as all the others. Running with 1.0.7 does catch this, so presumably it's just a coccinelle bug that was fixed in the interim. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-29commit: add repository argument to lookup_commit_referenceStefan Beller1-1/+2
Add a repository argument to allow callers of lookup_commit_reference to be more specific about which repository to handle. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet. As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-29commit: add repository argument to lookup_commit_reference_gentlyStefan Beller1-1/+1
Add a repository argument to allow callers of lookup_commit_reference_gently to be more specific about which repository to handle. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet. As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-29object: add repository argument to parse_objectStefan Beller1-2/+3
Add a repository argument to allow the callers of parse_object to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet. As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-29Merge branch 'sb/object-store-grafts' into sb/object-store-lookupJunio C Hamano1-0/+1
* sb/object-store-grafts: commit: allow lookup_commit_graft to handle arbitrary repositories commit: allow prepare_commit_graft to handle arbitrary repositories shallow: migrate shallow information into the object parser path.c: migrate global git_path_* to take a repository argument cache: convert get_graft_file to handle arbitrary repositories commit: convert read_graft_file to handle arbitrary repositories commit: convert register_commit_graft to handle arbitrary repositories commit: convert commit_graft_pos() to handle arbitrary repositories shallow: add repository argument to is_repository_shallow shallow: add repository argument to check_shallow_file_for_update shallow: add repository argument to register_shallow shallow: add repository argument to set_alternate_shallow_file commit: add repository argument to lookup_commit_graft commit: add repository argument to prepare_commit_graft commit: add repository argument to read_graft_file commit: add repository argument to register_commit_graft commit: add repository argument to commit_graft_pos object: move grafts to object parser object-store: move object access functions to object-store.h
2018-05-30Merge branch 'ma/lockfile-cleanup'Junio C Hamano1-1/+1
Code clean-up to adjust to a more recent lockfile API convention that allows lockfile instances kept on the stack. * ma/lockfile-cleanup: lock_file: move static locks into functions lock_file: make function-local locks non-static refs.c: do not die if locking fails in `delete_pseudoref()` refs.c: do not die if locking fails in `write_pseudoref()` t/helper/test-write-cache: clean up lock-handling
2018-05-16object-store: move object access functions to object-store.hStefan Beller1-0/+1
This should make these functions easier to find and cache.h less overwhelming to read. In particular, this moves: - read_object_file - oid_object_info - write_object_file As a result, most of the codebase needs to #include object-store.h. In this patch the #include is only added to files that would fail to compile otherwise. It would be better to #include wherever identifiers from the header are used. That can happen later when we have better tooling for it. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-10lock_file: make function-local locks non-staticMartin Ågren1-1/+1
Placing `struct lock_file`s on the stack used to be a bad idea, because the temp- and lockfile-machinery would keep a pointer into the struct. But after 076aa2cbd (tempfile: auto-allocate tempfiles on heap, 2017-09-05), we can safely have lockfiles on the stack. (This applies even if a user returns early, leaving a locked lock behind.) These `struct lock_file`s are local to their respective functions and we can drop their staticness. For good measure, I have inspected these sites and come to believe that they always release the lock, with the possible exception of bailing out using `die()` or `exit()` or by returning from a `cmd_foo()`. As pointed out by Jeff King, it would be bad if someone held on to a `struct lock_file *` for some reason. After some grepping, I agree with his findings: no-one appears to be doing that. Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-14sha1_file: convert read_sha1_file to struct object_idbrian m. carlson1-1/+1
Convert read_sha1_file to take a pointer to struct object_id and rename it read_object_file. Do the same for read_sha1_file_extended. Convert one use in grep.c to use the new function without any other code change, since the pointer being passed is a void pointer that is already initialized with a pointer to struct object_id. Update the declaration and definitions of the modified functions, and apply the following semantic patch to convert the remaining callers: @@ expression E1, E2, E3; @@ - read_sha1_file(E1.hash, E2, E3) + read_object_file(&E1, E2, E3) @@ expression E1, E2, E3; @@ - read_sha1_file(E1->hash, E2, E3) + read_object_file(E1, E2, E3) @@ expression E1, E2, E3, E4; @@ - read_sha1_file_extended(E1.hash, E2, E3, E4) + read_object_file_extended(&E1, E2, E3, E4) @@ expression E1, E2, E3, E4; @@ - read_sha1_file_extended(E1->hash, E2, E3, E4) + read_object_file_extended(E1, E2, E3, E4) Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-23Merge branch 'rs/lose-leak-pending'Junio C Hamano1-19/+16
API clean-up around revision traversal. * rs/lose-leak-pending: commit: remove unused function clear_commit_marks_for_object_array() revision: remove the unused flag leak_pending checkout: avoid using the rev_info flag leak_pending bundle: avoid using the rev_info flag leak_pending bisect: avoid using the rev_info flag leak_pending object: add clear_commit_marks_all() ref-filter: use clear_commit_marks_many() in do_merge_filter() commit: use clear_commit_marks_many() in remove_redundant() commit: avoid allocation in clear_commit_marks_many()
2017-12-28bundle: avoid using the rev_info flag leak_pendingRené Scharfe1-19/+16
The leak_pending flag is so awkward to use that multiple comments had to be added around each occurrence. We use it for remembering the prerequisites for the bundle. That is easy, though: We have the ref_list named "prerequisites" in the header for just that purpose. Use this original list of prerequisites to check if all of them are present and to clear their commit marks afterward. The two new loops are intentionally kept similar to the first one in the function. Calling parse_object() a second time is expected be quick and successful in each case -- any errors should have been handled in the first round. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-16refs: convert dwim_ref and expand_ref to struct object_idbrian m. carlson1-1/+1
All of the callers of these functions just pass the hash member of a struct object_id, so convert them to use a pointer to struct object_id directly. Insert a check for NULL in expand_ref on a temporary basis; this check can be removed when resolve_ref_unsafe is converted as well. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-16refs: convert read_ref and read_ref_full to object_idbrian m. carlson1-1/+1
All but two of the call sites already have parameters using the hash parameter of struct object_id, so convert them to take a pointer to the struct directly. Also convert refs_read_refs_full, the underlying implementation. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-24leak_pending: use `object_array_clear()`, not `free()`Martin Ågren1-1/+8
Setting `leak_pending = 1` tells `prepare_revision_walk()` not to release the `pending` array, and makes that the caller's responsibility. See 4a43d374f (revision: add leak_pending flag, 2011-10-01) and 353f5657a (bisect: use leak_pending flag, 2011-10-01). Commit 1da1e07c8 (clean up name allocation in prepare_revision_walk, 2014-10-15) fixed a memory leak in `prepare_revision_walk()` by switching from `free()` to `object_array_clear()`. However, where we use the `leak_pending`-mechanism, we're still only calling `free()`. Use `object_array_clear()` instead. Copy some helpful comments from 353f5657a to the other callers that we update to clarify the memory responsibilities, and to highlight that the commits are not affected when we clear the array -- it is indeed correct to both tidy up the commit flags and clear the object array. Document `leak_pending` in revision.h to help future users get this right. Signed-off-by: Martin Ågren <martin.agren@gmail.com> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-29Merge branch 'bc/object-id'Junio C Hamano1-18/+21
Conversion from uchar[20] to struct object_id continues. * bc/object-id: (53 commits) object: convert parse_object* to take struct object_id tree: convert parse_tree_indirect to struct object_id sequencer: convert do_recursive_merge to struct object_id diff-lib: convert do_diff_cache to struct object_id builtin/ls-tree: convert to struct object_id merge: convert checkout_fast_forward to struct object_id sequencer: convert fast_forward_to to struct object_id builtin/ls-files: convert overlay_tree_on_cache to object_id builtin/read-tree: convert to struct object_id sha1_name: convert internals of peel_onion to object_id upload-pack: convert remaining parse_object callers to object_id revision: convert remaining parse_object callers to object_id revision: rename add_pending_sha1 to add_pending_oid http-push: convert process_ls_object and descendants to object_id refs/files-backend: convert many internals to struct object_id refs: convert struct ref_update to use struct object_id ref-filter: convert some static functions to struct object_id Convert struct ref_array_item to struct object_id Convert the verify_pack callback to struct object_id Convert lookup_tag to struct object_id ...
2017-05-08object: convert parse_object* to take struct object_idbrian m. carlson1-4/+6
Make parse_object, parse_object_or_die, and parse_object_buffer take a pointer to struct object_id. Remove the temporary variables inserted earlier, since they are no longer necessary. Transform all of the callers using the following semantic patch: @@ expression E1; @@ - parse_object(E1.hash) + parse_object(&E1) @@ expression E1; @@ - parse_object(E1->hash) + parse_object(E1) @@ expression E1, E2; @@ - parse_object_or_die(E1.hash, E2) + parse_object_or_die(&E1, E2) @@ expression E1, E2; @@ - parse_object_or_die(E1->hash, E2) + parse_object_or_die(E1, E2) @@ expression E1, E2, E3, E4, E5; @@ - parse_object_buffer(E1.hash, E2, E3, E4, E5) + parse_object_buffer(&E1, E2, E3, E4, E5) @@ expression E1, E2, E3, E4, E5; @@ - parse_object_buffer(E1->hash, E2, E3, E4, E5) + parse_object_buffer(E1, E2, E3, E4, E5) Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-08Convert lookup_commit* to struct object_idbrian m. carlson1-1/+1
Convert lookup_commit, lookup_commit_or_die, lookup_commit_reference, and lookup_commit_reference_gently to take struct object_id arguments. Introduce a temporary in parse_object buffer in order to convert this function. This is required since in order to convert parse_object and parse_object_buffer, lookup_commit_reference_gently and lookup_commit_or_die would need to be converted. Not introducing a temporary would therefore require that lookup_commit_or_die take a struct object_id *, but lookup_commit would take unsigned char *, leaving a confusing and hard-to-use interface. parse_object_buffer will lose this temporary in a later patch. This commit was created with manual changes to commit.c, commit.h, and object.c, plus the following semantic patch: @@ expression E1, E2; @@ - lookup_commit_reference_gently(E1.hash, E2) + lookup_commit_reference_gently(&E1, E2) @@ expression E1, E2; @@ - lookup_commit_reference_gently(E1->hash, E2) + lookup_commit_reference_gently(E1, E2) @@ expression E1; @@ - lookup_commit_reference(E1.hash) + lookup_commit_reference(&E1) @@ expression E1; @@ - lookup_commit_reference(E1->hash) + lookup_commit_reference(E1) @@ expression E1; @@ - lookup_commit(E1.hash) + lookup_commit(&E1) @@ expression E1; @@ - lookup_commit(E1->hash) + lookup_commit(E1) @@ expression E1, E2; @@ - lookup_commit_or_die(E1.hash, E2) + lookup_commit_or_die(&E1, E2) @@ expression E1, E2; @@ - lookup_commit_or_die(E1->hash, E2) + lookup_commit_or_die(E1, E2) Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-02bundle: convert to struct object_idbrian m. carlson1-16/+17
Convert the bundle code, plus the sole external user of struct ref_list_entry, to use struct object_id. Include cache.h from within bundle.h to provide the definition. Convert some of the hash parsing code to use parse_oid_hex to avoid needing to hard-code constant values. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-27timestamp_t: a new data type for timestampsJohannes Schindelin1-1/+1
Git's source code assumes that unsigned long is at least as precise as time_t. Which is incorrect, and causes a lot of problems, in particular where unsigned long is only 32-bit (notably on Windows, even in 64-bit versions). So let's just use a more appropriate data type instead. In preparation for this, we introduce the new `timestamp_t` data type. By necessity, this is a very, very large patch, as it has to replace all timestamps' data type in one go. As we will use a data type that is not necessarily identical to `time_t`, we need to be very careful to use `time_t` whenever we interact with the system functions, and `timestamp_t` everywhere else. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-23parse_timestamp(): specify explicitly where we parse timestampsJohannes Schindelin1-1/+1
Currently, Git's source code represents all timestamps as `unsigned long`. In preparation for using a more appropriate data type, let's introduce a symbol `parse_timestamp` (currently being defined to `strtoul`) where appropriate, so that we can later easily switch to, say, use `strtoull()` instead. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-01bundle: don't leak an fd in case of early returnStefan Beller1-6/+17
In successful operation `write_pack_data` will close the `bundle_fd`, but when we exit early, we need to take care of the file descriptor as well as the lock file ourselves. The lock file may be deleted at the end of running the program, but we are in library code, so we should not rely on that. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-11-20Remove get_object_hash.brian m. carlson1-1/+1
Convert all instances of get_object_hash to use an appropriate reference to the hash member of the oid member of struct object. This provides no functional change, as it is essentially a macro substitution. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Jeff King <peff@peff.net>
2015-11-20Convert struct object to object_idbrian m. carlson1-9/+9
struct object is one of the major data structures dealing with object IDs. Convert it to use struct object_id instead of an unsigned char array. Convert get_object_hash to refer to the new member as well. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Jeff King <peff@peff.net>
2015-11-20Add several uses of get_object_hash.brian m. carlson1-1/+1
Convert most instances where the sha1 member of struct object is dereferenced to use get_object_hash. Most instances that are passed to functions that have versions taking struct object_id, such as get_sha1_hex/get_oid_hex, or instances that can be trivially converted to use struct object_id instead, are not converted. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Jeff King <peff@peff.net>
2015-08-10create_bundle(): duplicate file descriptor to avoid closing it twiceMichael Haggerty1-10/+16
write_pack_data() passes bundle_fd to start_command() to be used as the stdout of pack-objects. But start_command() closes its stdout if it is > 1. This is a problem if bundle_fd is the fd of a lock_file, because commit_lock_file() will also try to close the fd. So the old code suppressed commit_lock_file()'s usual behavior of closing the file descriptor by setting the lock_file object's fd field to -1. But this is not really kosher. Code here shouldn't be mutating fields within the lock_file object. Instead, duplicate the file descriptor before passing it to write_pack_data(). Then that function can close its copy without closing the copy held in the lock_file object. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-10bundle.c: fix memory leakStefan Beller1-6/+5
There was one continue statement without an accompanying `free(ref)`. Instead of adding that, replace all the free&&continue with a goto just after writing the refs, where we'd do the free anyway and then reloop. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-30bundle: split out ref writing from bundle_createJeff King1-39/+58
The bundle_create() function has a number of logical steps: process the input, write the refs, and write the packfile. Recent commits split the first and third into separate sub-functions. It's worth splitting the middle step out, too, if only because it makes the progression of the steps more obvious. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-30bundle: split out a helper function to compute and write prerequisitesJunio C Hamano1-24/+35
The new helper compute_and_write_prerequistes() is ugly, but it cannot be avoided. Ideally we should avoid a function that computes and does I/O at the same time, but the prerequisites lines in the output needs the human readable title only to help the recipient of the bundle. The code copies them straight from the rev-list output and immediately discards as no other internal computation needs that information. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-30bundle: split out a helper function to create pack dataJunio C Hamano1-27/+37
The create_bundle() function, while it does one single logical thing, takes a rather large implementation to do so. Let's start separating what it does into smaller steps to make it easier to see what is going on. This is a first step to separate out the actual pack-data generation, after the earlier part of the function figures out which part of the history to place in the bundle. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-28use child_process_init() to initialize struct child_process variablesRené Scharfe1-1/+1
Call child_process_init() instead of zeroing the memory of variables of type struct child_process by hand before use because the former is both clearer and shorter. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-15refs.c: change resolve_ref_unsafe reading argument to be a flags fieldRonnie Sahlberg1-1/+1
resolve_ref_unsafe takes a boolean argument for reading (a nonexistent ref resolves successfully for writing but not for reading). Change this to be a flags field instead, and pass the new constant RESOLVE_REF_READING when we want this behaviour. While at it, swap two of the arguments in the function to put output arguments at the end. As a nice side effect, this ensures that we can catch callers that were unaware of the new API so they can be audited. Give the wrapper functions resolve_refdup and read_ref_full the same treatment for consistency. Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-14Merge branch 'rs/plug-leak-in-bundle'Junio C Hamano1-7/+10
* rs/plug-leak-in-bundle: bundle: plug minor memory leak in is_tag_in_date_range()
2014-10-07bundle: plug minor memory leak in is_tag_in_date_range()René Scharfe1-7/+10
Free the buffer returned by read_sha1_file() even if no valid tagger line is found. Signed-off-by: Rene Scharfe <l.s.r@web.de> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-01lockfile.h: extract new header file for the functions in lockfile.cMichael Haggerty1-0/+1
Move the interface declaration for the functions in lockfile.c from cache.h to a new file, lockfile.h. Add #includes where necessary (and remove some redundant includes of cache.h by files that already include builtin.h). Move the documentation of the lock_file state diagram from lockfile.c to the new header file. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-19Merge branch 'lf/bundle-exclusion' into maintJunio C Hamano1-2/+2
* lf/bundle-exclusion: bundle: fix exclusion of annotated tags
2014-09-11Merge branch 'rs/child-process-init'Junio C Hamano1-4/+2
Code clean-up. * rs/child-process-init: run-command: inline prepare_run_command_v_opt() run-command: call run_command_v_opt_cd_env() instead of duplicating it run-command: introduce child_process_init() run-command: introduce CHILD_PROCESS_INIT
2014-09-09Merge branch 'lf/bundle-exclusion'Junio C Hamano1-2/+2
"git bundle create" with date-range specification were meant to exclude tags outside the range * lf/bundle-exclusion: bundle: fix exclusion of annotated tags
2014-08-20run-command: introduce CHILD_PROCESS_INITRené Scharfe1-4/+2
Most struct child_process variables are cleared using memset first after declaration. Provide a macro, CHILD_PROCESS_INIT, that can be used to initialize them statically instead. That's shorter, doesn't require a function call and is slightly more readable (especially given that we already have STRBUF_INIT, ARGV_ARRAY_INIT etc.). Helped-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-08-07bundle: fix exclusion of annotated tagsLukas Fleischer1-2/+2
In commit c9a42c4 (bundle: allow rev-list options to exclude annotated tags, 2009-01-02), support for excluding annotated tags outside the specified date range was added. However, the wrong order of parameters was chosen when calling memchr(). Fix this by swapping the character to search for with the maximum length parameter. Also cover this behavior with an additional test. Signed-off-by: Lukas Fleischer <git@cryptocrack.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-07-18bundle: use internal argv_array of struct child_process in create_bundle()René Scharfe1-10/+5
Use the existing argv_array member instead of providing our own. This way the argv_array is cleared after use automatically for us; it was leaking before. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-03Merge branch 'nd/log-show-linear-break'Junio C Hamano1-0/+1
Attempts to show where a single-strand-of-pearls break in "git log" output. * nd/log-show-linear-break: log: add --show-linear-break to help see non-linear history object.h: centralize object flag allocation
2014-03-25object.h: centralize object flag allocationNguyễn Thái Ngọc Duy1-0/+1
While the field "flags" is mainly used by the revision walker, it is also used in many other places. Centralize the whole flag allocation to one place for a better overview (and easier to move flags if we have too). Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-18Merge branch 'sh/use-hashcpy'Junio C Hamano1-1/+1
* sh/use-hashcpy: Use hashcpy() when copying object names
2014-03-06Use hashcpy() when copying object namesSun He1-1/+1
We invented hashcpy() to keep the abstraction of "object name" behind it. Use it instead of calling memcpy() with hard-coded 20-byte length when moving object names between pieces of memory. Leave ppc/sha1.c as-is, because the function is about the SHA-1 hash algorithm whose output is and will always be 20 bytes. Helped-by: Michael Haggerty <mhagger@alum.mit.edu> Helped-by: Duy Nguyen <pclouds@gmail.com> Signed-off-by: Sun He <sunheehnus@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-03bundle.c: use ALLOC_GROW() in add_to_ref_list()Dmitry S. Dolzhenko1-5/+1
Signed-off-by: Dmitry S. Dolzhenko <dmitrys.dolzhenko@yandex.ru> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-11-12bundle: use argv-arrayJunio C Hamano1-16/+16
Instead of hand-crafted arrays to manage command line arguments we create internally, use argv-array helpers. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-02object_array_entry: fix memory handling of the name fieldMichael Haggerty1-1/+1
Previously, the memory management of the object_array_entry::name field was inconsistent and undocumented. object_array_entries are ultimately created by a single function, add_object_array_with_mode(), which has an argument "const char *name". This function used to simply set the name field to reference the string pointed to by the name parameter, and nobody on the object_array side ever freed the memory. Thus, it assumed that the memory for the name field would be managed by the caller, and that the lifetime of that string would be at least as long as the lifetime of the object_array_entry. But callers were inconsistent: * Some passed pointers to constant strings or argv entries, which was OK. * Some passed pointers to newly-allocated memory, but didn't arrange for the memory ever to be freed. * Some passed the return value of sha1_to_hex(), which is a pointer to a statically-allocated buffer that can be overwritten at any time. * Some passed pointers to refnames that they received from a for_each_ref()-type iteration, but the lifetimes of such refnames is not guaranteed by the refs API. Bring consistency to this mess by changing object_array to make its own copy for the object_array_entry::name field and free this memory when an object_array_entry is deleted from the array. Many callers were passing the empty string as the name parameter, so as a performance optimization, treat the empty string specially. Instead of making a copy, store a pointer to a statically-allocated empty string to object_array_entry::name. When deleting such an entry, skip the free(). Change the callers that were already passing copies to add_object_array_with_mode() to either skip the copy, or (if the memory needed to be allocated anyway) freeing the memory itself. A part of this commit effectively reverts 70d26c6e76 read_revisions_from_stdin: make copies for handle_revision_arg because the copying introduced by that commit (which is still necessary) is now done at a deeper level. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-07bundle: Accept prerequisites without commit messagesLukas Fleischer1-1/+1
While explicitly stating that the commit message in a prerequisite line is optional, we required all lines with 40 or more characters to contain a space after the object name, bailing out if a line consisted of an object name only. This was to allow bundling a history to a commit without an message, but the code forgot that it already called rtrim() to remove that whitespace. As a workaround, only check for SP when the line has more than 40 characters. Signed-off-by: Lukas Fleischer <git@cryptocrack.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-03Merge branch 'jk/fully-peeled-packed-ref' into maint-1.8.1Junio C Hamano1-3/+3
* jk/fully-peeled-packed-ref: pack-refs: add fully-peeled trait pack-refs: write peeled entry for non-tags use parse_object_or_die instead of die("bad object") avoid segfaults on parse_object failure
2013-03-17avoid segfaults on parse_object failureJeff King1-3/+3
Many call-sites of parse_object assume that they will get a non-NULL return value; this is not the case if we encounter an error while parsing the object. This patch adds a wrapper function around parse_object that handles dying automatically, and uses it anywhere we immediately try to access the return value as a non-NULL pointer (i.e., anywhere that we would currently segfault). This wrapper may also be useful in other places. The most obvious one is code like: o = parse_object(sha1); if (!o) die(...); However, these should not be mechanically converted to parse_object_or_die, as the die message is sometimes customized. Later patches can address these sites on a case-by-case basis. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-03-08bundle: Add colons to list headings in "verify"Lukas Fleischer1-4/+4
These slightly improve the reading flow by making it obvious that a list follows. Also, make the wording of both headings consistent by changing "contains %d ref(s)" to "contains this ref"/"contains these %d refs". Signed-off-by: Lukas Fleischer <git@cryptocrack.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-03-07bundle: Fix "verify" output if history is completeLukas Fleischer1-1/+1
A more informative message for "complete" bundles was added in commit 8c3710fd3011 (tweak "bundle verify" of a complete history, 2012-06-04). However, the prerequisites ref list is currently read *after* we check if it equals zero, which means we never actually use the number of prerequisite refs to decide when to print the newly introduced message. The code incorrectly uses the number of references recorded in the bundle instead. Signed-off-by: Lukas Fleischer <git@cryptocrack.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-06-04tweak "bundle verify" of a complete historyJunio C Hamano1-6/+10
A bundle that records a complete history without prerequiste is a useful way to sneakernet the sources of your configuration files under your home directory, etc. E.g. $ GIT_DIR=/srv/git/homesrc.git git bundle create x.bndl HEAD master Running "git bundle verify" on such a "complete" bundle, however, gives somewhat a funny output. $ git bundle verify x.bndl The bundle contains 2 refs b2611f37ebc7ed6435a72d77fbc5f8b48a7d7146 HEAD b2611f37ebc7ed6435a72d77fbc5f8b48a7d7146 refs/heads/master The bundle requires these 0 refs x.bndl is okay Reword "requires these 0 refs" to say "The bundle records a complete history" instead. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-02Merge branch 'nd/i18n'Junio C Hamano1-17/+21
More message strings marked for i18n. By Nguyễn Thái Ngọc Duy (10) and Jonathan Nieder (1) * nd/i18n: help: replace underlining "help -a" headers using hyphens with a blank line i18n: bundle: mark strings for translation i18n: index-pack: mark strings for translation i18n: apply: update say_patch_name to give translators complete sentence i18n: apply: mark strings for translation i18n: remote: mark strings for translation i18n: make warn_dangling_symref() automatically append \n i18n: help: mark strings for translation i18n: mark relative dates for translation strbuf: convenience format functions with \n automatically appended Makefile: feed all header files to xgettext
2012-04-26Sync with 1.7.8.6Junio C Hamano1-1/+1
2012-04-26Sync with 1.7.7.7Junio C Hamano1-1/+1
2012-04-26bundle: remove stray single-quote from error messageJonathan Nieder1-1/+1
After running rev-list --boundary to retrieve the list of boundary commits, "git bundle create" runs its own revision walk. If in this stage git encounters an unfamiliar option, it writes a message with an unbalanced quotation mark: error: unrecognized argument: --foo' Drop the stray quote to match the "unrecognized argument: %s" message used elsewhere and save translators some work. This is mostly a futureproofing measure: for now, the "rev-list --boundary" command catches most strange arguments on its own and the above message is not seen unless you try something esoteric like "git bundle create test.bundle --header HEAD". Reported-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-24i18n: bundle: mark strings for translationNguyễn Thái Ngọc Duy1-17/+21
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-03-12Merge branch 'tr/maint-bundle-boundary' into maintJunio C Hamano1-1/+1
"git bundle" did not record boundary commits correctly when there are many of them. By Thomas Rast * tr/maint-bundle-boundary: bundle: keep around names passed to add_pending_object() t5510: ensure we stay in the toplevel test dir t5510: refactor bundle->pack conversion
2012-03-06Merge branch 'tr/maint-bundle-boundary'Junio C Hamano1-1/+1
By Thomas Rast * tr/maint-bundle-boundary: bundle: keep around names passed to add_pending_object() t5510: ensure we stay in the toplevel test dir t5510: refactor bundle->pack conversion
2012-03-04Merge branch 'tr/maint-bundle-long-subject' into maintJunio C Hamano1-26/+10
* tr/maint-bundle-long-subject: t5704: match tests to modern style strbuf: improve strbuf_get*line documentation bundle: use a strbuf to scan the log for boundary commits bundle: put strbuf_readline_fd in strbuf.c with adjustments
2012-03-01bundle: keep around names passed to add_pending_object()Thomas Rast1-1/+1
The 'name' field passed to add_pending_object() is used to later deduplicate in object_array_remove_duplicates(). git-bundle had a bug in this area since 18449ab (git-bundle: avoid packing objects which are in the prerequisites, 2007-03-08): it passed the name of each boundary object in a static buffer. In other words, all that object_array_remove_duplicates() saw was the name of the *last* added boundary object. The recent switch to a strbuf in bc2fed4 (bundle: use a strbuf to scan the log for boundary commits, 2012-02-22) made this slightly worse: we now free the buffer at the end, so it is not even guaranteed that it still points into addressable memory by the time object_array_remove_ duplicates looks at it. On the plus side however, it was now detectable by valgrind. The fix is easy: pass a copy of the string to add_pending_object. Signed-off-by: Thomas Rast <trast@student.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-02-26Merge branch 'tr/maint-bundle-long-subject'Junio C Hamano1-26/+10
* tr/maint-bundle-long-subject: t5704: match tests to modern style strbuf: improve strbuf_get*line documentation bundle: use a strbuf to scan the log for boundary commits bundle: put strbuf_readline_fd in strbuf.c with adjustments
2012-02-23bundle: use a strbuf to scan the log for boundary commitsThomas Rast1-7/+8
The first part of the bundle header contains the boundary commits, and could be approximated by # v2 git bundle $(git rev-list --pretty=oneline --boundary <ARGS> | grep ^-) git-bundle actually spawns exactly this rev-list invocation, and does the grepping internally. There was a subtle bug in the latter step: it used fgets() with a 1024-byte buffer. If the user has sufficiently long subjects (e.g., by not adhering to the git oneline-subject convention in the first place), the 'oneline' format can easily overflow the buffer. fgets() then returns the rest of the line in the next call(s). If one of these remaining parts started with '-', git-bundle would mistakenly insert it into the bundle thinking it was a boundary commit. Fix it by using strbuf_getwholeline() instead, which handles arbitrary line lengths correctly. Note that on the receiving side in parse_bundle_header() we were already using strbuf_getwholeline_fd(), so that part is safe. Reported-by: Jannis Pohlmann <jannis.pohlmann@codethink.co.uk> Signed-off-by: Thomas Rast <trast@student.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-02-22bundle: put strbuf_readline_fd in strbuf.c with adjustmentsThomas Rast1-19/+2
The comment even said that it should eventually go there. While at it, match the calling convention and name of the function to the strbuf_get*line family. So it now is strbuf_getwholeline_fd. Signed-off-by: Thomas Rast <trast@student.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-01-03Merge branch 'maint'Junio C Hamano1-2/+2
* maint: docs: describe behavior of relative submodule URLs fix hang in git fetch if pointed at a 0 length bundle Documentation: read-tree --prefix works with existing subtrees Add MYMETA.json to perl/.gitignore
2012-01-03fix hang in git fetch if pointed at a 0 length bundleBrian Harring1-2/+2
git-repo if interupted at the exact wrong time will generate zero length bundles- literal empty files. git-repo is wrong here, but git fetch shouldn't effectively spin loop if pointed at a zero length bundle. Signed-off-by: Brian Harring <ferringb@chromium.org> Helped-by: Johannes Sixt Helped-by: Nguyen Thai Ngoc Duy Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-11-13Convert many resolve_ref() calls to read_ref*() and ref_exists()Nguyễn Thái Ngọc Duy1-1/+1
resolve_ref() may return a pointer to a static buffer, which is not safe for long-term use because if another resolve_ref() call happens, the buffer may be changed. Many call sites though do not care about this buffer. They simply check if the return value is NULL or not. Convert all these call sites to new wrappers to reduce resolve_ref() calls from 57 to 34. If we change resolve_ref() prototype later on to avoid passing static buffer out, this helps reduce changes. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-10-21Merge branch 'jc/unseekable-bundle'Junio C Hamano1-36/+88
* jc/unseekable-bundle: bundle: add parse_bundle_header() helper function bundle: allowing to read from an unseekable fd Conflicts: transport.c
2011-10-13Merge branch 'rs/pending'Junio C Hamano1-7/+4
* rs/pending: commit: factor out clear_commit_marks_for_object_array checkout: use leak_pending flag bundle: use leak_pending flag bisect: use leak_pending flag revision: add leak_pending flag checkout: use add_pending_{object,sha1} in orphan check revision: factor out add_pending_sha1 checkout: check for "Previous HEAD" notice in t2020 Conflicts: builtin/checkout.c revision.c
2011-10-13bundle: add parse_bundle_header() helper functionJunio C Hamano1-8/+31
Move most of the code from read_bundle_header() to parse_bundle_header() that takes a file descriptor that is already opened for reading, and make the former responsible only for opening the file and noticing errors. As a logical consequence of this, is_bundle() helper function can be implemented as a non-complaining variant of read_bundle_header() that does not return an open file descriptor, and can be used to tighten the check used to decide the use of bundle transport in transport_get() function. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-10-13bundle: allowing to read from an unseekable fdJunio C Hamano1-35/+64
We wished that "git bundle" to eventually learn to read from a network socket which is not seekable. The current code opens with fopen(), reads the file halfway and run ftell(), and reopens the same file with open() and seeks, to skip the header. This patch by itself does not reach that goal yet, but I think it is a right step in that direction. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-10-03commit: factor out clear_commit_marks_for_object_arrayRené Scharfe1-2/+1
Factor out the code to clear the commit marks for a whole struct object_array from builtin/checkout.c into its own exported function clear_commit_marks_for_object_array and use it in bisect and bundle as well. It handles tags and commits and ignores objects of any other type. Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-10-03bundle: use leak_pending flagRené Scharfe1-5/+3
Instead of creating a copy of the list of pending objects, copy the struct object_array that points to it, turn on leak_pending, and thus cause prepare_revision_walk to leave it to us. And free it once we're done. Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-09-19Teach progress eye-candy to fetch_refs_from_bundle()Junio C Hamano1-2/+5
With the usual "git" transport, a large-ish transfer with "git fetch" and "git pull" give progress eye-candy to avoid boring users. However, not when they are reading from a bundle. I.e. $ git pull ../git-bundle.bndl master This teaches bundle.c:unbundle() to give "-v" option to index-pack and tell it to give progress bar when transport decides it is necessary. The operation in the other direction, "git bundle create", could also learn to honor --quiet but that is a separate issue. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-02-06bundle: Use OFS_DELTA in bundle filesShawn O. Pearce1-2/+3
git-bundle first appeared in 2e0afafe ("Add git-bundle") in Feb 2007, and first shipped in Git 1.5.1. However, OFS_DELTA is an even earlier invention, coming about in eb32d236 ("introduce delta objects with offset to base") in Sep 2006, and first shipped in Git 1.4.4.5. OFS_DELTA is smaller, about 3.2%-5% smaller, and is typically faster to access than REF_DELTA because the exact location of the delta base is available after parsing the object header. Since all bundle aware versions of Git are also OFS_DELTA aware, just make it the default. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Acked-by: Nicolas Pitre <nico@fluxnic.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-27bundle: detect if bundle file cannot be createdCsaba Henk1-2/+4
bundle command silently died with no sign of failure if it could not create the bundle file. (Eg.: its path resovles to a directory, or the parent dir is sticky while file already exists and is owned by someone else.) Signed-off-by: Csaba Henk <csaba@gluster.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-11-23Merge branch 'jc/log-stdin'Junio C Hamano1-10/+2
* jc/log-stdin: Add trivial tests for --stdin option to log family Make --stdin option to "log" family read also pathspecs setup_revisions(): do not call get_pathspec() too early Teach --stdin option to "log" family read_revision_from_stdin(): use strbuf Conflicts: revision.c
2009-11-23pack-objects: split implications of --all-progress from progress activationNicolas Pitre1-1/+1
Currently the --all-progress flag is used to use force progress display during the writing object phase even if output goes to stdout which is primarily the case during a push operation. This has the unfortunate side effect of forcing progress display even if stderr is not a terminal. Let's introduce the --all-progress-implied argument which has the same intent except for actually forcing the activation of any progress display. With this, progress display will be automatically inhibited whenever stderr is not a terminal, or full progress display will be included otherwise. This should let people use 'git push' within a cron job without filling their logs with useless percentage displays. Signed-off-by: Nicolas Pitre <nico@fluxnic.net> Tested-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-11-20Teach --stdin option to "log" familyJunio C Hamano1-10/+2
Move the logic to read revs from standard input that rev-list knows about from it to revision machinery, so that all the users of setup_revisions() can feed the list of revs from the standard input when "--stdin" is used on the command line. Allow some users of the revision machinery that want different semantics from the "--stdin" option to disable it by setting an option in the rev_info structure. This also cleans up the kludge made to bundle.c via cut and paste. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-09-13don't dereference NULL upon fdopen failureJim Meyering1-1/+1
There were several unchecked use of fdopen(); replace them with xfdopen() that checks and dies. Signed-off-by: Jim Meyering <meyering@redhat.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-05-21Terminate argv with NULL before calling setup_revisions()Nguyễn Thái Ngọc Duy1-1/+1
It is convention that argv should be terminated with NULL, even if argc is used to specify the size of argv. setup_revisions() requires this and may segfault otherwise. This patch makes sure that all argv (that I can find) is NULL terminated. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-01-25Merge branch 'js/maint-all-implies-HEAD'Junio C Hamano1-0/+2
* js/maint-all-implies-HEAD: bundle: allow the same ref to be given more than once revision walker: include a detached HEAD in --all
2009-01-17bundle: allow the same ref to be given more than onceJunio C Hamano1-0/+2
"git bundle create x master master" used to create a bundle that lists the same branch (master) twice. Cloning from such a bundle resulted in a needless warning "warning: Duplicated ref: refs/remotes/origin/master". Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-01-05bundle: allow rev-list options to exclude annotated tagsJohannes Schindelin1-0/+32
With options such as "--all --since=2.weeks.ago", annotated tags used to be included, when they should have been excluded. The reason is that we heavily abuse the revision walker to determine what needs to be included or excluded. And the revision walker does not show tags at all (and therefore never marks tags as uninteresting). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-11Fix non-literal format in printf-style callsDaniel Lowe1-2/+2
These were found using gcc 4.3.2-1ubuntu11 with the warning: warning: format not a string literal and no format arguments Incorporated suggestions from Brandon Casey <casey@nrlssc.navy.mil>. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-10-19Enhance hold_lock_file_for_{update,append}() APIJunio C Hamano1-1/+2
This changes the "die_on_error" boolean parameter to a mere "flags", and changes the existing callers of hold_lock_file_for_update/append() functions to pass LOCK_DIE_ON_ERROR. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-07-07Teach git-bundle to read revision arguments from stdin like git-rev-list.Adam Brewster1-2/+11
This patch allows the caller to feed the revision parameters to git-bundle from its standard input. This way, a script do not have to worry about limitation of the length of command line. Documentation/git-bundle.txt says that git-bundle takes arguments acceptable to git-rev-list. Obviously some arguments that git-rev-list handles don't make sense for git-bundle (e.g. --bisect) but --stdin is pretty reasonable. Signed-off-by: Adam Brewster <adambrewster@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-23start_command(), if .in/.out > 0, closes file descriptors, not the callersJohannes Sixt1-2/+3
Callers of start_command() can set the members .in and .out of struct child_process to a value > 0 to specify that this descriptor is used as the stdin or stdout of the child process. Previously, if start_command() was successful, this descriptor was closed upon return. Here we now make sure that the descriptor is also closed in case of failures. All callers are updated not to close the file descriptor themselves after start_command() was called. Note that earlier run_gpg_verify() of git-verify-tag set .out = 1, which worked because start_command() treated this as a special case, but now this is incorrect because it closes the descriptor. The intent here is to inherit stdout to the child, which is achieved by .out = 0. Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-23start_command(), .in/.out/.err = -1: Callers must close the file descriptorJohannes Sixt1-0/+1
By setting .in, .out, or .err members of struct child_process to -1, the callers of start_command() can request that a pipe is allocated that talks to the child process and one end is returned by replacing -1 with the file descriptor. Previously, a flag was set (for .in and .out, but not .err) to signal finish_command() to close the pipe end that start_command() had handed out, so it was optional for callers to close the pipe, and many already do so. Now we make it mandatory to close the pipe. Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-17check return code of prepare_revision_walkMartin Koegler1-1/+2
A failure in prepare_revision_walk can be caused by a not parseable object. Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-01-16Improve use of lockfile APIBrandon Casey1-4/+10
Remove remaining double close(2)'s. i.e. close() before commit_locked_index() or commit_lock_file(). Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-01-10bundle, fast-import: detect write failureJim Meyering1-3/+3
I noticed some unchecked writes. This fixes them. * bundle.c (create_bundle): Die upon write failure. * fast-import.c (keep_pack): Die upon write or close failure. Signed-off-by: Jim Meyering <meyering@redhat.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-01-04Don't access line[-1] for a zero-length "line" from fgets.Jim Meyering1-1/+1
A NUL byte at beginning of file, or just after a newline would provoke an invalid buf[-1] access in a few places. * builtin-grep.c (cmd_grep): Don't access buf[-1]. * builtin-pack-objects.c (get_object_list): Likewise. * builtin-rev-list.c (read_revisions_from_stdin): Likewise. * bundle.c (read_bundle_header): Likewise. * server-info.c (read_pack_info_file): Likewise. * transport.c (insert_packed_refs): Likewise. Signed-off-by: Jim Meyering <meyering@redhat.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-11-22Merge branch 'maint'Junio C Hamano1-1/+8
* maint: Make test scripts executable. bundle create: keep symbolic refs' names instead of resolving them
2007-11-08Style: place opening brace of a function definition at column 1Junio C Hamano1-1/+2
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-09-19Move bundle specific stuff into bundle.[ch]Johannes Schindelin1-0/+343
The transport specific stuff was moved into libgit.a, and the bundle specific stuff will not be left behind. This is a big code move, with one exception: the function unbundle() no longer outputs the list of refs. You have to call list_bundle_refs() yourself for that. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>