aboutsummaryrefslogtreecommitdiffstats
path: root/builtin/submodule--helper.c
AgeCommit message (Collapse)AuthorFilesLines
2022-06-30submodule--helper: remove unused SUPPORT_SUPER_PREFIX flagsÆvar Arnfjörð Bjarmason1-3/+3
Remove the SUPPORT_SUPER_PREFIX flag from "add", "init" and "summary". For the "add" command it hasn't been used since [1], likewise for "init" and "summary" since [2] and [3], respectively. As implemented in 74866d75793 (git: make super-prefix option, 2016-10-07) the SUPPORT_SUPER_PREFIX flag in git.c applies for the entire command, but as implemented in 89c86265576 (submodule helper: support super prefix, 2016-12-08) we assert here in cmd_submodule__helper() that we're not getting the flag unexpectedly. 1. 8c8195e9c3e (submodule--helper: introduce add-clone subcommand, 2021-07-10) 2. 6e7c14e65c8 (submodule update --init: display correct path from submodule, 2017-01-06) 3. 1cf823d8f00 (submodule: remove unnecessary `prefix` based option logic, 2021-06-22) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-30submodule--helper: use correct display path helperGlen Choo1-11/+3
Replace a chunk of code in update_submodule() with an equivalent do_get_submodule_displaypath() invocation. This is already tested by t/t7406-submodule-update.sh:'submodule update --init --recursive from subdirectory', so no tests are added. The two are equivalent because: - Exactly one of recursive_prefix|prefix is non-NULL at a time; prefix is set at the superproject level, and recursive_prefix is set when recursing into submodules. There is also a BUG() statement in get_submodule_displaypath() that asserts that both cannot be non-NULL. - In get_submodule_displaypath(), get_super_prefix() always returns NULL because "--super-prefix" is never passed. Thus calling it is equivalent to calling do_get_submodule_displaypath() with super_prefix = NULL. Therefore: - When recursive_prefix is non-NULL, prefix is NULL, and thus get_submodule_displaypath() just returns prefixed_path. This is identical to calling do_get_submodule_displaypath() with super_prefix = recursive_prefix because the return value is still the concatenation of recursive_prefix + update_data->sm_path. - When prefix is non-NULL, prefixed_path = update_data->sm_path. Thus calling get_submodule_displaypath() with prefixed_path is equivalent to calling do_get_submodule_displaypath() with update_data->sm_path Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-30submodule--helper: don't recreate recursive prefixGlen Choo1-11/+4
update_submodule() uses duplicated code to compute update_data->displaypath and next.recursive_prefix. The latter is just the former with "/" appended to it, and since update_data->displaypath not changed outside of this statement, we can just reuse the already computed result. We can go one step further and remove the reference to next.recursive_prefix altogether. Since it is only used in update_data_to_args() (to compute the "--recursive-prefix" flag for the recursive update child process) we can just use the already computed .displaypath value of there. Delete the duplicated code, and remove the unnecessary reference to next.recursive_prefix. As a bonus, this fixes a memory leak where prefixed_path was never freed (this leak was first reported in [1]). [1] https://lore.kernel.org/git/877a45867ae368bf9e053caedcb6cf421e02344d.1655336146.git.gitgitgadget@gmail.com Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-30submodule--helper update: use display path helperGlen Choo1-14/+5
There are two locations in prepare_to_clone_next_submodule() that manually calculate the submodule display path, but should just use do_get_submodule_displaypath() for consistency. Do this replacement and reorder the code slightly to avoid computing the display path twice. Until the preceding commit this code had never been tested, with our newly added tests we can see that both these sites have been computing the display path incorrectly ever since they were introduced in 48308681b0 (git submodule update: have a dedicated helper for cloning, 2016-02-29) [1]: - The first hunk puts a "/" between recursive_prefix and ce->name, but recursive_prefix already ends with "/". - The second hunk calls relative_path() on recursive_prefix and ce->name, but relative_path() only makes sense when both paths share the same base directory. This is never the case here: - recursive_prefix is the path from the topmost superproject to the current submodule - ce->name is the path from the root of the current submodule to its submodule. so, e.g. recursive_prefix="super" and ce->name="submodule" produces displayname="../super" instead of "super/submodule". [1] I verified this by applying the tests to 48308681b0. Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-30Merge branch 'ab/submodule-cleanup' into gc/submodule-use-super-prefixJunio C Hamano1-24/+50
* ab/submodule-cleanup: git-sh-setup.sh: remove "say" function, change last users git-submodule.sh: use "$quiet", not "$GIT_QUIET" submodule--helper: eliminate internal "--update" option submodule--helper: understand --checkout, --merge and --rebase synonyms submodule--helper: report "submodule" as our name in some "-h" output submodule--helper: rename "absorb-git-dirs" to "absorbgitdirs" submodule update: remove "-v" option submodule--helper: have --require-init imply --init git-submodule.sh: remove unused top-level "--branch" argument git-submodule.sh: make the "$cached" variable a boolean git-submodule.sh: remove unused $prefix variable git-submodule.sh: remove unused sanitize_submodule_env()
2022-06-28submodule--helper: eliminate internal "--update" optionGlen Choo1-20/+13
Follow-up on the preceding commit which taught "git submodule--helper update" to understand "--merge", "--checkout" and "--rebase" and use those options instead of "--update=(rebase|merge|checkout|none)" when the command invokes itself. Unlike the preceding change this isn't strictly necessary to eventually change "git-submodule.sh" so that it invokes "git submodule--helper update" directly, but let's remove this inconsistency in the command-line interface. We shouldn't need to carry special synonyms for existing options in "git submodule--helper" when that command can use the primary documented names instead. But, as seen in the post-image this makes the control flow within "builtin/submodule--helper.c" simpler, we can now write directly to the "update_default" member of "struct update_data" when parsing the options in "module_update()". Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-28submodule--helper: understand --checkout, --merge and --rebase synonymsÆvar Arnfjörð Bjarmason1-0/+30
Understand --checkout, --merge and --rebase synonyms for --update={checkout,merge,rebase}, as well as the short options that 'git submodule' itself understands. This removes a difference between the CLI API of "git submodule" and "git submodule--helper", making it easier to make the latter an alias for the former. See 48308681b07 (git submodule update: have a dedicated helper for cloning, 2016-02-29) for the initial addition of --update. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-28submodule--helper: report "submodule" as our name in some "-h" outputÆvar Arnfjörð Bjarmason1-9/+9
Change the user-facing "git submodule--helper" commands so that they'll report their name as being "git submodule". To a user these commands are internal implementation details, and it doesn't make sense to emit usage about an internal helper when "git submodule" is invoked with invalid options. Before this we'd emit e.g.: $ git submodule absorbgitdirs --blah error: unknown option `blah' usage: git submodule--helper absorbgitdirs [<options>] [<path>...] [...] And: $ git submodule set-url -- -- usage: git submodule--helper set-url [--quiet] <path> <newurl> [...] Now we'll start with "usage: git submodule [...]" in both of those cases. This change does not alter the "list", "name", "clone", "config" and "create-branch" commands, those are internal-only (as an aside; their usage info should probably invoke BUG(...)). This only changes the user-facing commands. The "status", "deinit" and "update" commands are not included in this change, because their usage information already used "submodule" rather than "submodule--helper". I don't think it's currently possible to emit some of this usage information in practice, as git-submodule.sh will catch unknown options, and e.g. it doesn't seem to be possible to get "add" to emit its usage information from "submodule--helper". Though that change may be superfluous now, it's also harmless, and will allow us to eventually dispatch further into "git submodule--helper" from git-submodule.sh, while emitting the correct usage output. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-28submodule--helper: rename "absorb-git-dirs" to "absorbgitdirs"Ævar Arnfjörð Bjarmason1-2/+2
Rename the "absorb-git-dirs" subcommand to "absorbgitdirs", which is what the "git submodule" command itself has called it since the subcommand was implemented in f6f85861400 (submodule: add absorb-git-dir function, 2016-12-12). Having these two be different will make it more tedious to dispatch to eventually dispatch "git submodule--helper" directly, as we'd need to retain this name mapping. So let's get rid of this needless inconsistency. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-28submodule--helper: have --require-init imply --initÆvar Arnfjörð Bjarmason1-1/+4
Adjust code added in 0060fd1511b (clone --recurse-submodules: prevent name squatting on Windows, 2019-09-12) to have the internal --require-init option imply --init, rather than having "git-submodule.sh" add it implicitly. This change doesn't make any difference now, but eliminates another special-case where "git submodule--helper update"'s behavior was different from "git submodule update". This will make it easier to eventually replace the cmd_update() function in git-submodule.sh. We'll still need to keep the distinction between "--init" and "--require-init" in git-submodule.sh. Once cmd_update() gets re-implemented in C we'll be able to change variables and other code related to that, but not yet. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-16submodule--helper: avoid memory leak when fetching submodulesJohannes Schindelin1-0/+1
In c51f8f94e5b3 (submodule--helper: run update procedures from C, 2021-08-24), we added code that first obtains the default remote, and then adds that to a `strvec`. However, we never released the default remote's memory. Reported by Coverity. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-10Merge branch 'ab/env-array'Junio C Hamano1-22/+22
Rename .env_array member to .env in the child_process structure. * ab/env-array: run-command API users: use "env" not "env_array" in comments & names run-command API: rename "env_array" to "env"
2022-06-07Merge branch 'ab/plug-leak-in-revisions'Junio C Hamano1-4/+10
Plug the memory leaks from the trickiest API of all, the revision walker. * ab/plug-leak-in-revisions: (27 commits) revisions API: add a TODO for diff_free(&revs->diffopt) revisions API: have release_revisions() release "topo_walk_info" revisions API: have release_revisions() release "date_mode" revisions API: call diff_free(&revs->pruning) in revisions_release() revisions API: release "reflog_info" in release revisions() revisions API: clear "boundary_commits" in release_revisions() revisions API: have release_revisions() release "prune_data" revisions API: have release_revisions() release "grep_filter" revisions API: have release_revisions() release "filter" revisions API: have release_revisions() release "cmdline" revisions API: have release_revisions() release "mailmap" revisions API: have release_revisions() release "commits" revisions API users: use release_revisions() for "prune_data" users revisions API users: use release_revisions() with UNLEAK() revisions API users: use release_revisions() in builtin/log.c revisions API users: use release_revisions() in http-push.c revisions API users: add "goto cleanup" for release_revisions() stash: always have the owner of "stash_info" free it revisions API users: use release_revisions() needing REV_INFO_INIT revision.[ch]: document and move code declared around "init" ...
2022-06-03Merge branch 'jx/l10n-workflow-change'Junio C Hamano1-1/+1
A workflow change for translators are being proposed. * jx/l10n-workflow-change: l10n: Document the new l10n workflow Makefile: add "po-init" rule to initialize po/XX.po Makefile: add "po-update" rule to update po/XX.po po/git.pot: don't check in result of "make pot" po/git.pot: this is now a generated file Makefile: remove duplicate and unwanted files in FOUND_SOURCE_FILES i18n CI: stop allowing non-ASCII source messages in po/git.pot Makefile: have "make pot" not "reset --hard" Makefile: generate "po/git.pot" from stable LOCALIZED_C Makefile: sort source files before feeding to xgettext
2022-06-03Merge branch 'ds/bundle-uri'Junio C Hamano1-129/+12
Preliminary code refactoring around transport and bundle code. * ds/bundle-uri: bundle.h: make "fd" version of read_bundle_header() public remote: allow relative_url() to return an absolute url remote: move relative_url() http: make http_get_file() external fetch-pack: move --keep=* option filling to a function fetch-pack: add a deref_without_lazy_fetch_extended() dir API: add a generalized path_match_flags() function connect.c: refactor sending of agent & object-format
2022-06-02run-command API users: use "env" not "env_array" in comments & namesÆvar Arnfjörð Bjarmason1-1/+1
Follow-up on a preceding commit which changed all references to the "env_array" when referring to the "struct child_process" member. These changes are all unnecessary for the compiler, but help the code's human readers. All the comments that referred to "env_array" have now been updated, as well as function names and variables that had "env_array" in their name, they now refer to "env". In addition the "out" name for the submodule.h prototype was inconsistent with the function definition's use of "env_array" in submodule.c. Both of them use "env" now. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-02run-command API: rename "env_array" to "env"Ævar Arnfjörð Bjarmason1-21/+21
Start following-up on the rename mentioned in c7c4bdeccf3 (run-command API: remove "env" member, always use "env_array", 2021-11-25) of "env_array" to "env". The "env_array" name was picked in 19a583dc39e (run-command: add env_array, an optional argv_array for env, 2014-10-19) because "env" was taken. Let's not forever keep the oddity of "*_array" for this "struct strvec", but not for its "args" sibling. This commit is almost entirely made with a coccinelle rule[1]. The only manual change here is in run-command.h to rename the struct member itself and to change "env_array" to "env" in the CHILD_PROCESS_INIT initializer. The rest of this is all a result of applying [1]: * make contrib/coccinelle/run_command.cocci.patch * patch -p1 <contrib/coccinelle/run_command.cocci.patch * git add -u 1. cat contrib/coccinelle/run_command.pending.cocci @@ struct child_process E; @@ - E.env_array + E.env @@ struct child_process *E; @@ - E->env_array + E->env I've avoided changing any comments and derived variable names here, that will all be done in the next commit. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-26i18n CI: stop allowing non-ASCII source messages in po/git.potÆvar Arnfjörð Bjarmason1-1/+1
In the preceding commit we moved away from using xgettext(1) to both generate the po/git.pot, and to merge the incrementally generated po/git.pot+ file as we sourced translations from C, shell and Perl. Doing it this way, which dates back to my initial implementation[1][2][3] was conflating two things: With xgettext(1) the --from-code both controls what encoding is specified in the po/git.pot's header, and what encoding we allow in source messages. We don't ever want to allow non-ASCII in *source messages*, and doing so has hid e.g. a buggy message introduced in a6226fd772b (submodule--helper: convert the bulk of cmd_add() to C, 2021-08-10) from us, we'd warn about it before, but only when running "make pot", but the operation would still succeed. Now we'll error out on it when running "make pot". Since the preceding Makefile changes made this easy: let's add a "make check-pot" target with the same prerequisites as the "po/git.pot" target, but without changing the file "po/git.pot". Running it as part of the "static-analysis" CI target will ensure that we catch any such issues in the future. E.g.: $ make check-pot XGETTEXT .build/pot/po/builtin/submodule--helper.c.po xgettext: Non-ASCII string at builtin/submodule--helper.c:3381. Please specify the source encoding through --from-code. make: *** [.build/pot/po/builtin/submodule--helper.c.po] Error 1 1. cd5513a7168 (i18n: Makefile: "pot" target to extract messages marked for translation, 2011-02-22) 2. adc3b2b2767 (Makefile: add xgettext target for *.sh files, 2011-05-14) 3. 5e9637c6297 (i18n: add infrastructure for translating Git with gettext, 2011-11-18) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-16remote: move relative_url()Derrick Stolee1-129/+12
This method was initially written in 63e95beb0 (submodule: port resolve_relative_url from shell to C, 2016-05-15). As we will need similar functionality in the bundle URI feature, extract this to be available in remote.h. The code is almost exactly the same, except for the following trivial differences: * Fix whitespace and wrapping issues with the prototype and argument lists. * Let's call starts_with_dot_{,dot_}slash_native() instead of the functionally identical "starts_with_dot_{,dot_}slash()" wrappers "builtin/submodule--helper.c". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-16dir API: add a generalized path_match_flags() functionÆvar Arnfjörð Bjarmason1-10/+10
Add a path_match_flags() function and have the two sets of starts_with_dot_{,dot_}slash() functions added in 63e95beb085 (submodule: port resolve_relative_url from shell to C, 2016-04-15) and a2b26ffb1a8 (fsck: convert gitmodules url to URL passed to curl, 2020-04-18) be thin wrappers for it. As the latter of those notes the fsck version was copied from the initial builtin/submodule--helper.c version. Since the code added in a2b26ffb1a8 was doing really doing the same as win32_is_dir_sep() added in 1cadad6f658 (git clone <url> C:\cygwin\home\USER\repo' is working (again), 2018-12-15) let's move the latter to git-compat-util.h is a is_xplatform_dir_sep(). We can then call either it or the platform-specific is_dir_sep() from this new function. Let's likewise change code in various other places that was hardcoding checks for "'/' || '\\'" with the new is_xplatform_dir_sep(). As can be seen in those callers some of them still concern themselves with ':' (Mac OS classic?), but let's leave the question of whether that should be consolidated for some other time. As we expect to make wider use of the "native" case in the future, define and use two starts_with_dot_{,dot_}slash_native() convenience wrappers. This makes the diff in builtin/submodule--helper.c much smaller. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-28Merge branch 'gc/submodule-update-part2'Junio C Hamano1-1/+0
"git submodule update" without pathspec should silently skip an uninitialized submodule, but it started to become noisy by mistake. * gc/submodule-update-part2: submodule--helper: fix initialization of warn_if_uninitialized
2022-04-26submodule--helper: fix initialization of warn_if_uninitializedOrgad Shaneh1-1/+0
The .warn_if_uninitialized member was introduced by 48308681 (git submodule update: have a dedicated helper for cloning, 2016-02-29) to submodule_update_clone struct and initialized to false. When c9911c93 (submodule--helper: teach update_data more options, 2022-03-15) moved it to update_data struct, it started to initialize it to true but this change was not explained in its log message. The member is set to true only when pathspec was given, and is used when a submodule that matched the pathspec is found uninitialized to give diagnostic message. "submodule update" without pathspec is supposed to iterate over all submodules (i.e. without pathspec limitation) and update only the initialized submodules, and finding uninitialized submodules during the iteration is a totally expected and normal thing that should not be warned. [jc: added tests] Signed-off-by: Orgad Shaneh <orgads@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-13revisions API users: add "goto cleanup" for release_revisions()Ævar Arnfjörð Bjarmason1-3/+7
Add a release_revisions() to various users of "struct rev_info" which requires a minor refactoring to a "goto cleanup" pattern to use that function. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-13revisions API users: use release_revisions() needing REV_INFO_INITÆvar Arnfjörð Bjarmason1-1/+2
Use release_revisions() to various users of "struct rev_list" which need to have their "struct rev_info" zero-initialized before we can start using it. For the bundle.c code see the early exit case added in 3bbbe467f29 (bundle verify: error out if called without an object database, 2019-05-27). For the relevant bisect.c code see 45b6370812c (bisect: libify `check_good_are_ancestors_of_bad` and its dependents, 2020-02-17). For the submodule.c code see the "goto" on "(!left || !right || !sub)" added in 8e6df65015f (submodule: refactor show_submodule_summary with helper function, 2016-08-31). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-13revisions API users: add straightforward release_revisions()Ævar Arnfjörð Bjarmason1-0/+1
Add a release_revisions() to various users of "struct rev_list" in those straightforward cases where we only need to add the release_revisions() call to the end of a block, and don't need to e.g. refactor anything to use a "goto cleanup" pattern. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-11i18n: fix some badly formatted i18n stringsJean-Noël Avila1-1/+1
String in submodule--helper is not correctly formatting placeholders. The string in git-send-email is partial. Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-07Merge branch 'js/apply-partial-clone-filters-recursively'Junio C Hamano1-1/+1
Typofix * js/apply-partial-clone-filters-recursively: submodule-helper: fix usage string
2022-04-07submodule-helper: fix usage stringFangyi Zhou1-1/+1
The missing space at the end of the line makes the closing square bracket sticking to the dash in the next line Found during localisation v2.36.0 round 1 Signed-off-by: Fangyi Zhou <me@fangyi.io> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-04Merge branch 'gc/branch-recurse-submodules-fix'Junio C Hamano1-3/+4
A handful of obvious clean-ups around a topic that is already in 'master'. * gc/branch-recurse-submodules-fix: branch.c: simplify advice-and-die sequence branch: rework comments for future developers branch: remove negative exit code branch --set-upstream-to: be consistent when advising branch: give submodule updating advice before exit branch: support more tracking modes when recursing
2022-03-30branch: support more tracking modes when recursingGlen Choo1-3/+4
"git branch --recurse-submodules" does not propagate "--track=inherit" or "--no-track" to submodules, which causes submodule branches to use the wrong tracking mode [1]. To fix this, pass the correct options to the "submodule--helper create-branch" child process and test for it. While we are refactoring the same code, replace "--track" with the synonymous, but more consistent-looking "--track=direct" option (introduced at the same time as "--track=inherit", d3115660b4 (branch: add flags and config to inherit tracking, 2021-12-20)). [1] This bug is partially a timing issue: "branch --recurse-submodules" was introduced around the same time as "--track=inherit", and even though I rebased "branch --recurse-submodules" on top of that, I had neglected to support the new tracking mode. Omitting "--no-track" was just a plain old mistake, though. Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-23Merge branch 'gc/submodule-update-part1'Junio C Hamano1-107/+141
Rewrite of "git submodule update" in C (early part). * gc/submodule-update-part1: submodule--helper update-clone: check for --filter and --init submodule update: add tests for --filter submodule--helper: remove ensure-core-worktree submodule--helper update-clone: learn --init submodule--helper: allow setting superprefix for init_submodule() submodule--helper: refactor get_submodule_displaypath() submodule--helper run-update-procedure: learn --remote submodule--helper: don't use bitfield indirection for parse_options() submodule--helper: get remote names from any repository submodule--helper run-update-procedure: remove --suboid submodule--helper: reorganize code for sh to C conversion submodule--helper: remove update-module-mode submodule tests: test for init and update failure output
2022-03-16submodule--helper: remove forward declarationGlen Choo1-197/+196
Rearrange functions so that submodule_update() no longer needs to be forward declared. Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-16submodule: move core cmd_update() logic to CAtharva Raykar1-105/+124
This patch completes the conversion past the flag parsing of `submodule update` by introducing a helper subcommand called `submodule--helper update`. The behaviour of `submodule update` should remain the same after this patch. Prior to this patch, `submodule update` was implemented by piping the output of `update-clone` (which clones all missing submodules, then prints relevant information for all submodules) into `run-update-procedure` (which reads the information and updates the submodule tree). With `submodule--helper update`, we iterate over the submodules and update the submodule tree in the same process. This reuses most of existing code structure, except that `update_submodule()` now updates the submodule tree (instead of printing submodule information to be consumed by another process). Recursing on a submodule is done by calling a subprocess that launches `submodule--helper update`, with a modified `--recursive-prefix` and `--prefix` parameter. Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-16submodule--helper: reduce logic in run_update_procedure()Glen Choo1-13/+17
A later commit will combine the "update-clone" and "run-update-procedure" commands, so run_update_procedure() will be removed. Prepare for this by moving as much logic as possible out of run_update_procedure() and into update_submodule2(). Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-16submodule--helper: teach update_data more optionsGlen Choo1-75/+75
Refactor 'struct update_data' to hold the parsed args needed by "git submodule--helper update" and refactor "update-clone" and "run-update-procedure" (the functions that will be combined to form "update") to use these options. For "run-update-procedure", 'struct update_data' already holds its args, so only arg parsing code needs to be updated. For "update-clone", move its args from 'struct submodule_update_clone' into 'struct update_data', and replace them with a pointer to 'struct update_data'. Its other members hold the submodule iteration state of "update-clone", so those are unchanged. Incidentally, since we reformat the designated initializers of the affected structs, also reformat MODULE_CLONE_DATA_INIT for consistency. Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-16builtin/submodule--helper.c: rename option struct to "opt"Ævar Arnfjörð Bjarmason1-26/+26
In a later commit, update_clone()'s options will be stored in a struct update_data instead of submodule_update_clone. Preemptively rename the options struct to "opt" to shrink that commit's diff. Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-16submodule update: use die_message()Glen Choo1-19/+14
Use die_message() to print the "fatal: " prefix instead of doing it in git-submodule.sh and remove a now-unnecessary exit code from "git submodule--helper run-update-procedure". Also, since die_message() adds the newline for us, replace an invocation of die_with_status() with printf + exit invocations that do not add a newline, but are otherwise identical to die_with_status(). Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-16submodule--helper: run update using child process structAtharva Raykar1-18/+16
We switch to using the run-command API function that takes a 'struct child process', since we are using a lot of the options. This will also make it simple to switch over to using 'capture_command()' when we start handling the output of the command completely in C. Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-16Merge branch 'gc/submodule-update-part1' into gc/submodule-update-part2Junio C Hamano1-107/+141
* gc/submodule-update-part1: submodule--helper update-clone: check for --filter and --init submodule update: add tests for --filter submodule--helper: remove ensure-core-worktree submodule--helper update-clone: learn --init submodule--helper: allow setting superprefix for init_submodule() submodule--helper: refactor get_submodule_displaypath() submodule--helper run-update-procedure: learn --remote submodule--helper: don't use bitfield indirection for parse_options() submodule--helper: get remote names from any repository submodule--helper run-update-procedure: remove --suboid submodule--helper: reorganize code for sh to C conversion submodule--helper: remove update-module-mode submodule tests: test for init and update failure output
2022-03-13Merge branch 'ab/plug-random-leaks'Junio C Hamano1-1/+4
Plug random memory leaks. * ab/plug-random-leaks: repository.c: free the "path cache" in repo_clear() range-diff: plug memory leak in read_patches() range-diff: plug memory leak in common invocation lockfile API users: simplify and don't leak "path" commit-graph: stop fill_oids_from_packs() progress on error and free() commit-graph: fix memory leak in misused string_list API submodule--helper: fix trivial leak in module_add() transport: stop needlessly copying bundle header references bundle: call strvec_clear() on allocated strvec remote-curl.c: free memory in cmd_main() urlmatch.c: add and use a *_release() function diff.c: free "buf" in diff_words_flush() merge-base: free() allocated "struct commit **" list index-pack: fix memory leaks
2022-03-06Merge branch 'ac/usage-string-fixups'Junio C Hamano1-1/+1
Usage-string normalization. * ac/usage-string-fixups: amend remaining usage strings according to style guide
2022-03-04submodule--helper update-clone: check for --filter and --initGlen Choo1-1/+19
"git submodule update --filter" also requires the "--init" option. Teach update-clone to do this usage check in C and remove the check from git-submodule.sh. In addition, change update-clone's usage string so that it teaches users about "git submodule update" instead of "git submodule--helper update-clone" (the string is copied from git-submodule.sh). This should be more helpful to users since they don't invoke update-clone directly. Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-04submodule--helper: remove ensure-core-worktreeGlen Choo1-10/+2
Move the logic of "git submodule--helper ensure-core-worktree" into run-update-procedure, and since this makes the ensure-core-worktree command obsolete, remove it. As a result, the order of two operations in git-submodule.sh is reversed: 'set the value of core.worktree' now happens after the call to "git submodule--helper relative-path". This is safe - "relative-path" does not depend on the value of core.worktree. Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-04submodule--helper update-clone: learn --initGlen Choo1-0/+26
Teach "git submodule--helper update-clone" the --init flag and remove the corresponding shell code. When the `--init` flag is passed to the subcommand, we do not spawn a new subprocess and call `submodule--helper init` on the submodule paths, because the Git machinery is not able to pick up the configuration changes introduced by that init call. So we instead run the `init_submodule_cb()` callback over each submodule in the same process. [1] https://lore.kernel.org/git/CAP8UFD0NCQ5w_3GtT_xHr35i7h8BuLX4UcHNY6VHPGREmDVObA@mail.gmail.com/ Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-04submodule--helper: allow setting superprefix for init_submodule()Atharva Raykar1-3/+7
We allow callers of the `init_submodule()` function to optionally override the superprefix from the environment. We need to enable this option because in our conversion of the update command that will follow, the '--init' option will be handled through this API. We will need to change the superprefix at that time to ensure the display paths show correctly in the output messages. Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-04submodule--helper: refactor get_submodule_displaypath()Atharva Raykar1-4/+10
We create a function called `do_get_submodule_displaypath()` that generates the display path required by several submodule functions, and takes a custom superprefix parameter, instead of reading it from the environment. We then redefine the existing `get_submodule_displaypath()` function as a call to this new function, where the superprefix is obtained from the environment. Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-04submodule--helper run-update-procedure: learn --remoteGlen Choo1-34/+22
Teach run-update-procedure to handle --remote instead of parsing --remote in git-submodule.sh. As a result, "git submodule--helper [print-default-remote|remote-branch]" have no more callers, so remove them. Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-04submodule--helper: don't use bitfield indirection for parse_options()Ævar Arnfjörð Bjarmason1-13/+10
Do away with the indirection of local variables added in c51f8f94e5b (submodule--helper: run update procedures from C, 2021-08-24). These were only needed because in C you can't get a pointer to a single bit, so we were using intermediate variables instead. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-04submodule--helper: get remote names from any repositoryAtharva Raykar1-15/+23
`get_default_remote()` retrieves the name of a remote by resolving the refs from of the current repository's ref store. Thus in order to use it for retrieving the remote name of a submodule, we have to start a new subprocess which runs from the submodule directory. Let's instead introduce a function called `repo_get_default_remote()` which takes any repository object and retrieves the remote accordingly. `get_default_remote()` is then defined as a call to `repo_get_default_remote()` with 'the_repository' passed to it. Now that we have `repo_get_default_remote()`, we no longer have to start a subprocess that called `submodule--helper get-default-remote` from within the submodule directory. So let's make a function called `get_default_remote_submodule()` which takes a submodule path, and returns the default remote for that submodule, all within the same process. We can now use this function to save an unnecessary subprocess spawn in `sync_submodule()`, and also in a subsequent patch, which will require this functionality. Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Helped-by: Glen Choo <chooglen@google.com> Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-04submodule--helper run-update-procedure: remove --suboidGlen Choo1-3/+6
Teach run-update-procedure to determine the oid of the submodule's HEAD instead of doing it in git-submodule.sh. Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-04submodule--helper: reorganize code for sh to C conversionGlen Choo1-5/+21
Introduce a function, update_submodule2(), that will implement the functionality of run-update-procedure and its surrounding shell code in submodule.sh. This name is temporary; it will replace update_submodule() when the sh to C conversion is complete. Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-04submodule--helper: remove update-module-modeGlen Choo1-24/+0
This is dead code - it has not been used since c51f8f94e5 (submodule--helper: run update procedures from C, 2021-08-24). Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-04submodule--helper: fix trivial leak in module_add()Ævar Arnfjörð Bjarmason1-1/+4
Fix a memory leak in code added in a6226fd772b (submodule--helper: convert the bulk of cmd_add() to C, 2021-08-10). If "realrepo" isn't a copy of the "repo" member we should free() it. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-25Merge branch 'js/apply-partial-clone-filters-recursively'Junio C Hamano1-3/+27
"git clone --filter=... --recurse-submodules" only makes the top-level a partial clone, while submodules are fully cloned. This behaviour is changed to pass the same filter down to the submodules. * js/apply-partial-clone-filters-recursively: clone, submodule: pass partial clone filters to submodules
2022-02-25Merge branch 'ja/i18n-common-messages'Junio C Hamano1-1/+1
Unify more messages to help l10n. * ja/i18n-common-messages: i18n: fix some misformated placeholders in command synopsis i18n: remove from i18n strings that do not hold translatable parts i18n: factorize "invalid value" messages i18n: factorize more 'incompatible options' messages
2022-02-23amend remaining usage strings according to style guideAbhradeep Chakraborty1-1/+1
Usage strings for git (sub)command flags has a style guide that suggests - first letter should not capitalized (unless required) and it should skip full-stop at the end of line. But there are some files where usage-strings do not follow the above mentioned guide. Amend the usage strings that don't follow the style convention/guide. Signed-off-by: Abhradeep Chakraborty <chakrabortyabhradeep79@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-18Merge branch 'gc/branch-recurse-submodules'Junio C Hamano1-0/+38
"git branch" learned the "--recurse-submodules" option. * gc/branch-recurse-submodules: branch.c: use 'goto cleanup' in setup_tracking() to fix memory leaks branch: add --recurse-submodules option for branch creation builtin/branch: consolidate action-picking logic in cmd_branch() branch: add a dry_run parameter to create_branch() branch: make create_branch() always create a branch branch: move --set-upstream-to behavior to dwim_and_setup_tracking()
2022-02-09clone, submodule: pass partial clone filters to submodulesJosh Steadmon1-3/+27
When cloning a repo with a --filter and with --recurse-submodules enabled, the partial clone filter only applies to the top-level repo. This can lead to unexpected bandwidth and disk usage for projects which include large submodules. For example, a user might wish to make a partial clone of Gerrit and would run: `git clone --recurse-submodules --filter=blob:5k https://gerrit.googlesource.com/gerrit`. However, only the superproject would be a partial clone; all the submodules would have all blobs downloaded regardless of their size. With this change, the same filter can also be applied to submodules, meaning the expected bandwidth and disk savings apply consistently. To avoid changing default behavior, add a new clone flag, `--also-filter-submodules`. When this is set along with `--filter` and `--recurse-submodules`, the filter spec is passed along to git-submodule and git-submodule--helper, such that submodule clones also have the filter applied. This applies the same filter to the superproject and all submodules. Users who need to customize the filter per-submodule would need to clone with `--no-recurse-submodules` and then manually initialize each submodule with the proper filter. Applying filters to submodules should be safe thanks to Jonathan Tan's recent work [1, 2, 3] eliminating the use of alternates as a method of accessing submodule objects, so any submodule object access now triggers a lazy fetch from the submodule's promisor remote if the accessed object is missing. This patch is a reworked version of [4], which was created prior to Jonathan Tan's work. [1]: 8721e2e (Merge branch 'jt/partial-clone-submodule-1', 2021-07-16) [2]: 11e5d0a (Merge branch 'jt/grep-wo-submodule-odb-as-alternate', 2021-09-20) [3]: 162a13b (Merge branch 'jt/no-abuse-alternate-odb-for-submodules', 2021-10-25) [4]: https://lore.kernel.org/git/52bf9d45b8e2b72ff32aa773f2415bf7b2b86da2.1563322192.git.steadmon@google.com/ Signed-off-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-04i18n: remove from i18n strings that do not hold translatable partsJean-Noël Avila1-1/+1
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-04branch: add --recurse-submodules option for branch creationGlen Choo1-0/+38
To improve the submodules UX, we would like to teach Git to handle branches in submodules. Start this process by teaching "git branch" the --recurse-submodules option so that "git branch --recurse-submodules topic" will create the `topic` branch in the superproject and its submodules. Although this commit does not introduce breaking changes, it does not work well with existing --recurse-submodules commands because "git branch --recurse-submodules" writes to the submodule ref store, but most commands only consider the superproject gitlink and ignore the submodule ref store. For example, "git checkout --recurse-submodules" will check out the commits in the superproject gitlinks (and put the submodules in detached HEAD) instead of checking out the submodule branches. Because of this, this commit introduces a new configuration value, `submodule.propagateBranches`. The plan is for Git commands to prioritize submodule ref store information over superproject gitlinks if this value is true. Because "git branch --recurse-submodules" writes to submodule ref stores, for the sake of clarity, it will not function unless this configuration value is set. This commit also includes changes that support working with submodules from a superproject commit because "branch --recurse-submodules" (and future commands) need to read .gitmodules and gitlinks from the superproject commit, but submodules are typically read from the filesystem's .gitmodules and the index's gitlinks. These changes are: * add a submodules_of_tree() helper that gives the relevant information of an in-tree submodule (e.g. path and oid) and initializes the repository * add is_tree_submodule_active() by adding a treeish_name parameter to is_submodule_active() * add the "submoduleNotUpdated" advice to advise users to update the submodules in their trees Incidentally, fix an incorrect usage string that combined the 'list' usage of git branch (-l) with the 'create' usage; this string has been incorrect since its inception, a8dfd5eac4 (Make builtin-branch.c use parse_options., 2007-10-07). Helped-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Glen Choo <chooglen@google.com> Reviewed-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-05i18n: refactor "foo and bar are mutually exclusive"Jean-Noël Avila1-2/+2
Use static strings for constant parts of the sentences. They are all turned into "cannot be used together". Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Reviewed-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-10Merge branch 'mp/absorb-submodule-git-dir-upon-deinit'Junio C Hamano1-10/+11
"git submodule deinit" for a submodule whose .git metadata directory is embedded in its working tree refused to work, until the submodule gets converted to use the "absorbed" form where the metadata directory is stored in superproject, and a gitfile at the top-level of the working tree of the submodule points at it. The command is taught to convert such submodules to the absorbed form as needed. * mp/absorb-submodule-git-dir-upon-deinit: submodule: absorb git dir instead of dying on deinit
2021-11-19submodule: absorb git dir instead of dying on deinitMugdha Pattnaik1-10/+11
Currently, running 'git submodule deinit' on repos where the submodule's '.git' is a directory, aborts with a message that is not exactly user friendly. Let's change this to instead warn the user that the .git/ directory has been absorbed into the superproject. The rest of the deinit function can operate as it already does with new-style submodules. In one test, we used to require "git submodule deinit" to fail even with the "--force" option when the submodule's .git/ directory is not absorbed. Adjust it to expect the operation to pass. Suggested-by: Atharva Raykar <raykar.ath@gmail.com> Signed-off-by: Mugdha Pattnaik <mugdhapattnaik@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-29Merge branch 'ks/submodule-add-message-fix'Junio C Hamano1-14/+22
Message regression fix. * ks/submodule-add-message-fix: submodule: drop unused sm_name parameter from append_fetch_remotes() submodule--helper: fix incorrect newlines in an error message
2021-10-27submodule: drop unused sm_name parameter from append_fetch_remotes()Jeff King1-3/+2
Commit c21fb4676f (submodule--helper: fix incorrect newlines in an error message, 2021-10-23) accidentally added a new, unused parameter while changing the name and signature of show_fetch_remotes() to append_fetch_remotes(). We can drop this to keep things simpler (and satisfy -Wunused-parameter). The error is likely because c21fb4676f is fixing a problem from 8c8195e9c3 (submodule--helper: introduce add-clone subcommand, 2021-07-10). An earlier iteration of that second commit introduced the same unused parameter (though it was dropped before it finally made it to 'next'), and the fix on top accidentally carried forward the extra parameter. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-23submodule--helper: fix incorrect newlines in an error messageKaartic Sivaraam1-14/+23
A refactoring[1] done as part of the recent conversion of 'git submodule add' to builtin, changed the error message shown when a Git directory already exists locally for a submodule name. Before the refactoring, the error used to appear like so: --- START OF OUTPUT --- $ git submodule add ../sub/ subm A git directory for 'subm' is found locally with remote(s): origin /me/git-repos-for-test/sub If you want to reuse this local git directory instead of cloning again from /me/git-repos-for-test/sub use the '--force' option. If the local git directory is not the correct repo or you are unsure what this means choose another name with the '--name' option. --- END OF OUTPUT --- After the refactoring the error started appearing like so: --- START OF OUTPUT --- $ git submodule add ../sub/ subm A git directory for 'subm' is found locally with remote(s): origin /me/git-repos-for-test/sub fatal: If you want to reuse this local git directory instead of cloning again from /me/git-repos-for-test/sub use the '--force' option. If the local git directory is not the correct repo or if you are unsure what this means, choose another name with the '--name' option. --- END OF OUTPUT --- As one could observe the remote information is printed along with the first line rather than on its own line. Also, there's an additional newline following output. Make the error message consistent with the error message that used to be printed before the refactoring. This also moves the 'fatal:' prefix that appears in the middle of the error message to the first line as it would more appropriate to have it in the first line. The output after the change would look like: --- START OF OUTPUT --- $ git submodule add ../sub/ subm fatal: A git directory for 'subm' is found locally with remote(s): origin /me/git-repos-for-test/sub If you want to reuse this local git directory instead of cloning again from /me/git-repos-for-test/sub use the '--force' option. If the local git directory is not the correct repo or you are unsure what this means choose another name with the '--name' option. --- END OF OUTPUT --- [1]: https://lore.kernel.org/git/20210710074801.19917-5-raykar.ath@gmail.com/#t Signed-off-by: Kaartic Sivaraam <kaartic.sivaraam@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-23submodule--helper: fix small memory leaksÆvar Arnfjörð Bjarmason1-0/+2
Add a missing strbuf_release() and a clear_pathspec() to the submodule--helper. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-13Merge branch 'en/removing-untracked-fixes'Junio C Hamano1-0/+4
Various fixes in code paths that move untracked files away to make room. * en/removing-untracked-fixes: Documentation: call out commands that nuke untracked files/directories Comment important codepaths regarding nuking untracked files/dirs unpack-trees: avoid nuking untracked dir in way of locally deleted file unpack-trees: avoid nuking untracked dir in way of unmerged file Change unpack_trees' 'reset' flag into an enum Remove ignored files by default when they are in the way unpack-trees: make dir an internal-only struct unpack-trees: introduce preserve_ignored to unpack_trees_options read-tree, merge-recursive: overwrite ignored files by default checkout, read-tree: fix leak of unpack_trees_options.dir t2500: add various tests for nuking untracked files
2021-10-11Merge branch 'ab/designated-initializers'Junio C Hamano1-10/+11
Code clean-up. * ab/designated-initializers: cbtree.h: define cb_init() in terms of CBTREE_INIT *.h: move some *_INIT to designated initializers *.h _INIT macros: don't specify fields equal to 0 *.[ch] *_INIT macros: use { 0 } for a "zero out" idiom submodule-config.h: remove unused SUBMODULE_INIT macro
2021-10-06Merge branch 'jt/add-submodule-odb-clean-up'Junio C Hamano1-6/+1
More code paths that use the hack to add submodule's object database to the set of alternate object store have been cleaned up. * jt/add-submodule-odb-clean-up: revision: remove "submodule" from opt struct repository: support unabsorbed in repo_submodule_init submodule: remove unnecessary unabsorbed fallback
2021-09-27*.h: move some *_INIT to designated initializersÆvar Arnfjörð Bjarmason1-2/+3
Move various *_INIT macros to use designated initializers. This helps readability. I've only picked those leftover macros that were not touched by another in-flight series of mine which changed others, but also how initialization was done. In the case of SUBMODULE_ALTERNATE_SETUP_INIT I've left an explicit initialization of "error_mode", even though SUBMODULE_ALTERNATE_ERROR_IGNORE itself is defined as "0". Let's not peek under the hood and assume that enum fields we know the value of will stay at "0". The change to "TESTSUITE_INIT" in "t/helper/test-run-command.c" was part of an earlier on-list version[1] of c90be786da9 (test-tool run-command: fix flip-flop init pattern, 2021-09-11). 1. https://lore.kernel.org/git/patch-1.1-0aa4523ab6e-20210909T130849Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-27*.[ch] *_INIT macros: use { 0 } for a "zero out" idiomÆvar Arnfjörð Bjarmason1-8/+8
In C it isn't required to specify that all members of a struct are zero'd out to 0, NULL or '\0', just providing a "{ 0 }" will accomplish that. Let's also change code that provided N zero'd fields to just provide one, and change e.g. "{ NULL }" to "{ 0 }" for consistency. I.e. even if the first member is a pointer let's use "0" instead of "NULL". The point of using "0" consistently is to pick one, and to not have the reader wonder why we're not using the same pattern everywhere. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-27Comment important codepaths regarding nuking untracked files/dirsElijah Newren1-0/+4
In the last few commits we focused on code in unpack-trees.c that mistakenly removed untracked files or directories. There may be more of those, but in this commit we change our focus: callers of toplevel commands that are expected to remove untracked files or directories. As noted previously, we have toplevel commands that are expected to delete untracked files such as 'read-tree --reset', 'reset --hard', and 'checkout --force'. However, that does not mean that other highlevel commands that happen to call these other commands thought about or conveyed to users the possibility that untracked files could be removed. Audit the code for such callsites, and add comments near existing callsites to mention whether these are safe or not. My auditing is somewhat incomplete, though; it skipped several cases: * git-rebase--preserve-merges.sh: is in the process of being deprecated/removed, so I won't leave a note that there are likely more bugs in that script. * contrib/git-new-workdir: why is the -f flag being used in a new empty directory?? It shouldn't hurt, but it seems useless. * git-p4.py: Don't see why -f is needed for a new dir (maybe it's not and is just superfluous), but I'm not at all familiar with the p4 stuff * git-archimport.perl: Don't care; arch is long since dead * git-cvs*.perl: Don't care; cvs is long since dead Also, the reset --hard in builtin/worktree.c looks safe, due to only running in an empty directory. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-23Merge branch 'jt/submodule-name-to-gitdir'Junio C Hamano1-2/+8
Code refactoring. * jt/submodule-name-to-gitdir: submodule: extract path to submodule gitdir func
2021-09-20Merge branch 'ar/submodule-run-update-procedure'Junio C Hamano1-0/+257
Reimplementation of parts of "git submodule" in C continues. * ar/submodule-run-update-procedure: submodule--helper: run update procedures from C
2021-09-20Merge branch 'ar/submodule-add-more'Junio C Hamano1-145/+162
More parts of "git submodule add" has been rewritten in C. * ar/submodule-add-more: submodule--helper: rename compute_submodule_clone_url() submodule--helper: remove resolve-relative-url subcommand submodule--helper: remove add-config subcommand submodule--helper: remove add-clone subcommand submodule--helper: convert the bulk of cmd_add() to C dir: libify and export helper functions from clone.c submodule--helper: remove repeated code in sync_submodule() submodule--helper: refactor resolve_relative_url() helper submodule--helper: add options for compute_submodule_clone_url()
2021-09-20Merge branch 'ar/submodule-add-config'Junio C Hamano1-0/+129
Large part of "git submodule add" gets rewritten in C. * ar/submodule-add-config: submodule--helper: introduce add-config subcommand
2021-09-15submodule: extract path to submodule gitdir funcJonathan Tan1-2/+8
We currently store each submodule gitdir in ".git/modules/<name>", but this has problems with some submodule naming schemes, as described in a comment in submodule_name_to_gitdir() in this patch. Extract the determination of the location of a submodule's gitdir into its own function submodule_name_to_gitdir(). For now, the problem remains unsolved, but this puts us in a better position for finding a solution. This was motivated, at $DAYJOB, by a part of Android's repo hierarchy [1]. In particular, there is a repo "build", and several repos of the form "build/<name>". This is based on earlier work by Brandon Williams [2]. [1] https://android.googlesource.com/platform/ [2] https://lore.kernel.org/git/20180808223323.79989-2-bmwill@google.com/ Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-09repository: support unabsorbed in repo_submodule_initJonathan Tan1-6/+1
In preparation for a subsequent commit that migrates code using add_submodule_odb() to repo_submodule_init(), teach repo_submodule_init() to support submodules with unabsorbed gitdirs. (See the documentation for "git submodule absorbgitdirs" for more information about absorbed and unabsorbed gitdirs.) Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-25advice: remove read uses of most global `advice_` variablesBen Boeckel1-1/+1
In c4a09cc9ccb (Merge branch 'hw/advise-ng', 2020-03-25), a new API for accessing advice variables was introduced and deprecated `advice_config` in favor of a new array, `advice_setting`. This patch ports all but two uses which read the status of the global `advice_` variables over to the new `advice_enabled` API. We'll deal with advice_add_embedded_repo and advice_graft_file_deprecated separately. Signed-off-by: Ben Boeckel <mathstuf@gmail.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-24submodule--helper: run update procedures from CAtharva Raykar1-0/+257
Add a new submodule--helper subcommand `run-update-procedure` that runs the update procedure if the SHA1 of the submodule does not match what the superproject expects. This is an intermediate change that works towards total conversion of `submodule update` from shell to C. Specific error codes are returned so that the shell script calling the subcommand can take a decision on the control flow, and preserve the error messages across subsequent recursive calls of `cmd_update`. This change is more focused on doing a faithful conversion, so for now we are not too concerned with trying to reduce subprocess spawns. Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-10submodule--helper: rename compute_submodule_clone_url()Atharva Raykar1-6/+6
Let's rename 'compute_submodule_clone_url()' to 'resolve_relative_url()' to make it clear that this internal helper need not be used exclusively for computing submodule clone URLs. Since the original 'resolve-relative-url' subcommand and its C entry point has been removed in c461095ae3 (submodule--helper: remove resolve-relative-url subcommand, 2021-07-02), this rename can be done without causing any confusion about which function it actually binds to. Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-10submodule--helper: remove resolve-relative-url subcommandAtharva Raykar1-20/+0
The shell subcommand `resolve-relative-url` is no longer required, as its last caller has been removed when it was converted to C. Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-10submodule--helper: remove add-config subcommandAtharva Raykar1-49/+0
Also no longer needed is this subcommand, as all of its functionality is being called by the newly-introduced `module_add()` directly within C. Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-10submodule--helper: remove add-clone subcommandAtharva Raykar1-60/+0
We no longer need this subcommand, as all of its functionality is being called by the newly-introduced `module_add()` directly within C. Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-10submodule--helper: convert the bulk of cmd_add() to CAtharva Raykar1-1/+164
Introduce the 'add' subcommand to `submodule--helper.c` that does all the work 'submodule add' past the parsing of flags. We also remove the constness of the sm_path field of the `add_data` struct. This is needed so that it can be modified by normalize_path_copy(). As with the previous conversions, this is meant to be a faithful conversion with no modification to the behaviour of `submodule add`. Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Mentored-by: Christian Couder <christian.couder@gmail.com> Helped-by: Kaartic Sivaraam <kaartic.sivaraam@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Based-on-patch-by: Shourya Shukla <periperidip@gmail.com> Based-on-patch-by: Prathamesh Chavan <pc44800@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-10submodule--helper: remove repeated code in sync_submodule()Atharva Raykar1-13/+3
This part of `sync_submodule()` is doing the same thing that `compute_submodule_clone_url()` is doing. Let's reuse that helper here. Note that this change adds a small overhead where we allocate and free the 'remote' twice, but that is a small price to pay for the higher level of abstraction we get. Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-10submodule--helper: refactor resolve_relative_url() helperAtharva Raykar1-36/+25
Refactor the helper function to resolve a relative url, by reusing the existing `compute_submodule_clone_url()` function. `compute_submodule_clone_url()` performs the same work that `resolve_relative_url()` is doing, so we eliminate this code repetition by moving the former function's definition up, and calling it inside `resolve_relative_url()`. Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-10submodule--helper: add options for compute_submodule_clone_url()Atharva Raykar1-7/+11
Let's modify the interface to `compute_submodule_clone_url()` function by adding two more arguments, so that we can reuse this in various parts of `submodule--helper.c` that follow a common pattern, which is--read the remote url configuration of the superproject and then call `relative_url()`. This function is nearly identical to `resolve_relative_url()`, the only difference being the extra warning message. We can add a quiet flag to it, to suppress that warning when not needed, and then refactor `resolve_relative_url()` by using this function, something we will do in the next patch. We also rename the local variable 'relurl' to avoid potential confusion with the 'rel_url' parameter while we are at it. Having this functionality factored out will be useful for converting the rest of `submodule add` in subsequent patches. Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-10Merge branch 'ar/submodule-add-config' into ar/submodule-addJunio C Hamano1-0/+129
* ar/submodule-add-config: submodule--helper: introduce add-config subcommand
2021-08-10submodule--helper: introduce add-config subcommandAtharva Raykar1-0/+129
Add a new "add-config" subcommand to `git submodule--helper` with the goal of converting part of the shell code in git-submodule.sh related to `git submodule add` into C code. This new subcommand sets the configuration variables of a newly added submodule, by registering the url in local git config, as well as the submodule name and path in the .gitmodules file. It also sets 'submodule.<name>.active' to "true" if the submodule path has not already been covered by any pathspec specified in 'submodule.active'. This is meant to be a faithful conversion from shell to C, although we add comments to areas that could be improved in future patches, after the conversion has settled. Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Based-on-patch-by: Shourya Shukla <periperidip@gmail.com> Based-on-patch-by: Prathamesh Chavan <pc44800@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-04Merge branch 'ah/plugleaks'Junio C Hamano1-2/+4
Leak plugging. * ah/plugleaks: reset: clear_unpack_trees_porcelain to plug leak builtin/rebase: fix options.strategy memory lifecycle builtin/merge: free found_ref when done builtin/mv: free or UNLEAK multiple pointers at end of cmd_mv convert: release strbuf to avoid leak read-cache: call diff_setup_done to avoid leak ref-filter: also free head for ATOM_HEAD to avoid leak diffcore-rename: move old_dir/new_dir definition to plug leak builtin/for-each-repo: remove unnecessary argv copy to plug leak builtin/submodule--helper: release unused strbuf to avoid leak environment: move strbuf into block to plug leak fmt-merge-msg: free newly allocated temporary strings when done
2021-08-04Merge branch 'ar/submodule-add'Junio C Hamano1-113/+304
Rewrite of "git submodule" in C continues. * ar/submodule-add: submodule: drop unused sm_name parameter from show_fetch_remotes() submodule--helper: introduce add-clone subcommand submodule--helper: refactor module_clone() submodule: prefix die messages with 'fatal' t7400: test failure to add submodule in tracked path
2021-07-26builtin/submodule--helper: release unused strbuf to avoid leakAndrzej Hunt1-2/+4
relative_url() populates sb. In the normal return path, its buffer is detached using strbuf_detach(). However the early return path does nothing with sb, which means that sb's memory is leaked - therefore we add a release to avoid this leak. The reset is also only necessary for the normal return path, hence we move it down to after the early-return to avoid unnecessary work. LSAN output from t0060: Direct leak of 121 byte(s) in 1 object(s) allocated from: #0 0x7f31246f28b0 in realloc (/usr/lib64/libasan.so.4+0xdc8b0) #1 0x98d7d6 in xrealloc wrapper.c:126 #2 0x909a60 in strbuf_grow strbuf.c:98 #3 0x90bf00 in strbuf_vaddf strbuf.c:401 #4 0x90c321 in strbuf_addf strbuf.c:335 #5 0x5cb78d in relative_url builtin/submodule--helper.c:182 #6 0x5cbe46 in resolve_relative_url_test builtin/submodule--helper.c:248 #7 0x410dcd in run_builtin git.c:475 #8 0x410dcd in handle_builtin git.c:729 #9 0x414087 in run_argv git.c:818 #10 0x414087 in cmd_main git.c:949 #11 0x40e9ec in main common-main.c:52 #12 0x7f3123c41349 in __libc_start_main (/lib64/libc.so.6+0x24349) SUMMARY: AddressSanitizer: 121 byte(s) leaked in 1 allocation(s). Signed-off-by: Andrzej Hunt <andrzej@ahunt.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-26submodule: drop unused sm_name parameter from show_fetch_remotes()Jeff King1-3/+2
This parameter has not been used since the function was introduced in 8c8195e9c3 (submodule--helper: introduce add-clone subcommand, 2021-07-10). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-16Merge branch 'ar/submodule-helper-include-cleanup'Junio C Hamano1-1/+0
Code clean-up. * ar/submodule-helper-include-cleanup: submodule--helper: remove redundant include
2021-07-12submodule--helper: introduce add-clone subcommandAtharva Raykar1-0/+177
Let's add a new "add-clone" subcommand to `git submodule--helper` with the goal of converting part of the shell code in git-submodule.sh related to `git submodule add` into C code. This new subcommand clones the repository that is to be added, and checks out to the appropriate branch. This is meant to be a faithful conversion that leaves the behaviour of 'cmd_add()' script unchanged. Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Based-on-patch-by: Shourya Shukla <periperidip@gmail.com> Based-on-patch-by: Prathamesh Chavan <pc44800@gmail.com> Helped-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-12submodule--helper: refactor module_clone()Atharva Raykar1-113/+128
Separate out the core logic of module_clone() from the flag parsing---this way we can call the equivalent of the `submodule--helper clone` subcommand directly within C, without needing to push arguments in a strvec. Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Suggested-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-06submodule--helper: remove redundant includeAtharva Raykar1-1/+0
"dir.h" should have been included only once. Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Shourya Shukla <periperidip@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-10Merge branch 'ah/submodule-helper-module-summary-parseopt'Junio C Hamano1-1/+1
Message update. * ah/submodule-helper-module-summary-parseopt: submodule: use the imperative mood to describe the --files option
2021-05-17submodule: use the imperative mood to describe the --files optionAlex Henrie1-1/+1
Signed-off-by: Alex Henrie <alexhenrie24@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-27hash: provide per-algorithm null OIDsbrian m. carlson1-14/+15
Up until recently, object IDs did not have an algorithm member, only a hash. Consequently, it was possible to share one null (all-zeros) object ID among all hash algorithms. Now that we're going to be handling objects from multiple hash algorithms, it's important to make sure that all object IDs have a correct algorithm field. Introduce a per-algorithm null OID, and add it to struct hash_algo. Introduce a wrapper function as well, and use it everywhere we used to use the null_oid constant. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-12Merge branch 'tb/precompose-prefix-too'Junio C Hamano1-1/+1
When commands are started from a subdirectory, they may have to compare the path to the subdirectory (called prefix and found out from $(pwd)) with the tracked paths. On macOS, $(pwd) and readdir() yield decomposed path, while the tracked paths are usually normalized to the precomposed form, causing mismatch. This has been fixed by taking the same approach used to normalize the command line arguments. * tb/precompose-prefix-too: MacOS: precompose_argv_prefix()
2021-02-03MacOS: precompose_argv_prefix()Torsten Bögershausen1-1/+1
The following sequence leads to a "BUG" assertion running under MacOS: DIR=git-test-restore-p Adiarnfd=$(printf 'A\314\210') DIRNAME=xx${Adiarnfd}yy mkdir $DIR && cd $DIR && git init && mkdir $DIRNAME && cd $DIRNAME && echo "Initial" >file && git add file && echo "One more line" >>file && echo y | git restore -p . Initialized empty Git repository in /tmp/git-test-restore-p/.git/ BUG: pathspec.c:495: error initializing pathspec_item Cannot close git diff-index --cached --numstat [snip] The command `git restore` is run from a directory inside a Git repo. Git needs to split the $CWD into 2 parts: The path to the repo and "the rest", if any. "The rest" becomes a "prefix" later used inside the pathspec code. As an example, "/path/to/repo/dir-inside-repå" would determine "/path/to/repo" as the root of the repo, the place where the configuration file .git/config is found. The rest becomes the prefix ("dir-inside-repå"), from where the pathspec machinery expands the ".", more about this later. If there is a decomposed form, (making the decomposing visible like this), "dir-inside-rep°a" doesn't match "dir-inside-repå". Git commands need to: (a) read the configuration variable "core.precomposeunicode" (b) precocompose argv[] (c) precompose the prefix, if there was any The first commit, 76759c7dff53 "git on Mac OS and precomposed unicode" addressed (a) and (b). The call to precompose_argv() was added into parse-options.c, because that seemed to be a good place when the patch was written. Commands that don't use parse-options need to do (a) and (b) themselfs. The commands `diff-files`, `diff-index`, `diff-tree` and `diff` learned (a) and (b) in commit 90a78b83e0b8 "diff: run arguments through precompose_argv" Branch names (or refs in general) using decomposed code points resulting in decomposed file names had been fixed in commit 8e712ef6fc97 "Honor core.precomposeUnicode in more places" The bug report from above shows 2 things: - more commands need to handle precomposed unicode - (c) should be implemented for all commands using pathspecs Solution: precompose_argv() now handles the prefix (if needed), and is renamed into precompose_argv_prefix(). Inside this function the config variable core.precomposeunicode is read into the global variable precomposed_unicode, as before. This reading is skipped if precomposed_unicode had been read before. The original patch for preocomposed unicode, 76759c7dff53, placed precompose_argv() into parse-options.c Now add it into git.c::run_builtin() as well. Existing precompose calls in diff-files.c and others may become redundant, and if we audit the callflows that reach these places to make sure that they can never be reached without going through the new call added to run_builtin(), we might be able to remove these existing ones. But in this commit, we do not bother to do so and leave these precompose callsites as they are. Because precompose() is idempotent and can be called on an already precomposed string safely, this is safer than removing existing calls without fully vetting the callflows. There is certainly room for cleanups - this change intends to be a bug fix. Cleanups needs more tests in e.g. t/t3910-mac-os-precompose.sh, and should be done in future commits. [1] git-bugreport-2021-01-06-1209.txt (git can't deal with special characters) [2] https://lore.kernel.org/git/A102844A-9501-4A86-854D-E3B387D378AA@icloud.com/ Reported-by: Daniel Troger <random_n0body@icloud.com> Helped-By: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-06builtin/*: update usage formatZheNing Hu1-12/+12
According to the guidelines in parse-options.h, we should not end in a full stop or start with a capital letter. Fix old error and usage messages to match this expectation. Signed-off-by: ZheNing Hu <adlternative@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-08doc: fix the bnf like style of some commandsJean-Noël Avila1-1/+1
In command line options, variables are entered between < and > Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-09Merge branch 'ss/submodule-summary-in-c-fixes'Junio C Hamano1-9/+8
Fixups to a topic in 'next'. * ss/submodule-summary-in-c-fixes: t7421: eliminate 'grep' check in t7421.4 for mingw compatibility submodule: fix style in function definition submodule: eliminate unused parameters from print_submodule_summary()
2020-09-09Merge branch 'ss/submodule-summary-in-c'Junio C Hamano1-3/+429
Yet another subcommand of "git submodule" is getting rewritten in C. * ss/submodule-summary-in-c: submodule: port submodule subcommand 'summary' from shell to C t7421: introduce a test script for verifying 'summary' output submodule: rename helper functions to avoid ambiguity submodule: remove extra line feeds between callback struct and macro
2020-08-27Merge branch 'jk/leakfix'Junio C Hamano1-8/+8
Code clean-up. * jk/leakfix: submodule--helper: fix leak of core.worktree value config: fix leak in git_config_get_expiry_in_days() config: drop git_config_get_string_const() config: fix leaks from git_config_get_string_const() checkout: fix leak of non-existent branch names submodule--helper: use strbuf_release() to free strbufs clear_pattern_list(): clear embedded hashmaps
2020-08-27t7421: eliminate 'grep' check in t7421.4 for mingw compatibilityShourya Shukla1-3/+4
The 'grep' check in test 4 of t7421 resulted in the failure of t7421 on Windows due to a different error message error: cannot spawn git: No such file or directory instead of fatal: exec 'rev-parse': cd to 'my-subm' failed: No such file or directory Tighten up the check to compute 'src_abbrev' by guarding the 'verify_submodule_committish()' call using `p->status !='D'`, so that the former isn't called in case of non-existent submodule directory, consequently, there is no such error message on any execution environment. The same need not be implemented for 'dst_abbrev' and is rather redundant since the conditional 'if (S_ISGITLINK(p->mod_dst))' already guards the 'verify_submodule_committish()' when we have a status of 'D'. Therefore, eliminate the 'grep' check in t7421. Instead, verify the absence of an error message by doing a 'test_must_be_empty' on the file containing the error. Reported-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Helped-by: Kaartic Sivaraam <kaartic.sivaraam@gmail.com> Mentored-by: Christian Couder <chriscool@tuxfamily.org> Mentored-by: Kaartic Sivaraam <kaartic.sivaraam@gmail.com> Signed-off-by: Shourya Shukla <shouryashukla.oo@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-25submodule: fix style in function definitionShourya Shukla1-3/+3
The definitions of 'verify_submodule_committish()' and 'print_submodule_summary()' had wrong styling in terms of the asterisk placement. Amend them. Also, the warning printed in case of an unexpected file mode printed the mode in decimal. Print it in octal for enhanced readability. Reported-by: Kaartic Sivaraam <kaartic.sivaraam@gmail.com> Mentored-by: Christian Couder <chriscool@tuxfamily.org> Mentored-by: Kaartic Sivaraam <kaartic.sivaraam@gmail.com> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Shourya Shukla <shouryashukla.oo@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-25submodule: eliminate unused parameters from print_submodule_summary()Shourya Shukla1-3/+1
Eliminate the parameters 'missing_{src,dst}' from the 'print_submodule_summary()' function call since they are not used anywhere in the function. Reported-by: Jeff King <peff@peff.net> Mentored-by: Christian Couder <chriscool@tuxfamily.org> Mentored-by: Kaartic Sivaraam <kaartic.sivaraam@gmail.com> Signed-off-by: Shourya Shukla <shouryashukla.oo@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-17submodule--helper: fix leak of core.worktree valueJeff King1-2/+2
In the ensure_core_worktree() function, we load the core.worktree value of the submodule repository using repo_config_get_string(). This function copies the string, but we never free it, leaking the memory. We can instead use the "tmp" version of that function to avoid the allocation at all. We don't have to worry about lifetime issues, since we never even look at the value (we just want to know if it's set). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-14config: fix leaks from git_config_get_string_const()Jeff King1-4/+4
There are two functions to get a single config string: - git_config_get_string() - git_config_get_string_const() One might naively think that the first one allocates a new string and the second one just points us to the internal configset storage. But in fact they both allocate a new copy; the second one exists only to avoid having to cast when using it with a const global which we never intend to free. The documentation for the function explains that clearly, but it seems I'm not alone in being surprised by this. Of 17 calls to the function, 13 of them leak the resulting value. We could obviously fix these by adding the appropriate free(). But it would be simpler still if we actually had a non-allocating way to get the string. There's git_config_get_value() but that doesn't quite do what we want. If the config key is present but is a boolean with no value (e.g., "[foo]bar" in the file), then we'll get NULL (whereas the string versions will print an error and die). So let's introduce a new variant, git_config_get_string_tmp(), that behaves as these callers expect. We need a new name because we have new semantics but the same function signature (so even if we converted the four remaining callers, topics in flight might be surprised). The "tmp" is because this value should only be held onto for a short time. In practice it's rare for us to clear and refresh the configset, invalidating the pointer, but hopefully the "tmp" makes callers think about the lifetime. In each of the converted cases here the value only needs to last within the local function or its immediate caller. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-14submodule--helper: use strbuf_release() to free strbufsJeff King1-2/+2
The prepare_to_clone_next_submodule() function has a few local-variable strbufs. We use strbuf_reset() throughout the function to reuse the buffers over and over. But at the end of the function we also use strbuf_reset() as they go out of scope, which means we end up leaking their heap buffers. This should be strbuf_release() instead. These were introduced by 48308681b0 (git submodule update: have a dedicated helper for cloning, 2016-02-29), but it doesn't seem to have the same mistake elsewhere. Likewise, I looked for other instances of the pattern in the submodule--helper file but couldn't find any. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-12submodule: port submodule subcommand 'summary' from shell to CPrathamesh Chavan1-0/+429
Convert submodule subcommand 'summary' to a builtin and call it via 'git-submodule.sh'. The shell version had to call $diff_cmd twice, once to find the modified modules cared by the user and then again, with that list of modules to do various operations for computing the summary of those modules. On the other hand, the C version does not need a second call to $diff_cmd since it reuses the module list from the first call to do the aforementioned tasks. In the C version, we use the combination of setting a child process' working directory to the submodule path and then calling 'prepare_submodule_repo_env()' which also sets the 'GIT_DIR' to '.git', so that we can be certain that those spawned processes will not access the superproject's ODB by mistake. A behavioural difference between the C and the shell version is that the shell version outputs two line feeds after the 'git log' output when run outside of the tests while the C version outputs one line feed in any case. The reason for this is that the shell version calls log with '--pretty=format:<fmt>' whose output is followed by two echo calls; 'format' does not have "terminator" semantics like its 'tformat' counterpart. So, the log output is terminated by a newline only when invoked by the user and not when invoked from the scripts. This results in the one & two line feed differences in the shell version. On the other hand, the C version calls log with '--pretty=<fmt>' which is equivalent to '--pretty:tformat:<fmt>' which is then followed by a 'printf("\n")'. Due to its "terminator" semantics the log output is always terminated by newline and hence one line feed in any case. Also, when we try to pass an option-like argument after a non-option argument, for instance: git submodule summary HEAD --foo-bar (or) git submodule summary HEAD --cached That argument would be treated like a path to the submodule for which the user is requesting a summary. So, the option ends up having no effect. Though, passing '--quiet' is an exception to this: git submodule summary HEAD --quiet While 'summary' doesn't support '--quiet', we don't get an output for the above command as '--quiet' is treated as a path which means we get an output only if a submodule whose path is '--quiet' exists. The error message in case of computing a summary for non-existent submodules in the C version is different from that of the shell version. Since the new error message is not marked for translation, change the 'test_i18ngrep' in t7421.4 to 'grep'. Mentored-by: Christian Couder <chriscool@tuxfamily.org> Mentored-by: Stefan Beller <stefanbeller@gmail.com> Mentored-by: Kaartic Sivaraam <kaartic.sivaraam@gmail.com> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com> Signed-off-by: Shourya Shukla <shouryashukla.oo@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-12submodule: remove extra line feeds between callback struct and macroShourya Shukla1-3/+0
Many `submodule--helper` subcommands follow the convention that a struct defines their callback data, and the declaration of that struct is followed immediately by a macro to use in static initializers, without any separating empty line. Let's align the `init`, `status` and `sync` subcommands with that convention. Mentored-by: Christian Couder <chriscool@tuxfamily.org> Mentored-by: Kaartic Sivaraam <kaartic.sivaraam@gmail.com> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Helped-by: Philip Oakley <philipoakley@iee.email> Signed-off-by: Shourya Shukla <shouryashukla.oo@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-30strvec: rename struct fieldsJeff King1-3/+3
The "argc" and "argv" names made sense when the struct was argv_array, but now they're just confusing. Let's rename them to "nr" (which we use for counts elsewhere) and "v" (which is rather terse, but reads well when combined with typical variable names like "args.v"). Note that we have to update all of the callers immediately. Playing tricks with the preprocessor is hard here, because we wouldn't want to rewrite unrelated tokens. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-28strvec: fix indentation in renamed callsJeff King1-10/+10
Code which split an argv_array call across multiple lines, like: argv_array_pushl(&args, "one argument", "another argument", "and more", NULL); was recently mechanically renamed to use strvec, which results in mis-matched indentation like: strvec_pushl(&args, "one argument", "another argument", "and more", NULL); Let's fix these up to align the arguments with the opening paren. I did this manually by sifting through the results of: git jump grep 'strvec_.*,$' and liberally applying my editor's auto-format. Most of the changes are of the form shown above, though I also normalized a few that had originally used a single-tab indentation (rather than our usual style of aligning with the open paren). I also rewrapped a couple of obvious cases (e.g., where previously too-long lines became short enough to fit on one), but I wasn't aggressive about it. In cases broken to three or more lines, the grouping of arguments is sometimes meaningful, and it wasn't worth my time or reviewer time to ponder each case individually. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-28strvec: convert builtin/ callers away from argv_array nameJeff King1-59/+59
We eventually want to drop the argv_array name and just use strvec consistently. There's no particular reason we have to do it all at once, or care about interactions between converted and unconverted bits. Because of our preprocessor compat layer, the names are interchangeable to the compiler (so even a definition and declaration using different names is OK). This patch converts all of the files in builtin/ to keep the diff to a manageable size. The conversion was done purely mechanically with: git ls-files '*.c' '*.h' | xargs perl -i -pe ' s/ARGV_ARRAY/STRVEC/g; s/argv_array/strvec/g; ' and then selectively staging files with "git add builtin/". We'll deal with any indentation/style fallouts separately. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-06Merge branch 'js/default-branch-name'Junio C Hamano1-1/+1
The name of the primary branch in existing repositories, and the default name used for the first branch in newly created repositories, is made configurable, so that we can eventually wean ourselves off of the hardcoded 'master'. * js/default-branch-name: contrib: subtree: adjust test to change in fmt-merge-msg testsvn: respect `init.defaultBranch` remote: use the configured default branch name when appropriate clone: use configured default branch name when appropriate init: allow setting the default for the initial branch name via the config init: allow specifying the initial branch name for the new repository docs: add missing diamond brackets submodule: fall back to remote's HEAD for missing remote.<name>.branch send-pack/transport-helper: avoid mentioning a particular branch fmt-merge-msg: stop treating `master` specially
2020-06-24submodule: fall back to remote's HEAD for missing remote.<name>.branchJohannes Schindelin1-1/+1
When `remote.<name>.branch` is not configured, `git submodule update` currently falls back to using the branch name `master`. A much better idea, however, is to use the remote `HEAD`: on all Git servers running reasonably recent Git versions, the symref `HEAD` points to the main branch. Note: t7419 demonstrates that there _might_ be use cases out there that _expect_ `git submodule update --remote` to update submodules to the remote `master` branch even if the remote `HEAD` points to another branch. Arguably, this patch makes the behavior more intuitive, but there is a slight possibility that this might cause regressions in obscure setups. Even so, it should be okay to fix this behavior without anything like a longer transition period: - The `git submodule update --remote` command is not really common. - Current Git's behavior when running this command is outright confusing, unless the remote repository's current branch _is_ `master` (in which case the proposed behavior matches the old behavior). - If a user encounters a regression due to the changed behavior, the fix is actually trivial: setting `submodule.<name>.branch` to `master` will reinstate the old behavior. Helped-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-02submodule: port subcommand 'set-branch' from shell to CShourya Shukla1-0/+44
Convert submodule subcommand 'set-branch' to a builtin and call it via 'git-submodule.sh'. Mentored-by: Christian Couder <chriscool@tuxfamily.org> Mentored-by: Kaartic Sivaraam <kaartic.sivaraam@gmail.com> Helped-by: Denton Liu <liu.denton@gmail.com> Helped-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Shourya Shukla <shouryashukla.oo@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-08submodule: port subcommand 'set-url' from shell to CShourya Shukla1-0/+32
Convert submodule subcommand 'set-url' to a builtin. Port 'set-url' to 'submodule--helper.c' and call the latter via 'git-submodule.sh'. Signed-off-by: Shourya Shukla <shouryashukla.oo@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-18submodule--helper.c: Rename 'cb_foreach' to 'foreach_cb'Shourya Shukla1-4/+4
In 'submodule--helper.c', the structures and macros for callbacks belonging to any subcommand are named in the format: 'subcommand_cb' and 'SUBCOMMAND_CB_INIT' respectively. This was an exception for the subcommand 'foreach' of the command 'submodule'. Rename the aforementioned structures and macros: 'struct cb_foreach' to 'struct foreach_cb' and 'CB_FOREACH_INIT' to 'FOREACH_CB_INIT'. Signed-off-by: Shourya Shukla <shouryashukla.oo@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-05Merge branch 'es/recursive-single-branch-clone'Junio C Hamano1-6/+25
"git clone --recurse-submodules --single-branch" now uses the same single-branch option when cloning the submodules. * es/recursive-single-branch-clone: clone: pass --single-branch during --recurse-submodules submodule--helper: use C99 named initializer
2020-02-25clone: pass --single-branch during --recurse-submodulesEmily Shaffer1-2/+18
Previously, performing "git clone --recurse-submodules --single-branch" resulted in submodules cloning all branches even though the superproject cloned only one branch. Pipe --single-branch through the submodule helper framework to make it to 'clone' later on. Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-25submodule--helper: use C99 named initializerEmily Shaffer1-4/+7
Start using a named initializer list for SUBMODULE_UPDATE_CLONE_INIT, as the struct is becoming cumbersome for a typical struct initializer list. Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-14Merge branch 'pk/status-of-uncloned-submodule'Junio C Hamano1-1/+11
The way "git submodule status" reports an initialized but not yet populated submodule has not been reimplemented correctly when a part of the "git submodule" command was rewritten in C, which has been corrected. * pk/status-of-uncloned-submodule: t7400: testcase for submodule status on unregistered inner git repos submodule: fix status of initialized but not cloned submodules t7400: add a testcase for submodule status on empty dirs
2020-01-27submodule: fix status of initialized but not cloned submodulesPeter Kaestle1-1/+11
Original bash helper for "submodule status" was doing a check for initialized but not cloned submodules and prefixed the status with a minus sign in case no .git file or folder was found inside the submodule directory. This check was missed when the original port of the functionality from bash to C was done. Signed-off-by: Peter Kaestle <peter.kaestle@nokia.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-10Merge branch 'jt/clone-recursesub-ref-advise'Junio C Hamano1-0/+10
The interaction between "git clone --recurse-submodules" and alternate object store was ill-designed. The documentation and code have been taught to make more clear recommendations when the users see failures. * jt/clone-recursesub-ref-advise: submodule--helper: advise on fatal alternate error Doc: explain submodule.alternateErrorStrategy
2019-12-09Sync with Git 2.24.1Junio C Hamano1-2/+18
2019-12-06Sync with 2.23.1Johannes Schindelin1-2/+18
* maint-2.23: (44 commits) Git 2.23.1 Git 2.22.2 Git 2.21.1 mingw: sh arguments need quoting in more circumstances mingw: fix quoting of empty arguments for `sh` mingw: use MSYS2 quoting even when spawning shell scripts mingw: detect when MSYS2's sh is to be spawned more robustly t7415: drop v2.20.x-specific work-around Git 2.20.2 t7415: adjust test for dubiously-nested submodule gitdirs for v2.20.x Git 2.19.3 Git 2.18.2 Git 2.17.3 Git 2.16.6 test-drop-caches: use `has_dos_drive_prefix()` Git 2.15.4 Git 2.14.6 mingw: handle `subst`-ed "DOS drives" mingw: refuse to access paths with trailing spaces or periods mingw: refuse to access paths with illegal characters ...
2019-12-06Sync with 2.22.2Johannes Schindelin1-2/+18
* maint-2.22: (43 commits) Git 2.22.2 Git 2.21.1 mingw: sh arguments need quoting in more circumstances mingw: fix quoting of empty arguments for `sh` mingw: use MSYS2 quoting even when spawning shell scripts mingw: detect when MSYS2's sh is to be spawned more robustly t7415: drop v2.20.x-specific work-around Git 2.20.2 t7415: adjust test for dubiously-nested submodule gitdirs for v2.20.x Git 2.19.3 Git 2.18.2 Git 2.17.3 Git 2.16.6 test-drop-caches: use `has_dos_drive_prefix()` Git 2.15.4 Git 2.14.6 mingw: handle `subst`-ed "DOS drives" mingw: refuse to access paths with trailing spaces or periods mingw: refuse to access paths with illegal characters unpack-trees: let merged_entry() pass through do_add_entry()'s errors ...
2019-12-06Sync with 2.21.1Johannes Schindelin1-2/+18
* maint-2.21: (42 commits) Git 2.21.1 mingw: sh arguments need quoting in more circumstances mingw: fix quoting of empty arguments for `sh` mingw: use MSYS2 quoting even when spawning shell scripts mingw: detect when MSYS2's sh is to be spawned more robustly t7415: drop v2.20.x-specific work-around Git 2.20.2 t7415: adjust test for dubiously-nested submodule gitdirs for v2.20.x Git 2.19.3 Git 2.18.2 Git 2.17.3 Git 2.16.6 test-drop-caches: use `has_dos_drive_prefix()` Git 2.15.4 Git 2.14.6 mingw: handle `subst`-ed "DOS drives" mingw: refuse to access paths with trailing spaces or periods mingw: refuse to access paths with illegal characters unpack-trees: let merged_entry() pass through do_add_entry()'s errors quote-stress-test: offer to test quoting arguments for MSYS2 sh ...
2019-12-06Sync with 2.20.2Johannes Schindelin1-2/+18
* maint-2.20: (36 commits) Git 2.20.2 t7415: adjust test for dubiously-nested submodule gitdirs for v2.20.x Git 2.19.3 Git 2.18.2 Git 2.17.3 Git 2.16.6 test-drop-caches: use `has_dos_drive_prefix()` Git 2.15.4 Git 2.14.6 mingw: handle `subst`-ed "DOS drives" mingw: refuse to access paths with trailing spaces or periods mingw: refuse to access paths with illegal characters unpack-trees: let merged_entry() pass through do_add_entry()'s errors quote-stress-test: offer to test quoting arguments for MSYS2 sh t6130/t9350: prepare for stringent Win32 path validation quote-stress-test: allow skipping some trials quote-stress-test: accept arguments to test via the command-line tests: add a helper to stress test argument quoting mingw: fix quoting of arguments Disallow dubiously-nested submodule git directories ...
2019-12-06submodule: defend against submodule.update = !command in .gitmodulesJonathan Nieder1-0/+2
In v2.15.4, we started to reject `submodule.update` settings in `.gitmodules`. Let's raise a BUG if it somehow still made it through from anywhere but the Git config. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2019-12-06Sync with 2.19.3Johannes Schindelin1-2/+16
* maint-2.19: (34 commits) Git 2.19.3 Git 2.18.2 Git 2.17.3 Git 2.16.6 test-drop-caches: use `has_dos_drive_prefix()` Git 2.15.4 Git 2.14.6 mingw: handle `subst`-ed "DOS drives" mingw: refuse to access paths with trailing spaces or periods mingw: refuse to access paths with illegal characters unpack-trees: let merged_entry() pass through do_add_entry()'s errors quote-stress-test: offer to test quoting arguments for MSYS2 sh t6130/t9350: prepare for stringent Win32 path validation quote-stress-test: allow skipping some trials quote-stress-test: accept arguments to test via the command-line tests: add a helper to stress test argument quoting mingw: fix quoting of arguments Disallow dubiously-nested submodule git directories protect_ntfs: turn on NTFS protection by default path: also guard `.gitmodules` against NTFS Alternate Data Streams ...
2019-12-06Sync with 2.18.2Johannes Schindelin1-2/+16
* maint-2.18: (33 commits) Git 2.18.2 Git 2.17.3 Git 2.16.6 test-drop-caches: use `has_dos_drive_prefix()` Git 2.15.4 Git 2.14.6 mingw: handle `subst`-ed "DOS drives" mingw: refuse to access paths with trailing spaces or periods mingw: refuse to access paths with illegal characters unpack-trees: let merged_entry() pass through do_add_entry()'s errors quote-stress-test: offer to test quoting arguments for MSYS2 sh t6130/t9350: prepare for stringent Win32 path validation quote-stress-test: allow skipping some trials quote-stress-test: accept arguments to test via the command-line tests: add a helper to stress test argument quoting mingw: fix quoting of arguments Disallow dubiously-nested submodule git directories protect_ntfs: turn on NTFS protection by default path: also guard `.gitmodules` against NTFS Alternate Data Streams is_ntfs_dotgit(): speed it up ...
2019-12-06Sync with 2.17.3Johannes Schindelin1-2/+16
* maint-2.17: (32 commits) Git 2.17.3 Git 2.16.6 test-drop-caches: use `has_dos_drive_prefix()` Git 2.15.4 Git 2.14.6 mingw: handle `subst`-ed "DOS drives" mingw: refuse to access paths with trailing spaces or periods mingw: refuse to access paths with illegal characters unpack-trees: let merged_entry() pass through do_add_entry()'s errors quote-stress-test: offer to test quoting arguments for MSYS2 sh t6130/t9350: prepare for stringent Win32 path validation quote-stress-test: allow skipping some trials quote-stress-test: accept arguments to test via the command-line tests: add a helper to stress test argument quoting mingw: fix quoting of arguments Disallow dubiously-nested submodule git directories protect_ntfs: turn on NTFS protection by default path: also guard `.gitmodules` against NTFS Alternate Data Streams is_ntfs_dotgit(): speed it up mingw: disallow backslash characters in tree objects' file names ...
2019-12-06Sync with 2.16.6Johannes Schindelin1-1/+16
* maint-2.16: (31 commits) Git 2.16.6 test-drop-caches: use `has_dos_drive_prefix()` Git 2.15.4 Git 2.14.6 mingw: handle `subst`-ed "DOS drives" mingw: refuse to access paths with trailing spaces or periods mingw: refuse to access paths with illegal characters unpack-trees: let merged_entry() pass through do_add_entry()'s errors quote-stress-test: offer to test quoting arguments for MSYS2 sh t6130/t9350: prepare for stringent Win32 path validation quote-stress-test: allow skipping some trials quote-stress-test: accept arguments to test via the command-line tests: add a helper to stress test argument quoting mingw: fix quoting of arguments Disallow dubiously-nested submodule git directories protect_ntfs: turn on NTFS protection by default path: also guard `.gitmodules` against NTFS Alternate Data Streams is_ntfs_dotgit(): speed it up mingw: disallow backslash characters in tree objects' file names path: safeguard `.git` against NTFS Alternate Streams Accesses ...
2019-12-06Sync with 2.15.4Johannes Schindelin1-1/+16
* maint-2.15: (29 commits) Git 2.15.4 Git 2.14.6 mingw: handle `subst`-ed "DOS drives" mingw: refuse to access paths with trailing spaces or periods mingw: refuse to access paths with illegal characters unpack-trees: let merged_entry() pass through do_add_entry()'s errors quote-stress-test: offer to test quoting arguments for MSYS2 sh t6130/t9350: prepare for stringent Win32 path validation quote-stress-test: allow skipping some trials quote-stress-test: accept arguments to test via the command-line tests: add a helper to stress test argument quoting mingw: fix quoting of arguments Disallow dubiously-nested submodule git directories protect_ntfs: turn on NTFS protection by default path: also guard `.gitmodules` against NTFS Alternate Data Streams is_ntfs_dotgit(): speed it up mingw: disallow backslash characters in tree objects' file names path: safeguard `.git` against NTFS Alternate Streams Accesses clone --recurse-submodules: prevent name squatting on Windows is_ntfs_dotgit(): only verify the leading segment ...
2019-12-06Sync with 2.14.6Johannes Schindelin1-1/+16
* maint-2.14: (28 commits) Git 2.14.6 mingw: handle `subst`-ed "DOS drives" mingw: refuse to access paths with trailing spaces or periods mingw: refuse to access paths with illegal characters unpack-trees: let merged_entry() pass through do_add_entry()'s errors quote-stress-test: offer to test quoting arguments for MSYS2 sh t6130/t9350: prepare for stringent Win32 path validation quote-stress-test: allow skipping some trials quote-stress-test: accept arguments to test via the command-line tests: add a helper to stress test argument quoting mingw: fix quoting of arguments Disallow dubiously-nested submodule git directories protect_ntfs: turn on NTFS protection by default path: also guard `.gitmodules` against NTFS Alternate Data Streams is_ntfs_dotgit(): speed it up mingw: disallow backslash characters in tree objects' file names path: safeguard `.git` against NTFS Alternate Streams Accesses clone --recurse-submodules: prevent name squatting on Windows is_ntfs_dotgit(): only verify the leading segment test-path-utils: offer to run a protectNTFS/protectHFS benchmark ...
2019-12-05Merge branch 'mg/submodule-status-from-a-subdirectory'Junio C Hamano1-1/+2
"git submodule status" that is run from a subdirectory of the superproject did not work well, which has been corrected. * mg/submodule-status-from-a-subdirectory: submodule: fix 'submodule status' when called from a subdirectory
2019-12-05Disallow dubiously-nested submodule git directoriesJohannes Schindelin1-0/+4
Currently it is technically possible to let a submodule's git directory point right into the git dir of a sibling submodule. Example: the git directories of two submodules with the names `hippo` and `hippo/hooks` would be `.git/modules/hippo/` and `.git/modules/hippo/hooks/`, respectively, but the latter is already intended to house the former's hooks. In most cases, this is just confusing, but there is also a (quite contrived) attack vector where Git can be fooled into mistaking remote content for file contents it wrote itself during a recursive clone. Let's plug this bug. To do so, we introduce the new function `validate_submodule_git_dir()` which simply verifies that no git dir exists for any leading directories of the submodule name (if there are any). Note: this patch specifically continues to allow sibling modules names of the form `core/lib`, `core/doc`, etc, as long as `core` is not a submodule name. This fixes CVE-2019-1387. Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-12-04clone --recurse-submodules: prevent name squatting on WindowsJohannes Schindelin1-1/+12
In addition to preventing `.git` from being tracked by Git, on Windows we also have to prevent `git~1` from being tracked, as the default NTFS short name (also known as the "8.3 filename") for the file name `.git` is `git~1`, otherwise it would be possible for malicious repositories to write directly into the `.git/` directory, e.g. a `post-checkout` hook that would then be executed _during_ a recursive clone. When we implemented appropriate protections in 2b4c6efc821 (read-cache: optionally disallow NTFS .git variants, 2014-12-16), we had analyzed carefully that the `.git` directory or file would be guaranteed to be the first directory entry to be written. Otherwise it would be possible e.g. for a file named `..git` to be assigned the short name `git~1` and subsequently, the short name generated for `.git` would be `git~2`. Or `git~3`. Or even `~9999999` (for a detailed explanation of the lengths we have to go to protect `.gitmodules`, see the commit message of e7cb0b4455c (is_ntfs_dotgit: match other .git files, 2018-05-11)). However, by exploiting two issues (that will be addressed in a related patch series close by), it is currently possible to clone a submodule into a non-empty directory: - On Windows, file names cannot end in a space or a period (for historical reasons: the period separating the base name from the file extension was not actually written to disk, and the base name/file extension was space-padded to the full 8/3 characters, respectively). Helpfully, when creating a directory under the name, say, `sub.`, that trailing period is trimmed automatically and the actual name on disk is `sub`. This means that while Git thinks that the submodule names `sub` and `sub.` are different, they both access `.git/modules/sub/`. - While the backslash character is a valid file name character on Linux, it is not so on Windows. As Git tries to be cross-platform, it therefore allows backslash characters in the file names stored in tree objects. Which means that it is totally possible that a submodule `c` sits next to a file `c\..git`, and on Windows, during recursive clone a file called `..git` will be written into `c/`, of course _before_ the submodule is cloned. Note that the actual exploit is not quite as simple as having a submodule `c` next to a file `c\..git`, as we have to make sure that the directory `.git/modules/b` already exists when the submodule is checked out, otherwise a different code path is taken in `module_clone()` that does _not_ allow a non-empty submodule directory to exist already. Even if we will address both issues nearby (the next commit will disallow backslash characters in tree entries' file names on Windows, and another patch will disallow creating directories/files with trailing spaces or periods), it is a wise idea to defend in depth against this sort of attack vector: when submodules are cloned recursively, we now _require_ the directory to be empty, addressing CVE-2019-1349. Note: the code path we patch is shared with the code path of `git submodule update --init`, which must not expect, in general, that the directory is empty. Hence we have to introduce the new option `--force-init` and hand it all the way down from `git submodule` to the actual `git submodule--helper` process that performs the initial clone. Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-12-03submodule--helper: advise on fatal alternate errorJonathan Tan1-0/+10
When recursively cloning a superproject with some shallow modules defined in its .gitmodules, then recloning with "--reference=<path>", an error occurs. For example: git clone --recurse-submodules --branch=master -j8 \ https://android.googlesource.com/platform/superproject \ master git clone --recurse-submodules --branch=master -j8 \ https://android.googlesource.com/platform/superproject \ --reference master master2 fails with: fatal: submodule '<snip>' cannot add alternate: reference repository '<snip>' is shallow When a alternate computed from the superproject's alternate cannot be added, whether in this case or another, advise about configuring the "submodule.alternateErrorStrategy" configuration option and using "--reference-if-able" instead of "--reference" when cloning. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-25submodule: fix 'submodule status' when called from a subdirectoryManish Goregaokar1-1/+2
When calling `git submodule status` while in a subdirectory, we are incorrectly not detecting modified submodules and thus reporting that all of the submodules are unchanged. This is because the submodule helper is calling `diff-index` with the submodule path assuming the path is relative to the current prefix directory, however the submodule path used is actually relative to the root. Always pass NULL as the `prefix` when running diff-files on the submodule, to make sure the submodule's path is interpreted as relative to the superproject's repository root. Signed-off-by: Manish Goregaokar <manishsmail@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-11Merge branch 'bc/object-id-part17'Junio C Hamano1-1/+1
Preparation for SHA-256 upgrade continues. * bc/object-id-part17: (26 commits) midx: switch to using the_hash_algo builtin/show-index: replace sha1_to_hex rerere: replace sha1_to_hex builtin/receive-pack: replace sha1_to_hex builtin/index-pack: replace sha1_to_hex packfile: replace sha1_to_hex wt-status: convert struct wt_status to object_id cache: remove null_sha1 builtin/worktree: switch null_sha1 to null_oid builtin/repack: write object IDs of the proper length pack-write: use hash_to_hex when writing checksums sequencer: convert to use the_hash_algo bisect: switch to using the_hash_algo sha1-lookup: switch hard-coded constants to the_hash_algo config: use the_hash_algo in abbrev comparison combine-diff: replace GIT_SHA1_HEXSZ with the_hash_algo bundle: switch to use the_hash_algo connected: switch GIT_SHA1_HEXSZ to the_hash_algo show-index: switch hard-coded constants to the_hash_algo blame: remove needless comparison with GIT_SHA1_HEXSZ ...
2019-09-29builtin/submodule--helper: fix usage string for 'update-clone'Bert Wesarg1-1/+1
Signed-off-by: Bert Wesarg <bert.wesarg@googlemail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-19cache: remove null_sha1brian m. carlson1-1/+1
All of the existing uses of null_sha1 can be converted into uses of null_oid, so do so. Remove null_sha1 and is_null_sha1, and define is_null_oid in terms of null_oid. This also has the additional benefit of removing several uses of sha1_to_hex. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-07-29Merge branch 'ms/submodule-foreach-fix' into maintJunio C Hamano1-0/+1
"git submodule foreach" did not protect command line options passed to the command to be run in each submodule correctly, when the "--recursive" option was in use. * ms/submodule-foreach-fix: submodule foreach: fix recursion of options
2019-07-09Merge branch 'ms/submodule-foreach-fix'Junio C Hamano1-0/+1
"git submodule foreach" did not protect command line options passed to the command to be run in each submodule correctly, when the "--recursive" option was in use. * ms/submodule-foreach-fix: submodule foreach: fix recursion of options
2019-06-25submodule foreach: fix recursion of optionsMorian Sonnet1-0/+1
Calling git submodule foreach --recursive <subcommand> --<option> leads to an error stating that the option --<option> is unknown to submodule--helper. That is of course only, when <option> is not a valid option for git submodule foreach. The reason for this is, that above call is internally translated into a call to submodule--helper: git submodule--helper foreach --recursive \ -- <subcommand> --<option> This call starts by executing the subcommand with its option inside the first level submodule and continues by calling the next iteration of the submodule foreach call git --super-prefix <submodulepath> submodule--helper \ foreach --recursive <subcommand> --<option> inside the first level submodule. Note that the double dash in front of the subcommand is missing. This problem starts to arise only recently, as the PARSE_OPT_KEEP_UNKNOWN flag for the argument parsing of git submodule foreach was removed in commit a282f5a906. Hence, the unknown option is complained about now, as the argument parsing is not properly ended by the double dash. This commit fixes the problem by adding the double dash in front of the subcommand during the recursion. Signed-off-by: Morian Sonnet <moriansonnet@googlemail.com> Acked-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-13Merge branch 'jk/unused-params-final-batch'Junio C Hamano1-2/+1
* jk/unused-params-final-batch: verify-commit: simplify parameters to run_gpg_verify() show-branch: drop unused parameter from show_independent() rev-list: drop unused void pointer from finish_commit() remove_all_fetch_refspecs(): drop unused "remote" parameter receive-pack: drop unused "commands" from prepare_shallow_update() pack-objects: drop unused rev_info parameters name-rev: drop unused parameters from is_better_name() mktree: drop unused length parameter wt-status: drop unused status parameter read-cache: drop unused parameter from threaded load clone: drop dest parameter from copy_alternates() submodule: drop unused prefix parameter from some functions builtin: consistently pass cmd_* prefix to parse_options cmd_{read,write}_tree: rename "unused" variable that is used
2019-05-19Merge branch 'nd/submodule-helper-incomplete-line-fix'Junio C Hamano1-1/+1
Typofix. * nd/submodule-helper-incomplete-line-fix: submodule--helper: add a missing \n
2019-05-13submodule: drop unused prefix parameter from some functionsJeff King1-2/+1
We stopped using the "prefix" parameter of relocate_single_git_dir_into_superproject() and its callers in 202275b96b (submodule.c: get_super_prefix_or_empty, 2017-03-14), where we switched to using the environment global directly. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-08submodule--helper: add a missing \nNguyễn Thái Ngọc Duy1-1/+1
This is a complete line. We're not expecting the next function to add anything to the same line. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-25Merge branch 'nd/submodule-foreach-quiet'Junio C Hamano1-4/+4
"git submodule foreach <command> --quiet" did not pass the option down correctly, which has been corrected. * nd/submodule-foreach-quiet: submodule foreach: fix "<command> --quiet" not being respected
2019-04-25Merge branch 'dl/submodule-set-branch'Junio C Hamano1-5/+12
"git submodule" learns "set-branch" subcommand that allows the submodule.*.branch settings to be modified. * dl/submodule-set-branch: submodule: teach set-branch subcommand submodule--helper: teach config subcommand --unset git-submodule.txt: "--branch <branch>" option defaults to 'master'
2019-04-15submodule foreach: fix "<command> --quiet" not being respectedNguyễn Thái Ngọc Duy1-4/+4
Robin reported that git submodule foreach --quiet git pull --quiet origin is not really quiet anymore [1]. "git pull" behaves as if --quiet is not given. This happens because parseopt in submodule--helper will try to parse both --quiet options as if they are foreach's options, not git-pull's. The parsed options are removed from the command line. So when we do pull later, we execute just this git pull origin When calling submodule helper, adding "--" in front of "git pull" will stop parseopt for parsing options that do not really belong to submodule--helper foreach. PARSE_OPT_KEEP_UNKNOWN is removed as a safety measure. parseopt should never see unknown options or something has gone wrong. There are also a couple usage string update while I'm looking at them. While at it, I also add "--" to other subcommands that pass "$@" to submodule--helper. "$@" in these cases are paths and less likely to be --something-like-this. But the point still stands, git-submodule has parsed and classified what are options, what are paths. submodule--helper should never consider paths passed by git-submodule to be options even if they look like one. The test case is also contributed by Robin. [1] it should be quiet before fc1b9243cd (submodule: port submodule subcommand 'foreach' from shell to C, 2018-05-10) because parseopt can't accidentally eat options then. Reported-by: Robin H. Johnson <robbat2@gentoo.org> Tested-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-20report_path_error(): drop unused prefix parameterJeff King1-1/+1
This hasn't been used since 17ddc66e70 (convert report_path_error to take struct pathspec, 2013-07-14), as the names in the struct will have already been prefixed when they were parsed. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22trace2: create new combined trace facilityJeff Hostetler1-5/+4
Create a new unified tracing facility for git. The eventual intent is to replace the current trace_printf* and trace_performance* routines with a unified set of git_trace2* routines. In addition to the usual printf-style API, trace2 provides higer-level event verbs with fixed-fields allowing structured data to be written. This makes post-processing and analysis easier for external tools. Trace2 defines 3 output targets. These are set using the environment variables "GIT_TR2", "GIT_TR2_PERF", and "GIT_TR2_EVENT". These may be set to "1" or to an absolute pathname (just like the current GIT_TRACE). * GIT_TR2 is intended to be a replacement for GIT_TRACE and logs command summary data. * GIT_TR2_PERF is intended as a replacement for GIT_TRACE_PERFORMANCE. It extends the output with columns for the command process, thread, repo, absolute and relative elapsed times. It reports events for child process start/stop, thread start/stop, and per-thread function nesting. * GIT_TR2_EVENT is a new structured format. It writes event data as a series of JSON records. Calls to trace2 functions log to any of the 3 output targets enabled without the need to call different trace_printf* or trace_performance* routines. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-08submodule--helper: teach config subcommand --unsetDenton Liu1-5/+12
This teaches submodule--helper config the --unset option, which removes the specified configuration key from the .gitmodule file. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-06Merge branch 'nd/the-index-final'Junio C Hamano1-0/+1
The assumption to work on the single "in-core index" instance has been reduced from the library-ish part of the codebase. * nd/the-index-final: cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch read-cache.c: remove the_* from index_has_changes() merge-recursive.c: remove implicit dependency on the_repository merge-recursive.c: remove implicit dependency on the_index sha1-name.c: remove implicit dependency on the_index read-cache.c: replace update_index_if_able with repo_& read-cache.c: kill read_index() checkout: avoid the_index when possible repository.c: replace hold_locked_index() with repo_hold_locked_index() notes-utils.c: remove the_repository references grep: use grep_opt->repo instead of explict repo argument
2019-01-29Merge branch 'sb/submodule-recursive-fetch-gets-the-tip'Junio C Hamano1-1/+1
"git fetch --recurse-submodules" may not fetch the necessary commit that is bound to the superproject, which is getting corrected. * sb/submodule-recursive-fetch-gets-the-tip: fetch: ensure submodule objects fetched submodule.c: fetch in submodules git directory instead of in worktree submodule: migrate get_next_submodule to use repository structs repository: repo_submodule_init to take a submodule struct submodule: store OIDs in changed_submodule_names submodule.c: tighten scope of changed_submodule_names struct submodule.c: sort changed_submodule_names before searching it submodule.c: fix indentation sha1-array: provide oid_array_filter
2019-01-24cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switchNguyễn Thái Ngọc Duy1-0/+1
By default, index compat macros are off from now on, because they could hide the_index dependency. Only those in builtin can use it. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-18Merge branch 'sb/submodule-unset-core-worktree-when-worktree-is-lost'Junio C Hamano1-1/+3
The core.worktree setting in a submodule repository should not be pointing at a directory when the submodule loses its working tree (e.g. getting deinit'ed), but the code did not properly maintain this invariant. * sb/submodule-unset-core-worktree-when-worktree-is-lost: submodule deinit: unset core.worktree submodule--helper: fix BUG message in ensure_core_worktree submodule: unset core.worktree if no working tree is present submodule update: add regression test with old style setups
2019-01-18Merge branch 'sb/submodule-fetchjobs-default-to-one'Junio C Hamano1-1/+1
"git submodule update" ought to use a single job unless asked, but by mistake used multiple jobs, which has been fixed. * sb/submodule-fetchjobs-default-to-one: submodule update: run at most one fetch job unless otherwise set
2019-01-04Merge branch 'jk/loose-object-cache'Junio C Hamano1-4/+5
Code clean-up with optimization for the codepath that checks (non-)existence of loose objects. * jk/loose-object-cache: odb_load_loose_cache: fix strbuf leak fetch-pack: drop custom loose object cache sha1-file: use loose object cache for quick existence check object-store: provide helpers for loose_objects_cache sha1-file: use an object_directory for the main object dir handle alternates paths the same as the main object dir sha1_file_name(): overwrite buffer instead of appending rename "alternate_object_database" to "object_directory" submodule--helper: prefer strip_suffix() to ends_with() fsck: do not reuse child_process structs
2018-12-26submodule deinit: unset core.worktreeStefan Beller1-0/+2
When a submodule is deinit'd, the working tree is gone, so the setting of core.worktree is bogus. Unset it. As we covered the only other case in which a submodule loses its working tree in the earlier step (i.e. switching branches of top-level project to move to a commit that did not have the submodule), this makes the code always maintain core.worktree correctly unset when there is no working tree for a submodule. This re-introduces 984cd77ddb (submodule deinit: unset core.worktree, 2018-06-18), which was reverted as part of f178c13fda (Revert "Merge branch 'sb/submodule-core-worktree'", 2018-09-07) The whole series was reverted as the offending commit e98317508c (submodule: ensure core.worktree is set after update, 2018-06-18) was relied on by other commits such as 984cd77ddb. Keep the offending commit reverted, but its functionality came back via 4d6d6ef1fc (Merge branch 'sb/submodule-update-in-c', 2018-09-17), such that we can reintroduce 984cd77ddb now. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-12-26submodule--helper: fix BUG message in ensure_core_worktreeStefan Beller1-1/+1
74d4731da1 (submodule--helper: replace connect-gitdir-workingtree by ensure-core-worktree, 2018-08-13) missed to update the BUG message. Fix it. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-12-14submodule update: run at most one fetch job unless otherwise setJunio C Hamano1-1/+1
In a028a1930c (fetching submodules: respect `submodule.fetchJobs` config option, 2016-02-29), we made sure to keep the default behavior of fetching at most one submodule at once when not setting the newly introduced `submodule.fetchJobs` config. This regressed in 90efe595c5 (builtin/submodule--helper: factor out submodule updating, 2018-08-03). Fix it. Reported-by: Sjon Hortensius <sjon@parse.nl> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-12-05repository: repo_submodule_init to take a submodule structStefan Beller1-1/+1
When constructing a struct repository for a submodule for some revision of the superproject where the submodule is not contained in the index, it may not be present in the working tree currently either. In that situation giving a 'path' argument is not useful. Upgrade the repo_submodule_init function to take a struct submodule instead. The submodule struct can be obtained via submodule_from_{path, name} or an artificial submodule struct can be passed in. While we are at it, rename the repository struct in the repo_submodule_init function, which is to be initialized, to a name that is not confused with the struct submodule as easily. Perform such renames in similar functions as well. Also move its documentation into the header file. Reviewed-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-13Merge branch 'ao/submodule-wo-gitmodules-checked-out'Junio C Hamano1-0/+40
The submodule support has been updated to read from the blob at HEAD:.gitmodules when the .gitmodules file is missing from the working tree. * ao/submodule-wo-gitmodules-checked-out: t/helper: add test-submodule-nested-repo-config submodule: support reading .gitmodules when it's not in the working tree submodule: add a helper to check if it is safe to write to .gitmodules t7506: clean up .gitmodules properly before setting up new scenario submodule: use the 'submodule--helper config' command submodule--helper: add a new 'config' subcommand t7411: be nicer to future tests and really clean things up t7411: merge tests 5 and 6 submodule: factor out a config_set_in_gitmodules_file_gently function submodule: add a print_config_from_gitmodules() helper
2018-11-13rename "alternate_object_database" to "object_directory"Jeff King1-3/+3
In preparation for unifying the handling of alt odb's and the normal repo object directory, let's use a more neutral name. This patch is purely mechanical, swapping the type name, and converting any variables named "alt" to "odb". There should be no functional change, but it will reduce the noise in subsequent diffs. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-13submodule--helper: prefer strip_suffix() to ends_with()Jeff King1-3/+4
Using strip_suffix() lets us avoid repeating ourselves. It also makes the handling of "/" a bit less subtle (we strip one less character than we matched in order to leave it in place, but we can just as easily include the "/" when we add more path components). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-06Merge branch 'sb/submodule-url-to-absolute'Junio C Hamano1-17/+34
Some codepaths failed to form a proper URL when .gitmodules record the URL to a submodule repository as relative to the repository of superproject, which has been corrected. * sb/submodule-url-to-absolute: submodule helper: convert relative URL to absolute URL if needed
2018-10-31submodule: support reading .gitmodules when it's not in the working treeAntonio Ospite1-1/+5
When the .gitmodules file is not available in the working tree, try using the content from the index and from the current branch. This covers the case when the file is part of the repository but for some reason it is not checked out, for example because of a sparse checkout. This makes it possible to use at least the 'git submodule' commands which *read* the gitmodules configuration file without fully populating the working tree. Writing to .gitmodules will still require that the file is checked out, so check for that before calling config_set_in_gitmodules_file_gently. Add a similar check also in git-submodule.sh::cmd_add() to anticipate the eventual failure of the "git submodule add" command when .gitmodules is not safely writeable; this prevents the command from leaving the repository in a spurious state (e.g. the submodule repository was cloned but .gitmodules was not updated because config_set_in_gitmodules_file_gently failed). Moreover, since config_from_gitmodules() now accesses the global object store, it is necessary to protect all code paths which call the function against concurrent access to the global object store. Currently this only happens in builtin/grep.c::grep_submodules(), so call grep_read_lock() before invoking code involving config_from_gitmodules(). Finally, add t7418-submodule-sparse-gitmodules.sh to verify that reading from .gitmodules succeeds and that writing to it fails when the file is not checked out. NOTE: there is one rare case where this new feature does not work properly yet: nested submodules without .gitmodules in their working tree. This has been documented with a warning and a test_expect_failure item in t7814, and in this case the current behavior is not altered: no config is read. Signed-off-by: Antonio Ospite <ao2@ao2.it> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-30Merge branch 'sb/submodule-helper-remove-cruft'Junio C Hamano1-2/+0
Code clean-up. * sb/submodule-helper-remove-cruft: builtin/submodule--helper: remove debugging leftover tracing
2018-10-19Merge branch 'nd/the-index'Junio C Hamano1-1/+1
Various codepaths in the core-ish part learn to work on an arbitrary in-core index structure, not necessarily the default instance "the_index". * nd/the-index: (23 commits) revision.c: reduce implicit dependency the_repository revision.c: remove implicit dependency on the_index ws.c: remove implicit dependency on the_index tree-diff.c: remove implicit dependency on the_index submodule.c: remove implicit dependency on the_index line-range.c: remove implicit dependency on the_index userdiff.c: remove implicit dependency on the_index rerere.c: remove implicit dependency on the_index sha1-file.c: remove implicit dependency on the_index patch-ids.c: remove implicit dependency on the_index merge.c: remove implicit dependency on the_index merge-blobs.c: remove implicit dependency on the_index ll-merge.c: remove implicit dependency on the_index diff-lib.c: remove implicit dependency on the_index read-cache.c: remove implicit dependency on the_index diff.c: remove implicit dependency on the_index grep.c: remove implicit dependency on the_index diff.c: remove the_index dependency in textconv() functions blame.c: rename "repo" argument to "r" combine-diff.c: remove implicit dependency on the_index ...
2018-10-18builtin/submodule--helper: remove debugging leftover tracingStefan Beller1-2/+0
I noticed 74d4731da1 (submodule--helper: replace connect-gitdir-workingtree by ensure-core-worktree, 2018-08-13) had two leftover debugging statements when reading The coverage report [1]. Remove them. https://public-inbox.org/git/e30a9c05-87d8-1f2b-182c-6d6a5fefe43c@gmail.com/ Signed-off-by: Stefan Beller <sbeller@google.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-18submodule helper: convert relative URL to absolute URL if neededStefan Beller1-17/+34
The submodule helper update_clone called by "git submodule update", clones submodules if needed. As submodules used to have the URL indicating if they were active, the step to resolve relative URLs was done in the "submodule init" step. Nowadays submodules can be configured active without calling an explicit init, e.g. via configuring submodule.active. When trying to obtain submodules that are set active this way, we'll fallback to the URL found in the .gitmodules, which may be relative to the superproject, but we do not resolve it, yet: git clone https://gerrit.googlesource.com/gerrit cd gerrit && grep url .gitmodules url = ../plugins/codemirror-editor ... git config submodule.active . git submodule update fatal: repository '../plugins/codemirror-editor' does not exist fatal: clone of '../plugins/codemirror-editor' into submodule path '/tmp/gerrit/plugins/codemirror-editor' failed Failed to clone 'plugins/codemirror-editor'. Retry scheduled [...] fatal: clone of '../plugins/codemirror-editor' into submodule path '/tmp/gerrit/plugins/codemirror-editor' failed Failed to clone 'plugins/codemirror-editor' a second time, aborting [...] To resolve the issue, factor out the function that resolves the relative URLs in "git submodule init" (in the submodule helper in the init_submodule function) and call it at the appropriate place in the update_clone helper. Reported-by: Jaewoong Jung <jungjw@google.com> Signed-off-by: Stefan Beller <sbeller@google.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-09submodule: add a helper to check if it is safe to write to .gitmodulesAntonio Ospite1-1/+23
Introduce a helper function named is_writing_gitmodules_ok() to verify that the .gitmodules file is safe to write. The function name follows the scheme of is_staging_gitmodules_ok(). The two symbolic constants GITMODULES_INDEX and GITMODULES_HEAD are used to get help from the C preprocessor in preventing typos, especially for future users. This is in preparation for a future change which teaches git how to read .gitmodules from the index or from the current branch if the file is not available in the working tree. The rationale behind the check is that writing to .gitmodules requires the file to be present in the working tree, unless a brand new .gitmodules is being created (in which case the .gitmodules file would not exist at all: neither in the working tree nor in the index or in the current branch). Expose the functionality also via a "submodule-helper config --check-writeable" command, as git scripts may want to perform the check before modifying submodules configuration. Signed-off-by: Antonio Ospite <ao2@ao2.it> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-09submodule--helper: add a new 'config' subcommandAntonio Ospite1-0/+14
Add a new 'config' subcommand to 'submodule--helper', this extra level of indirection makes it possible to add some flexibility to how the submodules configuration is handled. Signed-off-by: Antonio Ospite <ao2@ao2.it> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-27Sync with 2.19.1Junio C Hamano1-0/+1
* maint: Git 2.19.1 Git 2.18.1 Git 2.17.2 fsck: detect submodule paths starting with dash fsck: detect submodule urls starting with dash Git 2.16.5 Git 2.15.3 Git 2.14.5 submodule-config: ban submodule paths that start with a dash submodule-config: ban submodule urls that start with dash submodule--helper: use "--" to signal end of clone options
2018-09-27Sync with 2.18.1Junio C Hamano1-0/+1
* maint-2.18: Git 2.18.1 Git 2.17.2 fsck: detect submodule paths starting with dash fsck: detect submodule urls starting with dash Git 2.16.5 Git 2.15.3 Git 2.14.5 submodule-config: ban submodule paths that start with a dash submodule-config: ban submodule urls that start with dash submodule--helper: use "--" to signal end of clone options
2018-09-27Sync with 2.17.2Junio C Hamano1-0/+1
* maint-2.17: Git 2.17.2 fsck: detect submodule paths starting with dash fsck: detect submodule urls starting with dash Git 2.16.5 Git 2.15.3 Git 2.14.5 submodule-config: ban submodule paths that start with a dash submodule-config: ban submodule urls that start with dash submodule--helper: use "--" to signal end of clone options
2018-09-27Sync with 2.16.5Junio C Hamano1-0/+1
* maint-2.16: Git 2.16.5 Git 2.15.3 Git 2.14.5 submodule-config: ban submodule paths that start with a dash submodule-config: ban submodule urls that start with dash submodule--helper: use "--" to signal end of clone options
2018-09-27Sync with 2.15.3Junio C Hamano1-0/+1
* maint-2.15: Git 2.15.3 Git 2.14.5 submodule-config: ban submodule paths that start with a dash submodule-config: ban submodule urls that start with dash submodule--helper: use "--" to signal end of clone options
2018-09-27Sync with Git 2.14.4Junio C Hamano1-0/+1
* maint-2.14: Git 2.14.5 submodule-config: ban submodule paths that start with a dash submodule-config: ban submodule urls that start with dash submodule--helper: use "--" to signal end of clone options
2018-09-27submodule--helper: use "--" to signal end of clone optionsJeff King1-0/+1
When we clone a submodule, we call "git clone $url $path". But there's nothing to say that those components can't begin with a dash themselves, confusing git-clone into thinking they're options. Let's pass "--" to make it clear what we expect. There's no test here, because it's actually quite hard to make these names work, even with "git clone" parsing them correctly. And we're going to restrict these cases even further in future commits. So we'll leave off testing until then; this is just the minimal fix to prevent us from doing something stupid with a badly formed entry. Reported-by: joernchen <joernchen@phenoelit.de> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-21revision.c: remove implicit dependency on the_indexNguyễn Thái Ngọc Duy1-1/+1
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-17Merge branch 'sb/submodule-update-in-c'Junio C Hamano1-34/+158
"git submodule update" is getting rewritten piece-by-piece into C. * sb/submodule-update-in-c: submodule--helper: introduce new update-module-mode helper submodule--helper: replace connect-gitdir-workingtree by ensure-core-worktree builtin/submodule--helper: factor out method to update a single submodule builtin/submodule--helper: store update_clone information in a struct builtin/submodule--helper: factor out submodule updating git-submodule.sh: rename unused variables git-submodule.sh: align error reporting for update mode to use path
2018-09-07Revert "Merge branch 'sb/submodule-core-worktree'"Jonathan Nieder1-26/+0
This reverts commit 7e25437d35a70791b345872af202eabfb3e1a8bc, reversing changes made to 00624d608cc69bd62801c93e74d1ea7a7ddd6598. v2.19.0-rc0~165^2~1 (submodule: ensure core.worktree is set after update, 2018-06-18) assumes an "absorbed" submodule layout, where the submodule's Git directory is in the superproject's .git/modules/ directory and .git in the submodule worktree is a .git file pointing there. In particular, it uses $GIT_DIR/modules/$name to find the submodule to find out whether it already has core.worktree set, and it uses connect_work_tree_and_git_dir if not, resulting in fatal: could not open sub/.git for writing The context behind that patch: v2.19.0-rc0~165^2~2 (submodule: unset core.worktree if no working tree is present, 2018-06-12) unsets core.worktree when running commands like "git checkout --recurse-submodules" to switch to a branch without the submodule. If a user then uses "git checkout --no-recurse-submodules" to switch back to a branch with the submodule and runs "git submodule update", this patch is needed to ensure that commands using the submodule directly are aware of the path to the worktree. It is late in the release cycle, so revert the whole 3-patch series. We can try again later for 2.20. Reported-by: Allan Sandfeld Jensen <allan.jensen@qt.io> Helped-by: Stefan Beller <sbeller@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-23i18n: fix mistakes in translated stringsJean-Noël Avila1-1/+1
Fix typos and convert a question which does not expect to be replied to a simple advice. Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-20Merge branch 'sb/submodule-cleanup'Junio C Hamano1-1/+0
A few preliminary minor clean-ups in the area around submodules. * sb/submodule-cleanup: builtin/submodule--helper: remove stray new line t7410: update to new style
2018-08-16builtin/submodule--helper: remove stray new lineStefan Beller1-1/+0
Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-14submodule--helper: introduce new update-module-mode helperStefan Beller1-0/+61
This chews off a bit of the shell part of the update command in git-submodule.sh. When writing the C code, keep in mind that the submodule--helper part will go away eventually and we want to have a C function that is able to determine the submodule update strategy, it as a nicety, make determine_submodule_update_strategy accessible for arbitrary repositories. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-14submodule--helper: replace connect-gitdir-workingtree by ensure-core-worktreeStefan Beller1-24/+40
e98317508c0 (submodule: ensure core.worktree is set after update, 2018-06-18) was overly aggressive in calling connect_work_tree_and_git_dir as that ensures both the 'core.worktree' configuration is set as well as setting up correct gitlink file pointing at the git directory. We do not need to check for the gitlink in this part of the cmd_update in git-submodule.sh, as the initial call to update-clone will have ensured that. So we can reduce the work to only (check and potentially) set the 'core.worktree' setting. While at it move the check from shell to C as that proves to be useful in a follow up patch, as we do not need the 'name' in shell now. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>