aboutsummaryrefslogtreecommitdiffstats
path: root/contrib
AgeCommit message (Collapse)AuthorFilesLines
2024-03-03completion: reflog with implicit "show"Rubén Justo1-5/+4
When no subcommand is specified to "reflog", we assume "show" [1]: $ git reflog -h usage: git reflog [show] [<log-options>] [<ref>] ... This implicit "show" is not being completed correctly: $ git checkout -b default $ git reflog def<TAB><TAB> ... no completion options ... The expected result is: $ git reflog default This happens because we're completing references after seeing a valid subcommand in the command line. This prevents the implicit "show" from working properly, but also introduces a new problem: it keeps offering subcommand options when the subcommand is implicit: $ git checkout -b explore $ git reflog default ex<TAB> ... $ git reflog default expire The expected result is: $ git reflog default explore To fix this, complete references even if no subcommand is present, or in other words when the subcommand is implicit "show". Also, only include completion options for subcommands when completing the right position in the command line. 1. cf39f54efc (git reflog show, 2007-02-08) Signed-off-by: Rubén Justo <rjusto@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-27Merge branch 'mh/libsecret-empty-password-fix'Junio C Hamano1-0/+3
Credential helper based on libsecret (in contrib/) has been updated to handle an empty password correctly. * mh/libsecret-empty-password-fix: libsecret: retrieve empty password
2024-02-27Merge branch 'bb/completion-no-grep-into-awk'Junio C Hamano1-2/+4
Some parts of command line completion script (in contrib/) have been micro-optimized. * bb/completion-no-grep-into-awk: completion: use awk for filtering the config entries
2024-02-27completion: fix __git_complete_worktree_pathsRubén Justo1-1/+1
Use __git to invoke "worktree list" in __git_complete_worktree_paths, to respect any "-C" and "--git-dir" options present on the command line. Signed-off-by: Rubén Justo <rjusto@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-26Merge branch 'rs/use-xstrncmpz'Junio C Hamano1-0/+28
Code clean-up. * rs/use-xstrncmpz: use xstrncmpz()
2024-02-26Merge branch 'ps/reftable-backend'Junio C Hamano1-1/+1
Integrate the reftable code into the refs framework as a backend. * ps/reftable-backend: refs/reftable: fix leak when copying reflog fails ci: add jobs to test with the reftable backend refs: introduce reftable backend
2024-02-19libsecret: retrieve empty passwordM Hickford1-0/+3
Since 0ce02e2f (credential/libsecret: store new attributes, 2023-06-16) a test that stores empty username and password fails when t0303-credential-external.sh is run with GIT_TEST_CREDENTIAL_HELPER=libsecret. Retrieve empty password carefully. This fixes test: ok 14 - helper (libsecret) can store empty username Signed-off-by: M Hickford <mirth.hickford@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-16completion: use awk for filtering the config entriesBeat Bolli1-2/+4
Commits 1e0ee4087e (completion: add and use __git_compute_first_level_config_vars_for_section, 2024-02-10) and 6e32f718ff (completion: add and use __git_compute_second_level_config_vars_for_section, 2024-02-10) introduced new helpers for config completion. Both helpers use a pipeline of grep and awk to filter the list of config entries. awk is perfectly capable of filtering, so let's eliminate the grep process and move the filtering into the awk script. The "-E" grep option (extended syntax) was not necessary, as $section is a single word. While at it, wrap the over-long lines to make them more readable. Signed-off-by: Beat Bolli <dev+git@drbeat.li> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-14Merge branch 'pb/complete-config'Junio C Hamano1-44/+46
The command line completion script (in contrib/) learned to complete configuration variable names better. * pb/complete-config: completion: add and use __git_compute_second_level_config_vars_for_section completion: add and use __git_compute_first_level_config_vars_for_section completion: complete 'submodule.*' config variables completion: add space after config variable names also in Bash 3
2024-02-12Merge branch 'bk/complete-bisect'Junio C Hamano1-7/+58
Command line completion support (in contrib/) has been updated for "git bisect". * bk/complete-bisect: completion: bisect: recognize but do not complete view subcommand completion: bisect: complete log opts for visualize subcommand completion: new function __git_complete_log_opts completion: bisect: complete missing --first-parent and - -no-checkout options completion: bisect: complete custom terms and related options completion: bisect: complete bad, new, old, and help subcommands completion: tests: always use 'master' for default initial branch name
2024-02-12completion: add and use __git_compute_second_level_config_vars_for_sectionPhilippe Blain1-50/+21
In a previous commit we removed some hardcoded config variable names from function __git_complete_config_variable_name in the completion script by introducing a new function, __git_compute_first_level_config_vars_for_section. The remaining hardcoded config variables are "second level" configuration variables, meaning 'branch.<name>.upstream', 'remote.<name>.url', etc. where <name> is a user-defined name. Making use of the new existing --config flag to 'git help', add a new function, __git_compute_second_level_config_vars_for_section. This function takes as argument a config section name and computes the corresponding second-level config variables, i.e. those that contain a '<' which indicates the start of a placeholder. Note that as in __git_compute_first_level_config_vars_for_section added previsouly, we use indirect expansion instead of associative arrays to stay compatible with Bash 3 on which macOS is stuck for licensing reasons. As explained in the previous commit, we use the existing pattern in the completion script of using global variables to cache the list of variables for each section. Use this new function and the variables it defines in __git_complete_config_variable_name to remove hardcoded config variables, and add a test to verify the new function. Use a single 'case' for all sections with second-level variables names, since the code for each of them is now exactly the same. Adjust the name of a test added in a previous commit to reflect that it now tests the added function. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-12completion: add and use __git_compute_first_level_config_vars_for_sectionPhilippe Blain1-3/+21
The function __git_complete_config_variable_name in the Bash completion script hardcodes several config variable names. These variables are those in config sections where user-defined names can appear, such as "branch.<name>". These sections are treated first by the case statement, and the two last "catch all" cases are used for other sections, making use of the __git_compute_config_vars and __git_compute_config_sections function, which omit listing any variables containing wildcards or placeholders. Having hardcoded config variables introduces the risk of the completion code becoming out of sync with the actual config variables accepted by Git. To avoid these hardcoded config variables, introduce a new function, __git_compute_first_level_config_vars_for_section, making use of the existing __git_config_vars variable. This function takes as argument a config section name and computes the matching "first level" config variables for that section, i.e. those _not_ containing any placeholder, like 'branch.autoSetupMerge, 'remote.pushDefault', etc. Use this function and the variables it defines in the 'branch.*', 'remote.*' and 'submodule.*' switches of the case statement instead of hardcoding the corresponding config variables. Note that we use indirect expansion to create a variable for each section, instead of using a single associative array indexed by section names, because associative arrays are not supported in Bash 3, on which macOS is stuck for licensing reasons. Use the existing pattern in the completion script of using global variables to cache the list of config variables for each section. The rationale for such caching is explained in eaa4e6ee2a (Speed up bash completion loading, 2009-11-17), and the current approach to using and defining them via 'test -n' is explained in cf0ff02a38 (completion: work around zsh option propagation bug, 2012-02-02). Adjust the name of one of the tests added in the previous commit, reflecting that it now also tests the new function. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-12completion: complete 'submodule.*' config variablesPhilippe Blain1-0/+13
In the Bash completion script, function __git_complete_config_variable_name completes config variables and has special logic to deal with config variables involving user-defined names, like branch.<name>.* and remote.<name>.*. This special logic is missing for submodule-related config variables. Add the appropriate branches to the case statement, making use of the in-tree '.gitmodules' to list relevant submodules. Add corresponding tests in t9902-completion.sh, making sure we complete both first level submodule config variables as well as second level variables involving submodule names. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-12completion: add space after config variable names also in Bash 3Philippe Blain1-3/+3
In be6444d1ca (completion: bash: add correct suffix in variables, 2021-08-16), __git_complete_config_variable_name was changed to use "${sfx- }" instead of "$sfx" as the fourth argument of _gitcomp_nl and _gitcomp_nl_append, such that this argument evaluates to a space if sfx is unset. This was to ensure that e.g. git config branch.autoSetupMe[TAB] correctly completes to 'branch.autoSetupMerge ' with the trailing space. This commits notes that the fix only works in Bash 4 because in Bash 3 the 'local sfx' construct at the beginning of __git_complete_config_variable_name creates an empty string. Make the fix also work for Bash 3 by using the "unset or null' parameter expansion syntax ("${sfx:- }"), such that the parameter is also expanded to a space if it is set but null, as is the behaviour of 'local sfx' in Bash 3. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-12use xstrncmpz()René Scharfe1-0/+28
Add and apply a semantic patch for calling xstrncmpz() to compare a NUL-terminated string with a buffer of a known length instead of using strncmp() and checking the terminating NUL explicitly. This simplifies callers by reducing code duplication. I had to adjust remote.c manually because Coccinelle inexplicably changed the indent of the else branches. Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-08Merge branch 'sh/completion-with-reftable' into maint-2.43Junio C Hamano1-5/+38
Command line completion script (in contrib/) learned to work better with the reftable backend. * sh/completion-with-reftable: completion: support pseudoref existence checks for reftables completion: refactor existence checks for pseudorefs
2024-02-08Merge branch 'tz/send-email-negatable-options' into maint-2.43Junio C Hamano2-2/+2
Newer versions of Getopt::Long started giving warnings against our (ab)use of it in "git send-email". Bump the minimum version requirement for Perl to 5.8.1 (from September 2002) to allow simplifying our implementation. * tz/send-email-negatable-options: send-email: avoid duplicate specification warnings perl: bump the required Perl version to 5.8.1 from 5.8.0
2024-02-08Merge branch 'mh/credential-oauth-refresh-token-with-wincred'Junio C Hamano1-6/+40
The wincred credential backend has been taught to support oauth refresh token the same way as credential-cache and credential-libsecret backends. * mh/credential-oauth-refresh-token-with-wincred: credential/wincred: store oauth_refresh_token
2024-02-07refs: introduce reftable backendPatrick Steinhardt1-1/+1
Due to scalability issues, Shawn Pearce has originally proposed a new "reftable" format more than six years ago [1]. Initially, this new format was implemented in JGit with promising results. Around two years ago, we have then added the "reftable" library to the Git codebase via a4bbd13be3 (Merge branch 'hn/reftable', 2021-12-15). With this we have landed all the low-level code to read and write reftables. Notably missing though was the integration of this low-level code into the Git code base in the form of a new ref backend that ties all of this together. This gap is now finally closed by introducing a new "reftable" backend into the Git codebase. This new backend promises to bring some notable improvements to Git repositories: - It becomes possible to do truly atomic writes where either all refs are committed to disk or none are. This was not possible with the "files" backend because ref updates were split across multiple loose files. - The disk space required to store many refs is reduced, both compared to loose refs and packed-refs. This is enabled both by the reftable format being a binary format, which is more compact, and by prefix compression. - We can ignore filesystem-specific behaviour as ref names are not encoded via paths anymore. This means there is no need to handle case sensitivity on Windows systems or Unicode precomposition on macOS. - There is no need to rewrite the complete refdb anymore every time a ref is being deleted like it was the case for packed-refs. This means that ref deletions are now constant time instead of scaling linearly with the number of refs. - We can ignore file/directory conflicts so that it becomes possible to store both "refs/heads/foo" and "refs/heads/foo/bar". - Due to this property we can retain reflogs for deleted refs. We have previously been deleting reflogs together with their refs to avoid file/directory conflicts, which is not necessary anymore. - We can properly enumerate all refs. With the "files" backend it is not easily possible to distinguish between refs and non-refs because they may live side by side in the gitdir. Not all of these improvements are realized with the current "reftable" backend implementation. At this point, the new backend is supposed to be a drop-in replacement for the "files" backend that is used by basically all Git repositories nowadays. It strives for 1:1 compatibility, which means that a user can expect the same behaviour regardless of whether they use the "reftable" backend or the "files" backend for most of the part. Most notably, this means we artificially limit the capabilities of the "reftable" backend to match the limits of the "files" backend. It is not possible to create refs that would end up with file/directory conflicts, we do not retain reflogs, we perform stricter-than-necessary checks. This is done intentionally due to two main reasons: - It makes it significantly easier to land the "reftable" backend as tests behave the same. It would be tough to argue for each and every single test that doesn't pass with the "reftable" backend. - It ensures compatibility between repositories that use the "files" backend and repositories that use the "reftable" backend. Like this, hosters can migrate their repositories to use the "reftable" backend without causing issues for clients that use the "files" backend in their clones. It is expected that these artificial limitations may eventually go away in the long term. Performance-wise things very much depend on the actual workload. The following benchmarks compare the "files" and "reftable" backends in the current version: - Creating N refs in separate transactions shows that the "files" backend is ~50% faster. This is not surprising given that creating a ref only requires us to create a single loose ref. The "reftable" backend will also perform auto compaction on updates. In real-world workloads we would likely also want to perform pack loose refs, which would likely change the picture. Benchmark 1: update-ref: create refs sequentially (refformat = files, refcount = 1) Time (mean ± σ): 2.1 ms ± 0.3 ms [User: 0.6 ms, System: 1.7 ms] Range (min … max): 1.8 ms … 4.3 ms 133 runs Benchmark 2: update-ref: create refs sequentially (refformat = reftable, refcount = 1) Time (mean ± σ): 2.7 ms ± 0.1 ms [User: 0.6 ms, System: 2.2 ms] Range (min … max): 2.4 ms … 2.9 ms 132 runs Benchmark 3: update-ref: create refs sequentially (refformat = files, refcount = 1000) Time (mean ± σ): 1.975 s ± 0.006 s [User: 0.437 s, System: 1.535 s] Range (min … max): 1.969 s … 1.980 s 3 runs Benchmark 4: update-ref: create refs sequentially (refformat = reftable, refcount = 1000) Time (mean ± σ): 2.611 s ± 0.013 s [User: 0.782 s, System: 1.825 s] Range (min … max): 2.597 s … 2.622 s 3 runs Benchmark 5: update-ref: create refs sequentially (refformat = files, refcount = 100000) Time (mean ± σ): 198.442 s ± 0.241 s [User: 43.051 s, System: 155.250 s] Range (min … max): 198.189 s … 198.670 s 3 runs Benchmark 6: update-ref: create refs sequentially (refformat = reftable, refcount = 100000) Time (mean ± σ): 294.509 s ± 4.269 s [User: 104.046 s, System: 190.326 s] Range (min … max): 290.223 s … 298.761 s 3 runs - Creating N refs in a single transaction shows that the "files" backend is significantly slower once we start to write many refs. The "reftable" backend only needs to update two files, whereas the "files" backend needs to write one file per ref. Benchmark 1: update-ref: create many refs (refformat = files, refcount = 1) Time (mean ± σ): 1.9 ms ± 0.1 ms [User: 0.4 ms, System: 1.4 ms] Range (min … max): 1.8 ms … 2.6 ms 151 runs Benchmark 2: update-ref: create many refs (refformat = reftable, refcount = 1) Time (mean ± σ): 2.5 ms ± 0.1 ms [User: 0.7 ms, System: 1.7 ms] Range (min … max): 2.4 ms … 3.4 ms 148 runs Benchmark 3: update-ref: create many refs (refformat = files, refcount = 1000) Time (mean ± σ): 152.5 ms ± 5.2 ms [User: 19.1 ms, System: 133.1 ms] Range (min … max): 148.5 ms … 167.8 ms 15 runs Benchmark 4: update-ref: create many refs (refformat = reftable, refcount = 1000) Time (mean ± σ): 58.0 ms ± 2.5 ms [User: 28.4 ms, System: 29.4 ms] Range (min … max): 56.3 ms … 72.9 ms 40 runs Benchmark 5: update-ref: create many refs (refformat = files, refcount = 1000000) Time (mean ± σ): 152.752 s ± 0.710 s [User: 20.315 s, System: 131.310 s] Range (min … max): 152.165 s … 153.542 s 3 runs Benchmark 6: update-ref: create many refs (refformat = reftable, refcount = 1000000) Time (mean ± σ): 51.912 s ± 0.127 s [User: 26.483 s, System: 25.424 s] Range (min … max): 51.769 s … 52.012 s 3 runs - Deleting a ref in a fully-packed repository shows that the "files" backend scales with the number of refs. The "reftable" backend has constant-time deletions. Benchmark 1: update-ref: delete ref (refformat = files, refcount = 1) Time (mean ± σ): 1.7 ms ± 0.1 ms [User: 0.4 ms, System: 1.2 ms] Range (min … max): 1.6 ms … 2.1 ms 316 runs Benchmark 2: update-ref: delete ref (refformat = reftable, refcount = 1) Time (mean ± σ): 1.8 ms ± 0.1 ms [User: 0.4 ms, System: 1.3 ms] Range (min … max): 1.7 ms … 2.1 ms 294 runs Benchmark 3: update-ref: delete ref (refformat = files, refcount = 1000) Time (mean ± σ): 2.0 ms ± 0.1 ms [User: 0.5 ms, System: 1.4 ms] Range (min … max): 1.9 ms … 2.5 ms 287 runs Benchmark 4: update-ref: delete ref (refformat = reftable, refcount = 1000) Time (mean ± σ): 1.9 ms ± 0.1 ms [User: 0.5 ms, System: 1.3 ms] Range (min … max): 1.8 ms … 2.1 ms 217 runs Benchmark 5: update-ref: delete ref (refformat = files, refcount = 1000000) Time (mean ± σ): 229.8 ms ± 7.9 ms [User: 182.6 ms, System: 46.8 ms] Range (min … max): 224.6 ms … 245.2 ms 6 runs Benchmark 6: update-ref: delete ref (refformat = reftable, refcount = 1000000) Time (mean ± σ): 2.0 ms ± 0.0 ms [User: 0.6 ms, System: 1.3 ms] Range (min … max): 2.0 ms … 2.1 ms 3 runs - Listing all refs shows no significant advantage for either of the backends. The "files" backend is a bit faster, but not by a significant margin. When repositories are not packed the "reftable" backend outperforms the "files" backend because the "reftable" backend performs auto-compaction. Benchmark 1: show-ref: print all refs (refformat = files, refcount = 1, packed = true) Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.4 ms, System: 1.1 ms] Range (min … max): 1.5 ms … 2.0 ms 1729 runs Benchmark 2: show-ref: print all refs (refformat = reftable, refcount = 1, packed = true) Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.4 ms, System: 1.1 ms] Range (min … max): 1.5 ms … 1.8 ms 1816 runs Benchmark 3: show-ref: print all refs (refformat = files, refcount = 1000, packed = true) Time (mean ± σ): 4.3 ms ± 0.1 ms [User: 0.9 ms, System: 3.3 ms] Range (min … max): 4.1 ms … 4.6 ms 645 runs Benchmark 4: show-ref: print all refs (refformat = reftable, refcount = 1000, packed = true) Time (mean ± σ): 4.5 ms ± 0.2 ms [User: 1.0 ms, System: 3.3 ms] Range (min … max): 4.2 ms … 5.9 ms 643 runs Benchmark 5: show-ref: print all refs (refformat = files, refcount = 1000000, packed = true) Time (mean ± σ): 2.537 s ± 0.034 s [User: 0.488 s, System: 2.048 s] Range (min … max): 2.511 s … 2.627 s 10 runs Benchmark 6: show-ref: print all refs (refformat = reftable, refcount = 1000000, packed = true) Time (mean ± σ): 2.712 s ± 0.017 s [User: 0.653 s, System: 2.059 s] Range (min … max): 2.692 s … 2.752 s 10 runs Benchmark 7: show-ref: print all refs (refformat = files, refcount = 1, packed = false) Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.4 ms, System: 1.1 ms] Range (min … max): 1.5 ms … 1.9 ms 1834 runs Benchmark 8: show-ref: print all refs (refformat = reftable, refcount = 1, packed = false) Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.4 ms, System: 1.1 ms] Range (min … max): 1.4 ms … 2.0 ms 1840 runs Benchmark 9: show-ref: print all refs (refformat = files, refcount = 1000, packed = false) Time (mean ± σ): 13.8 ms ± 0.2 ms [User: 2.8 ms, System: 10.8 ms] Range (min … max): 13.3 ms … 14.5 ms 208 runs Benchmark 10: show-ref: print all refs (refformat = reftable, refcount = 1000, packed = false) Time (mean ± σ): 4.5 ms ± 0.2 ms [User: 1.2 ms, System: 3.3 ms] Range (min … max): 4.3 ms … 6.2 ms 624 runs Benchmark 11: show-ref: print all refs (refformat = files, refcount = 1000000, packed = false) Time (mean ± σ): 12.127 s ± 0.129 s [User: 2.675 s, System: 9.451 s] Range (min … max): 11.965 s … 12.370 s 10 runs Benchmark 12: show-ref: print all refs (refformat = reftable, refcount = 1000000, packed = false) Time (mean ± σ): 2.799 s ± 0.022 s [User: 0.735 s, System: 2.063 s] Range (min … max): 2.769 s … 2.836 s 10 runs - Printing a single ref shows no real difference between the "files" and "reftable" backends. Benchmark 1: show-ref: print single ref (refformat = files, refcount = 1) Time (mean ± σ): 1.5 ms ± 0.1 ms [User: 0.4 ms, System: 1.0 ms] Range (min … max): 1.4 ms … 1.8 ms 1779 runs Benchmark 2: show-ref: print single ref (refformat = reftable, refcount = 1) Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.4 ms, System: 1.1 ms] Range (min … max): 1.4 ms … 2.5 ms 1753 runs Benchmark 3: show-ref: print single ref (refformat = files, refcount = 1000) Time (mean ± σ): 1.5 ms ± 0.1 ms [User: 0.3 ms, System: 1.1 ms] Range (min … max): 1.4 ms … 1.9 ms 1840 runs Benchmark 4: show-ref: print single ref (refformat = reftable, refcount = 1000) Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.4 ms, System: 1.1 ms] Range (min … max): 1.5 ms … 2.0 ms 1831 runs Benchmark 5: show-ref: print single ref (refformat = files, refcount = 1000000) Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.4 ms, System: 1.1 ms] Range (min … max): 1.5 ms … 2.1 ms 1848 runs Benchmark 6: show-ref: print single ref (refformat = reftable, refcount = 1000000) Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.4 ms, System: 1.1 ms] Range (min … max): 1.5 ms … 2.1 ms 1762 runs So overall, performance depends on the usecases. Except for many sequential writes the "reftable" backend is roughly on par or significantly faster than the "files" backend though. Given that the "files" backend has received 18 years of optimizations by now this can be seen as a win. Furthermore, we can expect that the "reftable" backend will grow faster over time when attention turns more towards optimizations. The complete test suite passes, except for those tests explicitly marked to require the REFFILES prerequisite. Some tests in t0610 are marked as failing because they depend on still-in-flight bug fixes. Tests can be run with the new backend by setting the GIT_TEST_DEFAULT_REF_FORMAT environment variable to "reftable". There is a single known conceptual incompatibility with the dumb HTTP transport. As "info/refs" SHOULD NOT contain the HEAD reference, and because the "HEAD" file is not valid anymore, it is impossible for the remote client to figure out the default branch without changing the protocol. This shortcoming needs to be handled in a subsequent patch series. As the reftable library has already been introduced a while ago, this commit message will not go into the details of how exactly the on-disk format works. Please refer to our preexisting technical documentation at Documentation/technical/reftable for this. [1]: https://public-inbox.org/git/CAJo=hJtyof=HRy=2sLP0ng0uZ4=S-DpZ5dR1aF+VHVETKG20OQ@mail.gmail.com/ Original-idea-by: Shawn Pearce <spearce@spearce.org> Based-on-patch-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-06completion: bisect: recognize but do not complete view subcommandBritton Leo Kerin1-4/+11
The "view" alias for the visualize subcommand is neither completed nor recognized. It's undesirable to complete it because it's first letters are the same as for visualize, making completion less rather than more efficient without adding much in the way of interface discovery. However, it needs to be recognized in order to enable log option completion for it. Recognize but do not complete the view command by creating and using separate lists of completable_subcommands and all_subcommands. Add tests. Signed-off-by: Britton Leo Kerin <britton.kerin@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-06completion: bisect: complete log opts for visualize subcommandBritton Leo Kerin1-0/+4
Arguments passed to the "visualize" subcommand of git-bisect(1) get forwarded to git-log(1). It thus supports the same options as git-log(1) would, but our Bash completion script does not know to handle this. Make completion of porcelain git-log options and option arguments to the visualize subcommand work by calling __git_complete_log_opts when the start of an option to the subcommand is seen (visualize doesn't support any options besides the git-log options). Add test. Signed-off-by: Britton Leo Kerin <britton.kerin@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-06completion: new function __git_complete_log_optsBritton Leo Kerin1-3/+15
The options accepted by git-log are also accepted by at least one other command (git-bisect). Factor the common option completion code into a new function and use it from _git_log. The new function leaves COMPREPLY empty if no option candidates are found, so that callers can safely check it to determine if completion for other arguments should be attempted. Signed-off-by: Britton Leo Kerin <britton.kerin@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-06completion: bisect: complete missing --first-parent and - -no-checkout optionsBritton Leo Kerin1-1/+1
The --first-parent and --no-checkout options to the start subcommand of git-bisect(1) are not completed. Enable completion of the --first-parent and --no-checkout options to the start subcommand. Add test. Signed-off-by: Britton Leo Kerin <britton.kerin@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-06completion: bisect: complete custom terms and related optionsBritton Leo Kerin1-2/+30
git bisect supports the use of custom terms via the --term-(new|bad) and --term-(old|good) options, but the completion code doesn't know about these options or the new subcommands they define. Add support for these options and the custom subcommands by checking for BISECT_TERMS and adding them to the list of subcommands. Add tests. Signed-off-by: Britton Leo Kerin <britton.kerin@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-06completion: bisect: complete bad, new, old, and help subcommandsBritton Leo Kerin1-2/+2
The bad, new, old and help subcommands to git-bisect(1) are not completed. Add the bad, new, old, and help subcommands to the appropriate lists such that the commands and their possible ref arguments are completed. Add tests. Signed-off-by: Britton Leo Kerin <britton.kerin@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-02Merge branch 'zf/subtree-split-fix'Junio C Hamano2-1/+69
"git subtree" (in contrib/) update. * zf/subtree-split-fix: subtree: fix split processing with multiple subtrees present
2024-02-02Merge branch 'pb/complete-log-more'Junio C Hamano1-1/+14
The completion script (in contrib/) learned more options that can be used with "git log". * pb/complete-log-more: completion: complete missing 'git log' options completion: complete --encoding completion: complete --patch-with-raw completion: complete missing rev-list options
2024-01-29credential/wincred: store oauth_refresh_tokenM Hickford1-6/+40
a5c7656 (credential: new attribute oauth_refresh_token) introduced a new confidential credential attribute and added support to credential-cache. Later 0ce02e2f (credential/libsecret: store new attributes, 2023-06-16) added support in credential-libsecret. To add support in credential-wincred, we encode the new attribute in the CredentialBlob, separated by newline: hunter2 oauth_refresh_token=xyzzy This is extensible and backwards compatible. The credential protocol already assumes that attribute values do not contain newlines. This fixes test "helper (wincred) gets oauth_refresh_token" when t0303-credential-external.sh is run with GIT_TEST_CREDENTIAL_HELPER=wincred. This test was added in a5c76569e7 (credential: new attribute oauth_refresh_token, 2023-04-21). Alternatives considered: store oauth_refresh_token in a wincred attribute. This would be insecure because wincred assumes attribute values to be non-confidential. Signed-off-by: M Hickford <mirth.hickford@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-26Merge branch 'ps/completion-with-reftable-fix'Junio C Hamano1-6/+5
Completion update to prepare for reftable * ps/completion-with-reftable-fix: completion: treat dangling symrefs as existing pseudorefs completion: silence pseudoref existence check completion: improve existence check for pseudo-refs t9902: verify that completion does not print anything completion: discover repo path in `__git_pseudoref_exists ()`
2024-01-25subtree: fix split processing with multiple subtrees presentZach FettersMoore2-1/+69
When there are multiple subtrees present in a repository and they are all using 'git subtree split', the 'split' command can take a significant (and constantly growing) amount of time to run even when using the '--rejoin' flag. This is due to the fact that when processing commits to determine the last known split to start from when looking for changes, if there has been a split/merge done from another subtree there will be 2 split commits, one mainline and one subtree, for the second subtree that are part of the processing. The non-mainline subtree split commit will cause the processing to always need to search the entire history of the given subtree as part of its processing even though those commits are totally irrelevant to the current subtree split being run. To see this in practice you can use the open source GitHub repo 'apollo-ios-dev' and do the following in order: -Make a changes to a file in 'apollo-ios' and 'apollo-ios-codegen' directories -Create a commit containing these changes -Do a split on apollo-ios-codegen - Do a fetch on the subtree repo - git fetch git@github.com:apollographql/apollo-ios-codegen.git - git subtree split --prefix=apollo-ios-codegen --squash --rejoin - Depending on the current state of the 'apollo-ios-dev' repo you may see the issue at this point if the last split was on apollo-ios -Do a split on apollo-ios - Do a fetch on the subtree repo - git fetch git@github.com:apollographql/apollo-ios.git - git subtree split --prefix=apollo-ios --squash --rejoin -Make changes to a file in apollo-ios-codegen -Create a commit containing the change(s) -Do a split on apollo-ios-codegen - git subtree split --prefix=apollo-ios-codegen --squash --rejoin -To see that the patch fixes the issue you can use the custom subtree script in the repo so following the same steps as above, except instead of using 'git subtree ...' for the commands use 'git-subtree.sh ...' for the commands You will see that the final split is looking for the last split on apollo-ios-codegen to use as it's starting point to process commits. Since there is a split commit from apollo-ios in between the 2 splits run on apollo-ios-codegen, the processing ends up traversing the entire history of apollo-ios which increases the time it takes to do a split based on how long of a history apollo-ios has, while none of these commits are relevant to the split being done on apollo-ios-codegen. So this commit makes a change to the processing of commits for the split command in order to ignore non-mainline commits from other subtrees such as apollo-ios in the above breakdown by adding a new function 'should_ignore_subtree_commit' which is called during 'process_split_commit'. This allows the split/rejoin processing to still function as expected but removes all of the unnecessary processing that takes place currently which greatly inflates the processing time. In the above example, previously the final split would take ~10-12 minutes, while after this fix it takes seconds. Added a test to validate that the proposed fix solves the issue. The test accomplishes this by checking the output of the split command to ensure the output from the progress of 'process_split_commit' function that represents the 'extracount' of commits processed remains at 0, meaning none of the commits from the second subtree were processed. This was tested against the original functionality to show the test failed, and then with this fix to show the test passes. This illustrated that when using multiple subtrees, A and B, when doing a split on subtree B, the processing does not traverse the entire history of subtree A which is unnecessary and would cause the 'extracount' of processed commits to climb based on the number of commits in the history of subtree A. Signed-off-by: Zach FettersMoore <zach.fetters@apollographql.com> Reviewed-by: Christian Couder <christian.couder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-22completion: complete missing 'git log' optionsPhilippe Blain1-0/+2
Some options specific to 'git log' are missing from the Bash completion script. Add them to _git_log. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-22completion: complete --encodingPhilippe Blain1-0/+1
The option --encoding is supported by 'git log' and 'git show', so add it to __git_log_show_options. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-22completion: complete --patch-with-rawPhilippe Blain1-1/+1
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-22completion: complete missing rev-list optionsPhilippe Blain1-0/+10
Some options listed in rev-list-options.txt, and thus accepted by 'git log' and friends, are missing from the Bash completion script. Add them to __git_log_common_options. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-16completion: treat dangling symrefs as existing pseudorefsPatrick Steinhardt1-1/+1
The `__git_pseudoref_exists ()` helper function back to git-rev-parse(1) in case the reftable backend is in use. This is not in the same spirit as the simple existence check that the "files" backend does though, because there we only check for the pseudo-ref to exist with `test -f`. With git-rev-parse(1) we not only check for existence, but also verify that the pseudo-ref resolves to an object, which may not be the case when the pseudo-ref points to an unborn branch. Fix this issue by using `git show-ref --exists` instead. Note that we do not have to silence stdout anymore as git-show-ref(1) will not print anything. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-16completion: silence pseudoref existence checkPatrick Steinhardt1-1/+1
In 44dbb3bf29 (completion: support pseudoref existence checks for reftables, 2023-12-19), we have extended the Bash completion script to support future ref backends better by using git-rev-parse(1) to check for pseudo-ref existence. This conversion has introduced a bug, because even though we pass `--quiet` to git-rev-parse(1) it would still output the resolved object ID of the ref in question if it exists. Fix this by redirecting its stdout to `/dev/null` and add a test that catches this behaviour. Note that the test passes even without the fix for the "files" backend because we parse pseudo refs via the filesystem directly in that case. But the test will fail with the "reftable" backend. Helped-by: Jeff King <peff@peff.net> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-16completion: improve existence check for pseudo-refsPatrick Steinhardt1-3/+3
Improve the existence check along the following lines: - Stop stripping the "ref :" prefix and compare to the expected value directly. This allows us to drop a now-unused variable that was previously leaking into the user's shell. - Mark the "head" variable as local so that we don't leak its value into the user's shell. - Stop manually handling the `-C $__git_repo_path` option, which the `__git ()` wrapper aleady does for us. - In simlar spirit, stop redirecting stderr, which is also handled by the wrapper already. Suggested-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-16completion: discover repo path in `__git_pseudoref_exists ()`Patrick Steinhardt1-3/+2
The helper function `__git_pseudoref_exists ()` expects that the repo path has already been discovered by its callers, which makes for a rather fragile calling convention. Refactor the function to discover the repo path itself to make it more self-contained, which also removes the need to discover the path in some of its callers. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-08git-prompt: stop manually parsing HEAD with unknown ref formatsPatrick Steinhardt1-7/+24
We're manually parsing the HEAD reference in git-prompt to figure out whether it is a symbolic or direct reference. This makes it intimately tied to the on-disk format we use to store references and will stop working once we gain additional reference backends in the Git project. Ideally, we would refactor the code to exclusively use plumbing tools to read refs such that we do not have to care about the on-disk format at all. Unfortunately though, spawning processes can be quite expensive on some systems like Windows. As the Git prompt logic may be executed quite frequently we try very hard to spawn as few processes as possible. This refactoring is thus out of question for now. Instead, condition the logic on the repository's ref format: if the repo uses the the "files" backend we can continue to use the old logic and read the respective files from disk directly. If it's anything else, then we use git-symbolic-ref(1) to read the value of HEAD. This change makes the Git prompt compatible with the upcoming "reftable" format. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-02Merge branch 'sh/completion-with-reftable'Junio C Hamano1-5/+38
Command line completion script (in contrib/) learned to work better with the reftable backend. * sh/completion-with-reftable: completion: support pseudoref existence checks for reftables completion: refactor existence checks for pseudorefs
2023-12-20Merge branch 'en/complete-sparse-checkout'Junio C Hamano1-3/+120
Command line completion (in contrib/) learned to complete path arguments to the "add/set" subcommands of "git sparse-checkout" better. * en/complete-sparse-checkout: completion: avoid user confusion in non-cone mode completion: avoid misleading completions in cone mode completion: fix logic for determining whether cone mode is active completion: squelch stray errors in sparse-checkout completion
2023-12-19completion: support pseudoref existence checks for reftablesStan Hu1-0/+23
In contrib/completion/git-completion.bash, there are a bunch of instances where we read pseudorefs, such as HEAD, MERGE_HEAD, REVERT_HEAD, and others via the filesystem. However, the upcoming reftable refs backend won't use '.git/HEAD' at all but instead will write an invalid refname as placeholder for backwards compatibility, which will break the git-completion script. Update the '__git_pseudoref_exists' function to: 1. Recognize the placeholder '.git/HEAD' written by the reftable backend (its content is specified in the reftable specs). 2. If reftable is in use, use 'git rev-parse' to determine whether the given ref exists. 3. Otherwise, continue to use 'test -f' to check for the ref's filename. Signed-off-by: Stan Hu <stanhu@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-12-19completion: refactor existence checks for pseudorefsStan Hu1-5/+15
In preparation for the reftable backend, this commit introduces a '__git_pseudoref_exists' function that continues to use 'test -f' to determine whether a given pseudoref exists in the local filesystem. Signed-off-by: Stan Hu <stanhu@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-12-09Merge branch 'tz/send-email-negatable-options'Junio C Hamano2-2/+2
Newer versions of Getopt::Long started giving warnings against our (ab)use of it in "git send-email". Bump the minimum version requirement for Perl to 5.8.1 (from September 2002) to allow simplifying our implementation. * tz/send-email-negatable-options: send-email: avoid duplicate specification warnings perl: bump the required Perl version to 5.8.1 from 5.8.0
2023-12-09Merge branch 'ps/ban-a-or-o-operator-with-test'Junio C Hamano1-11/+23
Test and shell scripts clean-up. * ps/ban-a-or-o-operator-with-test: Makefile: stop using `test -o` when unlinking duplicate executables contrib/subtree: convert subtree type check to use case statement contrib/subtree: stop using `-o` to test for number of args global: convert trivial usages of `test <expr> -a/-o <expr>`
2023-12-09Merge branch 'js/doc-unit-tests-with-cmake'Junio C Hamano1-4/+34
Update the base topic to work with CMake builds. * js/doc-unit-tests-with-cmake: cmake: handle also unit tests cmake: use test names instead of full paths cmake: fix typo in variable name artifacts-tar: when including `.dll` files, don't forget the unit-tests unit-tests: do show relative file paths unit-tests: do not mistake `.pdb` files for being executable cmake: also build unit tests
2023-12-03completion: avoid user confusion in non-cone modeElijah Newren1-0/+89
It is tempting to think of "files and directories" of the current directory as valid inputs to the add and set subcommands of git sparse-checkout. However, in non-cone mode, they often aren't and using them as potential completions leads to *many* forms of confusion: Issue #1. It provides the *wrong* files and directories. For git sparse-checkout add we always want to add files and directories not currently in our sparse checkout, which means we want file and directories not currently present in the current working tree. Providing the files and directories currently present is thus always wrong. For git sparse-checkout set we have a similar problem except in the subset of cases where we are trying to narrow our checkout to a strict subset of what we already have. That is not a very common scenario, especially since it often does not even happen to be true for the first use of the command; for years we required users to create a sparse-checkout via git sparse-checkout init git sparse-checkout set <args...> (or use a clone option that did the init step for you at clone time). The init command creates a minimal sparse-checkout with just the top-level directory present, meaning the set command has to be used to expand the checkout. Thus, only in a special and perhaps unusual cases would any of the suggestions from normal file and directory completion be appropriate. Issue #2: Suggesting patterns that lead to warnings is unfriendly. If the user specifies any regular file and omits the leading '/', then the sparse-checkout command will warn the user that their command is problematic and suggest they use a leading slash instead. Issue #3: Completion gets confused by leading '/', and provides wrong paths. Users often want to anchor their patterns to the toplevel of the repository, especially when listing individual files. There are a number of reasons for this, but notably even sparse-checkout encourages them to do so (as noted above). However, if users do so (via adding a leading '/' to their pattern), then bash completion will interpret the leading slash not as a request for a path at the toplevel of the repository, but as a request for a path at the root of the filesytem. That means at best that completion cannot help with such paths, and if it does find any completions, they are almost guaranteed to be wrong. Issue #4: Suggesting invalid patterns from subdirectories is unfriendly. There is no per-directory equivalent to .gitignore with sparse-checkouts. There is only a single worktree-global $GIT_DIR/info/sparse-checkout file. As such, paths to files must be specified relative to the toplevel of a repository. Providing suggestions of paths that are relative to the current working directory, as bash completion defaults to, is wrong when the current working directory is not the worktree toplevel directory. Issue #5: Paths with special characters will be interpreted incorrectly The entries in the sparse-checkout file are patterns, not paths. While most paths also qualify as patterns (though even in such cases it would be better for users to not use them directly but prefix them with a leading '/'), there are a variety of special characters that would need special escaping beyond the normal shell escaping: '*', '?', '\', '[', ']', and any leading '#' or '!'. If completion suggests any such paths, users will likely expect them to be treated as an exact path rather than as a pattern that might match some number of files other than 1. However, despite the first four issues, we can note that _if_ users are using tab completion, then they are probably trying to specify a path in the index. As such, we transform their argument into a top-level-rooted pattern that matches such a file. For example, if they type: git sparse-checkout add Make<TAB> we could "complete" to git sparse-checkout add /Makefile or, if they ran from the Documentation/technical/ subdirectory: git sparse-checkout add m<TAB> we could "complete" it to: git sparse-checkout add /Documentation/technical/multi-pack-index.txt Note in both cases I use "complete" in quotes, because we actually add characters both before and after the argument in question, so we are kind of abusing "bash completions" to be "bash completions AND beginnings". The fifth issue is a bit stickier, especially when you consider that we not only need to deal with escaping issues because of special meanings of patterns in sparse-checkout & gitignore files, but also that we need to consider escaping issues due to ls-files needing to sometimes quote or escape characters, and because the shell needs to escape some characters. The multiple interacting forms of escaping could get ugly; this patch makes no attempt to do so and simply documents that we decided to not deal with those corner cases for now but at least get the common cases right. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-12-03completion: avoid misleading completions in cone modeElijah Newren1-0/+20
The "set" and "add" subcommands of "sparse-checkout", when in cone mode, should only complete on directories. For bash_completion in general, when no completions are returned for any subcommands, it will often fall back to standard completion of files and directories as a substitute. That is not helpful here. Since we have already looked for all valid completions, if none are found then falling back to standard bash file and directory completion is at best actively misleading. In fact, there are three different ways it can be actively misleading. Add a long comment in the code about how that fallback behavior can deceive, and disable the fallback by returning a fake result as the sole completion. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-12-03completion: fix logic for determining whether cone mode is activeElijah Newren1-2/+10
_git_sparse_checkout() was checking whether we were in cone mode by checking whether either: A) core.sparseCheckoutCone was "true" B) "--cone" was specified on the command line This code has 2 bugs I didn't catch in my review at the time 1) core.sparseCheckout must be "true" for core.sparseCheckoutCone to be relevant (which matters since "git sparse-checkout disable" only unsets core.sparseCheckout, not core.sparseCheckoutCone) 2) The presence of "--no-cone" should override any config setting Further, I forgot to update this logic as part of 2d95707a02 ("sparse-checkout: make --cone the default", 2022-04-22) for the new default. Update the code for the new default and make it be more careful in determining whether to complete based on cone mode or non-cone mode. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-12-03completion: squelch stray errors in sparse-checkout completionElijah Newren1-1/+1
If, in the root of a project, one types git sparse-checkout set --cone ../<TAB> then an error message of the form fatal: ../: '../' is outside repository at '/home/newren/floss/git' is written to stderr, which munges the users view of their own command. Squelch such messages by using the __git() wrapper, designed for this purpose; see commit e15098a314 (completion: consolidate silencing errors from git commands, 2017-02-03) for more on the wrapper. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-17perl: bump the required Perl version to 5.8.1 from 5.8.0Todd Zullinger2-2/+2
The following commit will make use of a Getopt::Long feature which is only present in Perl >= 5.8.1. Document that as the minimum version we support. Many of our Perl scripts will continue to run with 5.8.0 but this change allows us to adjust them as needed without breaking any promises to our users. The Perl requirement was last changed in d48b284183 (perl: bump the required Perl version to 5.8 from 5.6.[21], 2010-09-24). At that time, 5.8.0 was 8 years old. It is now over 21 years old. Signed-off-by: Todd Zullinger <tmz@pobox.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-11contrib/subtree: convert subtree type check to use case statementPatrick Steinhardt1-4/+10
The `subtree_for_commit ()` helper function asserts that the subtree identified by its parameters are either a commit or tree. This is done via the `-o` parameter of test, which is discouraged. Refactor the code to instead use a switch statement over the type. Despite being aligned with our coding guidelines, the resulting code is arguably also easier to read. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-11contrib/subtree: stop using `-o` to test for number of argsPatrick Steinhardt1-5/+11
Functions in git-subtree.sh all assert that they are being passed the correct number of arguments. In cases where we accept a variable number of arguments we assert this via a single call to `test` with `-o`, which is discouraged by our coding guidelines. Convert these cases to stop doing so. This requires us to decompose assertions of the style `assert test $# = 2 -o $# = 3` into two calls because we have no easy way to logically chain statements passed to the assert function. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-11global: convert trivial usages of `test <expr> -a/-o <expr>`Patrick Steinhardt1-2/+2
Our coding guidelines say to not use `test` with `-a` and `-o` because it can easily lead to bugs. Convert trivial cases where we still use these to instead instead concatenate multiple invocations of `test` via `&&` and `||`, respectively. While not all of the converted instances can cause ambiguity, it is worth getting rid of all of them regardless: - It becomes easier to reason about the code as we do not have to argue why one use of `-a`/`-o` is okay while another one isn't. - We don't encourage people to use these expressions. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-10cmake: handle also unit testsJohannes Schindelin1-0/+11
The unit tests should also be available e.g. in Visual Studio's Test Explorer when configuring Git's source code via CMake. Suggested-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-10cmake: use test names instead of full pathsJohannes Schindelin1-2/+3
The primary purpose of Git's CMake definition is to allow developing Git in Visual Studio. As part of that, the CTest feature allows running individual test scripts conveniently in Visual Studio's Test Explorer. However, this Test Explorer's design targets object-oriented languages and therefore expects the test names in the form `<namespace>.<class>.<testname>`. And since we specify the full path of the test scripts instead, including the ugly `/.././t/` part, these dots confuse the Test Explorer and it uses a large part of the path as "namespace". Let's just use `t.suite.<name>` instead. This presents the tests in Visual Studio's Test Explorer in the following form by default (i.e. unless the user changes the view via the "Group by" menu): ◢ ◈ git ◢ ◈ t ◢ ◈ suite ◈ t0000-basic ◈ t0001-init ◈ t0002-gitfile [...] Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-10cmake: fix typo in variable nameJohannes Schindelin1-2/+2
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-10cmake: also build unit testsJohannes Schindelin1-0/+18
A new, better way to run unit tests was just added to Git. This adds support for building those unit tests via CMake. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-08Merge branch 'jc/test-i18ngrep'Junio C Hamano1-1/+1
Another step to deprecate test_i18ngrep. * jc/test-i18ngrep: tests: teach callers of test_i18ngrep to use test_grep test framework: further deprecate test_i18ngrep
2023-11-02tests: teach callers of test_i18ngrep to use test_grepJunio C Hamano1-1/+1
They are equivalents and the former still exists, so as long as the only change this commit makes are to rewrite test_i18ngrep to test_grep, there won't be any new bug, even if there still are callers of test_i18ngrep remaining in the tree, or when merged to other topics that add new uses of test_i18ngrep. This patch was produced more or less with git grep -l -e 'test_i18ngrep ' 't/t[0-9][0-9][0-9][0-9]-*.sh' | xargs perl -p -i -e 's/test_i18ngrep /test_grep /' and a good way to sanity check the result yourself is to run the above in a checkout of c4603c1c (test framework: further deprecate test_i18ngrep, 2023-10-31) and compare the resulting working tree contents with the result of applying this patch to the same commit. You'll see that test_i18ngrep in a few t/lib-*.sh files corrected, in addition to the manual reproduction. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-10-30Merge branch 'sn/typo-grammo-phraso-fixes'Junio C Hamano2-3/+3
Many typos, ungrammatical sentences and wrong phrasing have been fixed. * sn/typo-grammo-phraso-fixes: t/README: fix multi-prerequisite example doc/gitk: s/sticked/stuck/ git-jump: admit to passing merge mode args to ls-files doc/diff-options: improve wording of the log.diffMerges mention doc: fix some typos, grammar and wording issues
2023-10-09completion: complete '--dd'Sergey Organov1-1/+1
'--dd' only makes sense for 'git log' and 'git show', so add it to __git_log_show_options which is referenced in the completion for these two commands. Signed-off-by: Sergey Organov <sorganov@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-10-05git-jump: admit to passing merge mode args to ls-filesŠtěpán Němec1-1/+1
There's even an example of such usage in the README. Fixes: 67ba13e5a4b2 ("git-jump: pass "merge" arguments to ls-files") Signed-off-by: Štěpán Němec <stepnem@smrk.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-10-05doc: fix some typos, grammar and wording issuesŠtěpán Němec1-2/+2
Signed-off-by: Štěpán Němec <stepnem@smrk.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-09-20completion: loosen and document the requirement around completing aliasJunio C Hamano1-2/+3
Recently we started to tell users to spell ": git foo ;" with space(s) around 'foo' for an alias to be completed similarly to the 'git foo' command. It however is easy to also allow users to spell it in a more natural way with the semicolon attached to 'foo', i.e. ": git foo;". Also, add a comment to note that 'git' is optional and writing ": foo;" would complete the alias just fine. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-09-20Merge branch 'pb/completion-aliases-doc'Junio C Hamano1-0/+1
Clarify how "alias.foo = : git cmd ; aliased-command-string" should be spelled with necessary whitespaces around punctuation marks to work. * pb/completion-aliases-doc: completion: improve doc for complex aliases
2023-09-20Merge branch 'pb/complete-commit-trailers'Junio C Hamano1-0/+9
The command-line complation support (in contrib/) learned to complete "git commit --trailer=" for possible trailer keys. * pb/complete-commit-trailers: completion: commit: complete trailers tokens more robustly completion: commit: complete configured trailer tokens
2023-09-18Merge branch 'js/complete-checkout-t'Junio C Hamano1-2/+2
The completion script (in contrib/) has been taught to treat the "-t" option to "git checkout" and "git switch" just like the "--track" option, to complete remote-tracking branches. * js/complete-checkout-t: completion(switch/checkout): treat --track and -t the same
2023-09-12completion: improve doc for complex aliasesPhilippe Blain1-0/+1
The completion code can be told to use a particular completion for aliases that shell out by using ': git <cmd> ;' as the first command of the alias. This only works if <cmd> and the semicolon are separated by a space, since if the space is missing __git_aliased_command returns (for example) 'checkout;' instead of just 'checkout', and then __git_complete_command fails to find a completion for 'checkout;'. The examples have that space but it's not clear if it's just for style or if it's mandatory. Explicitly mention it. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-09-12completion: commit: complete trailers tokens more robustlyPhilippe Blain1-1/+1
In the previous commit, we added support for completing configured trailer tokens in 'git commit --trailer'. Make the implementation more robust by: - using '__git' instead of plain 'git', as the rest of the completion script does - using a stricter pattern for --get-regexp to avoid false hits - using 'cut' and 'rev' instead of 'awk' to account for tokens including dots. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-09-08completion(switch/checkout): treat --track and -t the sameJohannes Schindelin1-2/+2
When `git switch --track ` is to be completed, only remote refs are eligible because that is what the `--track` option targets. And when the short-hand `-t` is used instead, the same _should_ happen. Let's make it so. Note that the bug exists both in the completions of `switch` and `completion`, even if it manifests in slightly different ways: While the completion of `git switch -t ` will not even look at remote refs, the completion of `git checkout -t ` will look at both remote _and_ local refs. Both should look only at remote refs. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-09-07completion: commit: complete configured trailer tokensPhilippe Blain1-0/+9
Since 2daae3d1d1 (commit: add --trailer option, 2021-03-23), 'git commit' can add trailers to commit messages. To make that feature more pleasant to use at the command line, update the Bash completion code to offer configured trailer tokens. Add a __git_trailer_tokens function to list the configured trailers tokens, and use it in _git_commit to suggest the configured tokens, suffixing the completion words with ':' so that the user only has to add the trailer value. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-08-28Merge branch 'mh/credential-erase-improvements-more'Junio C Hamano2-4/+33
Update two credential helpers to correctly match which credential to erase; they dropped not the ones with stale password. * mh/credential-erase-improvements-more: credential/wincred: erase matching creds only credential/libsecret: erase matching creds only
2023-08-28Merge branch 'mh/credential-libsecret-attrs'Junio C Hamano1-6/+72
The way authentication related data other than passwords (e.g. oath token and password expiration data) are stored in libsecret keyrings has been rethought. * mh/credential-libsecret-attrs: credential/libsecret: store new attributes
2023-08-25Merge branch 'rs/parse-options-negation-help'Junio C Hamano2-6/+6
"git cmd -h" learned to signal which options can be negated by listing such options like "--[no-]opt". * rs/parse-options-negation-help: parse-options: simplify usage_padding() parse-options: no --[no-]no-... parse-options: factor out usage_indent() and usage_padding() parse-options: show negatability of options in short help t1502: test option negation t1502: move optionspec help output to a file t1502, docs: disallow --no-help subtree: disallow --no-{help,quiet,debug,branch,message}
2023-08-15Merge branch 'ds/maintenance-on-windows-fix'Junio C Hamano3-2/+12
Windows updates. * ds/maintenance-on-windows-fix: git maintenance: avoid console window in scheduled tasks on Windows win32: add a helper to run `git.exe` without a foreground window
2023-08-09win32: add a helper to run `git.exe` without a foreground windowJohannes Schindelin3-2/+12
On Windows, there are two kinds of executables, console ones and non-console ones. Git's executables are all console ones. When launching the former e.g. in a scheduled task, a CMD window pops up. This is not what we want for the tasks installed via the `git maintenance` command. To work around this, let's introduce `headless-git.exe`, which is a non-console program that does _not_ pop up any window. All it does is to re-launch `git.exe`, suppressing that console window, passing through all command-line arguments as-are. Helped-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Helped-by: Yuyi Wang <Strawberry_Str@hotmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-08-06parse-options: show negatability of options in short helpRené Scharfe1-1/+1
Add a "[no-]" prefix to options without the flag PARSE_OPT_NONEG to document the fact that you can negate them. This looks a bit strange for options that already start with "no-", e.g. for the option --no-name of git show-branch: --[no-]no-name suppress naming strings You can actually use --no-no-name as an alias of --name, so the short help is not wrong. If we strip off any of the "no-"s, we lose either the ability to see if the remaining one belongs to the documented variant or to see if it can be negated. Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-08-06subtree: disallow --no-{help,quiet,debug,branch,message}René Scharfe1-5/+5
"git subtree" only handles the negated variant of the options annotate, prefix, onto, rejoin, ignore-joins and squash explicitly. help is handled by "git rev-parse --parseopt" implicitly, but not its negated form. Disable negation for it and the for the rest of the options to get a helpful error message when trying them. Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-07-26credential/wincred: erase matching creds onlyM Hickford1-4/+16
The credential erase request typically includes protocol, host, username and password. credential-wincred erases stored credentials that match protocol, host and username, regardless of password. This is confusing in the case the stored password differs from that in the request. This case can occur when multiple credential helpers are configured. Only erase credential if stored password matches request (or request omits password). This fixes test "helper (wincred) does not erase a password distinct from input" when t0303 is run with GIT_TEST_CREDENTIAL_HELPER set to "wincred". This test was added in aeb21ce22e (credential: avoid erasing distinct password, 2023-06-13). Signed-off-by: M Hickford <mirth.hickford@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-07-26credential/libsecret: erase matching creds onlyM Hickford1-0/+17
The credential erase request typically includes protocol, host, username and password. credential-libsecret erases a stored credential if it matches protocol, host and username, regardless of password. This is confusing in the case the stored password differs from that in the request. This case can occur when multiple credential helpers are configured. Only erase credential if stored password matches request (or request omits password). This fixes test "helper (libsecret) does not erase a password distinct from input" when t0303 is run with GIT_TEST_CREDENTIAL_HELPER set to "libsecret". This test was added in aeb21ce22e (credential: avoid erasing distinct password, 2023-06-13). Signed-off-by: M Hickford <mirth.hickford@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-07-06Merge branch 'gc/config-context'Junio C Hamano2-0/+171
Reduce reliance on a global state in the config reading API. * gc/config-context: config: pass source to config_parser_event_fn_t config: add kvi.path, use it to evaluate includes config.c: remove config_reader from configsets config: pass kvi to die_bad_number() trace2: plumb config kvi config.c: pass ctx with CLI config config: pass ctx with config files config.c: pass ctx in configsets config: add ctx arg to config_fn_t urlmatch.h: use config_fn_t type config: inline git_color_default_config
2023-07-06Merge branch 'pb/complete-diff-options'Junio C Hamano1-9/+48
Completion updates. * pb/complete-diff-options: (24 commits) diff.c: mention completion above add_diff_options completion: complete --remerge-diff completion: complete --diff-merges, its options and --no-diff-merges completion: move --pickaxe-{all,regex} to __git_diff_common_options completion: complete --ws-error-highlight completion: complete --unified completion: complete --output-indicator-{context,new,old} completion: complete --output completion: complete --no-stat completion: complete --no-relative completion: complete --line-prefix completion: complete --ita-invisible-in-index and --ita-visible-in-index completion: complete --irreversible-delete completion: complete --ignore-matching-lines completion: complete --function-context completion: complete --find-renames completion: complete --find-object completion: complete --find-copies completion: complete --default-prefix completion: complete --compact-summary ...
2023-06-28config: pass kvi to die_bad_number()Glen Choo1-0/+27
Plumb "struct key_value_info" through all code paths that end in die_bad_number(), which lets us remove the helper functions that read analogous values from "struct config_reader". As a result, nothing reads config_reader.config_kvi any more, so remove that too. In config.c, this requires changing the signature of git_configset_get_value() to 'return' "kvi" in an out parameter so that git_configset_get_<type>() can pass it to git_config_<type>(). Only numeric types will use "kvi", so for non-numeric types (e.g. git_configset_get_string()), pass NULL to indicate that the out parameter isn't needed. Outside of config.c, config callbacks now need to pass "ctx->kvi" to any of the git_config_<type>() functions that parse a config string into a number type. Included is a .cocci patch to make that refactor. The only exceptional case is builtin/config.c, where git_config_<type>() is called outside of a config callback (namely, on user-provided input), so config source information has never been available. In this case, die_bad_number() defaults to a generic, but perfectly descriptive message. Let's provide a safe, non-NULL for "kvi" anyway, but make sure not to change the message. Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-28config: add ctx arg to config_fn_tGlen Choo1-0/+144
Add a new "const struct config_context *ctx" arg to config_fn_t to hold additional information about the config iteration operation. config_context has a "struct key_value_info kvi" member that holds metadata about the config source being read (e.g. what kind of config source it is, the filename, etc). In this series, we're only interested in .kvi, so we could have just used "struct key_value_info" as an arg, but config_context makes it possible to add/adjust members in the future without changing the config_fn_t signature. We could also consider other ways of organizing the args (e.g. moving the config name and value into config_context or key_value_info), but in my experiments, the incremental benefit doesn't justify the added complexity (e.g. a config_fn_t will sometimes invoke another config_fn_t but with a different config value). In subsequent commits, the .kvi member will replace the global "struct config_reader" in config.c, making config iteration a global-free operation. It requires much more work for the machinery to provide meaningful values of .kvi, so for now, merely change the signature and call sites, pass NULL as a placeholder value, and don't rely on the arg in any meaningful way. Most of the changes are performed by contrib/coccinelle/config_fn_ctx.pending.cocci, which, for every config_fn_t: - Modifies the signature to accept "const struct config_context *ctx" - Passes "ctx" to any inner config_fn_t, if needed - Adds UNUSED attributes to "ctx", if needed Most config_fn_t instances are easily identified by seeing if they are called by the various config functions. Most of the remaining ones are manually named in the .cocci patch. Manual cleanups are still needed, but the majority of it is trivial; it's either adjusting config_fn_t that the .cocci patch didn't catch, or adding forward declarations of "struct config_context ctx" to make the signatures make sense. The non-trivial changes are in cases where we are invoking a config_fn_t outside of config machinery, and we now need to decide what value of "ctx" to pass. These cases are: - trace2/tr2_cfg.c:tr2_cfg_set_fl() This is indirectly called by git_config_set() so that the trace2 machinery can notice the new config values and update its settings using the tr2 config parsing function, i.e. tr2_cfg_cb(). - builtin/checkout.c:checkout_main() This calls git_xmerge_config() as a shorthand for parsing a CLI arg. This might be worth refactoring away in the future, since git_xmerge_config() can call git_default_config(), which can do much more than just parsing. Handle them by creating a KVI_INIT macro that initializes "struct key_value_info" to a reasonable default, and use that to construct the "ctx" arg. Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --remerge-diffPhilippe Blain1-1/+1
--remerge-diff only makes sense for 'git log' and 'git show', so add it to __git_log_show_options which is referenced in the completion for these two commands. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --diff-merges, its options and --no-diff-mergesPhilippe Blain1-0/+16
The flags --[no-]diff-merges only make sense for 'git log' and 'git show', so add a new variable __git_log_show_options for options only relevant to these two commands, and add them there. Also add __git_diff_merges_opts and list the accepted values for --diff-merges, and use it in _git_log and _git_show. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: move --pickaxe-{all,regex} to __git_diff_common_optionsPhilippe Blain1-2/+3
The options --pickaxe-all and --pickaxe-regex are listed in __git_diff_difftool_options and repeated in _git_log. Move them to __git_diff_common_options instead, which makes them available automatically in the completion of other commands referencing this variable. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --ws-error-highlightPhilippe Blain1-0/+14
Add --ws-error-highlight= to the list in __git_diff_common_options, and add the accepted values in a new list __git_ws_error_highlight_opts. Use __git_ws_error_highlight_opts in _git_diff, _git_log and _git_show to offer the accepted values. As noted in fd0bc17557 (completion: add diff --color-moved[-ws], 2020-02-21), there is no easy way to offer completion for several comma-separated values, so this is limited to completing a single value. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --unifiedPhilippe Blain1-1/+1
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --output-indicator-{context,new,old}Philippe Blain1-1/+2
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --outputPhilippe Blain1-0/+1
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --no-statPhilippe Blain1-1/+1
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --no-relativePhilippe Blain1-1/+2
Add --no-relative to __git_diff_common_options in the completion script, and move --relative from __git_diff_difftool_options to __git_diff_common_options since it applies to more than just diff and difftool. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --line-prefixPhilippe Blain1-1/+1
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --ita-invisible-in-index and --ita-visible-in-indexPhilippe Blain1-0/+1
The options --ita-invisible-in-index and --ita-visible-in-index are listed in diff-options.txt and so are included in the documentation of commands which include this file (diff, diff-*, log, show, format-patch) but they only make sense for diffs relating to the index. As such, add them to '__git_diff_difftool_options' instead of '__git_diff_common_options' since it makes more sense to add them there. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --irreversible-deletePhilippe Blain1-0/+1
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --ignore-matching-linesPhilippe Blain1-1/+1
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --function-contextPhilippe Blain1-1/+1
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --find-renamesPhilippe Blain1-1/+1
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --find-objectPhilippe Blain1-1/+1
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --find-copiesPhilippe Blain1-0/+1
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --default-prefixPhilippe Blain1-1/+1
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --compact-summaryPhilippe Blain1-1/+1
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --combined-all-pathsPhilippe Blain1-1/+1
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --ccPhilippe Blain1-1/+1
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: complete --break-rewritesPhilippe Blain1-1/+1
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-26completion: add comments describing __git_diff_* globalsPhilippe Blain1-0/+2
Add descriptive comments for '__git_diff_common_options' and '__git_diff_difftool_options', so that it is clearer when looking at these variables to know in which command's completion they are used. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-21cache.h: remove this no-longer-used headerElijah Newren1-1/+1
Since this header showed up in some places besides just #include statements, update/clean-up/remove those other places as well. Note that compat/fsmonitor/fsm-path-utils-darwin.c previously got away with violating the rule that all files must start with an include of git-compat-util.h (or a short-list of alternate headers that happen to include it first). This change exposed the violation and caused it to stop building correctly; fix it by having it include git-compat-util.h first, as per policy. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-20Merge branch 'pb/complete-and-document-auto-merge-and-friends'Junio C Hamano1-1/+1
Document more pseudo-refs and teach the command line completion machinery to complete AUTO_MERGE. * pb/complete-and-document-auto-merge-and-friends: completion: complete AUTO_MERGE Documentation: document AUTO_MERGE git-merge.txt: modernize word choice in "True merge" section completion: complete REVERT_HEAD and BISECT_HEAD revisions.txt: document more special refs revisions.txt: use description list for special refs
2023-06-16credential/libsecret: store new attributesM Hickford1-6/+72
d208bfd (credential: new attribute password_expiry_utc, 2023-02-18) and a5c76569e7 (credential: new attribute oauth_refresh_token) introduced new credential attributes. libsecret assumes attribute values are non-confidential and unchanging, so we encode the new attributes in the secret, separated by newline: hunter2 password_expiry_utc=1684189401 oauth_refresh_token=xyzzy This is extensible and backwards compatible. The credential protocol already assumes that attribute values do not contain newlines. Alternatives considered: store password_expiry_utc in a libsecret attribute. This has the problem that libsecret creates new items rather than overwrites when attribute values change. Signed-off-by: M Hickford <mirth.hickford@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-13Merge branch 'gc/doc-cocci-updates'Junio C Hamano1-4/+36
Update documentation regarding Coccinelle patches. * gc/doc-cocci-updates: cocci: codify authoring and reviewing practices cocci: add headings to and reword README
2023-05-23completion: complete AUTO_MERGEPhilippe Blain1-1/+1
The pseudoref AUTO_MERGE is documented since the previous commit. To make it easier to use, let __git_refs in the Bash completion code complete it. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-05-23completion: complete REVERT_HEAD and BISECT_HEADPhilippe Blain1-1/+1
The pseudorefs REVERT_HEAD and BISECT_HEAD are not suggested by the __git_refs function. Add them there. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-05-15Merge branch 'sg/retire-unused-cocci'Junio C Hamano3-170/+0
Retire a rather expensive-to-run Coccinelle check patch. * sg/retire-unused-cocci: cocci: remove 'unused.cocci'
2023-05-15Merge branch 'js/subtree-fully-spelt-quiet-and-debug-options'Junio C Hamano1-2/+2
"git subtree" (in contrib/) update. * js/subtree-fully-spelt-quiet-and-debug-options: subtree: support long global flags
2023-05-11Merge branch 'mh/credential-password-expiry-wincred'Junio C Hamano1-2/+23
Teach the recently invented "password expiry time" trait to the wincred credential helper. * mh/credential-password-expiry-wincred: credential/wincred: store password_expiry_utc
2023-05-11Merge branch 'mh/use-wincred-from-system'Junio C Hamano1-60/+1
Code clean-up. * mh/use-wincred-from-system: credential/wincred: include wincred.h
2023-05-10Merge branch 'tb/credential-long-lines'Junio C Hamano7-511/+32
The implementation of credential helpers used fgets() over fixed size buffers to read protocol messages, causing the remainder of the folded long line to trigger unexpected behaviour, which has been corrected. * tb/credential-long-lines: contrib/credential: embiggen fixed-size buffer in wincred contrib/credential: avoid fixed-size buffer in libsecret contrib/credential: .gitignore libsecret build artifacts contrib/credential: remove 'gnome-keyring' credential helper contrib/credential: avoid fixed-size buffer in osxkeychain t/lib-credential.sh: ensure credential helpers handle long headers credential.c: store "wwwauth[]" values in `credential_read()`
2023-05-08subtree: support long global flagsJosh Soref1-2/+2
The documentation at e75d1da38a claimed support, but it was never present Signed-off-by: Josh Soref <jsoref@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-05-02Merge branch 'ek/completion-use-read-r-to-read-literally'Junio C Hamano1-1/+1
The completion script used to use bare "read" without the "-r" option to read the contents of various state files, which risked getting confused with backslashes in them. This has been corrected. * ek/completion-use-read-r-to-read-literally: completion: suppress unwanted unescaping of `read`
2023-05-01contrib/credential: embiggen fixed-size buffer in wincredTaylor Blau1-4/+17
As in previous commits, harden the wincred credential helper against the aforementioned protocol injection attack. Unlike the approached used for osxkeychain and libsecret, where a fixed-size buffer was replaced with `getline()`, we must take a different approach here. There is no `getline()` equivalent in Windows, and the function is not available to us with ordinary compiler settings. Instead, allocate a larger (still fixed-size) buffer in which to process each line. The value of 100 KiB is chosen to match the maximum-length header that curl will allow, CURL_MAX_HTTP_HEADER. To ensure that we are reading complete lines at a time, and that we aren't susceptible to a similar injection attack (albeit with more padding), ensure that each read terminates at a newline (i.e., that no line is more than 100 KiB long). Note that it isn't sufficient to turn the old loop into something like: while (len && strchr("\r\n", buf[len - 1])) { buf[--len] = 0; ends_in_newline = 1; } because if an attacker sends something like: [aaaaa.....]\r host=example.com\r\n the credential helper would fill its buffer after reading up through the first '\r', call fgets() again, and then see "host=example.com\r\n" on its line. Note that the original code was written in a way that would trim an arbitrary number of "\r" and "\n" from the end of the string. We should get only a single "\n" (since the point of `fgets()` is to return the buffer to us when it sees one), and likewise would not expect to see more than one associated "\r". The new code trims a single "\r\n", which matches the original intent. [1]: https://curl.se/libcurl/c/CURLOPT_HEADERFUNCTION.html Tested-by: Matthew John Cheetham <mjcheetham@outlook.com> Helped-by: Matthew John Cheetham <mjcheetham@outlook.com> Co-authored-by: Jeff King <peff@peff.net> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-05-01contrib/credential: avoid fixed-size buffer in libsecretTaylor Blau1-8/+7
The libsecret credential helper reads the newline-delimited protocol stream one line at a time by repeatedly calling fgets() into a fixed-size buffer, and is thus affected by the vulnerability described in the previous commit. To mitigate this attack, avoid using a fixed-size buffer, and instead rely on getline() to allocate a buffer as large as necessary to fit the entire content of the line, preventing any protocol injection. In most parts of Git we don't assume that every platform has getline(). But libsecret is primarily used on Linux, where we do already assume it (using a knob in config.mak.uname). POSIX also added getline() in 2008, so we'd expect other recent Unix-like operating systems to have it (e.g., FreeBSD also does). Note that the buffer was already allocated on the heap in this case, but we'll swap `g_free()` for `free()`, since it will now be allocated by the system `getline()`, rather than glib's `g_malloc()`. Tested-by: Jeff King <peff@peff.net> Co-authored-by: Jeff King <peff@peff.net> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-05-01contrib/credential: .gitignore libsecret build artifactsTaylor Blau1-0/+1
The libsecret credential helper does not mark its build artifact as ignored, so running "make" results in a dirty working tree. Mark the "git-credential-libsecret" binary as ignored to avoid the above. Co-authored-by: Jeff King <peff@peff.net> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-05-01contrib/credential: remove 'gnome-keyring' credential helperTaylor Blau3-496/+0
libgnome-keyring was deprecated in 2014 (in favor of libsecret), more than nine years ago [1]. The credential helper implemented using libgnome-keyring has had a small handful of commits since 2013, none of which implemented or changed any functionality. The last commit to do substantial work in this area was 15f7221686 (contrib/git-credential-gnome-keyring.c: support really ancient gnome-keyring, 2013-09-23), just shy of nine years ago. This credential helper suffers from the same `fgets()`-related injection attack (using the new "wwwauth[]" feature) as in the previous commit. Instead of patching it, let's remove this helper as deprecated. [1]: https://mail.gnome.org/archives/commits-list/2014-January/msg01585.html Co-authored-by: Jeff King <peff@peff.net> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-05-01contrib/credential: avoid fixed-size buffer in osxkeychainTaylor Blau1-3/+7
The macOS Keychain-based credential helper reads the newline-delimited protocol stream one line at a time by repeatedly calling fgets() into a fixed-size buffer, and is thus affected by the vulnerability described in the previous commit. To mitigate this attack, avoid using a fixed-size buffer, and instead rely on getline() to allocate a buffer as large as necessary to fit the entire content of the line, preventing any protocol injection. We solved a similar problem in a5bb10fd5e (config: avoid fixed-sized buffer when renaming/deleting a section, 2023-04-06) by switching to strbuf_getline(). We can't do that here because the contrib helpers do not link with the rest of Git, and so can't use a strbuf. But we can use the system getline() directly, which works similarly. In most parts of Git we don't assume that every platform has getline(). But this helper is run only on OS X, and that platform added support in 10.7 ("Lion") which was released in 2011. Tested-by: Taylor Blau <me@ttaylorr.com> Co-authored-by: Jeff King <peff@peff.net> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-27cocci: codify authoring and reviewing practicesGlen Choo1-0/+30
These practices largely reflect what we are already doing on the mailing list, which should help new Coccinelle authors and reviewers get up to speed. Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-27cocci: add headings to and reword READMEGlen Choo1-4/+6
- Drop "examples" since we actually use the patches. - Drop sentences that could be headings instead Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-20completion: suppress unwanted unescaping of `read`Edwin Kofler1-1/+1
The function `__git_eread`, which reads the first line from the file, calls the `read` builtin without passing the flag option `-r`. When the `read` builtin is called without the flag `-r`, it processes the backslash escaping in the text that it reads. For this reason, it is generally considered the best practice to always use the `read` builtin with flag `-r` unless one intensionally processes the backslash escaping. For the present case in git-prompt.sh, in fact, all the occurrences of the calls of `__git_eread` intend to read the literal content of the first lines. To make it read the first line literally, pass the flag `-r` to the `read` builtin in the function `__git_eread`. Signed-off-by: Edwin Kofler <edwin@kofler.dev> Signed-off-by: Koichi Murase <myoga.murase@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-20cocci: remove 'unused.cocci'SZEDER Gábor3-170/+0
When 'unused.cocci' was added in 4f40f6cb73 (cocci: add and apply a rule to find "unused" strbufs, 2022-07-05) it found three unused strbufs, and when it was generalized in the next commit it managed to find an unused string_list as well. That's four unused variables in over 17 years, so apparently we rarely make this mistake. Unfortunately, applying 'unused.cocci' is quite expensive, e.g. it increases the from-scratch runtime of 'make coccicheck' by over 5:30 minutes or over 160%: $ make -s cocciclean $ time make -s coccicheck * new spatch flags real 8m56.201s user 0m0.420s sys 0m0.406s $ rm contrib/coccinelle/unused.cocci contrib/coccinelle/tests/unused.* $ make -s cocciclean $ time make -s coccicheck * new spatch flags real 3m23.893s user 0m0.228s sys 0m0.247s That's a lot of runtime spent for not much in return, and arguably an unused struct instance sneaking in is not that big of a deal to justify the significantly increased runtime. Remove 'unused.cocci', because we are not getting our CPU cycles' worth. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-06Merge branch 'ab/remove-implicit-use-of-the-repository'Junio C Hamano2-128/+123
Code clean-up around the use of the_repository. * ab/remove-implicit-use-of-the-repository: libs: use "struct repository *" argument, not "the_repository" post-cocci: adjust comments for recent repo_* migration cocci: apply the "revision.h" part of "the_repository.pending" cocci: apply the "rerere.h" part of "the_repository.pending" cocci: apply the "refs.h" part of "the_repository.pending" cocci: apply the "promisor-remote.h" part of "the_repository.pending" cocci: apply the "packfile.h" part of "the_repository.pending" cocci: apply the "pretty.h" part of "the_repository.pending" cocci: apply the "object-store.h" part of "the_repository.pending" cocci: apply the "diff.h" part of "the_repository.pending" cocci: apply the "commit.h" part of "the_repository.pending" cocci: apply the "commit-reach.h" part of "the_repository.pending" cocci: apply the "cache.h" part of "the_repository.pending" cocci: add missing "the_repository" macros to "pending" cocci: sort "the_repository" rules by header cocci: fix incorrect & verbose "the_repository" rules cocci: remove dead rule from "the_repository.pending.cocci"
2023-04-03credential/wincred: store password_expiry_utcM Hickford1-2/+23
This attribute is important when storing OAuth credentials which may expire after as little as one hour. d208bfdf (credential: new attribute password_expiry_utc, 2023-02-18) added support for this attribute in general so that individual credential backend like wincred can use it. Signed-off-by: M Hickford <mirth.hickford@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-28Merge branch 'fc/completion-colors-do-not-need-prompt-command'Junio C Hamano1-12/+7
Lift the limitation that colored prompts can only be used with PROMPT_COMMAND mode. * fc/completion-colors-do-not-need-prompt-command: completion: prompt: use generic colors
2023-03-28cocci: apply the "revision.h" part of "the_repository.pending"Ævar Arnfjörð Bjarmason2-14/+4
Apply the part of "the_repository.pending.cocci" pertaining to "revision.h". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-28cocci: apply the "rerere.h" part of "the_repository.pending"Ævar Arnfjörð Bjarmason2-4/+4
Apply the part of "the_repository.pending.cocci" pertaining to "rerere.h". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-28cocci: apply the "refs.h" part of "the_repository.pending"Ævar Arnfjörð Bjarmason2-4/+4
Apply the part of "the_repository.pending.cocci" pertaining to "refs.h". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-28cocci: apply the "promisor-remote.h" part of "the_repository.pending"Ævar Arnfjörð Bjarmason2-10/+10
Apply the part of "the_repository.pending.cocci" pertaining to "promisor-remote.h". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-28cocci: apply the "packfile.h" part of "the_repository.pending"Ævar Arnfjörð Bjarmason2-4/+4
Apply the part of "the_repository.pending.cocci" pertaining to "packfile.h". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-28cocci: apply the "pretty.h" part of "the_repository.pending"Ævar Arnfjörð Bjarmason2-4/+4
Apply the part of "the_repository.pending.cocci" pertaining to "pretty.h". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-28cocci: apply the "object-store.h" part of "the_repository.pending"Ævar Arnfjörð Bjarmason2-10/+10
Apply the part of "the_repository.pending.cocci" pertaining to "object-store.h". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-28cocci: apply the "diff.h" part of "the_repository.pending"Ævar Arnfjörð Bjarmason2-4/+4
Apply the part of "the_repository.pending.cocci" pertaining to "diff.h". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-28cocci: apply the "commit.h" part of "the_repository.pending"Ævar Arnfjörð Bjarmason2-19/+19
Apply the part of "the_repository.pending.cocci" pertaining to "commit.h". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-28cocci: apply the "commit-reach.h" part of "the_repository.pending"Ævar Arnfjörð Bjarmason2-16/+16
Apply the part of "the_repository.pending.cocci" pertaining to "commit-reach.h". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-28cocci: apply the "cache.h" part of "the_repository.pending"Ævar Arnfjörð Bjarmason2-37/+44
Apply the part of "the_repository.pending.cocci" pertaining to "cache.h". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-28cocci: add missing "the_repository" macros to "pending"Ævar Arnfjörð Bjarmason1-0/+70
In the case of diff.h, rerere.h and revision.h the macros were added in [1], [2] and [3] when "the_repository.pending.cocci" didn't exist. None of the subsequently added migration rules covered them. Let's add those missing rules. In the case of macros in "cache.h", "commit.h", "packfile.h", "promisor-remote.h" and "refs.h" those aren't guarded by "NO_THE_REPOSITORY_COMPATIBILITY_MACROS", but they're also macros that add "the_repository" as the first argument, so we should migrate away from them. 1. 2abf3503854 (revision.c: remove implicit dependency on the_index, 2018-09-21) 2. e6757652350 (diff.c: remove implicit dependency on the_index, 2018-09-21) 3. 35843b1123e (rerere.c: remove implicit dependency on the_index, 2018-09-21) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-28cocci: sort "the_repository" rules by headerÆvar Arnfjörð Bjarmason1-15/+19
Sort the "the_repository.pending.cocci" file by which header the macros are in, and add a comment to that effect in front of the rules. This will make subsequent commits easier to follow, as we'll be applying these rules on a header-by-header basis. Once we've fully applied "the_repository.pending.cocci" we'll keep this rules around for a while in "the_repository.cocci", to help any outstanding topics and out-of-tree code to resolve textual or semantic conflicts with these changes, but eventually we'll remove the "the_repository.cocci" as a follow-up. So even if some of these functions are subsequently moved and/or split into other or new headers there's no risk of this becoming stale, if and when that happens the we should be removing these rules anyway. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-28cocci: fix incorrect & verbose "the_repository" rulesÆvar Arnfjörð Bjarmason1-114/+46
When these rules started being added in [1] they didn't use a ";" after the ")", and would thus catch uses of these macros within expressions. But as of [2] the new additions were broken in that they'd only match a subset of the users of these macros. Rather than narrowly fixing that, let's have these use the much less verbose pattern introduced in my recent [3]: There's no need to exhaustively enumerate arguments if we use the "..." syntax. This means that we can fold all of these different rules into one. 1. afd69dcc219 (object-store: prepare read_object_file to deal with any repo, 2018-11-13) 2. 21a9651ba3f (commit-reach: prepare get_merge_bases to handle any repo, 2018-11-13) 3. 0e6550a2c63 (cocci: add a index-compatibility.pending.cocci, 2022-11-19) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-28cocci: remove dead rule from "the_repository.pending.cocci"Ævar Arnfjörð Bjarmason1-8/+0
The "parse_commit_gently" macro went away in [1], so we don't need to carry this for its migration. 1. ea3f7e598c8 (revision: use repository from rev_info when parsing commits, 2020-06-23) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-27credential/wincred: include wincred.hM Hickford1-60/+1
Delete redundant definitions. Mingw-w64 has wincred.h since 2007 [1]. [1] https://github.com/mingw-w64/mingw-w64/blob/9d937a7f4f766f903c9433044f77bfa97a0bc1d8/mingw-w64-headers/include/wincred.h Signed-off-by: M Hickford <mirth.hickford@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-16completion: prompt: use generic colorsFelipe Contreras1-12/+7
When the prompt command mode was introduced in 1bfc51ac81 (Allow __git_ps1 to be used in PROMPT_COMMAND, 2012-10-10), the assumption was that it was necessary in order to properly add colors to PS1 in bash, but this wasn't true. It's true that the \[ \] markers add the information needed to properly calculate the width of the prompt, and they have to be added directly to PS1, a function returning them doesn't work. But that is because bash coverts the \[ \] markers in PS1 to \001 \002, which is what readline ultimately needs in order to calculate the width. We don't need bash to do this conversion, we can use \001 \002 ourselves, and then the prompt command mode is not necessary to display colors. This is what functions returning colors are supposed to do [1]. [1] http://mywiki.wooledge.org/BashFAQ/053 Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Tested-by: Joakim Petersen <joak-pet@online.no> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-09test: don't print aggregate-results commandFelipe Contreras1-1/+1
There's no value in it. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-09test: simplify counts aggregationFelipe Contreras1-3/+1
When the list of files as input was implemented in 6508eedf67 (t/aggregate-results: accomodate systems with small max argument list length, 2010-06-01), a much simpler solution wasn't considered. Let's just pass the directory as an argument. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-10cocci & cache.h: remove "USE_THE_INDEX_COMPATIBILITY_MACROS"Ævar Arnfjörð Bjarmason2-21/+6
Have the last users of "USE_THE_INDEX_COMPATIBILITY_MACROS" use the underlying *_index() variants instead. Now all previous users of "USE_THE_INDEX_COMPATIBILITY_MACROS" have been migrated away from the wrapper macros, and if applicable to use the "USE_THE_INDEX_VARIABLE" added in [1]. Let's leave the "index-compatibility.cocci" in place, even though it won't be doing anything on "master". It will benefit any out-of-tree code that need to use these compatibility macros. We can eventually remove it. 1. bdafeae0b9c (cache.h & test-tool.h: add & use "USE_THE_INDEX_VARIABLE", 2022-11-19) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-10cache-tree API: remove redundant update_main_cache_tree()Ævar Arnfjörð Bjarmason1-0/+3
Remove the redundant update_main_cache_tree() function, and make its users use cache_tree_update() instead. The behavior of populating the "the_index.cache_tree" if it wasn't present already was needed when this function was introduced in [1], but it hasn't been needed since [2]; The "cache_tree_update()" will now lazy-allocate, so there's no need for the wrapper. 1. 996277c5206 (Refactor cache_tree_update idiom from commit, 2011-12-06) 2. fb0882648e0 (cache-tree: clean up cache_tree_update(), 2021-01-23) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-10cocci & cache-tree.h: migrate "write_cache_as_tree" to "*_index_*"Ævar Arnfjörð Bjarmason1-0/+11
Add a trivial rule for "write_cache_as_tree" to "index-compatibility.cocci", and apply it. This was left out of the rules added in 0e6550a2c63 (cocci: add a index-compatibility.pending.cocci, 2022-11-19) because this compatibility wrapper lived in "cache-tree.h", not "cache.h" But it's like the other "USE_THE_INDEX_COMPATIBILITY_MACROS", so let's migrate it too. The replacement of "USE_THE_INDEX_COMPATIBILITY_MACROS" here with "USE_THE_INDEX_VARIABLE" is a manual change on top, now that these files only use "&the_index", and don't need any compatibility macros (or functions). The wrapping of some argument lists is likewise manual, as coccinelle would otherwise give us overly long argument lists. The reason for putting the "O" in the cocci rule on the "-" and "+" lines is because I couldn't get correct whitespacing otherwise, i.e. I'd end up with "oid,&the_index", not "oid, &the_index". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-10cocci & cache.h: apply pending "index_cache_pos" ruleÆvar Arnfjörð Bjarmason2-3/+3
Apply the rule added in [1] to change "cache_name_pos" to "index_name_pos", which allows us to get rid of another "USE_THE_INDEX_COMPATIBILITY_MACROS" macro. The replacement of "USE_THE_INDEX_COMPATIBILITY_MACROS" here with "USE_THE_INDEX_VARIABLE" is a manual change on top, now that these files only use "&the_index", and don't need any compatibility macros (or functions). 1. 0e6550a2c63 (cocci: add a index-compatibility.pending.cocci, 2022-11-19) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-10cocci & cache.h: fully apply "active_nr" part of index-compatibilityÆvar Arnfjörð Bjarmason1-9/+4
Apply the "active_nr" part of "index-compatibility.pending.cocci", which was left out in [1] due to an in-flight conflict. As of [2] the topic we conflicted with has been merged to "master", so we can fully apply this rule. 1. dc594180d9e (cocci & cache.h: apply variable section of "pending" index-compatibility, 2022-11-19) 2. 9ea1378d046 (Merge branch 'ab/various-leak-fixes', 2022-12-14) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-09use DUP_ARRAYRené Scharfe1-0/+7
Add a semantic patch for replace ALLOC_ARRAY+COPY_ARRAY with DUP_ARRAY to reduce code duplication and apply its results. Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-19Merge branch 'aw/complete-case-insensitive'Junio C Hamano1-3/+23
Introduce a case insensitive mode to the Bash completion helpers. * aw/complete-case-insensitive: completion: add case-insensitive match of pseudorefs completion: add optional ignore-case when matching refs
2022-12-14Merge branch 'yn/git-jump-emacs'Junio C Hamano2-4/+51
"git jump" (in contrib/) learned to present the "quickfix list" to its standard output (instead of letting it consumed by the editor it invokes), and learned to also drive emacs/emacsclient. * yn/git-jump-emacs: git-jump: invoke emacs/emacsclient git-jump: move valid-mode check earlier git-jump: add an optional argument '--stdout'
2022-12-01Merge branch 'ab/fewer-the-index-macros'Junio C Hamano1-4/+8
Squelch warnings from Coccinelle * ab/fewer-the-index-macros: cocci: avoid "should ... be a metavariable" warnings
2022-12-01cocci: avoid "should ... be a metavariable" warningsÆvar Arnfjörð Bjarmason1-4/+8
Since [1] running "make coccicheck" has resulted in [2] being emitted to the *.log files for the "spatch" run, and in the case of "make coccicheck-test" we'd emit these to the user's terminal. Nothing was broken as a result, but let's refactor the relevant rules to eliminate the ambiguity between a possible variable and an identifier. 1. 0e6550a2c63 (cocci: add a index-compatibility.pending.cocci, 2022-11-19) 2. warning: line 257: should active_cache be a metavariable? warning: line 260: should active_cache_changed be a metavariable? warning: line 263: should active_cache_tree be a metavariable? warning: line 271: should active_nr be a metavariable? Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-30completion: add case-insensitive match of pseudorefsAlison Winters1-3/+11
When GIT_COMPLETION_IGNORE_CASE is set, also allow lowercase completion text like "head" to match uppercase HEAD and other pseudorefs. Signed-off-by: Alison Winters <alisonatwork@outlook.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-30completion: add optional ignore-case when matching refsAlison Winters1-0/+12
If GIT_COMPLETION_IGNORE_CASE is set, --ignore-case will be added to git for-each-ref calls so that refs can be matched case insensitively, even when running on case sensitive filesystems. Signed-off-by: Alison Winters <alisonatwork@outlook.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-28Merge branch 'ab/fewer-the-index-macros'Junio C Hamano2-0/+159
Progress on removing 'the_index' convenience wrappers. * ab/fewer-the-index-macros: cocci: apply "pending" index-compatibility to some "builtin/*.c" cache.h & test-tool.h: add & use "USE_THE_INDEX_VARIABLE" {builtin/*,repository}.c: add & use "USE_THE_INDEX_VARIABLE" cocci: apply "pending" index-compatibility to "t/helper/*.c" cocci & cache.h: apply variable section of "pending" index-compatibility cocci & cache.h: apply a selection of "pending" index-compatibility cocci: add a index-compatibility.pending.cocci read-cache API & users: make discard_index() return void cocci & cache.h: remove rarely used "the_index" compat macros builtin/{grep,log}.: don't define "USE_THE_INDEX_COMPATIBILITY_MACROS" cache.h: remove unused "the_index" compat macros
2022-11-27git-jump: invoke emacs/emacsclientYoichi Nakayama1-1/+16
It works with GIT_EDITOR="emacs", "emacsclient" or "emacsclient -t" Signed-off-by: Yoichi Nakayama <yoichi.nakayama@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-27git-jump: move valid-mode check earlierJeff King1-1/+2
We check if the "mode" argument supplied by the user is valid by seeing if we have a mode_$mode function defined. But we don't do that until after creating the tempfile. This is wasteful (we create a tempfile but never use it), and makes it harder to add new options (the recent stdout option exits before creating the tempfile, so it misses the check and "git jump --stdout foo" will produce "git-jump: 92: mode_foo: not found" rather than the regular usage message). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-27git-jump: add an optional argument '--stdout'Yoichi Nakayama2-2/+33
It can be used with M-x grep on Emacs. Signed-off-by: Yoichi Nakayama <yoichi.nakayama@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-23Merge branch 'ab/coccicheck-incremental'Junio C Hamano8-6/+358
"make coccicheck" is time consuming. It has been made to run more incrementally. * ab/coccicheck-incremental: Makefile: don't create a ".build/.build/" for cocci, fix output spatchcache: add a ccache-alike for "spatch" cocci: run against a generated ALL.cocci cocci rules: remove <id>'s from rules that don't need them Makefile: copy contrib/coccinelle/*.cocci to build/ cocci: optimistically use COMPUTE_HEADER_DEPENDENCIES cocci: make "coccicheck" rule incremental cocci: split off "--all-includes" from SPATCH_FLAGS cocci: split off include-less "tests" from SPATCH_FLAGS Makefile: split off SPATCH_BATCH_SIZE comment from "cocci" heading Makefile: have "coccicheck" re-run if flags change Makefile: add ability to TAB-complete cocci *.patch rules cocci rules: remove unused "F" metavariable from pending rule Makefile + shared.mak: rename and indent $(QUIET_SPATCH_T)
2022-11-21cocci: apply "pending" index-compatibility to some "builtin/*.c"Ævar Arnfjörð Bjarmason2-54/+53
Apply "index-compatibility.pending.cocci" rule to "builtin/*", but exclude those where we conflict with in-flight changes. As a result some of them end up using only "the_index", so let's have them use the more narrow "USE_THE_INDEX_VARIABLE" rather than "USE_THE_INDEX_COMPATIBILITY_MACROS". Manual changes not made by coccinelle, that were squashed in: * Whitespace-wrap argument lists for repo_hold_locked_index(), repo_read_index_preload() and repo_refresh_and_write_index(), in cases where the line became too long after the transformation. * Change "refresh_cache()" to "refresh_index()" in a comment in "builtin/update-index.c". * For those whose call was followed by perror("<macro-name>"), change it to perror("<function-name>"), referring to the new function. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-21cocci & cache.h: apply variable section of "pending" index-compatibilityÆvar Arnfjörð Bjarmason2-17/+22
Mostly apply the part of "index-compatibility.pending.cocci" that renames the global variables like "active_nr", which are a shorthand to referencing (in that case) a struct member as "the_index.cache_nr". In doing so move more of "index-compatibility.pending.cocci" to "index-compatibility.cocci". In the case of "active_nr" we'd have a textual conflict with "ab/various-leak-fixes" in "next"[1]. Let's exclude that specific case while moving the rule over from "pending". 1. 407b94280f8 (commit: discard partial cache before (re-)reading it, 2022-11-08) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-21cocci & cache.h: apply a selection of "pending" index-compatibilityÆvar Arnfjörð Bjarmason2-15/+15
Apply a selection of rules in "index-compatibility.pending.cocci" tree-wide, and in doing so migrate them to "index-compatibility.cocci". As in preceding commits the only manual changes here are the macro removals in "cache.h", and the update to the '*.cocci" rules. The rest of the C code changes are the result of applying those updated rules. Move rules for some rarely used cache compatibility macros from "index-compatibility.pending.cocci" to "index-compatibility.cocci" and apply them. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-21cocci: add a index-compatibility.pending.cocciÆvar Arnfjörð Bjarmason1-0/+110
Add a coccinelle rule which covers the rest of the macros guarded by "USE_THE_INDEX_COMPATIBILITY_MACROS" cache.h. If the result of this were applied it can be reduced down to just: #ifdef USE_THE_INDEX_COMPATIBILITY_MACROS extern struct index_state the_index; #endif But that patch is just under 2000 lines, so let's first add this as a "pending", and then incrementally pick changes from it in subsequent commits. In doing that we'll migrate rules from this "index-compatibility.pending.cocci" to the "index-compatibility.cocci" created in a preceding commit. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-21cocci & cache.h: remove rarely used "the_index" compat macrosÆvar Arnfjörð Bjarmason1-0/+45
Since 4aab5b46f44 (Make read-cache.c "the_index" free., 2007-04-01) we've been undergoing a slow migration away from these macros, but haven't made much progress since f8adbec9fea (cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch, 2019-01-24). Let's move forward a bit by changing the users of those macros that are rare enough that we can convert them in one go, and then remove the compatibility shim. The only manual change to the C code here is to "cache.h", the rest is all the result of applying the new "index-compatibility.cocci". Even though it's a one-off, let's keep the coccinelle rules for now. We'll extend them in subsequent commits, and this will help anything that's in-flight or out-of-tree to migrate. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-07Makefile & test-tool: replace "DC_SHA1" variable with a "define"Ævar Arnfjörð Bjarmason1-2/+0
Address the root cause of technical debt we've been carrying since sha1collisiondetection was made the default in [1]. In a preceding commit we narrowly fixed a bug where the "DC_SHA1" variable would be unset (in combination with "NO_APPLE_COMMON_CRYPTO=" on OSX), even though we had the sha1collisiondetection library enabled. But the only reason we needed to have such a user-exposed knob went away with [1], and it's been doing nothing useful since then. We don't care if you define DC_SHA1=*, we only care that you don't ask for any other SHA-1 implementation. If it turns out that you didn't, we'll use sha1collisiondetection, whether you had "DC_SHA1" set or not. As a result of this being confusing we had e.g. [2] for cmake and the recent [3] for ci/lib.sh setting "DC_SHA1" explicitly, even though this was always a NOOP. A much simpler way to do this is to stop having the Makefile and CMakeLists.txt set "DC_SHA1" to be picked up by the test-lib.sh, let's instead add a trivial "test-tool sha1-is-sha1dc". It returns zero if we're using sha1collisiondetection, non-zero otherwise. 1. e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17) 2. c4b2f41b5f5 (cmake: support for testing git with ctest, 2020-06-26) 3. 1ad5c3df35a (ci: use DC_SHA1=YesPlease on osx-clang job for CI, 2022-10-20) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-02spatchcache: add a ccache-alike for "spatch"Ævar Arnfjörð Bjarmason2-0/+324
Add a rather trivial "spatchcache", with this running e.g.: make cocciclean make contrib/coccinelle/free.cocci.patch \ SPATCH=contrib/coccicheck/spatchcache \ SPATCH_FLAGS=--very-quiet Is cut down from ~20s to ~5s on my system. Much of that is either fixable shell overhead, or the around 40 files we "CANTCACHE" (see the implementation). This uses "redis" as a cache by default, but it's configurable. See the embedded documentation. This is *not* like ccache in that we won't cache failed spatch invocations, or those where spatch suggests changes for us. Those cases are so rare that I didn't think it was worth the bother, by far the most common case is that it has no suggested changes. We'll also refuse to cache any "spatch" invocation that has output on stderr, which means that "--very-quiet" must be added to "SPATCH_FLAGS". Because we narrow the cache to that we don't need to save away stdout, stderr & the exit code. We simply cache the cases where we had no suggested changes. Another benchmark is to compare this with the previous SPATCH_BATCH_SIZE=N, as noted in [1]. Before this (on my 8 core system) running: make clean; time make contrib/coccinelle/array.cocci.patch SPATCH_BATCH_SIZE=0 Would take 33s, but with the preceding changes running without this "spatchcache" is slightly slower, or around 35s: make clean; time make contrib/coccinelle/array.cocci.patch Now doing the same with SPATCH=contrib/coccinelle/spatchcache will take around 6s, but we'll need to compile the *.o files first to take full advantage of it (which can be fast with "ccache"): make clean; make; time make contrib/coccinelle/array.cocci.patch SPATCH=contrib/coccinelle/spatchcache 1. https://lore.kernel.org/git/YwdRqP1CyUAzCEn2@coredump.intra.peff.net/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-02cocci: run against a generated ALL.cocciÆvar Arnfjörð Bjarmason1-0/+13
The preceding commits to make the "coccicheck" target incremental made it slower in some cases. As an optimization let's not have the many=many mapping of <*.cocci>=<*.[ch]>, but instead concat the <*.cocci> into an ALL.cocci, and then run one-to-many ALL.cocci=<*.[ch]>. A "make coccicheck" is now around 2x as fast as it was on "master", and around 1.5x as fast as the preceding change to make the run incremental: $ git hyperfine -L rev origin/master,HEAD~,HEAD -p 'make clean' 'make coccicheck SPATCH=spatch COCCI_SOURCES="$(echo $(ls o*.c builtin/h*.c))"' -r 3 Benchmark 1: make coccicheck SPATCH=spatch COCCI_SOURCES="$(echo $(ls o*.c builtin/h*.c))"' in 'origin/master Time (mean ± σ): 4.258 s ± 0.015 s [User: 27.432 s, System: 1.532 s] Range (min … max): 4.241 s … 4.268 s 3 runs Benchmark 2: make coccicheck SPATCH=spatch COCCI_SOURCES="$(echo $(ls o*.c builtin/h*.c))"' in 'HEAD~ Time (mean ± σ): 5.365 s ± 0.079 s [User: 36.899 s, System: 1.810 s] Range (min … max): 5.281 s … 5.436 s 3 runs Benchmark 3: make coccicheck SPATCH=spatch COCCI_SOURCES="$(echo $(ls o*.c builtin/h*.c))"' in 'HEAD Time (mean ± σ): 2.725 s ± 0.063 s [User: 14.796 s, System: 0.233 s] Range (min … max): 2.667 s … 2.792 s 3 runs Summary 'make coccicheck SPATCH=spatch COCCI_SOURCES="$(echo $(ls o*.c builtin/h*.c))"' in 'HEAD' ran 1.56 ± 0.04 times faster than 'make coccicheck SPATCH=spatch COCCI_SOURCES="$(echo $(ls o*.c builtin/h*.c))"' in 'origin/master' 1.97 ± 0.05 times faster than 'make coccicheck SPATCH=spatch COCCI_SOURCES="$(echo $(ls o*.c builtin/h*.c))"' in 'HEAD~' This can be turned off with SPATCH_CONCAT_COCCI, but as the beneficiaries of "SPATCH_CONCAT_COCCI=" would mainly be those developing the *.cocci rules themselves, let's leave this optimization on by default. For more information see my "Optimizing *.cocci rules by concat'ing them" (<220901.8635dbjfko.gmgdl@evledraar.gmail.com>) on the cocci@inria.fr mailing list. This potentially changes the results of our *.cocci rules, but as noted in that discussion it should be safe for our use. We don't name rules, or if we do their names don't conflict across our *.cocci files. To the extent that we'd have any inter-dependencies between rules this doesn't make that worse, as we'd have them now if we ran "make coccicheck", applied the results, and would then have (due to hypothetical interdependencies) suggested changes on the subsequent "make coccicheck". Our "coccicheck-test" target makes use of the ALL.cocci when running tests, e.g. when testing unused.{c,out} we test it against ALL.cocci, not unused.cocci. We thus assert (to the extent that we have test coverage) that this concatenation doesn't change the expected results of running these rules. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-02cocci rules: remove <id>'s from rules that don't need themÆvar Arnfjörð Bjarmason4-4/+4
The <id> in the <rulename> part of the coccinelle syntax[1] is for our purposes there to declares if we have inter-dependencies between different rules. But such <id>'s must be unique within a given semantic patch file. As we'll be processing a concatenated version of our rules in the subsequent commit let's remove these names. They weren't being used for the semantic patches themselves, and equated to a short comment about the rule. Both the filename and context of the rules makes it clear what they're doing, so we're not gaining anything from keeping these. Retaining them goes against recommendations that "contrib/coccinelle/README" will be making in the subsequent commit. This leaves only one named rule in our sources, where it's needed for a "<id> <-> <extends> <id>" relationship: $ git -P grep '^@ ' -- contrib/coccinelle/ contrib/coccinelle/swap.cocci:@ swap @ contrib/coccinelle/swap.cocci:@ extends swap @ 1. https://coccinelle.gitlabpages.inria.fr/website/docs/main_grammar.html Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-02cocci: optimistically use COMPUTE_HEADER_DEPENDENCIESÆvar Arnfjörð Bjarmason1-0/+16
Improve the incremental rebuilding support of "coccicheck" by piggy-backing on the computed dependency information of the corresponding *.o file, rather than rebuilding all <RULE>/<FILE> pairs if either their corresponding file changes, or if any header changes. This in effect uses the same method that the "sparse" target was made to use in c234e8a0ecf (Makefile: make the "sparse" target non-.PHONY, 2021-09-23), except that the dependency on the *.o file isn't a hard one, we check with $(wildcard) if the *.o file exists, and if so we'll depend on it. This means that the common case of: make make coccicheck Will benefit from incremental rebuilding, now changing e.g. a header will only re-run "spatch" on those those *.c files that make use of it: By depending on the *.o we piggy-back on COMPUTE_HEADER_DEPENDENCIES. See c234e8a0ecf (Makefile: make the "sparse" target non-.PHONY, 2021-09-23) for prior art of doing that for the *.sp files. E.g.: make contrib/coccinelle/free.cocci.patch make -W column.h contrib/coccinelle/free.cocci.patch Will take around 15 seconds for the second command on my 8 core box if I didn't run "make" beforehand to create the *.o files. But around 2 seconds if I did and we have those "*.o" files. Notes about the approach of piggy-backing on *.o for dependencies: * It *is* a trade-off since we'll pay the extra cost of running the C compiler, but we're probably doing that anyway. The compiler is much faster than "spatch", so even though we need to re-compile the *.o to create the dependency info for the *.c for "spatch" it's faster (especially if using "ccache"). * There *are* use-cases where some would like to have *.o files around, but to have the "make coccicheck" ignore them. See: https://lore.kernel.org/git/20220826104312.GJ1735@szeder.dev/ For those users a: make make coccicheck SPATCH_USE_O_DEPENDENCIES= Will avoid considering the *.o files. * If that *.o file doesn't exist we'll depend on an intermediate file of ours which in turn depends on $(FOUND_H_SOURCES). This covers both an initial build, or where "coccicheck" is run without running "all" beforehand, and because we run "coccicheck" on e.g. files in compat/* that we don't know how to build unless the requisite flag was provided to the Makefile. Most of the runtime of "incremental" runs is now spent on various compat/* files, i.e. we conditionally add files to COMPAT_OBJS, and therefore conflate whether we *can* compile an object and generate dependency information for it with whether we'd like to link it into our binary. Before this change the distinction didn't matter, but now one way to make this even faster on incremental builds would be to peel those concerns apart so that we can see that e.g. compat/mmap.c doesn't depend on column.h. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-02cocci: make "coccicheck" rule incrementalÆvar Arnfjörð Bjarmason1-1/+1
Optimize the very slow "coccicheck" target to take advantage of incremental rebuilding, and fix outstanding dependency problems with the existing rule. The rule is now faster both on the initial run as we can make better use of GNU make's parallelism than the old ad-hoc combination of make's parallelism combined with $(SPATCH_BATCH_SIZE) and/or the "--jobs" argument to "spatch(1)". It also makes us *much* faster when incrementally building, it's now viable to "make coccicheck" as topic branches are merged down. The rule didn't use FORCE (or its equivalents) before, so a: make coccicheck make coccicheck Would report nothing to do on the second iteration. But all of our patch output depended on all $(COCCI_SOURCES) files, therefore e.g.: make -W grep.c coccicheck Would do a full re-run, i.e. a a change in a single file would force us to do a full re-run. The reason for this (not the initial rationale, but my analysis) is: * Since we create a single "*.cocci.patch+" we don't know where to pick up where we left off, or how to incrementally merge e.g. a "grep.c" change with an existing *.cocci.patch. * We've been carrying forward the dependency on the *.c files since 63f0a758a06 (add coccicheck make target, 2016-09-15) the rule was initially added as a sort of poor man's dependency discovery. As we don't include other *.c files depending on other *.c files has always been broken, as could be trivially demonstrated e.g. with: make coccicheck make -W strbuf.h coccicheck However, depending on the corresponding *.c files has been doing something, namely that *if* an API change modified both *.c and *.h files we'd catch the change to the *.h we care about via the *.c being changed. For API changes that happened only via *.h files we'd do the wrong thing before this change, but e.g. for function additions (not "static inline" ones) catch the *.h change by proxy. Now we'll instead: * Create a <RULE>/<FILE> pair in the .build directory, E.g. for swap.cocci and grep.c we'll create .build/contrib/coccinelle/swap.cocci.patch/grep.c. That file is the diff we'll apply for that <RULE>-<FILE> combination, if there's no changes to me made (the common case) it'll be an empty file. * Our generated *.patch file (e.g. contrib/coccinelle/swap.cocci.patch) is now a simple "cat $^" of all of all of the <RULE>/<FILE> files for a given <RULE>. In the case discussed above of "grep.c" being changed we'll do the full "cat" every time, so they resulting *.cocci.patch will always be correct and up-to-date, even if it's "incrementally updated". See 1cc0425a27c (Makefile: have "make pot" not "reset --hard", 2022-05-26) for another recent rule that used that technique. As before we'll: * End up generating a contrib/coccinelle/swap.cocci.patch, if we "fail" by creating a non-empty patch we'll still exit with a zero exit code. Arguably we should move to a more Makefile-native way of doing this, i.e. fail early, and if we want all of the "failed" changes we can use "make -k", but as the current "ci/run-static-analysis.sh" expects us to behave this way let's keep the existing behavior of exhaustively discovering all cocci changes, and only failing if spatch itself errors out. Further implementation details & notes: * Before this change running "make coccicheck" would by default end up pegging just one CPU at the very end for a while, usually as we'd finish whichever *.cocci rule was the most expensive. This could be mitigated by combining "make -jN" with SPATCH_BATCH_SIZE, see 960154b9c17 (coccicheck: optionally batch spatch invocations, 2019-05-06). There will be cases where getting rid of "SPATCH_BATCH_SIZE" makes things worse, but a from-scratch "make coccicheck" with the default of SPATCH_BATCH_SIZE=1 (and tweaking it doesn't make a difference) is faster (~3m36s v.s. ~3m56s) with this approach, as we can feed the CPU more work in a less staggered way. * Getting rid of "SPATCH_BATCH_SIZE" particularly helps in cases where the default of 1 yields parallelism under "make coccicheck", but then running e.g.: make -W contrib/coccinelle/swap.cocci coccicheck I.e. before that would use only one CPU core, until the user remembered to adjust "SPATCH_BATCH_SIZE" differently than the setting that makes sense when doing a non-incremental run of "make coccicheck". * Before the "make coccicheck" rule would have to clean "contrib/coccinelle/*.cocci.patch*", since we'd create "*+" and "*.log" files there. Now those are created in .build/contrib/coccinelle/, which is covered by the "cocciclean" rule already. Outstanding issues & future work: * We could get rid of "--all-includes" in favor of manually specifying a list of includes to give to "spatch(1)". As noted upthread of [1] a naïve removal of "--all-includes" will result in broken *.cocci patches, but if we know the exhaustive list of includes via COMPUTE_HEADER_DEPENDENCIES we don't need to re-scan for them, we could grab the headers to include from the .depend.d/<file>.o.d and supply them with the "--include" option to spatch(1).q 1. https://lore.kernel.org/git/87ft18tcog.fsf@evledraar.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-02cocci rules: remove unused "F" metavariable from pending ruleÆvar Arnfjörð Bjarmason1-1/+0
Fix an issue with a rule added in 9b45f499818 (object-store: prepare has_{sha1, object}_file to handle any repo, 2018-11-13). We've been spewing out this warning into our $@.log since that rule was added: warning: rule starting on line 21: metavariable F not used in the - or context code We should do a better job of scouring our coccinelle log files for such issues, but for now let's fix this as a one-off. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-10-30Merge branch 'pb/subtree-split-and-merge-after-squashing-tag-fix'Taylor Blau3-69/+184
A bugfix to "git subtree" in its split and merge features. * pb/subtree-split-and-merge-after-squashing-tag-fix: subtree: fix split after annotated tag was squashed merged subtree: fix squash merging after annotated tag was squashed merged subtree: process 'git-subtree-split' trailer in separate function subtree: use named variables instead of "$@" in cmd_pull subtree: define a variable before its first use in 'find_latest_squash' subtree: prefix die messages with 'fatal' subtree: add 'die_incompatible_opt' function to reduce duplication subtree: use 'git rev-parse --verify [--quiet]' for better error messages test-lib-functions: mark 'test_commit' variables as 'local'
2022-10-27Merge branch 'js/cmake-updates'Junio C Hamano1-8/+8
Update to build procedure with VS using CMake/CTest. * js/cmake-updates: cmake: increase time-out for a long-running test cmake: avoid editing t/test-lib.sh add -p: avoid ambiguous signed/unsigned comparison cmake: copy the merge tools for testing cmake: make it easier to diagnose regressions in CTest runs
2022-10-21subtree: fix split after annotated tag was squashed mergedPhilippe Blain3-9/+36
The previous commit fixed a failure in 'git subtree merge --squash' when the previous squash-merge merged an annotated tag of the subtree repository which is missing locally. The same failure happens in 'git subtree split', either directly or when called by 'git subtree push', under the same circumstances: 'cmd_split' invokes 'find_existing_splits', which loops through previous commits and invokes 'git rev-parse' (via 'process_subtree_split_trailer') on the value of any 'git subtree-split' trailer it finds. This fails if this value is the hash of an annotated tag which is missing locally. Add a new optional argument 'repository' to 'cmd_split' and 'find_existing_splits', and invoke 'cmd_split' with that argument from 'cmd_push'. This allows 'process_subtree_split_trailer' to try to fetch the missing tag from the 'repository' if it's not available locally, mirroring the new behaviour of 'git subtree pull' and 'git subtree merge'. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-21subtree: fix squash merging after annotated tag was squashed mergedPhilippe Blain3-15/+86
When 'git subtree merge --squash $ref' is invoked, either directly or through 'git subtree pull --squash $repo $ref', the code looks for the latest squash merge of the subtree in order to create the new merge commit as a child of the previous squash merge. This search is done in function 'process_subtree_split_trailer', invoked by 'find_latest_squash', which looks for the most recent commit with a 'git-subtree-split' trailer; that trailer's value is the object name in the subtree repository of the ref that was last squash-merged. The function verifies that this object is present locally with 'git rev-parse', and aborts if it's not. The hash referenced by the 'git-subtree-split' trailer is guaranteed to correspond to a commit since it is the result of running 'git rev-parse -q --verify "$1^{commit}"' on the first argument of 'cmd_merge' (this corresponds to 'rev' in 'cmd_merge' which is passed through to 'new_squash_commit' and 'squash_msg'). But this is only the case since e4f8baa88a (subtree: parse revs in individual cmd_ functions, 2021-04-27), which went into Git 2.32. Before that commit, 'cmd_merge' verified the revision it was given using 'git rev-parse --revs-only "$@"'. Such an invocation, when fed the name of an annotated tag, would return the hash of the tag, not of the commit referenced by the tag. This leads to a failure in 'find_latest_squash' when squash-merging if the most recent squash-merge merged an annotated tag of the subtree repository, using a pre-2.32 version of 'git subtree', unless that previous annotated tag is present locally (which is not usually the case). We can fix this by fetching the object directly by its hash in 'process_subtree_split_trailer' when 'git rev-parse' fails, but in order to do so we need to know the name or URL of the subtree repository. This is not possible in general for 'git subtree merge', but is easy when it is invoked through 'git subtree pull' since in that case the subtree repository is passed by the user at the command line. Allow the 'git subtree pull' scenario to work out-of-the-box by adding an optional 'repository' argument to functions 'cmd_merge', 'find_latest_squash' and 'process_subtree_split_trailer', and invoke 'cmd_merge' with that 'repository' argument in 'cmd_pull'. If 'repository' is absent in 'process_subtree_split_trailer', instruct the user to try fetching the missing object directly. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-21subtree: process 'git-subtree-split' trailer in separate functionPhilippe Blain1-4/+11
Both functions 'find_latest_squash' (called by 'git subtree merge --squash' and 'git subtree split --rejoin') and 'find_existing_splits' (called by git 'subtree split') loop through commits that have a 'git-subtree-dir' trailer, and then process the 'git-subtree-mainline' and 'git-subtree-split' trailers for those commits. The processing done for the 'git-subtree-split' trailer is simple: we check if the object exists with 'rev-parse' and set the variable 'sub' to the object name, or we die if the object does not exist. In a future commit we will add more steps to the processing of this trailer in order to make the code more robust. To reduce code duplication, move the processing of the 'git-subtree-split' trailer to a dedicated function, 'process_subtree_split_trailer'. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-21subtree: use named variables instead of "$@" in cmd_pullPhilippe Blain1-2/+4
'cmd_pull' already checks that only two arguments are given, 'repository' and 'ref'. Define variables with these names instead of using the positional parameter $2 and "$@". This will allow a subsequent commit to pass 'repository' to 'cmd_merge'. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-21subtree: define a variable before its first use in 'find_latest_squash'Philippe Blain1-1/+1
The function 'find_latest_squash' takes a single argument, 'dir', but a debug statement uses this variable before it takes its value from $1. This statement thus gets the value of 'dir' from the calling function, which currently is the same as the 'dir' argument, so it works but it is confusing. Move the definition of 'dir' before its first use. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-21subtree: prefix die messages with 'fatal'Philippe Blain2-36/+36
Just as was done in 0008d12284 (submodule: prefix die messages with 'fatal', 2021-07-10) for 'git-submodule.sh', make the 'die' messages outputed by 'git-subtree.sh' more in line with the rest of the code base by prefixing them with "fatal: ", and do not capitalize their first letter. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-21subtree: add 'die_incompatible_opt' function to reduce duplicationPhilippe Blain1-12/+20
9a3e3ca2ba (subtree: be stricter about validating flags, 2021-04-27) added validation code to check that options given to 'git subtree <cmd>' made sense with the command being used. Refactor these checks by adding a 'die_incompatible_opt' function to reduce code duplication. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-21subtree: use 'git rev-parse --verify [--quiet]' for better error messagesPhilippe Blain1-3/+3
There are three occurences of 'git rev-parse <rev>' in 'git-subtree.sh' where the command expects a revision and the script dies or exits if the revision can't be found. In that case, the error message from 'git rev-parse' is: $ git rev-parse <bad rev> <bad rev> fatal: ambiguous argument '<bad rev>': unknown revision or path not in the working tree. Use '--' to separate paths from revisions, like this: 'git <command> [<revision>...] -- [<file>...]' This is a little confusing to the user, since this error message is outputed by 'git subtree'. At these points in the script, we know that we are looking for a single revision, so be explicit by using '--verify', resulting in a little better error message: $ git rev-parse --verify <bad rev> fatal: Needed a single revision In the two occurences where we 'die' if 'git rev-parse' fails, 'git subtree' outputs "could not rev-parse split hash $b from commit $sq", so we actually do not need the supplementary error message from 'git rev-parse'; add '--quiet' to silence it. In the third occurence, we 'exit', so keep the error message from 'git rev-parse'. Note that this messsage is still suboptimal since it can be understood to mean that 'git rev-parse' did not receive a single revision as argument, which is not the case here: the command did receive a single revision, but the revision is not resolvable to an available object. The alternative would be to use '--' after the revision, as suggested by the first error message, resulting in a clearer error message: $ git rev-parse <bad rev> -- fatal: bad revision '<bad rev>' Unfortunately we can't use that syntax because in the more common case of the revision resolving to a known object, the command outputs the object's hash, a newline, and the dashdash, which breaks the 'git subtree' script. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-19cmake: increase time-out for a long-running testJohannes Schindelin1-0/+4
As suggested in https://github.com/git-for-windows/git/issues/3966#issuecomment-1221264238, t7112 can run for well over one hour, which seems to be the default maximum run time at least when running CTest-based tests in Visual Studio. Let's increase the time-out as a stop gap to unblock developers wishing to run Git's test suite in Visual Studio. Note: The actual run time is highly dependent on the circumstances. For example, in Git's CI runs, the Windows-based tests typically take a bit over 5 minutes to run. CI runs have the added benefit that Windows Defender (the common anti-malware scanner on Windows) is turned off, something many developers are not at liberty to do on their work stations. When Defender is turned on, even on this developer's high-end Ryzen system, t7112 takes over 15 minutes to run. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-19cmake: avoid editing t/test-lib.shJohannes Schindelin1-6/+1
In 7f5397a07c6c (cmake: support for testing git when building out of the source tree, 2020-06-26), we implemented support for running Git's test scripts even after building Git in a different directory than the source directory. The way we did this was to edit the file `t/test-lib.sh` to override `GIT_BUILD_DIR` to point somewhere else than the parent of the `t/` directory. This is unideal because it always leaves a tracked file marked as modified, and it is all too easy to commit that change by mistake. Let's change the strategy by teaching `t/test-lib.sh` to detect the presence of a file called `GIT-BUILD-DIR` in the source directory. If it exists, the contents are interpreted as the location to the _actual_ build directory. We then write this file as part of the CTest definition. To support building Git via a regular `make` invocation after building it using CMake, we ensure that the `GIT-BUILD-DIR` file is deleted (for convenience, this is done as part of the Makefile rule that is already run with every `make` invocation to ensure that `GIT-BUILD-OPTIONS` is up to date). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-19cmake: copy the merge tools for testingJohannes Schindelin1-1/+2
Even when running the tests via CTest, t7609 and t7610 rely on more than only a few mergetools to be copied to the build directory. Let's make it so. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-19cmake: make it easier to diagnose regressions in CTest runsJohannes Schindelin1-1/+1
When a test script fails in Git's test suite, the usual course of action is to re-run it using options to increase the verbosity of the output, e.g. `-v` and `-x`. Like in Git's CI runs, when running the tests in Visual Studio via the CTest route, it is cumbersome or at least requires a very unintuitive approach to pass options to the test scripts: the CMakeLists.txt file would have to be modified, passing the desired options to _all_ test scripts, and then the CMake Cache would have to be reconfigured before running the test in question individually. Unintuitive at best, and opposite to the niceties IDE users expect. So let's just pass those options by default: This will not clutter any output window but the log that is written to a log file will have information necessary to figure out test failures. While at it, also imitate what the Windows jobs in Git's CI runs do to accelerate running the test scripts: pass the `--no-bin-wrappers` and `--no-chain-lint` options. This makes the test runs noticeably faster because the `bin-wrappers/` scripts as well as the `chain-lint` code make heavy use of POSIX shell scripting, which is really, really slow on Windows due to the need to emulate POSIX behavior via the MSYS2 runtime. In a test by Eric Sunshine, it added two minutes (!) just to perform the chain-lint task. The idea of adding a CMake config option (á la `GIT_TEST_OPTS`) was considered during the development of this patch, but then dropped: such a setting is global, across _all_ tests, where e.g. `--run=...` would not make sense. Users wishing to override these new defaults are better advised running the test script manually, in a Git Bash, with full control over the command line. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-17Merge branch 'ed/fsmonitor-on-networked-macos'Junio C Hamano1-0/+4
By default, use of fsmonitor on a repository on networked filesystem is disabled. Add knobs to make it workable on macOS. * ed/fsmonitor-on-networked-macos: fsmonitor: fix leak of warning message fsmonitor: add documentation for allowRemote and socketDir options fsmonitor: check for compatability before communicating with fsmonitor fsmonitor: deal with synthetic firmlinks on macOS fsmonitor: avoid socket location check if using hook fsmonitor: relocate socket file if .git directory is remote fsmonitor: refactor filesystem checks to common interface
2022-10-05fsmonitor: relocate socket file if .git directory is remoteEric DeCosta1-0/+2
If the .git directory is on a remote filesystem, create the socket file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME. Signed-off-by: Eric DeCosta <edecosta@mathworks.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-05fsmonitor: refactor filesystem checks to common interfaceEric DeCosta1-0/+2
Provide a common interface for getting basic filesystem information including filesystem type and whether the filesystem is remote. Refactor existing code for getting basic filesystem info and detecting remote file systems to the new interface. Refactor filesystem checks to leverage new interface. For macOS, error-out if the Unix Domain socket (UDS) file is on a remote filesystem. Signed-off-by: Eric DeCosta <edecosta@mathworks.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-09-22osxkeychain: clarify that we ignore unknown linesMatthew John Cheetham1-0/+5
Like in all the other credential helpers, the osxkeychain helper ignores unknown credential lines. Add a comment (a la the other helpers) to make it clear and explicit that this is the desired behaviour. Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-09-22netrc: ignore unknown lines (do not die)Matthew John Cheetham1-1/+4
Contrary to the documentation on credential helpers, as well as the help text for git-credential-netrc itself, this helper will `die` when presented with an unknown property/attribute/token. Correct the behaviour here by skipping and ignoring any tokens that are unknown. This means all helpers in the tree are consistent and ignore any unknown credential properties/attributes. Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>