aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2021-07-02perf: fix when running with TEST_OUTPUT_DIRECTORYPatrick Steinhardt3-13/+24
When the TEST_OUTPUT_DIRECTORY is defined, then all test data will be written in that directory instead of the default directory located in "t/". While this works as expected for our normal tests, performance tests fail to locate and aggregate performance data because they don't know to handle TEST_OUTPUT_DIRECTORY correctly and always look at the default location. Fix the issue by adding a `--results-dir` parameter to "aggregate.perl" which identifies the directory where results are and by making the "run" script awake of the TEST_OUTPUT_DIRECTORY variable. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-14The second batchJunio C Hamano1-2/+48
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-14Merge branch 'fc/doc-build-cleanup'Junio C Hamano1-48/+29
Preparatory build procedure clean-up for documentation. * fc/doc-build-cleanup: doc: avoid using rm directly doc: simplify Makefile using .DELETE_ON_ERROR doc: remove unnecessary rm instances doc: improve asciidoc dependencies doc: refactor common asciidoc dependencies
2021-06-14Merge branch 'ab/test-lib-updates'Junio C Hamano14-119/+89
Test clean-up. * ab/test-lib-updates: test-lib: split up and deprecate test_create_repo() test-lib: do not show advice about init.defaultBranch under --verbose test-lib: reformat argument list in test_create_repo() submodule tests: use symbolic-ref --short to discover branch name test-lib functions: add --printf option to test_commit describe tests: convert setup to use test_commit test-lib functions: add an --annotated option to "test_commit" test-lib-functions: document test_commit --no-tag test-lib-functions: reword "test_commit --append" docs test-lib tests: remove dead GIT_TEST_FRAMEWORK_SELFTEST variable test-lib: bring $remove_trash out of retirement
2021-06-14Merge branch 'dd/honor-users-tar-in-tests'Junio C Hamano2-4/+4
Test portability fix. * dd/honor-users-tar-in-tests: t: use configured TAR instead of tar
2021-06-14Merge branch 'ps/rev-list-object-type-filter'Junio C Hamano1-1/+1
Message update. * ps/rev-list-object-type-filter: help: fix small typo in error message
2021-06-14Merge branch 'ab/trace2-squelch-gcc-warning'Junio C Hamano1-10/+8
Workaround compiler warnings. * ab/trace2-squelch-gcc-warning: trace2: refactor to avoid gcc warning under -O3
2021-06-14Merge branch 'so/log-m-implies-p'Junio C Hamano12-25/+200
The "-m" option in "git log -m" that does not specify which format, if any, of diff is desired did not have any visible effect; it now implies some form of diff (by default "--patch") is produced. * so/log-m-implies-p: diff-merges: let "-m" imply "-p" diff-merges: rename "combined_imply_patch" to "merges_imply_patch" stash list: stop passing "-m" to "git log" git-svn: stop passing "-m" to "git rev-list" diff-merges: move specific diff-index "-m" handling to diff-index t4013: test "git diff-index -m" t4013: test "git diff-tree -m" t4013: test "git log -m --stat" t4013: test "git log -m --raw" t4013: test that "-m" alone has no effect in "git log"
2021-06-14Merge branch 'en/ort-perf-batch-11'Junio C Hamano8-37/+1804
Optimize out repeated rename detection in a sequence of mergy operations. * en/ort-perf-batch-11: merge-ort, diffcore-rename: employ cached renames when possible merge-ort: handle interactions of caching and rename/rename(1to1) cases merge-ort: add helper functions for using cached renames merge-ort: preserve cached renames for the appropriate side merge-ort: avoid accidental API mis-use merge-ort: add code to check for whether cached renames can be reused merge-ort: populate caches of rename detection results merge-ort: add data structures for in-memory caching of rename detection t6429: testcases for remembering renames fast-rebase: write conflict state to working tree, index, and HEAD fast-rebase: change assert() to BUG() Documentation/technical: describe remembering renames optimization t6423: rename file within directory that other side renamed
2021-06-14Merge branch 'jk/fetch-pack-v2-half-close-early'Junio C Hamano2-2/+13
"git fetch" over protocol v2 left its side of the socket open after it finished speaking, which unnecessarily wasted the resource on the other side. * jk/fetch-pack-v2-half-close-early: fetch-pack: signal v2 server that we are done making requests
2021-06-14Merge branch 'ds/write-index-with-hashfile-api'Junio C Hamano4-167/+134
Use the hashfile API in the codepath that writes the index file to reduce code duplication. * ds/write-index-with-hashfile-api: read-cache: delete unused hashing methods read-cache: use hashfile instead of git_hash_ctx csum-file.h: increase hashfile buffer size hashfile: use write_in_full()
2021-06-14Merge branch 'jk/clone-clean-upon-transport-error'Junio C Hamano2-7/+11
Recent "git clone" left a temporary directory behind when the transport layer returned an failure. * jk/clone-clean-upon-transport-error: clone: clean up directory after transport_fetch_refs() failure
2021-06-14Merge branch 'ga/send-email-sendmail-cmd'Junio C Hamano3-14/+76
"git send-email" learned the "--sendmail-cmd" command line option and the "sendemail.sendmailCmd" configuration variable, which is a more sensible approach than the current way of repurposing the "smtp-server" that is meant to name the server to instead name the command to talk to the server. * ga/send-email-sendmail-cmd: git-send-email: add option to specify sendmail command
2021-06-14Merge branch 'zh/ref-filter-atom-type'Junio C Hamano1-74/+140
The code to handle the "--format" option in "for-each-ref" and friends made too many string comparisons on %(atom)s used in the format string, which has been corrected by converting them into enum when the format string is parsed. * zh/ref-filter-atom-type: ref-filter: introduce enum atom_type ref-filter: add objectsize to used_atom
2021-06-10The first batch post Git 2.32Junio C Hamano2-1/+33
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-10Merge branch 'ah/setup-extensions-message-i18n-fix'Junio C Hamano1-2/+6
Message update. * ah/setup-extensions-message-i18n-fix: setup: split "extensions found" messages into singular and plural
2021-06-10Merge branch 'ah/fetch-reject-warning-grammofix'Junio C Hamano1-1/+1
Message update. * ah/fetch-reject-warning-grammofix: fetch: improve grammar of "shallow roots" message
2021-06-10Merge branch 'jk/doc-color-pager'Junio C Hamano1-2/+3
The documentation for "color.pager" configuration variable has been updated. * jk/doc-color-pager: doc: explain the use of color.pager
2021-06-10Merge branch 'tl/fix-packfile-uri-doc'Junio C Hamano1-7/+8
Doc fix. * tl/fix-packfile-uri-doc: packfile-uri.txt: fix blobPackfileUri description
2021-06-10Merge branch 'ry/clarify-fast-forward-in-glossary'Junio C Hamano1-2/+2
The description of "fast-forward" in the glossary has been updated. * ry/clarify-fast-forward-in-glossary: docs: improve fast-forward in glossary content
2021-06-10Merge branch 'wm/rev-parse-die-i18n'Junio C Hamano1-15/+15
Quite a many die() messages in rev-parse haven't been marked for translation. * wm/rev-parse-die-i18n: rev-parse: mark die() messages for translation
2021-06-10Merge branch 'jc/clarify-revision-range'Junio C Hamano1-0/+23
Doc update. * jc/clarify-revision-range: revisions(7): clarify that most commands take a single revision range
2021-06-10Merge branch 'ah/doc-describe'Junio C Hamano1-5/+9
Doc update. * ah/doc-describe: describe-doc: clarify default length of abbreviation
2021-06-10Merge branch 'ah/submodule-helper-module-summary-parseopt'Junio C Hamano1-1/+1
Message update. * ah/submodule-helper-module-summary-parseopt: submodule: use the imperative mood to describe the --files option
2021-06-10Merge branch 'ah/stash-usage-i18n-fix'Junio C Hamano1-2/+2
i18n update. * ah/stash-usage-i18n-fix: stash: don't translate literal commands
2021-06-10Merge branch 'ah/merge-usage-i18n-fix'Junio C Hamano1-2/+2
i18n update. * ah/merge-usage-i18n-fix: merge: don't translate literal commands
2021-06-10Merge branch 'jn/size-t-casted-to-off-t-fix'Junio C Hamano1-4/+2
Rewrite code that triggers undefined behaiour warning. * jn/size-t-casted-to-off-t-fix: xsize_t: avoid implementation defined behavior when len < 0
2021-06-10Merge branch 'mt/parallel-checkout-with-padded-oidcpy'Junio C Hamano2-7/+22
The parallel checkout codepath did not initialize object ID field used to talk to the worker processes in a futureproof way. * mt/parallel-checkout-with-padded-oidcpy: parallel-checkout: send the new object_id algo field to the workers
2021-06-10Merge branch 'ef/mailinfo-short-name'Junio C Hamano1-1/+1
We historically rejected a very short string as an author name while accepting a patch e-mail, which has been loosened. * ef/mailinfo-short-name: mailinfo: don't discard names under 3 characters
2021-06-06Git 2.32v2.32.0Junio C Hamano1-1/+1
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-06Merge tag 'l10n-2.32.0-rnd1.1' of git://github.com/git-l10n/git-poJunio C Hamano16-44421/+49753
l10n-2.32.0-rnd1.1 * tag 'l10n-2.32.0-rnd1.1' of git://github.com/git-l10n/git-po: (25 commits) l10n: es: 2.32.0 round 1 l10n: zh_CN: for git v2.32.0 l10n round 1 l10n: Update Catalan translation l10n: de.po: Update German translation for Git v2.32.0 l10n: README: note on fuzzy translations l10n: README: document l10n conventions l10n: README: document "core translation" l10n: README: document git-po-helper l10n: README: add file extention ".md" l10n: pt_PT: add Portuguese translations part 3 l10n: bg.po: Updated Bulgarian translation (5204t) l10n: id: po-id for 2.32.0 (round 1) l10n: vi.po(5204t): Updated Vietnamese translation for v2.32.0 l10n: zh_TW.po: localized l10n: zh_TW.po: v2.32.0 round 1 (11 untranslated) l10n: sv.po: Update Swedish translation (5204t0f0u) l10n: fix typos in po/TEAMS l10n: fr: v2.32.0 round 1 l10n: tr: v2.32.0-r1 l10n: fr: fixed inconsistencies ...
2021-06-06Merge branch 'rs/parallel-checkout-test-fix'Junio C Hamano1-1/+1
Test fix. * rs/parallel-checkout-test-fix: parallel-checkout: avoid dash local bug in tests
2021-06-06Merge branch 'jc/fsync-can-fail-with-eintr'Junio C Hamano1-2/+3
Last minute portability fix. * jc/fsync-can-fail-with-eintr: fsync(): be prepared to see EINTR
2021-06-06parallel-checkout: avoid dash local bug in testsRené Scharfe1-1/+1
Dash bug https://bugs.launchpad.net/ubuntu/+source/dash/+bug/139097 lets the shell erroneously perform field splitting on the expansion of a command substitution during declaration of a local variable. It causes the parallel-checkout tests to fail e.g. when running them with /bin/dash on MacOS 11.4, where they error out like this: ./t2080-parallel-checkout-basics.sh: 33: local: 0: bad variable name That's because the output of wc -l contains leading spaces and the returned number of lines is treated as another variable to declare, i.e. as in "local workers= 0". Work around it by enclosing the command substitution in quotes. Helped-by: Matheus Tavares Bernardino <matheus.bernardino@usp.br> Helped-by: SZEDER Gábor <szeder.dev@gmail.com> Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-05l10n: es: 2.32.0 round 1Christopher Diaz Riveros1-3300/+3806
Signed-off-by: Christopher Diaz Riveros <christopher.diaz.riv@gmail.com>
2021-06-05l10n: zh_CN: for git v2.32.0 l10n round 1Jiang Xin1-3207/+3655
Translate 126 new messages (5204t0f0u) for git 2.32.0. Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
2021-06-05Merge branch 'fix_typo' of github.com:e-yes/gitJiang Xin1-1/+1
* 'fix_typo' of github.com:e-yes/git: l10n: ru.po: fix typo in Russian translation
2021-06-05fsync(): be prepared to see EINTRJunio C Hamano1-2/+3
Some platforms, like NonStop do not automatically restart fsync() when interrupted by a signal, even when that signal is setup with SA_RESTART. This can lead to test breakage, e.g., where "--progress" is used, thus SIGALRM is sent often, and can interrupt an fsync() syscall. Make sure we deal with such a case by retrying the syscall ourselves. Luckily, we call fsync() fron a single wrapper, fsync_or_die(), so the fix is fairly isolated. Reported-by: Randall S. Becker <randall.becker@nexbridge.ca> Helped-by: Jeff King <peff@peff.net> Helped-by: Taylor Blau <me@ttaylorr.com> [jc: the above two did most of the work---I just tied the loose end] Helped-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-04Merge branch 'pt-PT' of github.com:git-l10n-pt-PT/git-poJiang Xin1-635/+602
* 'pt-PT' of github.com:git-l10n-pt-PT/git-po: l10n: pt_PT: add Portuguese translations part 3 l10n: pt_PT: add Portuguese translations part 2
2021-06-04l10n: Update Catalan translationJordi Mas1-4743/+5714
Signed-off-by: Jordi Mas <jmas@softcatala.org>
2021-06-02l10n: de.po: Update German translation for Git v2.32.0Matthias Rüster1-3239/+3717
Reviewed-by: Ralf Thielow <ralf.thielow@gmail.com> Signed-off-by: Matthias Rüster <matthias.ruester@gmail.com>
2021-06-02Git 2.32-rc3v2.32.0-rc3Junio C Hamano1-1/+1
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-02contrib/completion: fix zsh completion regression from 59d85a2a05David Aguilar2-2/+2
A recent change to make git-completion.bash use $__git_cmd_idx in more places broke a number of completions on zsh because it modified __git_main but did not update __git_zsh_main. Notably, completions for "add", "branch", "mv" and "push" were broken as a result of this change. In addition to the undefined variable usage, "git mv <tab>" also prints the following error: __git_count_arguments:7: bad math expression: operand expected at `"1"' _git_mv:[:7: unknown condition: -gt Remove the quotes around $__git_cmd_idx in __git_count_arguments and set __git_cmd_idx=1 early in __git_zsh_main to fix the regressions from 59d85a2a05. This was tested on zsh 5.7.1 (x86_64-apple-darwin19.0). Suggested-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: David Aguilar <davvid@gmail.com> Acked-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-02l10n: README: note on fuzzy translationsBagas Sanjaya1-0/+12
Fuzzy translation problem can occur when updating translations. Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com> Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
2021-06-02l10n: README: document l10n conventionsBagas Sanjaya1-0/+46
Document the conventions that l10n contributors must follow. Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com> Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
2021-06-02l10n: README: document "core translation"Bagas Sanjaya1-0/+24
Contributor for a new language must complete translations of a small set of l10n messages. Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com> Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
2021-06-02l10n: README: document git-po-helperBagas Sanjaya1-0/+31
Document the PO helper program (git-po-helper) with installation and basic usage. Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com> Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
2021-06-02l10n: README: add file extention ".md"Jiang Xin1-0/+0
Add file extension ".md" to "po/README" to help to display this markdown file properly. Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
2021-06-02Merge branch 'ab/fsck-api-cleanup'Junio C Hamano1-1/+2
Last minute compilation fix. * ab/fsck-api-cleanup: builtin/fsck.c: don't conflate "int" and "enum" in callback
2021-06-02builtin/fsck.c: don't conflate "int" and "enum" in callbackÆvar Arnfjörð Bjarmason1-1/+2
Fix a warning on AIX's xlc compiler that's been emitted since my a1aad71601a (fsck.h: use "enum object_type" instead of "int", 2021-03-28): "builtin/fsck.c", line 805.32: 1506-068 (W) Operation between types "int(*)(struct object*,enum object_type,void*,struct fsck_options*)" and "int(*)(struct object*,int,void*,struct fsck_options*)" is not allowed. I.e. it complains about us assigning a function with a prototype "int" where we're expecting "enum object_type". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-01l10n: pt_PT: add Portuguese translations part 3Daniel Santos1-14/+14
* Correct malformed strings * Transforming 'não' (no) into affirmative Signed-off-by: Daniel Santos <hello@brighterdan.com>
2021-05-30Merge branch 'l10n/zh_TW/21-05-20' of github.com:l10n-tw/git-poJiang Xin1-3310/+3794
* 'l10n/zh_TW/21-05-20' of github.com:l10n-tw/git-po: l10n: zh_TW.po: localized l10n: zh_TW.po: v2.32.0 round 1 (11 untranslated)
2021-05-30Merge branch 'master' of github.com:Softcatala/git-poJiang Xin1-5/+4
* 'master' of github.com:Softcatala/git-po: l10n: Update Catalan translation
2021-05-28l10n: bg.po: Updated Bulgarian translation (5204t)Alexander Shopov1-3309/+3791
Signed-off-by: Alexander Shopov <ash@kambanaria.org>
2021-05-28Git 2.32-rc2v2.32.0-rc2Junio C Hamano1-1/+1
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-28Merge branch 'en/dir-traversal'Junio C Hamano1-2/+1
Fix-up to a topic that is already in 'master'. * en/dir-traversal: dir: introduce readdir_skip_dot_and_dotdot() helper dir: update stale description of treat_directory() Revert "dir: update stale description of treat_directory()" Revert "dir: introduce readdir_skip_dot_and_dotdot() helper"
2021-05-27dir: introduce readdir_skip_dot_and_dotdot() helperElijah Newren11-45/+30
Many places in the code were doing while ((d = readdir(dir)) != NULL) { if (is_dot_or_dotdot(d->d_name)) continue; ...process d... } Introduce a readdir_skip_dot_and_dotdot() helper to make that a one-liner: while ((d = readdir_skip_dot_and_dotdot(dir)) != NULL) { ...process d... } This helper particularly simplifies checks for empty directories. Also use this helper in read_cached_dir() so that our statistics are consistent across platforms. (In other words, read_cached_dir() should have been using is_dot_or_dotdot() and skipping such entries, but did not and left it to treat_path() to detect and mark such entries as path_none.) Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-27dir: update stale description of treat_directory()Derrick Stolee1-7/+6
The documentation comment for treat_directory() was originally written in 095952 (Teach directory traversal about subprojects, 2007-04-11) which was before the 'struct dir_struct' split its bitfield of named options into a 'flags' enum in 7c4c97c0 (Turn the flags in struct dir_struct into a single variable, 2009-02-16). When those flags changed, the comment became stale, since members like 'show_other_directories' transitioned into flags like DIR_SHOW_OTHER_DIRECTORIES. Update the comments for treat_directory() to use these flag names rather than the old member names. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-27Revert "dir: update stale description of treat_directory()"Junio C Hamano1-6/+7
This reverts commit 4e689d81718eb6e939cace317ea3e33cb994dcbb, to be replaced with a reworked version.
2021-05-27Revert "dir: introduce readdir_skip_dot_and_dotdot() helper"Junio C Hamano11-31/+45
This reverts commit b548f0f1568f6b01e55ca69c24d3cb19489f92aa, to be replaced with a reworked version.
2021-05-27Merge branch 'ab/pack-linkage-fix'Junio C Hamano2-166/+167
"ld" on Solaris fails to link some test helpers, which has been worked around by reshuffling the inline function definitions from a header file to a source file that is the only user of them. * ab/pack-linkage-fix: pack-objects: move static inline from a header to the sole consumer
2021-05-27Merge branch 'mt/t2080-cp-symlink-fix'Junio C Hamano1-1/+1
Test portability fix. * mt/t2080-cp-symlink-fix: t2080: fix cp invocation to copy symlinks instead of following them
2021-05-27Merge branch 'ab/send-email-inline-hooks-path'Junio C Hamano3-17/+6
Code simplification. * ab/send-email-inline-hooks-path: send-email: move "hooks_path" invocation to git-send-email.perl send-email: don't needlessly abs_path() the core.hooksPath
2021-05-27Merge branch 'ds/t1092-fix-flake-from-progress'Junio C Hamano1-3/+3
Workaround flaky tests introduced recently. * ds/t1092-fix-flake-from-progress: t1092: revert the "-1" hack for emulating "no progress meter" t1092: use GIT_PROGRESS_DELAY for consistent results
2021-05-27pack-objects: move static inline from a header to the sole consumerÆvar Arnfjörð Bjarmason2-166/+167
Move the code that is only used in builtin/pack-objects.c out of pack-objects.h. This fixes an issue where Solaris's SunCC hasn't been able to compile git since 483fa7f42d9 (t/helper/test-bitmap.c: initial commit, 2021-03-31). The real origin of that issue is that in 898eba5e630 (pack-objects: refer to delta objects by index instead of pointer, 2018-04-14) utility functions only needed by builtin/pack-objects.c were added to pack-objects.h. Since then the header has been used in a few other places, but 483fa7f42d9 was the first time it was used by test helper. Since Solaris is stricter about linking and the oe_get_size_slow() function lives in builtin/pack-objects.c the build started failing with: Undefined first referenced symbol in file oe_get_size_slow t/helper/test-bitmap.o ld: fatal: symbol referencing errors. No output written to t/helper/test-tool On other platforms this is presumably OK because the compiler and/or linker detects that the "static inline" functions that reference oe_get_size_slow() aren't used. Let's solve this by moving the relevant code from pack-objects.h to builtin/pack-objects.c. This is almost entirely a code-only move, but because of the early macro definitions in that file referencing some of these inline functions we need to move the definition of "static struct packing_data to_pack" earlier, and declare these inline functions above the macros. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-27Merge branch 'fr_next' of github.com:jnavila/gitJiang Xin1-5528/+4576
* 'fr_next' of github.com:jnavila/git: l10n: fr: v2.32.0 round 1 l10n: fr: fixed inconsistencies l10n: fr.po fixed inconsistencies
2021-05-27t2080: fix cp invocation to copy symlinks instead of following themMatheus Tavares1-1/+1
t2080 makes a few copies of a test repository and later performs a branch switch on each one of the copies to verify that parallel checkout and sequential checkout produce the same results. However, the repository is copied with `cp -R` which, on some systems, defaults to following symlinks on the directory hierarchy and copying their target files instead of copying the symlinks themselves. AIX is one example of system where this happens. Because the symlinks are not preserved, the copied repositories have paths that do not match what is in the index, causing git to abort the checkout operation that we want to test. This makes the test fail on these systems. Fix this by copying the repository with the POSIX flag '-P', which forces cp to copy the symlinks instead of following them. Note that we already use this flag for other cp invocations in our test suite (see t7001). With this change, t2080 now passes on AIX. Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-27send-email: move "hooks_path" invocation to git-send-email.perlÆvar Arnfjörð Bjarmason2-13/+2
Move the newly added "hooks_path" API in Git.pm to its only user in git-send-email.perl. This was added in c8243933c74 (git-send-email: Respect core.hooksPath setting, 2021-03-23), meaning that it hasn't yet made it into a non-rc release of git. The consensus with Git.pm is that we need to be considerate of out-of-tree users who treat it as a public documented interface. We should therefore be less willing to add new functionality to it, least we be stuck supporting it after our own uses for it disappear. In this case the git-send-email.perl hook invocation will probably be replaced by a future "git hook run" command, and in the commit preceding this one the "hooks_path" become nothing but a trivial wrapper for "rev-parse --git-path hooks" anyway (with no Cwd::abs_path() call), so let's just inline this command in git-send-email.perl itself. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-27send-email: don't needlessly abs_path() the core.hooksPathÆvar Arnfjörð Bjarmason2-5/+5
In c8243933c74 (git-send-email: Respect core.hooksPath setting, 2021-03-23) we started supporting core.hooksPath in "send-email". It's been reported that on Windows[1] doing this by calling abs_path() results in different canonicalizations of the absolute path. This wasn't an issue in c8243933c74 itself, but was revealed by my ea7811b37e0 (git-send-email: improve --validate error output, 2021-04-06) when we started emitting the path to the hook, which was previously only internal to git-send-email.perl. The just-landed 53753a37d09 (t9001-send-email.sh: fix expected absolute paths on Windows, 2021-05-24) narrowly fixed this issue, but I believe we can do better here. We should not be relying on whatever changes Perl's abs_path() makes to the path "rev-parse --git-path hooks" hands to us. Let's instead trust it, and hand it to Perl's system() in git-send-email.perl. It will handle either a relative or absolute path. So let's revert most of 53753a37d09 and just have "hooks_path" return what we get from "rev-parse" directly without modification. This has the added benefit of making the error message friendlier in the common case, we'll no longer print an absolute path for repository-local hook errors. 1. http://lore.kernel.org/git/bb30fe2b-cd75-4782-24a6-08bb002a0367@kdbg.org Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-26t1092: revert the "-1" hack for emulating "no progress meter"Junio C Hamano1-3/+3
This looked like a good idea, but it seems to break tests on 32-bit builds rather badly. Revert to just use "100 thousands must be big enough" for now. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-25l10n: id: po-id for 2.32.0 (round 1)Bagas Sanjaya1-4437/+5372
Translate following components: * builtin/add.c * worktree.c * builtin/branch.c * builtin/commit.c * builtin/merge.c * builtin/rebase.c * builtin/pull.c * diff.c * add-interactive.c * builtin/log.c * builtin/stash.c * builtin/tag.c * config.c * builtin/config.c * reset.c * builtin/remote.c * builtin/rm.c * builtin/mv.c * builtin/clean.c * builtin/help.c * archive.c * submodule.c * builtin/submodule--helper.c * submodule-config.c Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com>
2021-05-25Merge branch 'mt/init-template-userpath-fix'Junio C Hamano2-9/+22
Regression fix. * mt/init-template-userpath-fix: init: fix bug regarding ~/ expansion in init.templateDir
2021-05-25Merge branch 'jt/send-email-validate-errors-fix'Junio C Hamano1-4/+3
Fix a test breakage. * jt/send-email-validate-errors-fix: t9001-send-email.sh: fix expected absolute paths on Windows
2021-05-25Merge branch 'ab/send-email-validate-errors-fix'Junio C Hamano2-3/+32
* ab/send-email-validate-errors-fix: send-email: fix missing error message regression
2021-05-25t1092: use GIT_PROGRESS_DELAY for consistent resultsDerrick Stolee1-3/+3
The t1092-sparse-checkout-compatibility.sh tests compare the stdout and stderr for several Git commands across both full checkouts, sparse checkouts with a full index, and sparse checkouts with a sparse index. Since these are direct comparisons, sometimes a progress indicator can flush at unpredictable points, especially on slower machines. This causes the tests to be flaky. One standard way to avoid this is to add GIT_PROGRESS_DELAY=0 to the Git commands that are run, as this will force every progress indicator created with start_progress_delay() to be created immediately. However, there are some progress indicators that are created in the case of a full index that are not created with a sparse index. Moreover, their values may be different as those indexes have a different number of entries. Instead, use GIT_PROGRESS_DELAY=-1 (which will turn into UINT_MAX) to ensure that any reasonable machine running these tests would never display delayed progress indicators. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-25init: fix bug regarding ~/ expansion in init.templateDirMatheus Tavares2-9/+22
We used to read the init.templateDir setting at builtin/init-db.c using a git_config() callback that, in turn, called git_config_pathname(). To simplify the config reading logic at this file and plug a memory leak, this was replaced by a direct call to git_config_get_value() at e4de4502e6 ("init: remove git_init_db_config() while fixing leaks", 2021-03-14). However, this function doesn't provide path expanding semantics, like git_config_pathname() does, so paths with '~/' and '~user/' are treated literally. This makes 'git init' fail to handle init.templateDir paths using these constructs: $ git config init.templateDir '~/templates_dir' $ git init 'warning: templates not found in ~/templates_dir' Replace the git_config_get_value() call by git_config_get_pathname(), which does the '~/' and '~user/' expansions. Also add a regression test. Note that unlike git_config_get_value(), the config cache does not own the memory for the path returned by git_config_get_pathname(), so we must free() it. Reported on IRC by rkta. Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-25send-email: fix missing error message regressionÆvar Arnfjörð Bjarmason2-3/+32
Fix a regression with the "the editor exited uncleanly, aborting everything" error message going missing after my d21616c0394 (git-send-email: refactor duplicate $? checks into a function, 2021-04-06). I introduced a $msg variable, but did not actually use it. This caused us to miss the optional error message supplied by the "do_edit" codepath. Fix that, and add tests to check that this works. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-25t9001-send-email.sh: fix expected absolute paths on WindowsJohannes Sixt1-4/+3
Git for Windows is a native Windows program that works with native absolute paths in the drive letter style C:\dir. The auxiliary infrastructure is based on MSYS2, which uses POSIX style /C/dir. When we test for output of absolute paths produced by git.exe, we usally have to expect C:\dir style paths. To produce such expected paths, we have to use $(pwd) in the test scripts; the alternative, $PWD, produces a POSIX style path. ($PWD is a shell variable, and the shell is bash, an MSYS2 program, and operates in the POSIX realm.) There are two recently added tests that were written to expect C:\dir paths. The output that is tested is produced by `git send-email`, but behind the scenes, this is a Perl script, which also works in the POSIX realm and produces /C/dir style output. In the first test case that is changed here, replace $(pwd) by $PWD so that the expected path is constructed using /C/dir style. The second test case sets core.hooksPath to an absolute path. Since the test script talks to native git.exe, it is supposed to place a C:/dir style path into the configuration; therefore, keep $(pwd). When this configuration value is consumed by the Perl script, it is transformed to /C/dir style by the MSYS2 layer and echoed back in this form in the error message. Hence, do use $PWD for the expected value. Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-25packfile-uri.txt: fix blobPackfileUri descriptionTeng Long1-7/+8
Fix the 'uploadpack.blobPackfileUri' description in packfile-uri.txt and the correct format also can be seen in t5702. Signed-off-by: Teng Long <dyroneteng@gmail.com> Reviewed-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-24doc: avoid using rm directlyFelipe Contreras1-3/+3
That's what we have $(RM) for. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-24doc: simplify Makefile using .DELETE_ON_ERRORFelipe Contreras1-28/+19
Currently GNU make already removes files when catching an interruption signal, however, in order to deal with other kinds of errors a workaround is in place to store target output to a temporary file, and only move it to its right place on success. By enabling the built-in .DELETE_ON_ERROR we let make do this task, so we don't have to. This way the rules can be simplified a lot. Suggested-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-24doc: remove unnecessary rm instancesFelipe Contreras1-27/+15
Commits 50cff52f1a (When generating manpages, delete outdated targets first., 2007-08-02) and f9286765b2 (Documentation/Makefile: remove cmd-list.made before redirecting to it., 2007-08-06) created these rm instances for a very rare corner-case: building as root by mistake. It's odd to have workarounds here, but nowhere else in the Makefile-- which already fails in this stuation, starting from Documentation/technical/. We gain nothing but complexity, so let's remove them. Comments-by: Jeff King <peff@peff.net> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-24doc: improve asciidoc dependenciesFelipe Contreras1-1/+2
asciidoc needs asciidoc.conf, asciidoctor asciidoctor-extensions.rb. Neither needs the other. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-24doc: refactor common asciidoc dependenciesFelipe Contreras1-3/+4
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-24l10n: vi.po(5204t): Updated Vietnamese translation for v2.32.0Tran Ngoc Quan1-3196/+3691
Signed-off-by: Tran Ngoc Quan <vnwildman@gmail.com>
2021-05-23l10n: zh_TW.po: localizedYi-Jyun Pan1-82/+82
Signed-off-by: Yi-Jyun Pan <pan93412@gmail.com>
2021-05-22Git 2.32-rc1v2.32.0-rc1Junio C Hamano2-1/+6
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-22Merge branch 'dl/stash-show-untracked-fixup'Junio C Hamano4-9/+10
Another brown paper bag inconsistency fix for a new feature introduced during this cycle. * dl/stash-show-untracked-fixup: stash show: use stash.showIncludeUntracked even when diff options given
2021-05-22Merge branch 'jh/simple-ipc-sans-pthread'Junio C Hamano6-14/+48
The "simple-ipc" did not compile without pthreads support, but the build procedure was not properly account for it. * jh/simple-ipc-sans-pthread: simple-ipc: correct ifdefs when NO_PTHREADS is defined
2021-05-22Merge branch 'wm/rev-parse-path-format-wo-arg'Junio C Hamano2-0/+6
The "rev-parse" command did not diagnose the lack of argument to "--path-format" option, which was introduced in v2.31 era, which has been corrected. * wm/rev-parse-path-format-wo-arg: rev-parse: fix segfault with missing --path-format argument
2021-05-22t: use configured TAR instead of tarĐoàn Trần Công Danh2-4/+4
Despite that tar is available everywhere, it's not required by POSIX. In our build system, users are allowed to specify which tar to be used in Makefile knobs. Furthermore, GNU tar (gtar) is prefered when autotools is being used. In our testsuite, 7 out of 9 tar-required-tests use "$TAR", the other two use "tar". Let's change the remaining two tests to "$TAR". Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-22stash show: use stash.showIncludeUntracked even when diff options givenDenton Liu4-9/+10
If options pertaining to how the diff is displayed is provided to `git stash show`, the command will ignore the stash.showIncludeUntracked configuration variable, defaulting to not showing any untracked files. This is unintuitive behaviour since the format of the diff output and whether or not to display untracked files are orthogonal. Use stash.showIncludeUntracked even when diff options are given. Of course, this is still overridable via the command-line options. Update the documentation to explicitly say which configuration variables will be overridden when a diff options are given. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-21l10n: zh_TW.po: v2.32.0 round 1 (11 untranslated)Yi-Jyun Pan1-3230/+3714
Signed-off-by: Yi-Jyun Pan <pan93412@gmail.com>
2021-05-21diff-merges: let "-m" imply "-p"Sergey Organov3-6/+7
Fix long standing inconsistency between -c/--cc that do imply -p on one side, and -m that did not imply -p on the other side. Change corresponding test accordingly, as "log -m" output should now match one from "log -m -p", rather than from just "log". Change documentation accordingly. NOTES: After this patch git log -m produces diffs without need to provide -p as well, that improves both consistency and usability. It gets even more useful if one sets "log.diffMerges" configuration variable to "first-parent" to force -m produce usual diff with respect to first parent only. This patch, however, does not change behavior when specific diff format is explicitly provided on the command-line, so that commands like git log -m --raw git log -m --stat are not affected, nor does it change commands where specific diff format is active by default, such as: git diff-tree -m It's also worth to be noticed that exact historical semantics of -m is still provided by --diff-merges=separate. Signed-off-by: Sergey Organov <sorganov@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-21diff-merges: rename "combined_imply_patch" to "merges_imply_patch"Sergey Organov2-6/+6
This is refactoring change in preparation for the next commit that will let -m imply -p. The old name doesn't match the intention to let not only -c/-cc imply -p, but also -m, that is not a "combined" format, so we rename the flag accordingly. Signed-off-by: Sergey Organov <sorganov@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-21stash list: stop passing "-m" to "git log"Sergey Organov2-2/+2
Passing "-m" in "git log --first-parent -m" is not needed as --first-parent implies --diff-merges=first-parent anyway. OTOH, it will stop being harmless once we let "-m" imply "-p". While we are at it, fix corresponding test description in t3903-stash to match what it actually tests. Signed-off-by: Sergey Organov <sorganov@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-21git-svn: stop passing "-m" to "git rev-list"Sergey Organov1-1/+1
rev-list doesn't utilize -m. It happens to eat it silently, so this bug went unnoticed. Signed-off-by: Sergey Organov <sorganov@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-21diff-merges: move specific diff-index "-m" handling to diff-indexSergey Organov3-12/+24
Move specific handling of "-m" for diff-index to diff-index.c, so diff-merges is left to handle only diff for merges options. Being a better design by itself, this is especially essential in preparation for letting -m imply -p, as "diff-index -m" obviously should not imply -p, as it's entirely unrelated. To handle this, in addition to moving specific diff-index "-m" code out of diff-merges, we introduce new diff_merges_suppress_options_parsing() and call it before generic options processing in cmd_diff_index(). This new diff_merges_suppress_options_parsing() could then be reused and called before invocations of setup_revisions() for other commands that don't need --diff-merges options, but that's outside of the scope of these patch series. Signed-off-by: Sergey Organov <sorganov@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-21t4013: test "git diff-index -m"Sergey Organov1-0/+13
-m in "git diff-index" means "match missing", that differs from its meaning in "git diff". Let's check it in diff-index. Signed-off-by: Sergey Organov <sorganov@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-21t4013: test "git diff-tree -m"Sergey Organov2-0/+12
We want to ensure we don't affect plumbing commands with our changes of "-m" semantics, so add corresponding test. Signed-off-by: Sergey Organov <sorganov@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-21t4013: test "git log -m --stat"Sergey Organov2-0/+67
This is to ensure we won't break different diff formats when we start to imply "-p" by "-m". Signed-off-by: Sergey Organov <sorganov@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-21t4013: test "git log -m --raw"Sergey Organov2-0/+62
This is to ensure we won't break different diff formats when we start to imply "-p" by "-m". Signed-off-by: Sergey Organov <sorganov@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-21t4013: test that "-m" alone has no effect in "git log"Sergey Organov1-0/+8
This is to notice current behavior that we are going to change when we start to imply "-p" by "-m". Signed-off-by: Sergey Organov <sorganov@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-21simple-ipc: correct ifdefs when NO_PTHREADS is definedJeff Hostetler6-14/+48
Simple IPC always requires threads (in addition to various platform-specific IPC support). Fix the ifdefs in the Makefile to define SUPPORTS_SIMPLE_IPC when appropriate. Previously, the Unix version of the code would only verify that Unix domain sockets were available. This problem was reported here: https://lore.kernel.org/git/YKN5lXs4AoK%2FJFTO@coredump.intra.peff.net/T/#m08be8f1942ea8a2c36cfee0e51cdf06489fdeafc Reported-by: Randall S. Becker <rsbecker@nexbridge.com> Helped-by: Jeff King <peff@peff.net> Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-21help: fix small typo in error messageJean-Noël Avila1-1/+1
Classic string concatenation while forgetting a space character. Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-21trace2: refactor to avoid gcc warning under -O3Ævar Arnfjörð Bjarmason1-10/+8
Refactor tr2_dst_try_uds_connect() to avoid a gcc warning[1] that appears under -O3 (but not -O2). This makes the build pass under DEVELOPER=1 without needing a DEVOPTS=no-error. This can be reproduced with GCC Debian 8.3.0-6, but not e.g. with clang 7.0.1-8+deb10u2. We've had this warning since ee4512ed481 (trace2: create new combined trace facility, 2019-02-22). As noted in [2] this warning happens because the compiler doesn't assume that errno must be non-zero after a failed syscall. Let's work around by using the well-established "saved_errno" pattern, along with returning -1 ourselves instead of "errno". The caller can thus rely on our "errno" on failure. See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61846 for a related bug report against GCC. 1. trace2/tr2_dst.c: In function ‘tr2_dst_get_trace_fd.part.5’: trace2/tr2_dst.c:296:10: warning: ‘fd’ may be used uninitialized in this function [-Wmaybe-uninitialized] dst->fd = fd; ~~~~~~~~^~~~ trace2/tr2_dst.c:229:6: note: ‘fd’ was declared here int fd; ^~ 2. https://lore.kernel.org/git/20200404142131.GA679473@coredump.intra.peff.net/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-21Merge branch 'ds/sparse-index-protections'Junio C Hamano1-1/+1
Fix access to uninitialized piece of memory, introduced during this cycle. * ds/sparse-index-protections: sparse-index: fix uninitialized jump
2021-05-21Merge branch 'tz/c-locale-output-is-no-more'Junio C Hamano1-1/+1
Test update. * tz/c-locale-output-is-no-more: t7500: remove non-existant C_LOCALE_OUTPUT prereq
2021-05-21Merge branch 'cs/http-use-basic-after-failed-negotiate'Junio C Hamano3-13/+48
Regression fix for a change made during this cycle. * cs/http-use-basic-after-failed-negotiate: Revert "remote-curl: fall back to basic auth if Negotiate fails" t5551: test http interaction with credential helpers
2021-05-20l10n: sv.po: Update Swedish translation (5204t0f0u)Peter Krefting1-3192/+3684
Signed-off-by: Peter Krefting <peter@softwolves.pp.se>
2021-05-20merge-ort, diffcore-rename: employ cached renames when possibleElijah Newren4-30/+90
When there are many renames between the old base of a series of commits and the new base, the way sequencer.c, merge-recursive.c, and diffcore-rename.c have traditionally split the work resulted in redetecting the same renames with each and every commit being transplanted. To address this, the last several commits have been creating a cache of rename detection results, determining when it was safe to use such a cache in subsequent merge operations, adding helper functions, and so on. See the previous half dozen commit messages for additional discussion of this optimization, particularly the message a few commits ago entitled "add code to check for whether cached renames can be reused". This commit finally ties all of that work together, modifying the merge algorithm to make use of these cached renames. For the testcases mentioned in commit 557ac0350d ("merge-ort: begin performance work; instrument with trace2_region_* calls", 2020-10-28), this change improves the performance as follows: Before After no-renames: 5.665 s ± 0.129 s 5.622 s ± 0.059 s mega-renames: 11.435 s ± 0.158 s 10.127 s ± 0.073 s just-one-mega: 494.2 ms ± 6.1 ms 500.3 ms ± 3.8 ms That's a fairly small improvement, but mostly because the previous optimizations were so effective for these particular testcases; this optimization only kicks in when the others don't. If we undid the basename-guided rename detection and skip-irrelevant-renames optimizations, then we'd see that this series by itself improved performance as follows: Before Basename Series After Just This Series no-renames: 13.815 s ± 0.062 s 5.697 s ± 0.080 s mega-renames: 1799.937 s ± 0.493 s 205.709 s ± 0.457 s Since this optimization kicks in to help accelerate cases where the previous optimizations do not apply, this last comparison shows that this cached-renames optimization has the potential to help signficantly in cases that don't meet the requirements for the other optimizations to be effective. The changes made in this optimization also lay some important groundwork for a future optimization around having collect_merge_info() avoid recursing into subtrees in more cases. However, for this optimization to be effective, merge_switch_to_result() should only be called when the rebase or cherry-pick operation has either completed or hit a case where the user needs to resolve a conflict or edit the result. If it is called after every commit, as sequencer.c does, then the working tree and index are needlessly updated with every commit and the cached metadata is tossed, defeating this optimization. Refactoring sequencer.c to only call merge_switch_to_result() at the end of the operation is a bigger undertaking, and the practical benefits of this optimization will not be realized until that work is performed. Since `test-tool fast-rebase` only updates at the end of the operation, it was used to obtain the timings above. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20merge-ort: handle interactions of caching and rename/rename(1to1) casesElijah Newren1-1/+29
As documented in Documentation/technical/remembering-renames.txt, and as tested for in the two testcases in t6429 with "rename same file identically" in their description, there is one case where we need to have renames in one commit NOT be cached for the next commit in our rebase sequence -- namely, rename/rename(1to1) cases. Rather than specifically trying to uncache those and fix up dir_rename_counts() to match (which would also be valid but more work), we simply disable the optimization when this really rare type of rename occurs. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20merge-ort: add helper functions for using cached renamesElijah Newren1-0/+47
If we have a usable rename cache, then we can remove from relevant_sources all the paths that were cached; diffcore_rename_extended() can then consider an even smaller set of relevant_sources in its rename detection. However, when diffcore_rename_extended() is done, we will need to take the renames it detected and then add back in all the ones we had cached from before. Add helper functions for doing these two operations; the next commit will make use of them. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20merge-ort: preserve cached renames for the appropriate sideElijah Newren1-9/+11
Previous commits created an in-memory cache of the results of rename detection, and added logic to detect when that cache could appropriately be used in a subsequent merge operation -- but we were still unconditionally clearing the cache with each new merge operation anyway. If it is valid to reuse the cache from one of the two sides of history, preserve that side. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20merge-ort: avoid accidental API mis-useElijah Newren2-0/+9
Previously, callers of the merge-ort API could have passed an uninitialized value for struct merge_result *result. However, we want to check result to see if it has cached renames from a previous merge that we can reuse; such values would be found behind result->priv. However, if result->priv is uninitialized, attempting to access behind it will give a segfault. So, we need result->priv to be NULL (which will be the case if the caller does a memset(&result, 0)), or be written by a previous call to the merge-ort machinery. Documenting this requirement may help, but despite being the person who introduced this requirement, I still missed it once and it did not fail in a very clear way and led to a long debugging session. Add a _properly_initialized field to merge_result; that value will be 0 if the caller zero'ed the merge_result, it will be set to a very specific value by a previous run by the merge-ort machinery, and if it's uninitialized it will most likely either be 0 or some value that does not match the specific one we'd expect allowing us to throw a much more meaningful error. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20merge-ort: add code to check for whether cached renames can be reusedElijah Newren1-2/+64
We need to know when renames detected in a previous merge operation can be reused in a later merge operation. Consider the following setup (from the git-rebase manpage): A---B---C topic / D---E---F---G master After rebasing, this will appear as: A'--B'--C' topic / D---E---F---G master Further, let's say that 'oldfile' was renamed to 'newfile' between E and G. The rebase or cherry-pick of A onto G will involve a three-way merge between E (as the merge base) and G and A. After detecting the rename between E:oldfile and G:newfile, there will be a three-way content merge of the following: E:oldfile G:newfile A:oldfile and produce a new result: A':newfile Now, when we want to pick B onto A', we will need to do a three-way merge between A (as the merge-base) and A' and B. This will involve a three-way content merge of A:oldfile A':newfile B:oldfile but only if we can detect that A:oldfile is similar enough to A':newfile to be used together in a three-way content merge, i.e. only if we can detect that A:oldfile and A':newfile are a rename. But we already know that A:oldfile and A':newfile are similar enough to be used in a three-way content merge, because that is precisely where A':newfile came from in the previous merge. Note that A & A' both appear in both merges. That gives us the condition under which we can reuse renames. There are a couple important points about this optimization: - If the rebase or cherry-pick halts for user conflicts, these caches are NOT saved anywhere. Thus, resuming a halted rebase or cherry-pick will result in no reused renames for the next commit. This is intentional, as user resolution can change files significantly and in ways that violate the similarity assumptions here. - Technically, in a *very* narrow case this might give slightly different results for rename detection. Using the example above, if: * E:oldfile had 20 lines * G:newfile added 10 new lines at the beginning of the file * A:oldfile deleted all but the first three lines of the file then => A':newfile would have 13 lines, 3 of which matches those in A:oldfile. Consider the two cases: * Without this optimization: - the next step of the rebase operation (moving B to B') would not detect the rename betwen A:oldfile and A':newfile - we'd thus get a modify/delete conflict with the rebase operation halting for the user to resolve, and have both A':newfile and B:oldfile sitting in the working tree. * With this optimization: - the rename between A:oldfile and A':newfile would be detected via the cache of renames - a three-way merge between A:oldfile, A':newfile, and B:oldfile would commence and be written to A':newfile Now, is the difference in behavior a bug...or a bugfix? I can't tell. Given that A:oldfile and A':newfile are not very similar, when we three-way merge with B:oldfile it seems likely we'll hit a conflict for the user to resolve. And it shouldn't be too hard for users to see why we did that three-way merge; oldfile and newfile *were* renames somewhere in the sequence. So, most of these corner cases will still behave similarly -- namely, a conflict given to the user to resolve. Also, consider the interesting case when commit B is a clean revert of commit A. Without this optimization, a rebase could not both apply a weird patch like A and then immediately revert it; users would be forced to resolve merge conflicts. With this optimization, it would successfully apply the clean revert. So, there is certainly at least one case that behaves better. Even if it's considered a "difference in behavior", I think both behaviors are reasonable, and the time savings provided by this optimization justify using the slightly altered rename heuristics. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20merge-ort: populate caches of rename detection resultsElijah Newren1-1/+72
Fill in cache_pairs, cached_target_names, and cached_irrelevant based on rename detection results. Future commits will make use of these values. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20merge-ort: add data structures for in-memory caching of rename detectionElijah Newren1-0/+53
When there are many renames between the old base of a series of commits and the new base for a series of commits, the sequence of merges employed to transplant those commits (from a cherry-pick or rebase operation) will repeatedly detect the exact same renames. This is wasted effort. Add data structures which will be used to cache rename detection results, along with the initialization and deallocation of these data structures. Future commits will populate these caches, detect the appropriate circumstances when they can be used, and employ them to avoid re-detecting the same renames repeatedly. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20t6429: testcases for remembering renamesElijah Newren2-6/+700
We will soon be adding an optimization that caches (in memory only, never written to disk) upstream renames during a sequence of merges such as occurs during a cherry-pick or rebase operation. Add several tests meant to stress such an implementation to ensure it does the right thing, and include a test whose outcome we will later change due to this optimization as well. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20fast-rebase: write conflict state to working tree, index, and HEADElijah Newren1-19/+32
Previously, when fast-rebase hit a conflict, it simply aborted and left HEAD, the index, and the working tree where they were before the operation started. While fast-rebase does not support restarting from a conflicted state, write the conflicted state out anyway as it gives us a way to see what the conflicts are and write tests that check for them. This will be important in the upcoming commits, because sequencer.c is only superficially integrated with merge-ort.c; in particular, it calls merge_switch_to_result() after EACH merge instead of only calling it at the end of all the sequence of merges (or when a conflict is hit). This not only causes needless updates to the working copy and index, but also causes all intermediate data to be freed and tossed, preventing caching information from one merge to the next. However, integrating sequencer.c more deeply with merge-ort.c is a big task, and making this small extension to fast-rebase.c provides us with a simple way to test the edge and corner cases that we want to make sure continue working. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20fast-rebase: change assert() to BUG()Elijah Newren1-1/+2
assert() can succinctly document expectations for the code, and do so in a way that may be useful to future folks trying to refactor the code and change basic assumptions; it allows them to more quickly find some places where their violations of previous assumptions trips things up. Unfortunately, assert() can surround a function call with important side-effects, which is a huge mistake since some users will compile with assertions disabled. I've had to debug such mistakes before in other codebases, so I should know better. Luckily, this was only in test code, but it's still very embarrassing. Change an assert() to an if (...) BUG (...). Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20Documentation/technical: describe remembering renames optimizationElijah Newren1-0/+669
Remembering renames on the upstream side of history in an early merge of a rebase or cherry-pick for re-use in a latter merge of the same operation makes pretty good intuitive sense. However, trying to show that it doesn't cause some subtle behavioral difference or some funny edge or corner case is much more involved. And, in fact, it does introduce a subtle behavioral change. Document all the assumptions, special cases, and logic involved in such an optimization, and describe why this optimization is safe under the current optimizations/features/etc. -- even when the subtle behavioral change is triggered. Part of the point of adding this document that goes over the optimization in such laborious detail, is that it is possible that significant future changes (optimizations or feature changes) could interact with this optimization in interesting ways; this document is here to help folks making big changes sanity check that the assumptions and arguments underlying this optimization are still valid. (As a side note, creating this document forced me to review things in sufficient detail that I found I was not properly caching directory-rename-induced renames, resulting in the code not being aware of those renames and causing unnecessary diffcore_rename_extended() calls in subsequent merges.) A subsequent commit will add several testcases based on this document meant to stress-test the implementation and also document the case with the subtle behavioral change. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20doc: explain the use of color.pagerJeff King1-2/+3
The current documentation for color.pager is technically correct, but slightly misleading and doesn't really clarify the purpose of the variable. As explained in the original thread which added it: https://lore.kernel.org/git/E1G6zPH-00062L-Je@moooo.ath.cx/ the point is to deal with pagers that don't understand colors. And hence it being set to "true" is necessary for colorizing output to the pager, but not sufficient by itself (you must also have enabled one of the other color options, though note that these are set to "auto" by default these days). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20l10n: fix typos in po/TEAMSJiang Xin1-2/+2
Find typos in "po/TEAMS" file using the "git-po-helper" program. These typos were introduced from commit v2.24.0-1-g9917eca794 (l10n: zh_TW: add translation for v2.24.0, 2019-11-20 19:14:22 +0800). Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
2021-05-20setup: split "extensions found" messages into singular and pluralAlex Henrie1-2/+6
It's easier to translate this way. Signed-off-by: Alex Henrie <alexhenrie24@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20fetch: improve grammar of "shallow roots" messageAlex Henrie1-1/+1
Signed-off-by: Alex Henrie <alexhenrie24@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20A handful more topics before -rc1Junio C Hamano1-0/+19
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20Merge branch 'jk/test-chainlint-softer'Junio C Hamano4-3/+21
The "chainlint" feature in the test framework is a handy way to catch common mistakes in writing new tests, but tends to get expensive. An knob to selectively disable it has been introduced to help running tests that the developer has not modified. * jk/test-chainlint-softer: t: avoid sed-based chain-linting in some expensive cases
2021-05-20Merge branch 'en/prompt-under-set-u'Junio C Hamano1-3/+3
The bash prompt script (in contrib/) did not work under "set -u". * en/prompt-under-set-u: git-prompt: work under set -u
2021-05-20Merge branch 'zh/ref-filter-push-remote-fix'Junio C Hamano2-1/+20
The handling of "%(push)" formatting element of "for-each-ref" and friends was broken when the same codepath started handling "%(push:<what>)", which has been corrected. * zh/ref-filter-push-remote-fix: ref-filter: fix read invalid union member bug
2021-05-20Merge branch 'ew/sha256-clone-remote-curl-fix'Junio C Hamano1-0/+2
"git clone" from SHA256 repository by Git built with SHA-1 as the default hash algorithm over the dumb HTTP protocol did not correctly set up the resulting repository, which has been corrected. * ew/sha256-clone-remote-curl-fix: remote-curl: fix clone on sha256 repos
2021-05-20Merge branch 'en/dir-traversal'Junio C Hamano18-172/+298
"git clean" and "git ls-files -i" had confusion around working on or showing ignored paths inside an ignored directory, which has been corrected. * en/dir-traversal: dir: introduce readdir_skip_dot_and_dotdot() helper dir: update stale description of treat_directory() dir: traverse into untracked directories if they may have ignored subfiles dir: avoid unnecessary traversal into ignored directory t3001, t7300: add testcase showcasing missed directory traversal t7300: add testcase showing unnecessary traversal into ignored directory ls-files: error out on -i unless -o or -c are specified dir: report number of visited directories and paths with trace2 dir: convert trace calls to trace2 equivalents
2021-05-20Merge branch 'ab/perl-makefile-cleanup'Junio C Hamano2-4/+19
Build procedure clean-up. * ab/perl-makefile-cleanup: Makefile: make PERL_DEFINES recursively expanded perl: use mock i18n functions under NO_GETTEXT=Y Makefile: regenerate *.pm on NO_PERL_CPAN_FALLBACKS change Makefile: regenerate perl/build/* if GIT-PERL-DEFINES changes Makefile: don't re-define PERL_DEFINES
2021-05-20fetch-pack: signal v2 server that we are done making requestsJeff King2-2/+13
When fetching with the v0 protocol over ssh (or a local upload-pack with pipes), the server closes the connection as soon as it is finished sending the pack. So even though the client may still be operating on the data via index-pack (e.g., resolving deltas, checking connectivity, etc), the server has released all resources. With the v2 protocol, however, the server considers the ssh session only as a transport, with individual requests coming over it. After sending the pack, it goes back to its main loop, waiting for another request to come from the client. As a result, the ssh session hangs around until the client process ends, which may be much later (because resolving deltas, etc, may consume a lot of CPU). This is bad for two reasons: - it's consuming resources on the server to leave open a connection that won't see any more use - if something bad happens to the ssh connection in the meantime (say, it gets killed by the network because it's idle, as happened in a real-world report), then ssh will exit non-zero, and we'll propagate the error up the stack. The server is correct here not to hang up after serving the pack. The v2 protocol's design is meant to allow multiple requests like this, and hanging up would be the wrong thing for a hypothetical client which was planning to make more requests (though in practice, the git.git client never would, and I doubt any other implementations would either). The right thing is instead for the client to signal to the server that it's not interested in making more requests. We can do that by closing the pipe descriptor we use to write to ssh. This will propagate to the server upload-pack as an EOF when it tries to read the next request (and then it will close its half, and the whole connection will go away). It's important to do this "half duplex" shutdown, because we have to do it _before_ we actually receive the pack. This is an artifact of the way fetch-pack and index-pack (or unpack-objects) interact. We hand the connection off to index-pack (really, a sideband demuxer which feeds it), and then wait until it returns. And it doesn't do that until it has resolved all of the deltas in the pack, even though it was done reading from the server long before. So just closing the connection fully after index-pack returns would be too late; we'd have held it open much longer than was necessary. And teaching index-pack to close the connection is awkward. It's not even seeing the whole conversation (the sideband demuxer is, but it doesn't actually know what's in the packets, or when the end comes). Note that this close() is happening deep within the transport code. It's possible that a caller would want to perform other operations over the same ssh transport after receiving the pack. But as of the current code, none of the callers do, and there haven't been discussions of any plans to change this. If we need to support that later, we can probably do so by passing down a flag for "you're the last request on the transport; it's OK to close" instead of the code just assuming that's true. The description above all discusses v2 ssh, so it's worth thinking about how this interacts with other protocols: - in v0 protocols, we could do the same half-duplex shutdown (it just goes into the v0 do_fetch_pack() instead). This does work, but since it doesn't have the same persistence problem in the first place, there's little reason to change it at this point. - local fetches against git-upload-pack on the same machine will behave the same as ssh (they are talking over two pipes, and see EOF on their input pipe) - fetches against git-daemon will run this same code, and close one of the descriptors. In practice, this won't do anything, since there our two descriptors are dups of each other, and not part of a half-duplex pair. The right thing would probably be to call shutdown(SHUT_WR) on it. I didn't bother with that here. It doesn't face the same error-code problem (since it's just a TCP connection), so it's really only an optimization problem. And git:// is not that widely used these days, and has less impact on server resources than an ssh termination. - v2 http doesn't suffer from this problem in the first place, as our pipes terminate at a local git-remote-https, which is passing data along as individual requests via curl. Probably curl is keeping the TCP/TLS connection open for more requests, and we might be able to tell it manually "hey, we are done making requests now". But I think that's much less important. It again doesn't suffer from the error-code problem, and HTTP keepalive is pretty well understood (importantly, the timeouts can be set low, because clients like curl know how to reconnect for subsequent requests if necessary). So it's probably not worth figuring out how to tell curl that we're done (though if we do, this patch is probably the first step anyway; fetch-pack closes the pipe back to remote-https, which would be the signal that it should tell curl we're done). The code is pretty straightforward. We close the pipe at the right moment, and set it to -1 to mark it as invalid. I modified the later cleanup code to avoid calling close(-1). That's not strictly necessary, since close(-1) is a noop, but hopefully makes things a bit more obvious to a reader. I suspect that trying to call more transport functions after the close() (e.g., calling transport_fetch_refs() again) would fail, as it's not smart enough to realize we need to re-open the ssh connection. But that's already true when v0 is in use. And no current callers want to do that (and again, the solution is probably a flag in the transport code to keep things open, which can be added later). There's no test here, as the situation it covers is inherently racy (the question is when upload-pack exits, compared to when index-pack finishes resolving deltas and exits). The rather gross shell snippet below does recreate the problematic situation; when run on a sufficiently-large repository (git.git works fine), it kills an "idle" upload-pack while the client is resolving deltas, leading to a failed clone. ( git clone --no-local --progress . foo.git 2>&1 echo >&2 "clone exit code=$?" ) | tr '\r' '\n' | while read line do case "$done,$line" in ,Resolving*) echo "hit resolving deltas; killing upload-pack" killall -9 git-upload-pack done=t ;; esac done Reported-by: Greg Pflaum <greg.pflaum@pnp-hcl.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-19l10n: fr: v2.32.0 round 1Jean-Noël Avila1-5526/+4574
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr>
2021-05-19clone: clean up directory after transport_fetch_refs() failureJeff King2-7/+11
git-clone started respecting errors from the transport subsystem in aab179d937 (builtin/clone.c: don't ignore transport_fetch_refs() errors, 2020-12-03). However, that commit didn't handle the cleanup of the filesystem quite right. The cleanup of the directory that cmd_clone() creates is done by an atexit() handler, which we control with a flag. It starts as JUNK_LEAVE_NONE ("clean up everything"), then progresses to JUNK_LEAVE_REPO when we know we have a valid repo but not working tree, and then finally JUNK_LEAVE_ALL when we have a successful checkout. Most errors cause us to die(), which then triggers the handler to do the right thing based on how far into cmd_clone() we got. But the checks added by aab179d937 instead set the "err" variable and then jump to a new "cleanup" label, which then returns our non-zero status. However, the code after the cleanup label includes setting the flag to JUNK_LEAVE_ALL, and so we accidentally leave the repository and working tree in place. One obvious option to fix this is to reorder the end of the function to set the flag first, before cleanup code, and put the label between them. But we can observe another small bug: the error return from transport_fetch_refs() is generally "-1", and we propagate that to the return value of cmd_clone(), which ultimately becomes the exit code of the process. And we try to avoid transmitting negative values via exit codes (only the low 8 bits are passed along as an unsigned value, though in practice for "-1" this at least retains the property that it's non-zero). Instead, let's just die(). That makes us consistent with rest of the code in the function. It does add a new "fatal:" line to the output, but I'd argue that's a good thing: - in the rare case that the transport code didn't say anything, now the user gets _some_ error message - even if the transport code said something like "error: ssh died of signal 9", it's nice to also say "fatal" to indicate that we considered that to be a show-stopper. Triggering this in the test suite turns out to be surprisingly difficult. Almost every error we'd encounter, including ones deep inside the transport code, cause us to just die() right there! However, one way is to put a fake wrapper around git-upload-pack that sends the complete packfile but exits with a failure code. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-19docs: improve fast-forward in glossary contentReuven Y1-2/+2
The text was somewhat confusing between the revision itself and the author. Signed-off-by: Reuven Yagel <robi@post.jce.ac.il> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-19read-cache: delete unused hashing methodsDerrick Stolee1-64/+0
These methods were marked as MAYBE_UNUSED in the previous change to avoid a complicated diff. Delete them entirely, since we now use the hashfile API instead of this custom hashing code. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-19read-cache: use hashfile instead of git_hash_ctxDerrick Stolee1-71/+66
The do_write_index() method in read-cache.c has its own hashing logic and buffering mechanism. Specifically, the ce_write() method was introduced by 4990aadc (Speed up index file writing by chunking it nicely, 2005-04-20) and similar mechanisms were introduced a few months later in c38138cd (git-pack-objects: write the pack files with a SHA1 csum, 2005-06-26). Based on the timing, in the early days of the Git codebase, I figured that these roughly equivalent code paths were never unified only because it got lost in the shuffle. The hashfile API has since been used extensively in other file formats, such as pack-indexes, multi-pack-indexes, and commit-graphs. Therefore, it seems prudent to unify the index writing code to use the same mechanism. I discovered this disparity while trying to create a new index format that uses the chunk-format API. That API uses a hashfile as its base, so it is incompatible with the custom code in read-cache.c. This rewrite is rather straightforward. It replaces all writes to the temporary file with writes to the hashfile struct. This takes care of many of the direct interactions with the_hash_algo. There are still some git_hash_ctx uses remaining: the extension headers are hashed for use in the End of Index Entries (EOIE) extension. This use of the git_hash_ctx is left as-is. There are multiple reasons to not use a hashfile here, including the fact that the data is not actually writing to a file, just a hash computation. These hashes do not block our adoption of the chunk-format API in a future change to the index, so leave it as-is. The internals of the algorithms are mostly identical. Previously, the hashfile API used a smaller 8KB buffer instead of the 128KB buffer from read-cache.c. The previous change already unified these sizes. There is one subtle point: we do not pass the CSUM_FSYNC to the finalize_hashfile() method, which differs from most consumers of the hashfile API. The extra fsync() call indicated by this flag causes a significant peformance degradation that is noticeable for quick commands that write the index, such as "git add". Other consumers can absorb this cost with their more complicated data structure organization, and further writing structures such as pack-files and commit-graphs is rarely in the critical path for common user interactions. Some static methods become orphaned in this diff, so I marked them as MAYBE_UNUSED. The diff is much harder to read if they are deleted during this change. Instead, they will be deleted in the following change. In addition to the test suite passing, I computed indexes using the previous binaries and the binaries compiled after this change, and found the index data to be exactly equal. Finally, I did extensive performance testing of "git update-index --force-write" on repos of various sizes, including one with over 2 million paths at HEAD. These tests demonstrated less than 1% difference in behavior. As expected, the performance should be considered unchanged. The previous changes to increase the hashfile buffer size from 8K to 128K ensured this change would not create a peformance regression. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-19csum-file.h: increase hashfile buffer sizeDerrick Stolee3-25/+68
The hashfile API uses a hard-coded buffer size of 8KB and has ever since it was introduced in c38138c (git-pack-objects: write the pack files with a SHA1 csum, 2005-06-26). It performs a similar function to the hashing buffers in read-cache.c, but that code was updated from 8KB to 128KB in f279894 (read-cache: make the index write buffer size 128K, 2021-02-18). The justification there was that do_write_index() improves from 1.02s to 0.72s. Since our end goal is to have the index writing code use the hashfile API, we need to unify this buffer size to avoid a performance regression. There is a buffer, 'check_buffer', that is used to verify the check_fd file descriptor. When this buffer increases to 128K to fit the data being flushed, it causes the stack to overflow the limits placed in the test suite. To avoid issues with stack size, move both 'buffer' and 'check_buffer' to be heap pointers within 'struct hashfile'. The 'check_buffer' member is left as NULL unless check_fd is set in hashfd_check(). Both buffers are cleared as part of finalize_hashfile() which also frees the full structure. Since these buffers are now on the heap, we can adjust their size based on the needs of the consumer. In particular, callers to hashfd_throughput() are expecting to report progress indicators as the buffer flushes. These callers would prefer the smaller 8k buffer to avoid large delays between updates, especially for users with slower networks. When the progress indicator is not used, the larger buffer is preferrable. By adding a new trace2 region in the chunk-format API, we can see that the writing portion of 'git multi-pack-index write' lowers from ~1.49s to ~1.47s on a Linux machine. These effects may be more pronounced or diminished on other filesystems. The end-to-end timing is too noisy to have a definitive change either way. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-19xsize_t: avoid implementation defined behavior when len < 0Jonathan Nieder1-4/+2
The xsize_t helper aims to safely convert an off_t to a size_t, erroring out when a file offset is too large to fit into a memory address. It does this by using two casts: size_t size = (size_t) len; if (len != (off_t) size) ... error out ... On a platform with sizeof(size_t) < sizeof(off_t), this check is safe and correct. The first cast truncates to a size_t by finding the remainder modulo SIZE_MAX+1 (see C99 section 6.3.1.3 Signed and unsigned integers) and the second promotes to an off_t, meaning the result is true if and only if len is representable as a size_t. On other platforms, this two-casts strategy still works well (always succeeds) for len >= 0. But for len < 0, when the first cast succeeds and produces SIZE_MAX + 1 + len, the resulting value is too large to be represented as an off_t, so the second cast produces implementation defined behavior. In practice, it is likely to produce a result of true despite len not being representable as size_t. Simplify by replacing with a more straightforward check: compare len to the relevant bounds and then cast it. (To avoid a -Wsign-compare warning, after checking that len >= 0, we explicitly convert to a sufficiently-large unsigned type before comparing to SIZE_MAX.) In practice, this is not likely to come up since typical callers use nonnegative len. Still, it's helpful to handle this case to make the behavior easy to reason about. Historical note: the original bounds-checking in 46be82dfd0 (xsize_t: check whether we lose bits, 2010-07-28) did not produce this implementation-defined behavior, though it still did not handle negative offsets. It was not until 73560c793a (git-compat-util.h: xsize_t() - avoid -Wsign-compare warnings, 2017-09-21) introduced the double cast that the implementation-defined behavior was triggered. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-19Revert "remote-curl: fall back to basic auth if Negotiate fails"Jeff King2-9/+8
This reverts commit 1b0d9545bb85912a16b367229d414f55d140d3be. That commit does fix the situation it intended to (avoiding Negotiate even when the credentials were provided in the URL), but it creates a more serious regression: we now never hit the conditional for "we had a username and password, tried them, but the server still gave us a 401". That has two bad effects: 1. we never call credential_reject(), and thus a bogus credential stored by a helper will live on forever 2. we never return HTTP_NOAUTH, so the error message the user gets is "The requested URL returned error: 401", instead of "Authentication failed". Doing this correctly seems non-trivial, as we don't know whether the Negotiate auth was a problem. Since this is a regression in the upcoming v2.23.0 release (for which we're in -rc0), let's revert for now and work on a fix separately. (Note that this isn't a pure revert; the previous commit added a test showing the regression, so we can now flip it to expect_success). Reported-by: Ben Humphreys <behumphreys@atlassian.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-19t5551: test http interaction with credential helpersJeff King1-0/+41
We test authentication with http, and we independently test that credential helpers work, but we don't have any tests that cover the two features working together. Let's add two: 1. Make sure that a successful request asks the helper to save the credential. This works as expected. 2. Make sure that a failed request asks the helper to forget the credential. This is marked as expect_failure, as it was recently regressed by 1b0d9545bb (remote-curl: fall back to basic auth if Negotiate fails, 2021-03-22). The symptom here is that the second request should prompt the user, but doesn't. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-18revisions(7): clarify that most commands take a single revision rangeJunio C Hamano1-0/+23
Sometimes new people are confused by how a revision "range" works, in that it is not a random collection of commits but a set of commits that are all connected to each other, and most Git commands work on a single such "range". Give an example to clarify it. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-18hashfile: use write_in_full()Derrick Stolee1-12/+5
The flush() logic in csum-file.c was introduced originally by c38138c (git-pack-objects: write the pack files with a SHA1 csum, 2005-06-26) and a portion of the logic performs similar utility to write_in_full() in wrapper.c. The history of write_in_full() is full of moves and renames, but was originally introduced by 7230e6d (Add write_or_die(), a helper function, 2006-08-21). The point of these sections of code are to flush a write buffer using xwrite() and report errors in the case of disk space issues or other generic input/output errors. The logic in flush() can interpret the output of write_in_full() to provide the correct error messages to users. The logic in the hashfile API has an additional set of logic to augment the progress indicator between calls to xwrite(). This was introduced by 2a128d6 (add throughput display to git-push, 2007-10-30). It seems that since the hashfile's buffer is only 8KB, these additional progress indicators might not be incredibly necessary. Instead, update the progress only when write_in_full() complete. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-18sparse-index: fix uninitialized jumpDerrick Stolee1-1/+1
While testing the sparse-index, I verified a test with --valgrind and it complained about an uninitialized value being used in a jump in the path_matches_pattern_list() method. The line was this one: if (*dtype == DT_UNKNOWN) In the call stack, the culprit was the initialization of the dtype variable in convert_to_sparse_rec(). Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-18parallel-checkout: send the new object_id algo field to the workersMatheus Tavares2-7/+22
An object_id storing a SHA-1 name has some unused bytes at the end of the hash array. Since these bytes are not used, they are usually not initialized to any value either. However, at parallel_checkout.c:send_one_item() the object_id of a cache entry is copied into a buffer which is later sent to a checkout worker through a pipe write(). This makes Valgrind complain about passing uninitialized bytes to a syscall. The worker won't use these uninitialized bytes either, but the warning could confuse someone trying to debug this code; So instead of using oidcpy(), send_one_item() uses hashcpy() to only copy the used/initialized bytes of the object_id, and leave the remaining part with zeros. However, since cf0983213c ("hash: add an algo member to struct object_id", 2021-04-26), using hashcpy() is no longer sufficient here as it won't copy the new algo field from the object_id. Let's add and use a new function which meets both our requirements of copying all the important object_id data while still avoiding the uninitialized bytes, by padding the end of the hash array in the destination object_id. With this change, we also no longer need the destination buffer from send_one_item() to be initialized with zeros, so let's switch from xcalloc() to xmalloc() to make this clear. Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-18t7500: remove non-existant C_LOCALE_OUTPUT prereqTodd Zullinger1-1/+1
The C_LOCALE_OUTPUT prerequisite was removed in b1e079807b (tests: remove last uses of C_LOCALE_OUTPUT, 2021-02-11), where Ævar noted: I'm not leaving the prerequisite itself in place for in-flight changes as there currently are none that introduce new tests that rely on it, and because C_LOCALE_OUTPUT is currently a noop on the master branch we likely won't have any new submissions that use it. One more use of C_LOCALE_OUTPUT did creep in with 3d1bda6b5b (t7500: add tests for --fixup=[amend|reword] options, 2021-03-15). This causes a number of the tests to be skipped by default: ok 35 # SKIP --fixup=reword: incompatible with --all (missing C_LOCALE_OUTPUT) ok 36 # SKIP --fixup=reword: incompatible with --include (missing C_LOCALE_OUTPUT) ok 37 # SKIP --fixup=reword: incompatible with --only (missing C_LOCALE_OUTPUT) ok 38 # SKIP --fixup=reword: incompatible with --interactive (missing C_LOCALE_OUTPUT) ok 39 # SKIP --fixup=reword: incompatible with --patch (missing C_LOCALE_OUTPUT) Remove the C_LOCALE_OUTPUT prerequisite from these tests so they are not skipped. Signed-off-by: Todd Zullinger <tmz@pobox.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-17l10n: tr: v2.32.0-r1Emir Sarı1-3177/+3642
Signed-off-by: Emir Sarı <bitigchi@me.com>
2021-05-17l10n: fr: fixed inconsistenciesrlespinasse1-1/+1
Signed-off-by: rlespinasse <romain.lespinasse@gmail.com>
2021-05-17l10n: fr.po fixed inconsistenciesVincent Tam1-1/+1
Signed-off-by: Vincent Tam <sere@live.hk>
2021-05-17rev-parse: mark die() messages for translationWolfgang Müller1-15/+15
These error messages are intended for the user. Let's touch them up since we're here from the previous commit. Signed-off-by: Wolfgang Müller <wolf@oriole.systems> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-17rev-parse: fix segfault with missing --path-format argumentWolfgang Müller2-0/+6
Calling "git rev-parse --path-format" without an argument segfaults instead of giving an error message. Commit fac60b8925 (rev-parse: add option for absolute or relative path formatting, 2020-12-13) added the argument parsing code but forgot to handle NULL. Returning an error makes sense here because there is no default value we could use. Add a test case to verify. Signed-off-by: Wolfgang Müller <wolf@oriole.systems> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-17l10n: pt_PT: add Portuguese translations part 2Daniel Santos1-625/+592
* Eliminated 'Negation of emptiness' of 'nenhum' (not one/none) * Eliminated 'Negation of emptiness' of 'nada' (nothing) * Transformed 'Não' (No) into affirmative * Some other translations * Transforming 'não' (no) into affirmative * From junção-de-3 to tri-junção Signed-off-by: Daniel Santos <hello@brighterdan.com>
2021-05-17l10n: git.pot: v2.32.0 round 1 (126 new, 26 removed)Jiang Xin1-3141/+3590
Generate po/git.pot from v2.32.0-rc0 for git v2.32.0 l10n round 1. Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
2021-05-17describe-doc: clarify default length of abbreviationAnders Höckersten1-5/+9
Clarify the default length used for the abbreviated form used for commits in git describe. The behavior was modified in Git 2.11.0, but the documentation was not updated to clarify the new behavior. Signed-off-by: Anders Höckersten <anders@hockersten.se> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-17mailinfo: don't discard names under 3 charactersedef1-1/+1
I sometimes receive patches from people with short mononyms, and in my cultural environment these are not uncommon. To my dismay, git-am currently discards their names, and replaces them with their email addresses. Link: https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ Signed-off-by: edef <edef@edef.eu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-17submodule: use the imperative mood to describe the --files optionAlex Henrie1-1/+1
Signed-off-by: Alex Henrie <alexhenrie24@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-17stash: don't translate literal commandsAlex Henrie1-2/+2
Signed-off-by: Alex Henrie <alexhenrie24@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-17git-send-email: add option to specify sendmail commandGregory Anders3-14/+76
The sendemail.smtpServer configuration option and --smtp-server command line option both support using a sendmail-like program to send emails by specifying an absolute file path. However, this is not ideal for the following reasons: 1. It overloads the meaning of smtpServer (now a program is being used for the server?) 2. It doesn't allow for non-absolute paths, arguments, or arbitrary scripting Requiring an absolute path is bad for portability, as the same program may be in different locations on different systems. If a user wishes to pass arguments to their program, they have to use the smtpServerOption option, which is cumbersome (as it must be repeated for each option) and doesn't adhere to normal git conventions. Introduce a new configuration option sendemail.sendmailCmd as well as a command line option --sendmail-cmd that can be used to specify a command (with or without arguments) or shell expression to run to send email. The name of this option is consistent with --to-cmd and --cc-cmd. This invocation honors the user's $PATH so that absolute paths are not necessary. Arbitrary shell expressions are also supported, allowing users to do basic scripting. Give this option a higher precedence over --smtp-server and sendemail.smtpServer, as the new interface is more flexible. For backward compatibility, continue to support absolute paths in --smtp-server and sendemail.smtpServer. Signed-off-by: Gregory Anders <greg@gpanders.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-16Git 2.32-rc0v2.32.0-rc0Junio C Hamano2-1/+17
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-16Merge branch 'ls/typofix'Junio C Hamano1-1/+1
* ls/typofix: pretty: fix a typo in the documentation for %(trailers)
2021-05-16Merge branch 'dl/stash-show-untracked-fixup'Junio C Hamano2-3/+22
The code to handle options recently added to "git stash show" around untracked part of the stash segfaulted when these options were used on a stash entry that does not record untracked part. * dl/stash-show-untracked-fixup: stash show: fix segfault with --{include,only}-untracked t3905: correct test title
2021-05-16Merge branch 'wc/packed-ref-removal-cleanup'Junio C Hamano2-6/+15
When "git update-ref -d" removes a ref that is packed, it left empty directories under $GIT_DIR/refs/ for * wc/packed-ref-removal-cleanup: refs: cleanup directories when deleting packed ref
2021-05-16Merge branch 'lh/maintenance-leakfix'Junio C Hamano1-2/+8
* lh/maintenance-leakfix: maintenance: fix two memory leaks
2021-05-16Merge branch 'ma/typofixes'Junio C Hamano2-2/+2
A couple of trivial typofixes. * ma/typofixes: pretty-formats.txt: add missing space git-repack.txt: remove spurious ")"
2021-05-16Merge branch 'ah/merge-ort-i18n'Junio C Hamano1-6/+15
An i18n fix. * ah/merge-ort-i18n: merge-ort: split "distinct types" message into two translatable messages
2021-05-16Merge branch 'dd/mailinfo-quoted-cr'Junio C Hamano14-34/+376
"git mailinfo" (hence "git am") learned the "--quoted-cr" option to control how lines ending with CRLF wrapped in base64 or qp are handled. * dd/mailinfo-quoted-cr: am: learn to process quoted lines that ends with CRLF mailinfo: allow stripping quoted CR without warning mailinfo: allow squelching quoted CRLF warning mailinfo: warn if CRLF found in decoded base64/QP email mailinfo: stop parsing options manually mailinfo: load default metainfo_charset lazily
2021-05-16Merge branch 'ab/sparse-index-cleanup'Junio C Hamano1-10/+1
Code clean-up. * ab/sparse-index-cleanup: sparse-index.c: remove set_index_sparse_config()
2021-05-16Merge branch 'ab/streaming-simplify'Junio C Hamano1-153/+115
Code clean-up. * ab/streaming-simplify: streaming.c: move {open,close,read} from vtable to "struct git_istream" streaming.c: stop passing around "object_info *" to open() streaming.c: remove {open,close,read}_method_decl() macros streaming.c: remove enum/function/vtbl indirection streaming.c: avoid forward declarations
2021-05-16Merge branch 'mt/parallel-checkout-part-3'Junio C Hamano16-49/+734
The final part of "parallel checkout". * mt/parallel-checkout-part-3: ci: run test round with parallel-checkout enabled parallel-checkout: add tests related to .gitattributes t0028: extract encoding helpers to lib-encoding.sh parallel-checkout: add tests related to path collisions parallel-checkout: add tests for basic operations checkout-index: add parallel checkout support builtin/checkout.c: complete parallel checkout support make_transient_cache_entry(): optionally alloc from mem_pool
2021-05-16Merge branch 'jt/push-negotiation'Junio C Hamano14-100/+455
"git push" learns to discover common ancestor with the receiving end over protocol v2. * jt/push-negotiation: send-pack: support push negotiation fetch: teach independent negotiation (no packfile) fetch-pack: refactor command and capability write fetch-pack: refactor add_haves() fetch-pack: refactor process_acks()
2021-05-16merge: don't translate literal commandsAlex Henrie1-2/+2
These strings have not been modified in any translation, nor should they be. Signed-off-by: Alex Henrie <alexhenrie24@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-14l10n: Update Catalan translationAlex Henrie1-5/+4
Signed-off-by: Alex Henrie <alexhenrie24@gmail.com>
2021-05-14The seventeenth batchJunio C Hamano1-0/+15
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-14Merge branch 'mt/clean-clean'Junio C Hamano1-3/+1
Code clean-up. * mt/clean-clean: clean: remove unnecessary variable
2021-05-14Merge branch 'ow/no-dryrun-in-add-i'Junio C Hamano2-0/+6
"git add -i --dry-run" does not dry-run, which was surprising. The combination of options has taught to error out. * ow/no-dryrun-in-add-i: add: die if both --dry-run and --interactive are given
2021-05-14Merge branch 'jk/p4-locate-branch-point-optim'Junio C Hamano2-11/+12
"git p4" learned to find branch points more efficiently. * jk/p4-locate-branch-point-optim: git-p4: speed up search for branch parent git-p4: ensure complex branches are cloned correctly
2021-05-14Merge branch 'ba/object-info'Junio C Hamano6-0/+183
Over-the-wire protocol learns a new request type to ask for object sizes given a list of object names. * ba/object-info: object-info: support for retrieving object info
2021-05-14Merge branch 'pw/patience-diff-clean-up'Junio C Hamano1-11/+3
Code clean-up. * pw/patience-diff-clean-up: patience diff: remove unused variable patience diff: remove unnecessary string comparisons
2021-05-14Merge branch 'pw/word-diff-zero-width-matches'Junio C Hamano2-3/+12
The word-diff mode has been taught to work better with a word regexp that can match an empty string. * pw/word-diff-zero-width-matches: word diff: handle zero length matches
2021-05-14ref-filter: introduce enum atom_typeZheNing Hu1-69/+128
In the original ref-filter design, it will copy the parsed atom's name and attributes to `used_atom[i].name` in the atom's parsing step, and use it again for string matching in the later specific ref attributes filling step. It use a lot of string matching to determine which atom we need. Introduce the enum "atom_type", each enum value is named as `ATOM_*`, which is the index of each corresponding valid_atom entry. In the first step of the atom parsing, `used_atom.atom_type` will record corresponding enum value from valid_atom entry index, and then in specific reference attribute filling step, only need to compare the value of the `used_atom[i].atom_type` to check the atom type. Helped-by: Junio C Hamano <gitster@pobox.com> Helped-by: Christian Couder <christian.couder@gmail.com> Signed-off-by: ZheNing Hu <adlternative@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-14ref-filter: add objectsize to used_atomZheNing Hu1-6/+13
When the support for "objectsize:disk" was bolted onto the existing support for "objectsize", it didn't follow the usual pattern for handling "atomtype:modifier", which reads the <modifier> part just once while parsing the format string, and store the parsed result in the union in the used_atom structure, so that the string form of it does not have to be parsed over and over at runtime (e.g. in grab_common_values()). Add a new member `objectsize` to the union `used_atom.u`, so that we can separate the check of <modifier> from the check of <atomtype>, this will bring scalability to atom `%(objectsize)`. Signed-off-by: ZheNing Hu <adlternative@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-13l10n: ru.po: fix typo in Russian translationAlexey Roslyakov1-1/+1
Signed-off-by: Alexey Roslyakov <alexey.roslyakov@gmail.com>
2021-05-13t: avoid sed-based chain-linting in some expensive casesJeff King4-3/+21
Commit 878f988350 (t/test-lib: teach --chain-lint to detect broken &&-chains in subshells, 2018-07-11) introduced additional chain-lint tests which add an extra "sed" pipeline to each test we run. This has a measurable impact on runtime. Here are timings with and without a new environment variable (added by this patch) that lets you disable just the additional sed-based chain-lint tests: Benchmark #1: GIT_TEST_CHAIN_LINT_HARDER=1 make test Time (mean ± σ): 64.202 s ± 1.030 s [User: 622.469 s, System: 301.402 s] Range (min … max): 61.571 s … 65.662 s 10 runs Benchmark #2: GIT_TEST_CHAIN_LINT_HARDER=0 make test Time (mean ± σ): 57.591 s ± 0.333 s [User: 529.368 s, System: 270.618 s] Range (min … max): 57.143 s … 58.309 s 10 runs Summary 'GIT_TEST_CHAIN_LINT_HARDER=0 make test' ran 1.11 ± 0.02 times faster than 'GIT_TEST_CHAIN_LINT_HARDER=1 make test' Of course those extra lint checks are doing something useful, so paying a few extra seconds (at least on Linux) isn't so bad (though note the CPU time; we're bounded in our parallel run here by the slowest test, so it really is ~120s of CPU improvement). But we can observe that there are some test scripts where they produce a much stronger effect, and provide less value. In t0027 and t3070 we run a very large number of small tests, all driven by a series of functions/loops which are filling in the test bodies. There we get much less bang for our buck in terms of bug-finding versus CPU cost. This patch introduces a mechanism for controlling when those extra lint checks are run, at two levels: - a user can ask to disable or to force-enable the checks by setting GIT_TEST_CHAIN_LINT_HARDER - if the user hasn't specified a preference, individual scripts can disable the checks by setting GIT_TEST_CHAIN_LINT_HARDER_DEFAULT; scripts which don't set that get the current behavior of enabling them. In addition, this patch flips the default for t0027 and t3070's mass-generated sections to disable the extra checks. Here are the timing results for t0027: Benchmark #1: GIT_TEST_CHAIN_LINT_HARDER=1 ./t0027-auto-crlf.sh Time (mean ± σ): 17.078 s ± 0.848 s [User: 14.878 s, System: 7.075 s] Range (min … max): 15.952 s … 18.421 s 10 runs Benchmark #2: GIT_TEST_CHAIN_LINT_HARDER=0 ./t0027-auto-crlf.sh Time (mean ± σ): 9.063 s ± 0.759 s [User: 7.890 s, System: 3.362 s] Range (min … max): 7.747 s … 10.619 s 10 runs Benchmark #3: ./t0027-auto-crlf.sh Time (mean ± σ): 9.186 s ± 0.881 s [User: 7.957 s, System: 3.427 s] Range (min … max): 7.796 s … 10.498 s 10 runs Summary 'GIT_TEST_CHAIN_LINT_HARDER=0 ./t0027-auto-crlf.sh' ran 1.01 ± 0.13 times faster than './t0027-auto-crlf.sh' 1.88 ± 0.18 times faster than 'GIT_TEST_CHAIN_LINT_HARDER=1 ./t0027-auto-crlf.sh' We can see that disabling the checks for the whole script buys us an almost 2x speedup. But the new default behavior, disabling them only for the mass-generated part, gets us most of that speedup (but still leaves the checks on for further manual tests people might write). As a side note, I'd caution about comparing runtimes and CPU seconds between this timing and the earlier "make test" one. In "make test", we're running a lot of scripts in parallel, so the CPU is throttling down (and thus a CPU second saved here would count for more during a parallel run; the same work takes more CPU seconds there). We get similar results for t3070: Benchmark #1: GIT_TEST_CHAIN_LINT_HARDER=1 ./t3070-wildmatch.sh Time (mean ± σ): 20.054 s ± 3.967 s [User: 16.003 s, System: 8.286 s] Range (min … max): 11.891 s … 23.671 s 10 runs Benchmark #2: GIT_TEST_CHAIN_LINT_HARDER=0 ./t3070-wildmatch.sh Time (mean ± σ): 12.399 s ± 2.256 s [User: 7.542 s, System: 5.342 s] Range (min … max): 9.606 s … 15.727 s 10 runs Benchmark #3: ./t3070-wildmatch.sh Time (mean ± σ): 10.726 s ± 3.476 s [User: 6.790 s, System: 4.365 s] Range (min … max): 5.444 s … 15.376 s 10 runs Summary './t3070-wildmatch.sh' ran 1.16 ± 0.43 times faster than 'GIT_TEST_CHAIN_LINT_HARDER=0 ./t3070-wildmatch.sh' 1.87 ± 0.71 times faster than 'GIT_TEST_CHAIN_LINT_HARDER=1 ./t3070-wildmatch.sh' Again, we get almost a 2x speedup disabling these. In this case, there are no tests not covered by the script's "default to disable" behavior, so the second two benchmarks should be the same (and while they do differ, you can see the variance is quite high but they're within one standard deviation). So it seems like for these two scripts, at least, disabling the extra checks is a reasonable tradeoff. Sadly, the overall runtime of "make test" on my system doesn't get much faster. But that's because we're mostly limited by the cost of the single biggest test. Here are the top-5 tests by wall-clock time from a parallel run, before my patch: 57.9192368984222 t9001-send-email.sh 45.6329638957977 t0027-auto-crlf.sh 32.5278220176697 t3070-wildmatch.sh 22.2701289653778 t7610-mergetool.sh 20.8635759353638 t1701-racy-split-index.sh And after: 57.1476998329163 t9001-send-email.sh 33.776211977005 t0027-auto-crlf.sh 21.3116669654846 t7610-mergetool.sh 20.7748689651489 t1701-racy-split-index.sh 19.6957249641418 t7112-reset-submodule.sh We dropped 12s from t0027, and t3070 dropped off our list entirely at around 16s. In both cases we're bound by t9001, but its slowness is due to the actual tests, so we'll have to deal with it in a different way. But this reduces overall CPU, and means that dealing with t9001 (by improving the speed of send-email or splitting it apart) will let us reduce our overall runtime even on multi-core machines. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-13git-prompt: work under set -uElijah Newren1-3/+3
Commit afda36dbf3 ("git-prompt: include sparsity state as well", 2020-06-21) added the use of some variables to control how to show sparsity state in the git prompt, but implicitly assumed that undefined variables would be treated as the empty string. This breaks users who run under 'set -u'; fix the code to be more explicit. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-13stash show: fix segfault with --{include,only}-untrackedDenton Liu2-2/+21
When `git stash show --include-untracked` or `git stash show --only-untracked` is run on a stash that doesn't include an untracked entry, a segfault occurs. This happens because we do not check whether the untracked entry is actually present and just attempt to blindly dereference it. Ensure that the untracked entry is present before actually attempting to dereference it. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-13t3905: correct test titleDenton Liu1-1/+1
We reference the non-existent option `git stash show --show-untracked` when we really meant `--only-untracked`. Correct the test title accordingly. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-13dir: introduce readdir_skip_dot_and_dotdot() helperElijah Newren11-45/+31
Many places in the code were doing while ((d = readdir(dir)) != NULL) { if (is_dot_or_dotdot(d->d_name)) continue; ...process d... } Introduce a readdir_skip_dot_and_dotdot() helper to make that a one-liner: while ((d = readdir_skip_dot_and_dotdot(dir)) != NULL) { ...process d... } This helper particularly simplifies checks for empty directories. Also use this helper in read_cached_dir() so that our statistics are consistent across platforms. (In other words, read_cached_dir() should have been using is_dot_or_dotdot() and skipping such entries, but did not and left it to treat_path() to detect and mark such entries as path_none.) Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-13dir: update stale description of treat_directory()Derrick Stolee1-7/+6
The documentation comment for treat_directory() was originally written in 095952 (Teach directory traversal about subprojects, 2007-04-11) which was before the 'struct dir_struct' split its bitfield of named options into a 'flags' enum in 7c4c97c0 (Turn the flags in struct dir_struct into a single variable, 2009-02-16). When those flags changed, the comment became stale, since members like 'show_other_directories' transitioned into flags like DIR_SHOW_OTHER_DIRECTORIES. Update the comments for treat_directory() to use these flag names rather than the old member names. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-13dir: traverse into untracked directories if they may have ignored subfilesElijah Newren3-6/+8
A directory that is untracked does not imply that all files under it should be categorized as untracked; in particular, if the caller is interested in ignored files, many files or directories underneath the untracked directory may be ignored. We previously partially handled this right with DIR_SHOW_IGNORED_TOO, but missed DIR_SHOW_IGNORED. It was not obvious, though, because the logic for untracked and excluded files had been fused together making it harder to reason about. The previous commit split that logic out, making it easier to notice that DIR_SHOW_IGNORED was missing. Add it. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-13dir: avoid unnecessary traversal into ignored directoryElijah Newren2-16/+30
The show_other_directories case in treat_directory() tried to handle both excludes and untracked files with the same logic, and mishandled both the excludes and the untracked files in the process, in different ways. Split that logic apart, and then focus on the logic for the excludes; a subsequent commit will address the logic for untracked files. For show_other_directories, an excluded directory means that every path underneath that directory will also be excluded. Given that the calling code requested to just show directories when everything under a directory had the same state (that's what the "DIR_SHOW_OTHER_DIRECTORIES" flag means), we generally do not need to traverse into such directories and can just immediately mark them as ignored (i.e. as path_excluded). The only reason we cannot just immediately return path_excluded is the DIR_HIDE_EMPTY_DIRECTORIES flag and the possibility that the ignored directory is an empty directory. The code previously treated DIR_SHOW_IGNORED_TOO in most cases as an exception as well, which was wrong. It can sometimes reduce the number of cases where we need to recurse (namely if DIR_SHOW_IGNORED_TOO_MODE_MATCHING is also set), but should not be able to increase the number of cases where we need to recurse. Fix the logic accordingly. Some sidenotes about possible confusion with dir.c: * "ignored" often refers to an untracked ignore", i.e. a file which is not tracked which matches one of the ignore/exclusion rules. But you can also have a "tracked ignore", a tracked file that happens to match one of the ignore/exclusion rules and which dir.c has to worry about since "git ls-files -c -i" is supposed to list them. * The dir code often uses "ignored" and "excluded" interchangeably, which you need to keep in mind while reading the code. * "exclude" is used multiple ways in the code: * As noted above, "exclude" is often a synonym for "ignored". * The logic for parsing .gitignore files was re-used in .git/info/sparse-checkout, except there it is used to mark paths that the user wants to *keep*. This was mostly addressed by commit 65edd96aec ("treewide: rename 'exclude' methods to 'pattern'", 2019-09-03), but every once in a while you'll find a comment about "exclude" referring to these patterns that might in fact be in use by the sparse-checkout machinery for inclusion rules. * The word "EXCLUDE" is also used for pathspec negation, as in (pathspec->items[3].magic & PATHSPEC_EXCLUDE) Thus if a user had a .gitignore file containing *~ *.log !settings.log And then ran git add -- 'settings.*' ':^settings.log' Then :^settings.log is a pathspec negation making settings.log not be requested to be added even though all other settings.* files are being added. Also, !settings.log in the gitignore file is a negative exclude pattern meaning that settings.log is normally a file we want to track even though all other *.log files are ignored. Sometimes it feels like dir.c needs its own glossary with its many definitions, including the multiply-defined terms. Reported-by: Jason Gore <Jason.Gore@microsoft.com> Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-13t3001, t7300: add testcase showcasing missed directory traversalElijah Newren2-0/+24
In the last commit, we added a testcase showing that the directory traversal machinery sometimes traverses into directories unnecessarily. Here we show that there are cases where it does the opposite: it does not traverse into directories, despite those directories having important files that need to be flagged. Add a testcase showing that `git ls-files -o -i --directory` can omit some of the files it should be listing, and another showing that `git clean -fX` can fail to clean out some of the expected files. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-13t7300: add testcase showing unnecessary traversal into ignored directoryElijah Newren1-0/+23
The PNPM package manager is apparently creating deeply nested (but ignored) directory structures; traversing them is costly performance-wise, unnecessary, and in some cases is even throwing warnings/errors because the paths are too long to handle on various platforms. Add a testcase that checks for such unnecessary directory traversal. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-13ls-files: error out on -i unless -o or -c are specifiedElijah Newren3-3/+6
ls-files --ignored can be used together with either --others or --cached. After being perplexed for a bit and digging in to the code, I assumed that ls-files -i was just broken and not printing anything and I had a nice patch ready to submit when I finally realized that -i can be used with --cached to find tracked ignores. While that was a mistake on my part, and a careful reading of the documentation could have made this more clear, I suspect this is an error others are likely to make as well. In fact, of two uses in our testsuite, I believe one of the two did make this error. In t1306.13, there are NO tracked files, and all the excludes built up and used in that test and in previous tests thus have to be about untracked files. However, since they were looking for an empty result, the mistake went unnoticed as their erroneous command also just happened to give an empty answer. -i will most the time be used with -o, which would suggest we could just make -i imply -o in the absence of either a -o or -c, but that would be a backward incompatible break. Instead, let's just flag -i without either a -o or -c as an error, and update the two relevant testcases to specify their intent. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-13dir: report number of visited directories and paths with trace2Elijah Newren3-1/+15
Provide more statistics in trace2 output that include the number of directories and total paths visited by the directory traversal logic. Subsequent patches will take advantage of this to ensure we do not unnecessarily traverse into ignored directories. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-13dir: convert trace calls to trace2 equivalentsElijah Newren3-101/+162
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-13pretty: fix a typo in the documentation for %(trailers)Louis Sautier1-1/+1
Signed-off-by: Louis Sautier <sautier.louis@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-13Makefile: make PERL_DEFINES recursively expandedÆvar Arnfjörð Bjarmason1-1/+1
Since 07d90eadb50 (Makefile: add Perl runtime prefix support, 2018-04-10) PERL_DEFINES has been a simply-expanded variable, let's make it recursively expanded instead. This change doesn't matter for the correctness of the logic. Whether we used simply-expanded or recursively expanded didn't change what we wrote out in GIT-PERL-DEFINES, but being consistent with other rules makes this easier to understand. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-12remote-curl: fix clone on sha256 reposEric Wong1-0/+2
The remote-https process needs to update it's own instance of `the_repository' when it sees an HTTP(S) remote is using sha256. Without this, parse_oid_hex() fails to handle sha256 OIDs when it's eventually called by parse_fetch(). Tested with: git clone https://yhbt.net/sha256test.git GIT_SMART_HTTP=0 git clone https://yhbt.net/sha256test.git (plain http:// also works) Cloning the URL via git:// required no changes Signed-off-by: Eric Wong <e@80x24.org> Acked-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>