aboutsummaryrefslogtreecommitdiffstats
path: root/t
AgeCommit message (Collapse)AuthorFilesLines
2019-05-13Merge branch 'pw/clean-sequencer-state-upon-final-commit'Junio C Hamano2-0/+75
"git chery-pick" (and "revert" that shares the same runtime engine) that deals with multiple commits got confused when the final step gets stopped with a conflict and the user concluded the sequence with "git commit". Attempt to fix it by cleaning up the state files used by these commands in such a situation. * pw/clean-sequencer-state-upon-final-commit: fix cherry-pick/revert status after commit commit/reset: try to clean up sequencer state
2019-05-13Merge branch 'jk/perf-aggregate-wo-libjson'Junio C Hamano1-2/+2
The script to aggregate perf result unconditionally depended on libjson-perl even though it did not have to, which has been corrected. * jk/perf-aggregate-wo-libjson: t/perf: depend on perl JSON only when using --codespeed
2019-05-13Merge branch 'jk/p5302-avoid-collision-check-cost'Junio C Hamano1-13/+18
Fix index-pack perf test so that the repeated invocations always run in an empty repository, which emulates the initial clone situation better. * jk/p5302-avoid-collision-check-cost: p5302: create the repo in each index-pack test
2019-05-13Merge branch 'ew/repack-with-bitmaps-by-default'Junio C Hamano4-7/+21
The connectivity bitmaps are created by default in bare repositories now; also the pathname hash-cache is created by default to avoid making crappy deltas when repacking. * ew/repack-with-bitmaps-by-default: pack-objects: default to writing bitmap hash-cache t5310: correctly remove bitmaps for jgit test repack: enable bitmaps by default on bare repos
2019-05-13Merge branch 'js/partial-clone-connectivity-check'Junio C Hamano1-0/+26
During an initial "git clone --depth=..." partial clone, it is pointless to spend cycles for a large portion of the connectivity check that enumerates and skips promisor objects (which by definition is all objects fetched from the other side). This has been optimized out. * js/partial-clone-connectivity-check: t/perf: add perf script for partial clones clone: do faster object check for partial clones
2019-05-13Merge branch 'jh/trace2-sid-fix'Junio C Hamano3-21/+113
Polishing of the new trace2 facility continues. The system-level configuration can specify site-wide trace2 settings, which can be overridden with per-user configuration and environment variables. * jh/trace2-sid-fix: trace2: fixup access problem on /etc/gitconfig in read_very_early_config trace2: update docs to describe system/global config settings trace2: make SIDs more unique trace2: clarify UTC datetime formatting trace2: report peak memory usage of the process trace2: use system/global config for default trace2 settings config: add read_very_early_config() trace2: find exec-dir before trace2 initialization trace2: add absolute elapsed time to start event trace2: refactor setting process starting time config: initialize opts structure in repo_read_config()
2019-05-09Merge branch 'jt/submodule-repo-is-with-worktree'Junio C Hamano1-0/+24
The logic to tell if a Git repository has a working tree protects "git branch -D" from removing the branch that is currently checked out by mistake. The implementation of this logic was broken for repositories with unusual name, which unfortunately is the norm for submodules these days. This has been fixed. * jt/submodule-repo-is-with-worktree: worktree: update is_bare heuristics
2019-05-09Merge branch 'jk/prune-optim'Junio C Hamano1-0/+8
A follow-up test for an earlier "git prune" improvements. * jk/prune-optim: t5304: add a test for pruning with bitmaps
2019-05-09Merge branch 'js/misc-doc-fixes'Junio C Hamano8-24/+230
"make check-docs", "git help -a", etc. did not account for cases where a particular build may deliberately omit some subcommands, which has been corrected. * js/misc-doc-fixes: Turn `git serve` into a test helper test-tool: handle the `-C <directory>` option just like `git` check-docs: do not bother checking for legacy scripts' documentation docs: exclude documentation for commands that have been excluded check-docs: allow command-list.txt to contain excluded commands help -a: do not list commands that are excluded from the build Makefile: drop the NO_INSTALL variable remote-testgit: move it into the support directory for t5801
2019-05-09Merge branch 'dr/ref-filter-push-track-fix'Junio C Hamano1-2/+11
%(push:track) token used in the --format option to "git for-each-ref" and friends was not showing the right branch, which has been fixed. * dr/ref-filter-push-track-fix: ref-filter: use correct branch for %(push:track)
2019-05-09Merge branch 'jt/clone-server-option'Junio C Hamano1-0/+41
"git clone" learned a new --server-option option when talking over the protocol version 2. * jt/clone-server-option: clone: send server options when using protocol v2 transport: die if server options are unsupported
2019-05-09Merge branch 'tb/unexpected'Junio C Hamano5-12/+133
Code tightening against a "wrong" object appearing where an object of a different type is expected, instead of blindly assuming that the connection between objects are correctly made. * tb/unexpected: rev-list: detect broken root trees rev-list: let traversal die when --missing is not in use get_commit_tree(): return NULL for broken tree list-objects.c: handle unexpected non-tree entries list-objects.c: handle unexpected non-blob entries t: introduce tests for unexpected object types t: move 'hex2oct' into test-lib-functions.sh
2019-05-09Merge branch 'nd/sha1-name-c-wo-the-repository'Junio C Hamano1-5/+1
Further code clean-up to allow the lowest level of name-to-object mapping layer to work with a passed-in repository other than the default one. * nd/sha1-name-c-wo-the-repository: (34 commits) sha1-name.c: remove the_repo from get_oid_mb() sha1-name.c: remove the_repo from other get_oid_* sha1-name.c: remove the_repo from maybe_die_on_misspelt_object_name submodule-config.c: use repo_get_oid for reading .gitmodules sha1-name.c: add repo_get_oid() sha1-name.c: remove the_repo from get_oid_with_context_1() sha1-name.c: remove the_repo from resolve_relative_path() sha1-name.c: remove the_repo from diagnose_invalid_index_path() sha1-name.c: remove the_repo from handle_one_ref() sha1-name.c: remove the_repo from get_oid_1() sha1-name.c: remove the_repo from get_oid_basic() sha1-name.c: remove the_repo from get_describe_name() sha1-name.c: remove the_repo from get_oid_oneline() sha1-name.c: add repo_interpret_branch_name() sha1-name.c: remove the_repo from interpret_branch_mark() sha1-name.c: remove the_repo from interpret_nth_prior_checkout() sha1-name.c: remove the_repo from get_short_oid() sha1-name.c: add repo_for_each_abbrev() sha1-name.c: store and use repo in struct disambiguate_state sha1-name.c: add repo_find_unique_abbrev_r() ...
2019-05-09Merge branch 'cc/replace-graft-peel-tags'Junio C Hamano1-3/+28
When given a tag that points at a commit-ish, "git replace --graft" failed to peel the tag before writing a replace ref, which did not make sense because the old graft mechanism the feature wants to mimick only allowed to replace one commit object with another. This has been fixed. * cc/replace-graft-peel-tags: replace: peel tag when passing a tag first to --graft replace: peel tag when passing a tag as parent to --graft t6050: redirect expected error output to a file t6050: use test_line_count instead of wc -l
2019-05-09Merge branch 'js/trace2-to-directory'Junio C Hamano1-0/+15
The trace2 tracing facility learned to auto-generate a filename when told to log to a directory. * js/trace2-to-directory: trace2: write to directory targets
2019-05-09Merge branch 'dl/merge-cleanup-scissors-fix'Junio C Hamano5-80/+297
The list of conflicted paths shown in the editor while concluding a conflicted merge was shown above the scissors line when the clean-up mode is set to "scissors", even though it was commented out just like the list of updated paths and other information to help the user explain the merge better. * dl/merge-cleanup-scissors-fix: cherry-pick/revert: add scissors line on merge conflict sequencer.c: save and restore cleanup mode merge: add scissors line on merge conflict merge: cleanup messages like commit parse-options.h: extract common --cleanup option commit: extract cleanup_mode functions to sequencer t7502: clean up style t7604: clean up style t3507: clean up style t7600: clean up style
2019-05-09Merge branch 'pw/sequencer-cleanup-with-signoff-x-fix'Junio C Hamano1-0/+20
"git cherry-pick" run with the "-x" or the "--signoff" option used to (and more importantly, ought to) clean up the commit log message with the --cleanup=space option by default, but this has been broken since late 2017. This has been fixed. * pw/sequencer-cleanup-with-signoff-x-fix: sequencer: fix cleanup with --signoff and -x
2019-05-09Merge branch 'km/empty-repo-is-still-a-repo'Junio C Hamano3-1/+72
Running "git add" on a repository created inside the current repository is an explicit indication that the user wants to add it as a submodule, but when the HEAD of the inner repository is on an unborn branch, it cannot be added as a submodule. Worse, the files in its working tree can be added as if they are a part of the outer repository, which is not what the user wants. These problems are being addressed. * km/empty-repo-is-still-a-repo: add: error appropriately on repository with no commits dir: do not traverse repositories with no commits submodule: refuse to add repository with no commits
2019-05-09Merge branch 'dl/warn-tagging-a-tag'Junio C Hamano1-0/+11
"git tag" learned to give an advice suggesting it might be a mistake when creating an annotated or signed tag that points at another tag. * dl/warn-tagging-a-tag: tag: advise on nested tags tag: fix formatting
2019-05-09Merge branch 'en/merge-directory-renames'Junio C Hamano3-63/+415
"git merge-recursive" backend recently learned a new heuristics to infer file movement based on how other files in the same directory moved. As this is inherently less robust heuristics than the one based on the content similarity of the file itself (rather than based on what its neighbours are doing), it sometimes gives an outcome unexpected by the end users. This has been toned down to leave the renamed paths in higher/conflicted stages in the index so that the user can examine and confirm the result. * en/merge-directory-renames: merge-recursive: switch directory rename detection default merge-recursive: give callers of handle_content_merge() access to contents merge-recursive: track information associated with directory renames t6043: fix copied test description to match its purpose merge-recursive: switch from (oid,mode) pairs to a diff_filespec merge-recursive: cleanup handle_rename_* function signatures merge-recursive: track branch where rename occurred in rename struct merge-recursive: remove ren[12]_other fields from rename_conflict_info merge-recursive: shrink rename_conflict_info merge-recursive: move some struct declarations together merge-recursive: use 'ci' for rename_conflict_info variable name merge-recursive: rename locals 'o' and 'a' to 'obuf' and 'abuf' merge-recursive: rename diff_filespec 'one' to 'o' merge-recursive: rename merge_options argument from 'o' to 'opt' Use 'unsigned short' for mode, like diff_filespec does
2019-05-05t/perf: add perf script for partial clonesJeff King1-0/+26
We don't cover the partial clone feature at all in t/perf. Let's at least run a few basic tests so that we'll notice any regressions. We'll do a no-blob clone, and split it into two parts: the actual object transfer, and the subsequent checkout (which will of course require another transfer to get the blobs). That will help us more clearly assess the performance of each. There are obviously a lot more possibilities besides just a no-blob partial clone, but this should serve as a canary that alerts us to any generic slow-downs (and we can add more tests later for cases that aren't exercised here). There are a few non-ideal things here that make this not an entirely accurate test, but are probably OK for our purposes: 1. We have to do some extra prep/cleanup work inside the timing tests, since they impact the on-disk state and the perf harness may run each one multiple times. In practice this is probably OK, since these bits should be much less expensive than the operations we are measuring. 2. The clone time is likely to be dominated by the server's object enumeration. In the real world, a repo large enough to drive people to partial clones is likely to have reachability bitmaps enabled. And in the opposite direction, our object transfer is happening at the speed of a local pipe, whereas in the real world it would bottle-neck on the network. So any percentage speedups should be taken with a grain of salt. But hopefully any regressions will produce enough of an effect to be noticeable. This script also demonstrates the recent improvement from dfa33a298d (clone: do faster object check for partial clones, 2019-04-19): Test dfa33a298d^ dfa33a298d ------------------------------------------------------------------------- 5600.2: clone without blobs 18.41(22.72+1.09) 6.83(11.65+0.50) -62.9% 5600.3: checkout of result 1.82(3.24+0.26) 1.84(3.24+0.26) +1.1% Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-25Merge branch 'bc/send-email-qp-cr'Junio C Hamano1-0/+14
"git send-email" has been taught to use quoted-printable when the payload contains carriage-return. The use of the mechanism is in line with the design originally added the codepath that chooses QP when the payload has overly long lines. * bc/send-email-qp-cr: send-email: default to quoted-printable when CR is present
2019-04-25Merge branch 'nd/submodule-foreach-quiet'Junio C Hamano1-0/+10
"git submodule foreach <command> --quiet" did not pass the option down correctly, which has been corrected. * nd/submodule-foreach-quiet: submodule foreach: fix "<command> --quiet" not being respected
2019-04-25Merge branch 'js/iso8895-test-on-apfs'Junio C Hamano1-0/+7
Test fix on APFS that is incapable of store paths in Latin-1. * js/iso8895-test-on-apfs: t9822: skip tests if file names cannot be ISO-8859-1 encoded
2019-04-25Merge branch 'jc/gettext-test-fix'Junio C Hamano1-0/+1
The GETTEXT_POISON test option has been quite broken ever since it was made runtime-tunable, which has been fixed. * jc/gettext-test-fix: gettext tests: export the restored GIT_TEST_GETTEXT_POISON
2019-04-25Merge branch 'jk/fetch-reachability-error-fix'Junio C Hamano2-7/+31
Code clean-up and a fix for "git fetch" by an explicit object name (as opposed to fetching refs by name). * jk/fetch-reachability-error-fix: fetch: do not consider peeled tags as advertised tips remote.c: make singular free_ref() public fetch: use free_refs() pkt-line: prepare buffer before handling ERR packets upload-pack: send ERR packet for non-tip objects t5530: check protocol response for "not our ref" t5516: drop ok=sigpipe from unreachable-want tests
2019-04-25Merge branch 'jk/xmalloc'Junio C Hamano1-1/+1
The code is updated to check the result of memory allocation before it is used in more places, by using xmalloc and/or xcalloc calls. * jk/xmalloc: progress: use xmalloc/xcalloc xdiff: use xmalloc/xrealloc xdiff: use git-compat-util test-prio-queue: use xmalloc
2019-04-25Merge branch 'km/t3000-retitle'Junio C Hamano1-1/+1
A test update. * km/t3000-retitle: t3000 (ls-files -o): widen description to reflect current tests
2019-04-25Merge branch 'js/t3301-unbreak-notes-test'Junio C Hamano1-2/+3
Test fix. * js/t3301-unbreak-notes-test: t3301: fix false negative
2019-04-25Merge branch 'sg/blame-in-bare-start-at-head'Junio C Hamano1-0/+8
"git blame -- path" in a non-bare repository starts blaming from the working tree, and the same command in a bare repository errors out because there is no working tree by definition. The command has been taught to instead start blaming from the commit at HEAD, which is more useful. * sg/blame-in-bare-start-at-head: blame: default to HEAD in a bare repo when no start commit is given
2019-04-25Merge branch 'jt/batch-fetch-blobs-in-diff'Junio C Hamano1-0/+103
While running "git diff" in a lazy clone, we can upfront know which missing blobs we will need, instead of waiting for the on-demand machinery to discover them one by one. Aim to achieve better performance by batching the request for these promised blobs. * jt/batch-fetch-blobs-in-diff: diff: batch fetching of missing blobs sha1-file: support OBJECT_INFO_FOR_PREFETCH
2019-04-25Merge branch 'dl/submodule-set-branch'Junio C Hamano2-0/+102
"git submodule" learns "set-branch" subcommand that allows the submodule.*.branch settings to be modified. * dl/submodule-set-branch: submodule: teach set-branch subcommand submodule--helper: teach config subcommand --unset git-submodule.txt: "--branch <branch>" option defaults to 'master'
2019-04-25Merge branch 'jk/revision-rewritten-parents-in-prio-queue'Junio C Hamano1-0/+18
Performance fix for "rev-list --parents -- pathspec". * jk/revision-rewritten-parents-in-prio-queue: revision: use a prio_queue to hold rewritten parents
2019-04-25Merge branch 'bc/hash-transition-16'Junio C Hamano1-1/+2
Conversion from unsigned char[20] to struct object_id continues. * bc/hash-transition-16: (35 commits) gitweb: make hash size independent Git.pm: make hash size independent read-cache: read data in a hash-independent way dir: make untracked cache extension hash size independent builtin/difftool: use parse_oid_hex refspec: make hash size independent archive: convert struct archiver_args to object_id builtin/get-tar-commit-id: make hash size independent get-tar-commit-id: parse comment record hash: add a function to lookup hash algorithm by length remote-curl: make hash size independent http: replace sha1_to_hex http: compute hash of downloaded objects using the_hash_algo http: replace hard-coded constant with the_hash_algo http-walker: replace sha1_to_hex http-push: remove remaining uses of sha1_to_hex http-backend: allow 64-character hex names http-push: convert to use the_hash_algo builtin/pull: make hash-size independent builtin/am: make hash size independent ...
2019-04-25Merge branch 'en/fast-import-parsing-fix'Junio C Hamano1-0/+37
"git fast-import" update. * en/fast-import-parsing-fix: fast-import: fix erroneous handling of get-mark with empty orphan commits fast-import: only allow cat-blob requests where it makes sense fast-import: check most prominent commands first git-fast-import.txt: fix wording about where ls command can appear t9300: demonstrate bug with get-mark and empty orphan commits
2019-04-25Merge branch 'ab/test-lib-pass-trace2-env'Junio C Hamano1-0/+1
Allow tracing of Git executable while running the testsuite. * ab/test-lib-pass-trace2-env: test-lib: whitelist GIT_TR2_* in the environment
2019-04-25Merge branch 'ab/commit-graph-fixes'Junio C Hamano1-6/+36
Code cleanup with more careful error checking before using data read from the commit-graph file. * ab/commit-graph-fixes: commit-graph: improve & i18n error messages commit-graph write: don't die if the existing graph is corrupt commit-graph verify: detect inability to read the graph commit-graph: don't pass filename to load_commit_graph_one_fd_st() commit-graph: don't early exit(1) on e.g. "git status" commit-graph: fix segfault on e.g. "git status" commit-graph tests: test a graph that's too small commit-graph tests: split up corrupt_graph_and_verify()
2019-04-25Merge branch 'ab/gc-reflog'Junio C Hamano2-8/+36
Fix various glitches in "git gc" around reflog handling. * ab/gc-reflog: gc: handle & check gc.reflogExpire config reflog tests: assert lack of early exit with expiry="never" reflog tests: test for the "points nowhere" warning reflog tests: make use of "test_config" idiom gc: refactor a "call me once" pattern gc: convert to using the_hash_algo gc: remove redundant check for gc_auto_threshold
2019-04-25Merge branch 'nd/checkout-m'Junio C Hamano1-7/+2
"git checkout -m <other>" was about carrying the differences between HEAD and the working-tree files forward while checking out another branch, and ignored the differences between HEAD and the index. The command has been taught to abort when the index and the HEAD are different. * nd/checkout-m: checkout: prevent losing staged changes with --merge read-tree: add --quiet unpack-trees: rename "gently" flag to "quiet" unpack-trees: keep gently check inside add_rejected_path
2019-04-25Merge branch 'js/difftool-no-index'Junio C Hamano2-1/+11
"git difftool" can now run outside a repository. * js/difftool-no-index: difftool: allow running outside Git worktrees with --no-index parse-options: make OPT_ARGUMENT() more useful difftool: remove obsolete (and misleading) comment
2019-04-25Merge branch 'pw/cherry-pick-continue'Junio C Hamano1-0/+24
"git cherry-pick --options A..B", after giving control back to the user to ask help resolving a conflicted step, did not honor the options it originally received, which has been corrected. * pw/cherry-pick-continue: cherry-pick --continue: remember options cherry-pick: demonstrate option amnesia sequencer: break some long lines
2019-04-25Merge branch 'jk/server-info-rabbit-hole'Junio C Hamano2-7/+23
Code clean-up around a much-less-important-than-it-used-to-be update_server_info() funtion. * jk/server-info-rabbit-hole: update_info_refs(): drop unused force parameter server-info: drop objdirlen pointer arithmetic server-info: drop nr_alloc struct member server-info: use strbuf to read old info/packs file server-info: simplify cleanup in parse_pack_def() server-info: fix blind pointer arithmetic http: simplify parsing of remote objects/info/packs packfile: fix pack basename computation midx: check both pack and index names for containment t5319: drop useless --buffer from cat-file t5319: fix bogus cat-file argument pack-revindex: open index if necessary packfile.h: drop extern from function declarations
2019-04-25Merge branch 'nd/diff-parseopt-4'Junio C Hamano1-2/+1
Fourth batch to teach the diff machinery to use the parse-options API. * nd/diff-parseopt-4: am: avoid diff_opt_parse() diff --no-index: use parse_options() instead of diff_opt_parse() range-diff: use parse_options() instead of diff_opt_parse() diff.c: allow --no-color-moved-ws diff-parseopt: convert --color-moved-ws diff-parseopt: convert --[no-]color-moved diff-parseopt: convert --inter-hunk-context diff-parseopt: convert --no-prefix diff-parseopt: convert --line-prefix diff-parseopt: convert --[src|dst]-prefix diff-parseopt: convert --[no-]abbrev diff-parseopt: convert --diff-filter diff-parseopt: convert --find-object diff-parseopt: convert -O diff-parseopt: convert --pickaxe-all|--pickaxe-regex diff-parseopt: convert -S|-G diff-parseopt: convert -l diff-parseopt: convert -z diff-parseopt: convert --ita-[in]visible-in-index diff-parseopt: convert --ws-error-highlight
2019-04-25Merge branch 'jk/unused-params-even-more'Junio C Hamano1-2/+2
Code cleanup. * jk/unused-params-even-more: parse_opt_ref_sorting: always use with NONEG flag pretty: drop unused strbuf from parse_padding_placeholder() pretty: drop unused "type" parameter in needs_rfc2047_encoding() parse-options: drop unused ctx parameter from show_gitcomp() fetch_pack(): drop unused parameters report_path_error(): drop unused prefix parameter unpack-trees: drop unused error_type parameters unpack-trees: drop name_entry from traverse_by_cache_tree() test-date: drop unused "now" parameter from parse_dates() update-index: drop unused prefix_length parameter from do_reupdate() log: drop unused "len" from show_tagger() log: drop unused rev_info from early output revision: drop some unused "revs" parameters
2019-04-25Merge branch 'sg/test-atexit'Junio C Hamano68-247/+136
Test framework update to more robustly clean up leftover files and processes after tests are done. * sg/test-atexit: t9811-git-p4-label-import: fix pipeline negation git p4 test: disable '-x' tracing in the p4d watchdog loop git p4 test: simplify timeout handling git p4 test: clean up the p4d cleanup functions git p4 test: use 'test_atexit' to kill p4d and the watchdog process t0301-credential-cache: use 'test_atexit' to stop the credentials helper tests: use 'test_atexit' to stop httpd git-daemon: use 'test_atexit` to stop 'git-daemon' test-lib: introduce 'test_atexit' t/lib-git-daemon: make sure to kill the 'git-daemon' process test-lib: fix interrupt handling with 'dash' and '--verbose-log -x'
2019-04-25Merge branch 'bp/post-index-change-hook'Junio C Hamano1-0/+144
A new hook "post-index-change" is called when the on-disk index file changes, which can help e.g. a virtualized working tree implementation. * bp/post-index-change-hook: read-cache: add post-index-change hook
2019-04-24t/perf: depend on perl JSON only when using --codespeedJeff King1-2/+2
Commit 05eb1c37ed (perf/aggregate: implement codespeed JSON output, 2018-01-05) added a dependency on the perl JSON module to show output from aggregate.perl, but we only need it when the user asks for --codespeed output. While the module is pretty common, it's not part of the base system, and this dependency can get in the way of producing the default human-readable output. Let's bump the "use" down to a "require" in the code path that needs it, which will be interpreted at run-time instead of compile-time. People not using "--codespeed" won't even load the module, and anybody using it should see the same results (including the same perl error if they don't have it). Note that this skips the importing step, so we'll have to fully qualify our function call. We could accomplish the same thing in other ways. E.g., calling JSON->import() ourselves, or wrapping "use JSON" in an eval. Since there's only one such call, this seems like the least-magical way of doing it. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-23p5302: create the repo in each index-pack testJeff King1-13/+18
The p5302 script runs "index-pack --stdin" in each timing test. It does two things to try to get good timings: 1. we do the repo creation in a separate (non-timed) setup test, so that our timing is purely the index-pack run 2. we use a separate repo for each test; this is important because the presence of existing objects in the repo influences the result (because we'll end up doing collision checks against them) But this forgets one thing: we generally run each timed test multiple times to reduce the impact of noise. Which means that repeats of each test after the first will be subject to the collision slowdown from point 2, and we'll generally just end up taking the first time anyway. Instead, let's create the repo in the test (effectively undoing point 1). That does add a constant amount of extra work to each iteration, but it's quite small compared to the actual effects we're interested in measuring. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-22Merge branch 'js/spell-out-options-in-tests'Junio C Hamano11-43/+66
The tests have been updated not to rely on the abbreviated option names the parse-options API offers, to protect us from an abbreviated form of an option that used to be unique within the command getting non-unique when a new option that share the same prefix is added. * js/spell-out-options-in-tests: tests: disallow the use of abbreviated options (by default) tests (pack-objects): use the full, unabbreviated `--revs` option tests (status): spell out the `--find-renames` option in full tests (push): do not abbreviate the `--follow-tags` option t5531: avoid using an abbreviated option t7810: do not abbreviate `--no-exclude-standard` nor `--invert-match` tests (rebase): spell out the `--force-rebase` option tests (rebase): spell out the `--keep-empty` option
2019-04-22Merge branch 'nd/include-if-wildmatch'Junio C Hamano1-0/+13
A buglet in configuration parser has been fixed. * nd/include-if-wildmatch: config: correct '**' matching in includeIf patterns
2019-04-22Merge branch 'tg/stash-in-c-show-default-to-p-fix'Junio C Hamano1-0/+18
A regression fix. * tg/stash-in-c-show-default-to-p-fix: stash: setup default diff output format if necessary
2019-04-22Merge branch 'js/stash-in-c-pathspec-fix'Junio C Hamano1-0/+6
Further fixes to "git stash" reimplemented in C. * js/stash-in-c-pathspec-fix: stash: pass pathspec as pointer built-in stash: handle :(glob) pathspecs again legacy stash: fix "rudimentary backport of -q"
2019-04-22Merge branch 'ps/stash-in-c'Junio C Hamano3-66/+227
"git stash" rewritten in C. * ps/stash-in-c: (28 commits) tests: add a special setup where stash.useBuiltin is off stash: optionally use the scripted version again stash: add back the original, scripted `git stash` stash: convert `stash--helper.c` into `stash.c` stash: replace all `write-tree` child processes with API calls stash: optimize `get_untracked_files()` and `check_changes()` stash: convert save to builtin stash: make push -q quiet stash: convert push to builtin stash: convert create to builtin stash: convert store to builtin stash: convert show to builtin stash: convert list to builtin stash: convert pop to builtin stash: convert branch to builtin stash: convert drop and clear to builtin stash: convert apply to builtin stash: mention options in `show` synopsis stash: add tests for `git stash show` config stash: rename test cases to be more descriptive ...
2019-04-21worktree: update is_bare heuristicsJonathan Tan1-0/+24
When "git branch -D <name>" is run, Git usually first checks if that branch is currently checked out. But this check is not performed if the Git directory of that repository is not at "<repo>/.git", which is the case if that repository is a submodule that has its Git directory stored as "super/.git/modules/<repo>", for example. This results in the branch being deleted even though it is checked out. This is because get_main_worktree() in worktree.c sets is_bare on a worktree only using the heuristic that a repo is bare if the worktree's path does not end in "/.git", and not bare otherwise. This is_bare code was introduced in 92718b7438 ("worktree: add details to the worktree struct", 2015-10-08), following a pre-core.bare heuristic. This patch does 2 things: - Teach get_main_worktree() to use is_bare_repository() instead, introduced in 7d1864ce67 ("Introduce is_bare_repository() and core.bare configuration variable", 2007-01-07) and updated in e90fdc39b6 ("Clean up work-tree handling", 2007-08-01). This solves the "git branch -D <name>" problem described above. However... - If a repository has core.bare=1 but the "git" command is being run from one of its secondary worktrees, is_bare_repository() returns false (which is fine, since there is a worktree available). However, treating the main worktree as non-bare when it is bare causes issues: for example, failure to delete a branch from a secondary worktree that is referred to by a main worktree's HEAD, even if that main worktree is bare. In order to avoid that, also check core.bare when setting is_bare. If core.bare=1, trust it, and otherwise, use is_bare_repository(). Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-19t5304: add a test for pruning with bitmapsJeff King1-0/+8
Commit fde67d6896 (prune: use bitmaps for reachability traversal, 2019-02-13) uses bitmaps for pruning when they're available, but only covers this functionality in the t/perf tests. This makes a kind of sense, since the point is that the behaviour is indistinguishable before and after the patch, just faster. But since the bitmap code path is not exercised at all in the regular test suite, it leaves us open to a regression where the behavior does in fact change. The most thorough way to test that would be running the whole suite with bitmaps enabled. But we don't yet have a way to do that, and anyway it's expensive to do so. Let's at least add a basic test that exercises this path and make sure we prune an object we should (and not one that we shouldn't). That would hopefully catch the most obvious breakages early. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-19Turn `git serve` into a test helperJohannes Schindelin6-24/+62
The `git serve` built-in was introduced in ed10cb952d31 (serve: introduce git-serve, 2018-03-15) as a backend to serve Git protocol v2, probably originally intended to be spawned by `git upload-pack`. However, in the version that the protocol v2 patches made it into core Git, `git upload-pack` calls the `serve()` function directly instead of spawning `git serve`; The only reason in life for `git serve` to survive as a built-in command is to provide a way to test the protocol v2 functionality. Meaning that it does not even have to be a built-in that is installed with end-user facing Git installations, but it can be a test helper instead. Let's make it so. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-19test-tool: handle the `-C <directory>` option just like `git`Johannes Schindelin1-0/+19
In preparation for moving `git serve` into `test-tool` (because it really is only used by the test suite), we teach the `test-tool` the useful trick to change the working directory before running the test command, which will avoid introducing subshells in the test code. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-19cherry-pick/revert: add scissors line on merge conflictDenton Liu1-0/+86
Fix a bug where the scissors line is placed after the Conflicts: section, in the case where a merge conflict occurs and commit.cleanup = scissors. Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18merge: add scissors line on merge conflictDenton Liu1-0/+46
This fixes a bug where the scissors line is placed after the Conflicts: section, in the case where a merge conflict occurs and commit.cleanup = scissors. Next, if commit.cleanup = scissors is specified, don't produce a scissors line in commit if one already exists in the MERGE_MSG file. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18merge: cleanup messages like commitDenton Liu2-0/+71
This change allows git-merge messages to be cleaned up with the commit.cleanup configuration or --cleanup option, just like how git-commit does it. We also give git-pull the option of --cleanup so that it can also take advantage of this change. Finally, add testing to ensure that messages are properly cleaned up. Note that some newlines that were added to the commit message were removed so that if a file were read via -F, it would be copied faithfully. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18t7502: clean up styleDenton Liu1-40/+53
Refactor out Git commands that were upstream of a pipe. Remove spaces after "> ". Indent here-docs appropriately. Convert echo chains to use the test_write_lines function. Refactor 'sign off' test to use test_cmp instead of comparing variables. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18t7604: clean up styleDenton Liu1-5/+7
Before, we had some Git commands which were upstream of the pipe. This meant that if it produced an error, it would've gone unnoticed. Refactor to place Git commands on their own. Also, while we're at it, remove spaces after redirection operators. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18t3507: clean up styleDenton Liu1-21/+21
Remove space after redirection operators for style. Also, remove a git command which was upstream of a pipe. Finally, let grep and sed open their own input instead of letting the shell redirect the input. Helped-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18t7600: clean up styleDenton Liu1-14/+13
Clean up the 'merge --squash c3 with c7' test by removing some unnecessary braces and removing a pipe. Also, generally cleanup style by unindenting a here-doc, removing stray spaces after a redirection operator and allowing sed to open its own input instead of redirecting input from the shell. Helped-by: SZEDER Gábor <szeder.dev@gmail.com> Helped-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18sequencer: fix cleanup with --signoff and -xPhillip Wood1-0/+20
Before commit 356ee4659b ("sequencer: try to commit without forking 'git commit'", 2017-11-24) when --signoff or -x were given on the command line the commit message was cleaned up with --cleanup=space or commit.cleanup if it was set. Unfortunately this behavior was lost when I implemented committing without forking. Fix this and add some tests to catch future regressions. Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18clone: send server options when using protocol v2Jonathan Tan1-0/+22
Commit 5e3548ef16 ("fetch: send server options when using protocol v2", 2018-04-24) taught "fetch" the ability to send server options when using protocol v2, but not "clone". This ability is triggered by "-o" or "--server-option". Teach "clone" the same ability, except that because "clone" already has "-o" for another parameter, teach "clone" only to receive "--server-option". Explain in the documentation, both for clone and for fetch, that server handling of server options are server-specific. This is similar to receive-pack's handling of push options - currently, they are just sent to hooks to interpret as they see fit. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18transport: die if server options are unsupportedJonathan Tan1-0/+19
Server options were added in commit 5e3548ef16 ("fetch: send server options when using protocol v2", 2018-04-24), supported only for protocol version 2. But if the user specifies server options, and the protocol version being used doesn't support them, the server options are silently ignored. Teach any transport users to die instead in this situation, just like how "push" dies if push options are provided when the server doesn't support them. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18ref-filter: use correct branch for %(push:track)Damien Robert1-2/+11
In ref-filter.c, when processing the atom %(push:track), the ahead/behind values are computed using `stat_tracking_info` which refers to the upstream branch. Fix that by introducing a new flag `for_push` in `stat_tracking_info` in remote.c, which does the same thing but for the push branch. Update the few callers of `stat_tracking_info` to handle this flag. This ensure that whenever we use this function in the future, we are careful to specify is this should apply to the upstream or the push branch. This bug was not detected in t/t6300-for-each-ref.sh because in the test for push:track, both the upstream and the push branches were behind by 1 from the local branch. Change the test so that the upstream branch is behind by 1 while the push branch is ahead by 1. This allows us to test that %(push:track) refers to the correct branch. This changes the expected value of some following tests (by introducing new references), so update them too. Signed-off-by: Damien Robert <damien.olivier.robert+git@gmail.com> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-17fix cherry-pick/revert status after commitPhillip Wood1-0/+36
If the user commits a conflict resolution using `git commit` in the middle of a sequence of cherry-picks/reverts then `git status` missed the fact that a cherry-pick/revert is still in progress. Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-17commit/reset: try to clean up sequencer statePhillip Wood1-0/+39
When cherry-picking or reverting a sequence of commits and if the final pick/revert has conflicts and the user uses `git commit` to commit the conflict resolution and does not run `git cherry-pick --continue` then the sequencer state is left behind. This can cause problems later. In my case I cherry-picked a sequence of commits the last one of which I committed with `git commit` after resolving some conflicts, then a while later, on a different branch I aborted a revert which rewound my HEAD to the end of the cherry-pick sequence on the previous branch. Avoid this potential problem by removing the sequencer state if we're committing or resetting the final pick in a sequence. Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16Merge branch 'sg/t5318-cleanup'Junio C Hamano1-1/+1
Code cleanup. * sg/t5318-cleanup: t5318-commit-graph: remove unused variable
2019-04-16Merge branch 'jt/t5551-protocol-v2-does-not-have-half-auth'Junio C Hamano1-1/+11
Test update. * jt/t5551-protocol-v2-does-not-have-half-auth: t5551: mark half-auth no-op fetch test as v0-only
2019-04-16Merge branch 'jk/http-walker-status-fix'Junio C Hamano1-0/+16
dumb-http walker has been updated to share more error recovery strategy with the normal codepath. * jk/http-walker-status-fix: http: use normalize_curl_result() instead of manual conversion http: normalize curl results for dumb loose and alternates fetches http: factor out curl result code normalization
2019-04-16Merge branch 'jk/refs-double-abort'Junio C Hamano1-0/+16
A corner case bug in the refs API has been corrected. * jk/refs-double-abort: refs/files-backend: don't look at an aborted transaction refs/files-backend: handle packed transaction prepare failure
2019-04-16Merge branch 'tz/completion'Junio C Hamano1-0/+6
The completion helper code now pays attention to repository-local configuration (when available), which allows --list-cmds to honour a repository specific setting of completion.commands, for example. * tz/completion: completion: use __git when calling --list-cmds completion: fix multiple command removals t9902: test multiple removals via completion.commands git: read local config in --list-cmds
2019-04-16Merge branch 'tz/t4038-bash-redirect-target-workaround'Junio C Hamano1-4/+4
Work-around extra warning from bash in our tests. * tz/t4038-bash-redirect-target-workaround: t4038-diff-combined: quote paths with whitespace
2019-04-16Merge branch 'ab/drop-scripted-rebase'Junio C Hamano3-10/+19
Retire scripted "git rebase" implementation. * ab/drop-scripted-rebase: rebase: remove the rebase.useBuiltin setting
2019-04-16Merge branch 'jk/perf-lib-tee'Junio C Hamano1-23/+11
Code cleanup in the test framework. * jk/perf-lib-tee: perf-lib.sh: rely on test-lib.sh for --tee handling
2019-04-16Merge branch 'nd/checkout-f-while-conflicted-fix'Junio C Hamano1-0/+24
"git checkout -f <branch>" while the index has an unmerged path incorrectly left some paths in an unmerged state, which has been corrected. * nd/checkout-f-while-conflicted-fix: unpack-trees: fix oneway_merge accidentally carry over stage index
2019-04-16Merge branch 'jc/format-patch-error-check'Junio C Hamano1-0/+6
"git format-patch" used overwrite an existing patch/cover-letter file. A new "--no-clobber" option stops it. * jc/format-patch-error-check: format-patch: notice failure to open cover letter for writing builtin/log: downcase the beginning of error messages
2019-04-16Merge branch 'js/get-short-oid-drop-cache'Junio C Hamano1-0/+22
A corner-case object name ambiguity while the sequencer machinery is working (e.g. "rebase -i -x") has been (half) fixed. * js/get-short-oid-drop-cache: get_oid(): when an object was not found, try harder sequencer: move stale comment into correct location sequencer: improve error message when an OID could not be parsed rebase -i: demonstrate obscure loose object cache bug
2019-04-16Merge branch 'js/init-db-update-for-mingw'Junio C Hamano1-0/+11
"git init" forgot to read platform-specific repository configuration, which made Windows port to ignore settings of core.hidedotfiles, for example. * js/init-db-update-for-mingw: mingw: respect core.hidedotfiles = false in git-init again
2019-04-16Merge branch 'jt/test-protocol-version'Junio C Hamano14-37/+113
Help developers by making it easier to run most of the tests under different versions of over-the-wire protocols. * jt/test-protocol-version: t5552: compensate for v2 filtering ref adv. tests: fix protocol version for overspecifications t5700: only run with protocol version 1 t5512: compensate for v0 only sending HEAD symrefs t5503: fix overspecification of trace expectation tests: always test fetch of unreachable with v0 t5601: check ssh command only with protocol v0 tests: define GIT_TEST_PROTOCOL_VERSION
2019-04-16submodule-config.c: use repo_get_oid for reading .gitmodulesNguyễn Thái Ngọc Duy1-5/+1
Since 76e9bdc437 (submodule: support reading .gitmodules when it's not in the working tree - 2018-10-25), every time you do git grep --recurse-submodules you are likely to see one warning line per submodule (unless all those submodules also have submodules). On a superproject with plenty of submodules (I've seen one with 67) this is really annoying. The warning was there because we could not resolve extended SHA-1 syntax on a submodule. We can now. Make use of the new API and get rid of the warning. It would be even better if config_with_options() supports multiple repositories too. But one step at a time. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16packfile: fix pack basename computationJeff King2-0/+15
When we have a multi-pack-index that covers many packfiles, we try to avoid opening the .idx for those packfiles. To do that we feed the pack name to midx_contains_pack(). But that function wants to see only the basename, which we compute using strrchr() to find the final slash. But that leaves an extra "/" at the start of our string. We can fix this by incrementing the pointer. That also raises the question of what to do when the name does not have a '/' at all. This should generally not happen (we always find files in "pack/"), but it doesn't hurt to be defensive here. Let's wrap all of that up in a helper function and make it publicly available, since a later patch will need to use it, too. The tests don't notice because there's nothing about opening those .idx files that would cause us to give incorrect output. It's just a little slower. The new test checks this case by corrupting the covered .idx, and then making sure we don't complain about it. We also have to tweak t5570, which intentionally corrupts a .idx file and expects us to notice it. When run with GIT_TEST_MULTI_PACK_INDEX, this will fail since we now will (correctly) not bother opening the .idx at all. We can fix that by unconditionally dropping any midx that's there, which ensures we'll have to read the .idx. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16t5319: drop useless --buffer from cat-fileJeff King1-2/+2
The cat-file --buffer option is the default already when using --batch-all-objects. It doesn't hurt to specify it, but it's nice for the test scripts to model good usage. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16t5319: fix bogus cat-file argumentJeff King1-6/+7
There's no such argument as "--unsorted"; it's spelled "--unordered". But our test failed to notice that cat-file didn't run at all because: 1. It lost the exit code of git on the left-hand side of a pipe. 2. It was comparing two runs of the broken invocation with and without a particular config variable (and indeed, both cases produced no output!). Let's fix the option, but also tweak the helper function to check the exit code. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16trace2: use system/global config for default trace2 settingsJeff Hostetler3-15/+107
Teach git to read the system and global config files for default Trace2 settings. This allows system-wide Trace2 settings to be installed and inherited to make it easier to manage a collection of systems. The original GIT_TR2* environment variables are loaded afterwards and can be used to override the system settings. Only the system and global config files are used. Repo and worktree local config files are ignored. Likewise, the "-c" command line arguments are also ignored. These limits are for performance reasons. (1) For users not using Trace2, there should be minimal overhead to detect that Trace2 is not enabled. In particular, Trace2 should not allocate lots of otherwise unused data strucutres. (2) For accurate performance measurements, Trace2 should be initialized as early in the git process as possible, and before most of the normal git process initialization (which involves discovering the .git directory and reading a hierarchy of config files). Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16trace2: add absolute elapsed time to start eventJeff Hostetler1-6/+6
Add elapsed process time to "start" event to measure the performance of early process startup. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15remote-testgit: move it into the support directory for t5801Johannes Schindelin2-0/+149
The `git-remote-testgit` script is really only used in `t5801-remote-helpers.sh`. It does not even contain any `@@<MAGIC>@@` placeholders that would need to be interpolated via `make git-remote-testgit`. Let's just move it to a new home, decluttering the top-level directory and clarifying that this is just a test helper, not an official Git command that we would want to ever support. Suggested by Ævar Arnfjörð Bjarmason. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15fetch: do not consider peeled tags as advertised tipsJeff King1-0/+11
Our filter_refs() function accidentally considers the target of a peeled tag to be advertised by the server, even though upload-pack on the server side does not consider it so. This can result in the client making a bogus fetch to the server, which will end with the server complaining "not our ref". Whereas the correct behavior is for the client to notice that the server will not allow the request and error out immediately. So as bugs go, this is not very serious (the outcome is the same either way -- the fetch fails). But it's worth making the logic here correct and consistent with other related cases (e.g., fetching an oid that the server did not mention at all). The crux of the issue comes from fdb69d33c4 (fetch-pack: always allow fetching of literal SHA1s, 2017-05-15). After that, the strategy of filter_refs() is basically: - for each advertised ref, try to match it with a "sought" ref provided by the user. Skip any malformed refs (which includes peeled values like "refs/tags/foo^{}"), and place any unmatched items onto the unmatched list. - if there are unmatched sought refs, then put all of the advertised tips into an oidset, including the unmatched ones. - for each sought ref, see if it's in the oidset, in which case it's legal for us to ask the server for it The problem is in the second step. Our list of unmatched refs includes the peeled refs, even though upload-pack does not allow them to be directly fetched. So the simplest fix would be to exclude them during that step. However, we can observe that the unmatched list isn't used for anything else, and is freed at the end. We can just free those malformed refs immediately. That saves us having to check each ref a second time to see if it's malformed. Note that this code only kicks in when "strict" is in effect. I.e., if we are using the v0 protocol and uploadpack.allowReachableSHA1InWant is not in effect. With v2, all oids are allowed, and we do not bother creating or consulting the oidset at all. To future-proof our test against the upcoming GIT_TEST_PROTOCOL_VERSION flag, we'll manually mark it as a v0-only test. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15pkt-line: prepare buffer before handling ERR packetsJeff King1-1/+1
Since 2d103c31c2 (pack-protocol.txt: accept error packets in any context, 2018-12-29), the pktline code will detect an ERR packet and die automatically, saving the caller from dealing with it. But we do so too early in the function, before we have terminated the buffer with a NUL. As a result, passing the ERR message to die() may result in us printing random cruft from a previous packet. This doesn't trigger memory tools like ASan because we reuse the same buffer over and over (so the contents are valid and initialized; they're just stale). We can see demonstrate this by tightening the regex we use to match the error message in t5516; without this patch, git-fetch will accidentally print the capabilities from the (much longer) initial packet we received. By moving the ERR code later in the function we get a few other benefits, too: - we'll now chomp any newline sent by the other side (which is what we want, since die() will add its own newline) - we'll now mention the ERR packet with GIT_TRACE_PACKET Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15upload-pack: send ERR packet for non-tip objectsJeff King2-2/+14
Commit bdb31eada7 (upload-pack: report "not our ref" to client, 2017-02-23) catches the case where a client asks for an object we don't have, and issues a message that the client can show to the user (in addition to dying and writing to stderr). There's a similar case (with the same message) when the client asks for an object which we _do_ have, but which isn't a ref tip (or isn't reachable, when uploadpack.allowReachableSHA1InWant is true). Let's give that one the same treatment, for the same reason (namely that it's more informative to the client than just hanging up, since they won't see our stderr over some protocols). There are two tests here. We cover it most directly in t5530 by invoking upload-pack, which matches the existing "not our ref" test. But a more end-to-end check is that "git fetch" actually shows the message to the client. We're already checking in t5516 that this case fails, so we can just check stderr there, too. Note that even after we started ignoring SIGPIPE in 8bf4becf0c, this could in theory still be racy as described in that commit (because we die() on write failures before pumping the connection for any ERR packets). In practice this should be OK for this case. The server will not actually check reachability until it has received our whole group of "want" lines. And since we have no objects in the repository, we won't send any "have" lines, meaning we're always waiting to read the server response. Note also that this case cannot happen in the v2 protocol, since it allows any available object to be requested. However, we don't have to take any steps to protect against the upcoming GIT_TEST_PROTOCOL_VERSION in our tests: - the tests in t5516 would already need to be skipped under v2, and that is covered by ab0c5f5096 (tests: always test fetch of unreachable with v0, 2019-02-25) - the tests in t5530 invoke upload-pack directly, which will continue to default to v0. Eventually we may have a test setting which uses v2 even for bare upload-pack calls, but we can't override it here until we know what the setting looks like. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15t5530: check protocol response for "not our ref"Jeff King1-2/+3
Back in 9f9aa76130 (upload-pack: Improve error message when bad ref requested, 2010-07-31), we added a test to make sure that we die with a sensible message when the client asks for an object we don't have. Much later, in bdb31eada7 (upload-pack: report "not our ref" to client, 2017-02-23), we started reporting that information via an "ERR" line in the protocol. Let's check that part, as well. While we're touching this test, let's drop the "-q" on the grep calls. Our usual test style just relies on --verbose to control output. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15t5516: drop ok=sigpipe from unreachable-want testsJeff King1-3/+3
We annotated our test_must_fail calls in 8bf4becf0c (add "ok=sigpipe" to test_must_fail and use it to fix flaky tests, 2015-11-27) because the abrupt hangup of the server meant that we'd sometimes fail on read() and sometimes get SIGPIPE on write(). But since 143588949c (fetch: ignore SIGPIPE during network operation, 2019-03-03), we make sure that we end up with a real die(), and our tests no longer need to work around the race. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15gettext tests: export the restored GIT_TEST_GETTEXT_POISONJunio C Hamano1-0/+1
6cdccfce ("i18n: make GETTEXT_POISON a runtime option", 2018-11-08) made the gettext-poison test a runtime option (which was a good move) and adjusted the test framework so that Git commands we run as part of the framework, as opposed to the ones that are part of the test proper, are not affected by the setting. The original value for the GIT_TEST_GETTEXT_POISON environment variable is saved away in another variable and gets unset, and then later the saved value is restored to the environment variable. But the code forgot to export the variable again, which is necessary to restore the "export" bit that was lost when the variable was unset. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15t9822: skip tests if file names cannot be ISO-8859-1 encodedJohannes Schindelin1-0/+7
Most notably, it seems that macOS' APFS does not allow that. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15submodule foreach: fix "<command> --quiet" not being respectedNguyễn Thái Ngọc Duy1-0/+10
Robin reported that git submodule foreach --quiet git pull --quiet origin is not really quiet anymore [1]. "git pull" behaves as if --quiet is not given. This happens because parseopt in submodule--helper will try to parse both --quiet options as if they are foreach's options, not git-pull's. The parsed options are removed from the command line. So when we do pull later, we execute just this git pull origin When calling submodule helper, adding "--" in front of "git pull" will stop parseopt for parsing options that do not really belong to submodule--helper foreach. PARSE_OPT_KEEP_UNKNOWN is removed as a safety measure. parseopt should never see unknown options or something has gone wrong. There are also a couple usage string update while I'm looking at them. While at it, I also add "--" to other subcommands that pass "$@" to submodule--helper. "$@" in these cases are paths and less likely to be --something-like-this. But the point still stands, git-submodule has parsed and classified what are options, what are paths. submodule--helper should never consider paths passed by git-submodule to be options even if they look like one. The test case is also contributed by Robin. [1] it should be quiet before fc1b9243cd (submodule: port submodule subcommand 'foreach' from shell to C, 2018-05-10) because parseopt can't accidentally eat options then. Reported-by: Robin H. Johnson <robbat2@gentoo.org> Tested-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15tests: disallow the use of abbreviated options (by default)Johannes Schindelin3-1/+24
Git's command-line parsers support uniquely abbreviated options, e.g. `git init --ba` would automatically expand `--ba` to `--bare`. This is a very convenient feature in every day life for Git users, in particular when tab completion is not available. However, it is not a good idea to rely on that in Git's test suite, as something that is a unique abbreviation of a command line option today might no longer be a unique abbreviation tomorrow. For example, if a future contribution added a new mode `git init --babyproofing` and a previously-introduced test case used the fact that `git init --ba` expanded to `git init --bare`, that future contribution would now have to touch seemingly unrelated tests just to keep the test suite from failing. So let's disallow abbreviated options in the test suite by default. Note: for ease of implementation, this patch really only touches the `parse-options` machinery: more and more hand-rolled option parsers are converted to use that internal API, and more and more scripts are converted to built-ins (naturally using the parse-options API, too), so in practice this catches most issues, and is definitely the biggest bang for the buck. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15replace: peel tag when passing a tag first to --graftChristian Couder1-0/+11
When passing a tag as the first argument to `git replace --graft`, it can be useful to accept it and use the underlying commit as a the commit that will be replaced. This already works for lightweight tags, but unfortunately for annotated tags we have been using the hash of the tag object instead of the hash of the underlying commit. Especially we would pass the hash of the tag object to replace_object_oid() where we would likely fail with an error like: "error: Objects must be of the same type. 'annotated_replaced_object' points to a replaced object of type 'tag' while 'replacement' points to a replacement object of type 'commit'." This patch fixes that by using the hash of the underlying commit when an annotated tag is passed. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15replace: peel tag when passing a tag as parent to --graftChristian Couder1-0/+11
When passing a tag as a parent argument to `git replace --graft`, it can be useful to accept it and use the underlying commit as a parent. This already works for lightweight tags, but unfortunately for annotated tags we have been using the hash of the tag object instead of the hash of the underlying commit as a parent in the replacement object we create. This created invalid objects, but the replace succeeded even if it showed an error like: error: object A is a tag, not a commit This patch fixes that by using the hash of the underlying commit when an annotated tag is passed. While at it, let's also update an error message to make it clearer. Reviewed-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-14send-email: default to quoted-printable when CR is presentbrian m. carlson1-0/+14
In 7a36987fff ("send-email: add an auto option for transfer encoding", 2018-07-08), git send-email learned how to automatically determine the transfer encoding for a patch. However, the only criterion considered was the length of the lines. Another case we need to consider is that of carriage returns. Because emails have CRLF endings when canonicalized, we don't want to write raw carriage returns into a patch, lest they be stripped off as an artifact of the transport. Ensure that we choose quoted-printable encoding if the patch we're sending contains carriage returns. Note that we are guaranteed to always correctly encode carriage returns when writing quoted-printable since we explicitly specify the line ending as "\n", forcing MIME::QuotedPrint to encode our carriage return as "=0D". Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-12test-prio-queue: use xmallocJeff King1-1/+1
test-prio-queue.c doesn't check the return value of malloc, and could segfault. It's unlikely for this to matter in practice; it's a small allocation, and this code isn't even installed alongside the rest of Git. But let's use xmalloc(), which makes auditing for other accidental uses of bare malloc() easier. Reported-by: 王健强 <jianqiang.wang@securitygossip.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-12tag: advise on nested tagsDenton Liu1-0/+11
Robert Dailey reported confusion on the mailing list about a nested tag which was most likely created by mistake. Jeff King noted that this isn't a very common case and creating a tag-to-a-tag can be a user-error. Suggest that it may be a mistake with an advice message when creating such a tag. Those who do want to create a tag that point at another tag regularly can turn it off with the usual advice mechanism. Reported-by: Robert Dailey <rcdailey.lists@gmail.com> Helped-by: Jeff King <peff@peff.net> Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Denton Liu <liu.denton@gmail.com> [jc: fixed test style and tweaked the log message] Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-12t3000 (ls-files -o): widen description to reflect current testsKyle Meyer1-1/+1
Remove the mention of symlinks from the test description because several tests that are not related to symlinks have been added since this file was introduced long ago. Signed-off-by: Kyle Meyer <kyle@kyleam.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-10rev-list: detect broken root treesJeff King1-2/+4
When the traversal machinery sees a commit without a root tree, it assumes that the tree was part of a BOUNDARY commit, and quietly ignores the tree. But it could also be caused by a commit whose root tree is broken or missing. Instead, let's die() when we see a NULL root tree. We can differentiate it from the BOUNDARY case by seeing if the commit was actually parsed. This covers that case, plus future-proofs us against any others where we might try to show an unparsed commit. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-10rev-list: let traversal die when --missing is not in useJeff King1-2/+2
Commit 7c0fe330d5 (rev-list: handle missing tree objects properly, 2018-10-05) taught the traversal machinery used by git-rev-list to ignore missing trees, so that rev-list could handle them itself. However, it does so only by checking via oid_object_info_extended() that the object exists at all. This can miss several classes of errors that were previously detected by rev-list: - type mismatches (e.g., we expected a tree but got a blob) - failure to read the object data (e.g., due to bitrot on disk) This is especially important because we use "rev-list --objects" as our connectivity check to admit new objects to the repository, and it will now miss these cases (though the bitrot one is less important here, because we'd typically have just hashed and stored the object). There are a few options to fix this: 1. we could check these properties in rev-list when we do the existence check. This is probably too expensive in practice (perhaps even for a type check, but definitely for checking the whole content again, which implies loading each object into memory twice). 2. teach the traversal machinery to differentiate between a missing object, and one that could not be loaded as expected. This probably wouldn't be too hard to detect type mismatches, but detecting bitrot versus a truly missing object would require deep changes to the object-loading code. 3. have the traversal machinery communicate the failure to the caller, so that it can decide how to proceed without re-evaluting the object itself. Of those, I think (3) is probably the best path forward. However, this patch does none of them. In the name of expediently fixing the regression to a normal "rev-list --objects" that we use for connectivity checks, this simply restores the pre-7c0fe330d5 behavior of having the traversal die as soon as it fails to load a tree (when --missing is set to MA_ERROR, which is the default). Note that we can't get rid of the object-existence check in finish_object(), because this also handles blobs (which are not otherwise checked at all by the traversal code). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-10list-objects.c: handle unexpected non-tree entriesTaylor Blau1-2/+3
Apply similar treatment as the previous commit for non-tree entries, too. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-10list-objects.c: handle unexpected non-blob entriesTaylor Blau1-2/+3
Fix one of the cases described in the previous commit where a tree-entry that is promised to a blob is in fact a non-blob. When 'lookup_blob()' returns NULL, it is because Git has cached the requested object as a non-blob. In this case, prevent a SIGSEGV by 'die()'-ing immediately before attempting to dereference the result. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-10t: introduce tests for unexpected object typesTaylor Blau1-0/+123
Call an object's type "unexpected" when the actual type of an object does not match Git's contextual expectation. For example, a tree entry whose mode differs from the object's actual type, or a commit's parent which is not another commit, and so on. This can manifest itself in various unfortunate ways, including Git SIGSEGV-ing under specific conditions. Consider the following example: Git traverses a blob (say, via `git rev-list`), and then tries to read out a tree-entry which lists that object as something other than a blob. In this case, `lookup_blob()` will return NULL, and the subsequent dereference will result in a SIGSEGV. Introduce tests that present objects of "unexpected" type in the above fashion to 'git rev-list'. Mark as failures the combinations that are already broken (i.e., they exhibit the segfault described above). In the cases that are not broken (i.e., they have NULL-ness checks or similar), mark these as expecting success. We might hit an unexpected type in two different ways (imagine we have a tree entry that claims to be a tree but actually points to a blob): - when we call lookup_tree(), we might find that we've already seen the object referenced as a blob, in which case we'd get NULL. We can exercise this with "git rev-list --objects $blob $tree", which guarantees that the blob will have been parsed before we look in the tree. These tests are marked as "seen" in the test script. - we call lookup_tree() successfully, but when we try to read the object, we find out it's something else. We construct our tests such that $blob is not otherwise mentioned in $tree. These tests are marked as "lone" in the script. We should check that we behave sensibly in both cases (especially because it is easy for a malicious actor to provoke one case or the other). Co-authored-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-10add: error appropriately on repository with no commitsKyle Meyer1-0/+11
The previous commit made 'git add' abort when given a repository that doesn't have a commit checked out. However, the output upon failure isn't appropriate: % git add repo warning: adding embedded git repository: repo hint: You've added another git repository inside your current repository. hint: [...] error: unable to index file 'repo/' fatal: adding files failed The hint doesn't apply in this case, and the error message doesn't tell the user why 'repo' couldn't be added to the index. Provide better output by teaching add_to_index() to error when given a git directory where HEAD can't be resolved. To avoid the embedded repository warning and hint, call check_embedded_repo() only after add_file_to_index() succeeds because, in general, its output doesn't make sense if adding to the index fails. Signed-off-by: Kyle Meyer <kyle@kyleam.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-10dir: do not traverse repositories with no commitsKyle Meyer2-0/+51
When treat_directory() encounters a directory that is not in the index and DIR_NO_GITLINKS is unset, it calls resolve_gitlink_ref() to decide if a directory looks like a repository, in which case the directory won't be traversed. As a result, 'status -uall' and 'ls-files -o' will show only the directory, even when there are untracked files within the directory. For the unusual case where a repository doesn't have a commit checked out, resolve_gitlink_ref() returns -1 because HEAD cannot be resolved, and the directory is treated as a normal directory (i.e. traversal does not stop at the repository boundary). The status and ls-files commands above list untracked files within the repository rather than showing only the top-level directory. And if 'git add' is called on a repository with no commit checked out, any untracked files under the repository are added as blobs in the top-level project, a behavior that is unlikely to be what the caller intended. The above case is a corner case in an already unusual situation of the working tree containing a repository that is not a tracked submodule, but we might as well treat anything that looks like a repository consistently. Loosen the "looks like a repository" criteria in treat_directory() by replacing resolve_gitlink_ref() with is_nonbare_repository_dir(), one of the checks that is performed downstream when resolve_gitlink_ref() is called. As the required update to t3700-add shows, calling 'git add' on a repository with no commit checked out will now raise an error. While this is the desired behavior, note that the output isn't yet appropriate. The next commit will improve this output. Signed-off-by: Kyle Meyer <kyle@kyleam.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-10submodule: refuse to add repository with no commitsKyle Meyer1-1/+10
When the path given to 'git submodule add' is an existing repository that is not in the index, the repository is passed to 'git add'. If this repository doesn't have a commit checked out, we don't get a useful result: there is no subproject OID to track, and any untracked files in the sub-repository are added as blobs in the top-level repository. To avoid getting into this state, abort if the path is a repository that doesn't have a commit checked out. Note that this check must come before the 'git add --dry-run' check because the next commit will make 'git add' fail when given a repository that doesn't have a commit checked out. Signed-off-by: Kyle Meyer <kyle@kyleam.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-10submodule: teach set-branch subcommandDenton Liu1-0/+93
This teaches git-submodule the set-branch subcommand which allows the branch of a submodule to be set through a porcelain command without having to manually manipulate the .gitmodules file. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-10Merge branch 'ar/t4150-remove-cruft'Junio C Hamano1-3/+1
Test cleanup. * ar/t4150-remove-cruft: t4150: remove unused variable
2019-04-10Merge branch 'jk/line-log-with-patch'Junio C Hamano1-0/+17
"git log -L<from>,<to>:<path>" with "-s" did not suppress the patch output as it should. This has been corrected. * jk/line-log-with-patch: line-log: detect unsupported formats line-log: suppress diff output with "-s"
2019-04-10Merge branch 'ra/t3600-test-path-funcs'Junio C Hamano2-171/+187
A GSoC micro. * ra/t3600-test-path-funcs: t3600: use helpers to replace test -d/f/e/s <path> t3600: modernize style test functions: add function `test_file_not_empty`
2019-04-10Merge branch 'nd/rewritten-ref-is-per-worktree'Junio C Hamano1-0/+35
"git rebase" uses the refs/rewritten/ hierarchy to store its intermediate states, which inherently makes the hierarchy per worktree, but it didn't quite work well. * nd/rewritten-ref-is-per-worktree: Make sure refs/rewritten/ is per-worktree files-backend.c: reduce duplication in add_per_worktree_entries_to_dir() files-backend.c: factor out per-worktree code in loose_fill_ref_dir()
2019-04-09t3301: fix false negativeJohannes Schindelin1-2/+3
In 6956f858f6 (notes: implement helpers needed for note copying during rewrite, 2010-03-12), we introduced a test case that verifies that the config setting `notes.rewriteRef` can be overridden via the environment variable `GIT_NOTES_REWRITE_REF`. Back when it was introduced, it relied on a side effect of an earlier test case that configured `core.noteRef` to point to `refs/notes/other`. In 908a320363 (t3301: modernize style, 2014-11-12), this side effect was removed. The test case *still* passed, but for the wrong reason: we no longer overrode the rewrite ref, but there simply was nothing to rewrite anymore, as the overridden notes ref was "modernized" away. Let's let that test case pass for the correct reason again. To make sure of that, let's change the idea of the original test case: it configured `notes.rewriteRef` to point to the actual notes ref, forced that to be ignored and then verified that the notes were *not* rewritten. By turning that idea upside down (configure the `notes.rewriteRef` to another notes ref, override it via the environment variable to force the notes to be copied, and then verify that the notes *were* rewritten), we make it much harder for that test case to pass for the wrong reason. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-08blame: default to HEAD in a bare repo when no start commit is givenSZEDER Gábor1-0/+8
When 'git blame' is invoked without specifying the commit to start blaming from, it starts from the given file's state in the work tree. However, when invoked in a bare repository without a start commit, then there is no work tree state to start from, and it dies with the following error message: $ git rev-parse --is-bare-repository true $ git blame file.c fatal: this operation must be run in a work tree This is misleading, because it implies that 'git blame' doesn't work in bare repositories at all, but it does, in fact, work just fine when it is given a commit to start from. We could improve the error message, of course, but let's just default to HEAD in a bare repository instead, as most likely that is what the user wanted anyway (if they wanted to start from an other commit, then they would have specified that in the first place). 'git annotate' is just a thin wrapper around 'git blame', so in the same situation it printed the same misleading error message, and this patch fixes it, too. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-08merge-recursive: switch directory rename detection defaultElijah Newren3-61/+413
When all of x/a, x/b, and x/c have moved to z/a, z/b, and z/c on one branch, there is a question about whether x/d added on a different branch should remain at x/d or appear at z/d when the two branches are merged. There are different possible viewpoints here: A) The file was placed at x/d; it's unrelated to the other files in x/ so it doesn't matter that all the files from x/ moved to z/ on one branch; x/d should still remain at x/d. B) x/d is related to the other files in x/, and x/ was renamed to z/; therefore x/d should be moved to z/d. Since there was no ability to detect directory renames prior to git-2.18, users experienced (A) regardless of context. Choice (B) was implemented in git-2.18, with no option to go back to (A), and has been in use since. However, one user reported that the merge results did not match their expectations, making the change of default problematic, especially since there was no notice printed when directory rename detection moved files. Note that there is also a third possibility here: C) There are different answers depending on the context and content that cannot be determined by git, so this is a conflict. Use a higher stage in the index to record the conflict and notify the user of the potential issue instead of silently selecting a resolution for them. Add an option for users to specify their preference for whether to use directory rename detection, and default to (C). Even when directory rename detection is on, add notice messages about files moved into new directories. As a sidenote, x/d did not have to be a new file here; it could have already existed at some other path and been renamed to x/d, with directory rename detection just renaming it again to z/d. Thus, it's not just new files, but also a modification to all rename types (normal renames, rename/add, rename/delete, rename/rename(1to1), rename/rename(1to2), and rename/rename(2to1)). Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-08t6043: fix copied test description to match its purposeElijah Newren1-2/+2
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-08diff: batch fetching of missing blobsJonathan Tan1-0/+103
When running a command like "git show" or "git diff" in a partial clone, batch all missing blobs to be fetched as one request. This is similar to c0c578b33c ("unpack-trees: batch fetching of missing blobs", 2017-12-08), but for another command. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-05t: move 'hex2oct' into test-lib-functions.shTaylor Blau4-12/+6
The helper 'hex2oct' is used to convert base-16 encoded data into a base-8 binary form, and is useful for preparing data for commands that accept input in a binary format, such as 'git hash-object', via 'printf'. This helper is defined identically in three separate places throughout 't'. Move the definition to test-lib-function.sh, so that it can be used in new test suites, and its definition is not redundant. This will likewise make our job easier in the subsequent commit, which also uses 'hex2oct'. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-04revision: use a prio_queue to hold rewritten parentsJeff King1-0/+18
This patch fixes a quadratic list insertion in rewrite_one() when pathspec limiting is combined with --parents. What happens is something like this: 1. We see that some commit X touches the path, so we try to rewrite its parents. 2. rewrite_one() loops forever, rewriting parents, until it finds a relevant parent (or hits the root and decides there are none). The heavy lifting is done by process_parent(), which uses try_to_simplify_commit() to drop parents. 3. process_parent() puts any intermediate parents into the &revs->commits list, inserting by commit date as usual. So if commit X is recent, and then there's a large chunk of history that doesn't touch the path, we may add a lot of commits to &revs->commits. And insertion by commit date is O(n) in the worst case, making the whole thing quadratic. We tried to deal with this long ago in fce87ae538 (Fix quadratic performance in rewrite_one., 2008-07-12). In that scheme, we cache the oldest commit in the list; if the new commit to be added is older, we can start our linear traversal there. This often works well in practice because parents are older than their descendants, and thus we tend to add older and older commits as we traverse. But this isn't guaranteed, and in fact there's a simple case where it is not: merges. Imagine we look at the first parent of a merge and see a very old commit (let's say 3 years old). And on the second parent, as we go back 3 years in history, we might have many commits. That one first-parent commit has polluted our oldest-commit cache; it will remain the oldest while we traverse a huge chunk of history, during which we have to fall back to the slow, linear method of adding to the list. Naively, one might imagine that instead of caching the oldest commit, we'd start at the last-added one. But that just makes some cases faster while making others slower (and indeed, while it made a real-world test case much faster, it does quite poorly in the perf test include here). Fundamentally, these are just heuristics; our worst case is still quadratic, and some cases will approach that. Instead, let's use a data structure with better worst-case performance. Swapping out revs->commits for something else would have repercussions all over the code base, but we can take advantage of one fact: for the rewrite_one() case, nobody actually needs to see those commits in revs->commits until we've finished generating the whole list. That leaves us with two obvious options: 1. We can generate the list _unordered_, which should be O(n), and then sort it afterwards, which would be O(n log n) total. This is "sort-after" below. 2. We can insert the commits into a separate data structure, like a priority queue. This is "prio-queue" below. I expected that sort-after would be the fastest (since it saves us the extra step of copying the items into the linked list), but surprisingly the prio-queue seems to be a bit faster. Here are timings for the new p0001.6 for all three techniques across a few repositories, as compared to master: master cache-last sort-after prio-queue -------------------------------------------------------------------------------------------- GIT_PERF_REPO=git.git 0.52(0.50+0.02) 0.53(0.51+0.02) +1.9% 0.37(0.33+0.03) -28.8% 0.37(0.32+0.04) -28.8% GIT_PERF_REPO=linux.git 20.81(20.74+0.07) 20.31(20.24+0.07) -2.4% 0.94(0.86+0.07) -95.5% 0.91(0.82+0.09) -95.6% GIT_PERF_REPO=llvm-project.git 83.67(83.57+0.09) 4.23(4.15+0.08) -94.9% 3.21(3.15+0.06) -96.2% 2.98(2.91+0.07) -96.4% A few items to note: - the cache-list tweak does improve the bad case for llvm-project.git that started my digging into this problem. But it performs terribly on linux.git, barely helping at all. - the sort-after and prio-queue techniques work well. They approach the timing for running without --parents at all, which is what you'd expect (see below for more data). - prio-queue just barely outperforms sort-after. As I said, I'm not really sure why this is the case, but it is. You can see it even more prominently in this real-world case on llvm-project.git: git rev-list --parents 07ef786652e7 -- llvm/test/CodeGen/Generic/bswap.ll where prio-queue routinely outperforms sort-after by about 7%. One guess is that the prio-queue may just be more efficient because it uses a compact array. There are three new perf tests: - "rev-list --parents" gives us a baseline for running with --parents. This isn't sped up meaningfully here, because the bad case is triggered only with simplification. But it's good to make sure we don't screw it up (now, or in the future). - "rev-list -- dummy" gives us a baseline for just traversing with pathspec limiting. This gives a lower bound for the next test (and it's also a good thing for us to be checking in general for regressions, since we don't seem to have any existing tests). - "rev-list --parents -- dummy" shows off the problem (and our fix) Here are the timings for those three on llvm-project.git, before and after the fix: Test master prio-queue ------------------------------------------------------------------------------ 0001.3: rev-list --parents 2.24(2.12+0.12) 2.22(2.11+0.11) -0.9% 0001.5: rev-list -- dummy 2.89(2.82+0.07) 2.92(2.89+0.03) +1.0% 0001.6: rev-list --parents -- dummy 83.67(83.57+0.09) 2.98(2.91+0.07) -96.4% Changes in the first two are basically noise, and you can see we approach our lower bound in the final one. Note that we can't fully get rid of the list argument from process_parents(). Other callers do have lists, and it would be hard to convert them. They also don't seem to have this problem (probably because they actually remove items from the list as they loop, meaning it doesn't grow so large in the first place). So this basically just drops the "cache_ptr" parameter (which was used only by the one caller we're fixing here) and replaces it with a prio_queue. Callers are free to use either data structure, depending on what they're prepared to handle. Reported-by: Björn Pettersson A <bjorn.a.pettersson@ericsson.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02tests (pack-objects): use the full, unabbreviated `--revs` optionJohannes Schindelin1-22/+22
To use the singular form of a word, when the option wants the plural form (and quietly expands it because it thinks it was abbreviated), is an easy mistake to make, and t5317 contains almost two dozen of them. However, using abbreviated options in tests is a bit fragile, so we will disallow use of abbreviated options in our test suite. In preparation for this change, let's fix `t5317-pack-objects-filter-objects.sh`. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02tests (status): spell out the `--find-renames` option in fullJohannes Schindelin1-4/+4
To avoid future ambiguities, we really want to use full option names in the test suite. `t7525-status-rename.sh` used an abbreviated form of the `--find-renames` option, though, so let's change that. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02tests (push): do not abbreviate the `--follow-tags` optionJohannes Schindelin1-2/+2
We really want to spell out the option in the full form, to avoid any ambiguity that might be introduced by future patches. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02t5531: avoid using an abbreviated optionJohannes Schindelin1-1/+1
It was probably just an oversight: the `--recurse-submodules` option puts the term "submodules" in the plural form, not the singular one. To avoid future problems in case that another option is introduced that starts with the prefix `--recurse-submodule`, let's just fix this. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02t7810: do not abbreviate `--no-exclude-standard` nor `--invert-match`Johannes Schindelin1-8/+8
This script used abbreviated options, which is unnecessarily fragile. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02tests (rebase): spell out the `--force-rebase` optionJohannes Schindelin2-3/+3
In quite a few test cases, we were sloppy and used the abbreviation `--force`, but we really should be precise in what we want to test. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02tests (rebase): spell out the `--keep-empty` optionJohannes Schindelin1-2/+2
This test wants to run `git rebase` with the `--keep-empty` option, but it really only spelled out `--keep` and trusted Git's option parsing to determine that this was a unique abbreviation of the real option. However, Denton Liu contributed a patch series in https://public-inbox.org/git/cover.1553354374.git.liu.denton@gmail.com/ that introduces a new `git rebase` option called `--keep-base`, which makes this previously unique abbreviation non-unique. Whether this patch series is accepted or not, it is actually a bad practice to use abbreviated options in our test suite, because of the issue that those unique option names are not guaranteed to stay unique in the future. So let's just not use abbreviated options in the test suite. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01t6050: redirect expected error output to a fileChristian Couder1-1/+2
Otherwise the error from `git rev-parse` is uselessly polluting the debug output. Redirecting to a file, instead of /dev/null, makes it possible to check that we got the error we expected, so let's check that too. Reviewed-by: Taylor Blau <me@ttaylorr.com> Helped-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01t6050: use test_line_count instead of wc -lChristian Couder1-2/+4
This modernizes a test and makes it more portable. Reviewed-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01test-lib: whitelist GIT_TR2_* in the environmentÆvar Arnfjörð Bjarmason1-0/+1
Add GIT_TR2_* to the whitelist of environment variables that we don't clear when running the test suite. This allows us to use the test suite to produce trace2 test data, which is handy to e.g. write consumers that collate the trace data itself. One caveat here is that we produce trace output for not *just* the tests, but also e.g. from this line in test-lib.sh: # It appears that people try to run tests without building... "${GIT_TEST_INSTALLED:-$GIT_BUILD_DIR}/git$X" >/dev/null [...] I consider this not just OK but a feature. Let's log *all* the git commands we're going to execute, not just those within test_expect_*(). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01config: correct '**' matching in includeIf patternsNguyễn Thái Ngọc Duy1-0/+13
The current wildmatch() call for includeIf's gitdir pattern does not pass the WM_PATHNAME flag. Without this flag, '*' is treated _almost_ the same as '**' (because '*' also matches slashes) with one exception: '/**/' can match a single slash. The pattern 'foo/**/bar' matches 'foo/bar'. But '/*/', which is essentially what wildmatch engine sees without WM_PATHNAME, has to match two slashes (and '*' matches nothing). Which means 'foo/*/bar' cannot match 'foo/bar'. It can only match 'foo//bar'. The result of this is the current wildmatch() call works most of the time until the user depends on '/**/' matching no path component. And also '*' matches slashes while it should not, but people probably haven't noticed this yet. The fix is straightforward. Reported-by: Jason Karns <jason.karns@gmail.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01gc: handle & check gc.reflogExpire configÆvar Arnfjörð Bjarmason1-0/+19
Don't redundantly run "git reflog expire --all" when gc.reflogExpire and gc.reflogExpireUnreachable are set to "never", and die immediately if those configuration valuer are bad. As an earlier "assert lack of early exit" change to the tests for "git reflog expire" shows, an early check of gc.reflogExpire{Unreachable,} isn't wanted in general for "git reflog expire", but it makes sense for "gc" because: 1) Similarly to 8ab5aa4bd8 ("parseopt: handle malformed --expire arguments more nicely", 2018-04-21) we'll now die early if the config variables are set to invalid values. We run "pack-refs" before "reflog expire", which can take a while, only to then die on an invalid gc.reflogExpire{Unreachable,} configuration. 2) Not invoking the command at all means it won't show up in trace output, which makes what's going on more obvious when the two are set to "never". 3) As a later change documents we lock the refs when looping over the refs to expire, even in cases where we end up doing nothing due to this config. For the reasons noted in the earlier "assert lack of early exit" change I don't think it's worth it to bend over backwards in "git reflog expire" itself to carefully detect if we'll really do nothing given the combination of all its possible options and skip that locking, but that's easy to detect here in "gc" where we'll only run "reflog expire" in a relatively simple mode. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01reflog tests: assert lack of early exit with expiry="never"Ævar Arnfjörð Bjarmason1-1/+3
When gc.reflogExpire and gc.reflogExpireUnreachable are set to "never" and --stale-fix isn't in effect we *could* exit early without pointlessly looping over all the reflogs. However, as an earlier change to add a test for the "points nowhere" warning shows even in such a mode we might want to print out a warning. So while it's conceivable to implement this, I don't think it's worth it. It's going to be too easy to inadvertently add some flag that'll make the expiry happen anyway, and even with "never" we'd like to see all the lines we're going to keep. So let's assert that we're going to loop over all the references even when this configuration is in effect. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01commit-graph write: don't die if the existing graph is corruptÆvar Arnfjörð Bjarmason1-2/+9
When the commit-graph is written we end up calling parse_commit(). This will in turn invoke code that'll consult the existing commit-graph about the commit, if the graph is corrupted we die. We thus get into a state where a failing "commit-graph verify" can't be followed-up with a "commit-graph write" if core.commitGraph=true is set, the graph either needs to be manually removed to proceed, or core.commitGraph needs to be set to "false". Change the "commit-graph write" codepath to use a new parse_commit_no_graph() helper instead of parse_commit() to avoid this. The latter will call repo_parse_commit_internal() with use_commit_graph=1 as seen in 177722b344 ("commit: integrate commit graph with commit parsing", 2018-04-10). Not using the old graph at all slows down the writing of the new graph by some small amount, but is a sensible way to prevent an error in the existing commit-graph from spreading. Just fixing the current issue would be likely to result in code that's inadvertently broken in the future. New code might use the commit-graph at a distance. To detect such cases introduce a "GIT_TEST_COMMIT_GRAPH_DIE_ON_LOAD" setting used when we do our corruption tests, and test that a "write/verify" combo works after every one of our current test cases where we now detect commit-graph corruption. Some of the code changes here might be strictly unnecessary, e.g. I was unable to find cases where the parse_commit() called from write_graph_chunk_data() didn't exit early due to "item->object.parsed" being true in repo_parse_commit_internal() (before the use_commit_graph=1 has any effect). But let's also convert those cases for good measure, we do not have exhaustive tests for all possible types of commit-graph corruption. This might need to be re-visited if we learn to write the commit-graph incrementally, but probably not. Hopefully we'll just start by finding out what commits we have in total, then read the old graph(s) to see what they cover, and finally write a new graph file with everything that's missing. In that case the new graph writing code just needs to continue to use e.g. a parse_commit() that doesn't consult the existing commit-graphs. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01commit-graph verify: detect inability to read the graphÆvar Arnfjörð Bjarmason1-0/+6
Change "commit-graph verify" to error on open() failures other than ENOENT. As noted in the third paragraph of 283e68c72f ("commit-graph: add 'verify' subcommand", 2018-06-27) and the test it added it's intentional that "commit-graph verify" doesn't error out when the file doesn't exist. But let's not be overly promiscuous in what we accept. If we can't read the file for other reasons, e.g. permission errors, bad file descriptor etc. we'd like to report an error to the user. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01commit-graph: don't early exit(1) on e.g. "git status"Ævar Arnfjörð Bjarmason1-1/+1
Make the commit-graph loading code work as a library that returns an error code instead of calling exit(1) when the commit-graph is corrupt. This means that e.g. "status" will now report commit-graph corruption as an "error: [...]" at the top of its output, but then proceed to work normally. This required splitting up the load_commit_graph_one() function so that the code that deals with open()-ing and stat()-ing the graph can now be called independently as open_commit_graph(). This is needed because "commit-graph verify" where the graph doesn't exist isn't an error. See the third paragraph in 283e68c72f ("commit-graph: add 'verify' subcommand", 2018-06-27). There's a bug in that logic where we conflate the intended ENOENT with other errno values (e.g. EACCES), but this change doesn't address that. That'll be addressed in a follow-up change. I'm then splitting most of the logic out of load_commit_graph_one() into load_commit_graph_one_fd_st(), which allows for providing an existing file descriptor and stat information to the loading code. This isn't strictly needed, but it would be redundant and confusing to open() and stat() the file twice for some of the codepaths, this allows for calling open_commit_graph() followed by load_commit_graph_one_fd_st(). The "graph_file" still needs to be passed to that function for the the "graph file %s is too small" error message. This leaves load_commit_graph_one() unused by everything except the internal prepare_commit_graph_one() function, so let's mark it as "static". If someone needs it in the future we can remove the "static" attribute. I could also rewrite its sole remaining user ("prepare_commit_graph_one()") to use load_commit_graph_one_fd_st() instead, but let's leave it at this. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01commit-graph: fix segfault on e.g. "git status"Ævar Arnfjörð Bjarmason1-1/+2
When core.commitGraph=true is set, various common commands now consult the commit graph. Because the commit-graph code is very trusting of its input data, it's possibly to construct a graph that'll cause an immediate segfault on e.g. "status" (and e.g. "log", "blame", ...). In some other cases where git immediately exits with a cryptic error about the graph being broken. The root cause of this is that while the "commit-graph verify" sub-command exhaustively verifies the graph, other users of the graph simply trust the graph, and will e.g. deference data found at certain offsets as pointers, causing segfaults. This change does the bare minimum to ensure that we don't segfault in the common fill_commit_in_graph() codepath called by e.g. setup_revisions(), to do this instrument the "commit-graph verify" tests to always check if "status" would subsequently segfault. This fixes the following tests which would previously segfault: not ok 50 - detect low chunk count not ok 51 - detect missing OID fanout chunk not ok 52 - detect missing OID lookup chunk not ok 53 - detect missing commit data chunk Those happened because with the commit-graph enabled setup_revisions() would eventually call fill_commit_in_graph(), where e.g. g->chunk_commit_data is used early as an offset (and will be 0x0). With this change we get far enough to detect that the graph is broken, and show an error instead. E.g.: $ git status; echo $? error: commit-graph is missing the Commit Data chunk 1 That also sucks, we should *warn* and not hard-fail "status" just because the commit-graph is corrupt, but fixing is left to a follow-up change. A side-effect of changing the reporting from graph_report() to error() is that we now have an "error: " prefix for these even for "commit-graph verify". Pseudo-diff before/after: $ git commit-graph verify -commit-graph is missing the Commit Data chunk +error: commit-graph is missing the Commit Data chunk Changing that is OK. Various errors it emits now early on are prefixed with "error: ", moving these over and changing the output doesn't break anything. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01fast-import: fix erroneous handling of get-mark with empty orphan commitsElijah Newren1-2/+2
When get-mark was introduced in commit 28c7b1f7b7b7 ("fast-import: add a get-mark command", 2015-07-01), it followed the precedent of the cat-blob command to be allowed on any line other than in the middle of a data directive; see commit 777f80d7429b ("fast-import: Allow cat-blob requests at arbitrary points in stream", 2010-11-28). It was useful to allow cat-blob directives in the middle of a commit to get more data that would be used in writing the current commit object. get-mark is not similarly useful since fast-import can already use either object id or mark. Further, trying to allow this command anywhere caused parsing bugs. Fix the parsing problems by only allowing get-mark commands to appear when other commands have completed. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01t9300: demonstrate bug with get-mark and empty orphan commitsElijah Newren1-0/+37
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01t/lib-submodule-update: use appropriate length constantbrian m. carlson1-1/+2
Instead of using a specific invalid hard-coded object ID, produce one of the appropriate length by using test_oid. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-24diff --no-index: use parse_options() instead of diff_opt_parse()Nguyễn Thái Ngọc Duy1-2/+1
While at there, move exit() back to the caller. It's easier to see the flow that way than burying it in diff-no-index.c Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-24t5318-commit-graph: remove unused variableSZEDER Gábor1-1/+1
This is a remnant from early versions of the commit-graph patch series [1], when 'git commit-graph --write' printed the hash of the created commit-graph file, and tests did look at the command's output, because the commit-graph file's name included that hash as well. [1] https://public-inbox.org/git/1517348383-112294-6-git-send-email-dstolee@microsoft.com/ Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-24checkout: prevent losing staged changes with --mergeNguyễn Thái Ngọc Duy1-9/+1
When --merge is specified, we may need to do a real merge (instead of three-way tree unpacking), the steps are best seen in git-checkout.sh version before it's removed: # Match the index to the working tree, and do a three-way. git diff-files --name-only | git update-index --remove --stdin && work=`git write-tree` && git read-tree $v --reset -u $new || exit git merge-recursive $old -- $new $work # Do not register the cleanly merged paths in the index yet. # this is not a real merge before committing, but just carrying # the working tree changes along. unmerged=`git ls-files -u` git read-tree $v --reset $new case "$unmerged" in '') ;; *) ( z40=0000000000000000000000000000000000000000 echo "$unmerged" | sed -e 's/^[0-7]* [0-9a-f]* /'"0 $z40 /" echo "$unmerged" ) | git update-index --index-info ;; esac Notice the last 'read-tree --reset' step. We restore worktree back to 'new' tree after worktree's messed up by merge-recursive. If there are staged changes before this whole command sequence is executed, they are lost because they are unlikely part of the 'new' tree to be restored. There is no easy way to fix this. Elijah may have something up his sleeves [1], but until then, check if there are staged changes and refuse to run and lose them. The user would need to do "git reset" to continue in this case. A note about the test update. 'checkout -m' in that test will fail because a deletion is staged. This 'checkout -m' was previously needed to verify quietness behavior of unpack-trees. But a different check has been put in place in the last patch. We can safely drop 'checkout -m' now. [1] CABPp-BFoL_U=bzON4SEMaQSKU2TKwnOgNqjt5MUaOejTKGUJxw@mail.gmail.com Reported-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-24read-tree: add --quietNguyễn Thái Ngọc Duy1-0/+3
read-tree is basically the front end of unpack-trees code and shoud expose all of its functionality (unless it's designed for internal use). This "opts.quiet" (formerly "opts.gently") was added for builtin/checkout.c but there is no reason why other read-tree users won't find this useful. The test that is updated to run 'read-tree --quiet' was added because unpack-trees was accidentally not being quiet [1] in 6a143aa2b2 (checkout -m: attempt merge when deletion of path was staged - 2014-08-12). Because checkout is the only "opts.quiet" user, there was no other way to test quiet behavior. But we can now test it directly. 6a143aa2b2 was manually reverted to verify that read-tree --quiet works correctly (i.e. test_must_be_empty fails). [1] the commit message there say "errors out instead of performing a merge" but I'm pretty sure the "performing a merge" happens anyway even before that commit. That line should say "errors out _in addition to_ performing a merge" Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-24t5551: mark half-auth no-op fetch test as v0-onlyJonathan Tan1-1/+11
When using protocol v0, upload-pack over HTTP permits a "half-auth" configuration in which, at the web server layer, the info/refs path is not protected by authentication but the git-upload-pack path is, so that a user can perform fetches that do not download any objects without authentication, but still needs authentication to download objects. But protocol v2 does not support this, because both ref and pack are obtained from the git-upload-pack path. Mark the test verifying this behavior as protocol v0-only, with a description of what needs to be done to make v2 support this. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-24http: normalize curl results for dumb loose and alternates fetchesJeff King1-0/+16
If the dumb-http walker encounters a 404 when fetching a loose object, it then looks at any http-alternates for the object. The 404 check is implemented by missing_target(), which checks not only the http code, but also that we got an http error from the CURLcode. That broke when we stopped using CURLOPT_FAILONERROR in 17966c0a63 (http: avoid disconnecting on 404s for loose objects, 2016-07-11), since our CURLcode will now be CURLE_OK. As a result, fetching over dumb-http from a repository with alternates could result in Git printing "Unable to find abcd1234..." and aborting. We could probably fix this just by loosening missing_target(). However, there's other code which looks at the curl result, and it would have to be tweaked as well. Instead, let's just normalize the result the same way the smart-http code does. There's a similar case in processing the alternates (where we failover from "info/http-alternates" to "info/alternates"). We'll give it the same treatment. After this patch, we should be hitting all code paths that need this normalization (notably absent here is the http_pack_request path, but it does not use FAILONERROR, nor missing_target()). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-22refs/files-backend: handle packed transaction prepare failureJeff King1-0/+16
In files_transaction_prepare(), if we have to delete some refs, we use a subordinate packed_transaction to do so. It's rare for that sub-transaction's prepare step to fail, since we hold the packed-refs lock. But if it does, we trigger a BUG() due to these steps: - we've attached the packed transaction to the files transaction as backend_data->packed_transaction - when the prepare step fails, the packed transaction cleans itself up, putting itself into the CLOSED state - the error value from preparing the packed transaction lets us know in files_transaction_prepare() that we should also clean up and return an error. We call files_transaction_cleanup(), which tries to abort backend_data->packed_transaction. Since it's already CLOSED, that triggers an assertion in ref_transaction_abort(). We can fix that by disconnecting the packed transaction from the outer files transaction, and then free-ing (not aborting!) it ourselves. A few other options/alternatives I considered: - we could just make it a noop to abort a CLOSED transaction. But that seems less safe, since clearly this code expects (and enforces) a particular set of state transitions. - we could have files_transaction_cleanup() selectively call abort() vs free() based on the state of the on the packed transaction. That's basically a more restricted version of the above, but also potentially unsafe. - instead of disconnecting backend_data->packed_transaction on error, we could wait to install it until we successfully prepare. That might make the flow a little simpler, but it introduces a hassle. Earlier parts of files_transaction_prepare() that encounter an error will jump to the cleanup label, and expect that cleaning up the outer transaction will clean up the packed transaction, too. We'd have to adjust those sites to clean up the packed transaction. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-22trace2: write to directory targetsJosh Steadmon1-0/+15
When the value of a trace2 environment variable is an absolute path referring to an existing directory, write output to files (one per process) underneath the given directory. Files will be named according to the final component of the trace2 SID, followed by a counter to avoid potential collisions. This makes it more convenient to collect traces for every git invocation by unconditionally setting the relevant trace2 envvar to a constant directory name. Signed-off-by: Josh Steadmon <steadmon@google.com> Reviewed-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-21unpack-trees: fix oneway_merge accidentally carry over stage indexNguyễn Thái Ngọc Duy1-0/+24
Phillip found out that 'git checkout -f <branch>' does not restore conflict/unmerged files correctly. All tracked files should be taken from <branch> and all non-zero stages removed. Most of this is true, except that the final file could be in stage one instead of zero. "checkout -f" (among other commands) does this with one-way merge, which is supposed to take stat info from the index and everything else from the given tree. The add_entry(.., old, ...) call in oneway_merge() though will keep stage index from the index. This is normally not a problem if the entry from the index is normal (stage #0). But if there is a conflict, stage #0 does not exist and we'll get stage #1 entry as "old" variable, which gets recorded in the final index. Fix it by clearing stage mask. This bug probably comes from b5b425074e (git-read-tree: make one-way merge also honor the "update" flag, 2005-06-07). Before this commit, we may create the final ("dst") index entry from the one in index, but we do clear CE_STAGEMASK. I briefly checked two- and three-way merge functions. I think we don't have the same problem in those. Reported-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-21completion: fix multiple command removalsJeff King1-1/+1
Commit 6532f3740b ("completion: allow to customize the completable command list", 2018-05-20) tried to allow multiple space-separated entries in completion.commands. To do this, it copies each parsed token into a strbuf so that the result is NUL-terminated. However, for tokens starting with "-", it accidentally passes the original non-terminated string, meaning that only the final one worked. Switch to using the strbuf. Reported-by: Todd Zullinger <tmz@pobox.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-21t9902: test multiple removals via completion.commandsTodd Zullinger1-0/+6
6532f3740b ("completion: allow to customize the completable command list", 2018-05-20) added the completion.commands config variable. Multiple commands may be added or removed, separated by a space. Demonstrate the failure of multiple removals. Signed-off-by: Todd Zullinger <tmz@pobox.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-21stash: setup default diff output format if necessaryThomas Gummerer1-0/+18
In the scripted 'git stash show' when no arguments are passed, we just pass '--stat' to 'git diff'. When any argument is passed to 'stash show', we no longer pass '--stat' to 'git diff', and pass whatever flags are passed directly through to 'git diff'. By default 'git diff' shows the patch output. So when a user uses 'git stash show --patience', they would be shown the diff as expected, using the patience algorithm. '--patience' in this case only changes the diff algorithm, but does not cause 'git diff' to show the diff by itself. The diff is shown because that's the default behaviour of 'git diff'. In the C version of 'git stash show', we try to emulate that behaviour using the internal diff API. However we forgot to set up the default output format, in case it wasn't set by any of the flags that were passed through. So 'git stash show --patience' in the builtin version of stash would be completely silent, while it would show the diff in the scripted version. The same thing would happen for other flags that only affect the way a patch is displayed, rather than switching to a different output format than the default one. Fix this by setting up the default output format for 'git diff'. Reported-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-20test-date: drop unused "now" parameter from parse_dates()Jeff King1-2/+2
We only need the current time for relative dates like "5 minutes ago", and those are parsed only through approxidate, not the strict parser used by parse_dates(). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-20Merge branch 'jk/virtual-objects-do-exist'Junio C Hamano1-0/+10
A recent update broke "is this object available to us?" check for well-known objects like an empty tree (which should yield "yes", even when there is no on-disk object for an empty tree), which has been corrected. * jk/virtual-objects-do-exist: rev-list: allow cached objects in existence check
2019-03-20Merge branch 'jk/fsck-doc'Junio C Hamano1-2/+17
"git fsck --connectivity-only" omits computation necessary to sift the objects that are not reachable from any of the refs into unreachable and dangling. This is now enabled when dangling objects are requested (which is done by default, but can be overridden with the "--no-dangling" option). * jk/fsck-doc: fsck: always compute USED flags for unreachable objects doc/fsck: clarify --connectivity-only behavior
2019-03-20Merge branch 'js/stress-test-ui-tweak'Junio C Hamano2-4/+13
Dev support. * js/stress-test-ui-tweak: tests: introduce --stress-jobs=<N> tests: let --stress-limit=<N> imply --stress
2019-03-20Merge branch 'js/rebase-orig-head-fix'Junio C Hamano1-0/+8
"git rebase" that was reimplemented in C did not set ORIG_HEAD correctly, which has been corrected. * js/rebase-orig-head-fix: built-in rebase: set ORIG_HEAD just once, before the rebase built-in rebase: demonstrate that ORIG_HEAD is not set correctly built-in rebase: use the correct reflog when switching branches built-in rebase: no need to check out `onto` twice
2019-03-20Merge branch 'jk/bisect-final-output'Junio C Hamano1-3/+3
The final report from "git bisect" used to show the suspected culprit using a raw "diff-tree", with which there is no output for a merge commit. This has been updated to use a more modern and human readable output that still is concise enough. * jk/bisect-final-output: bisect: make diff-tree output prettier bisect: fix internal diff-tree config loading bisect: use string arguments to feed internal diff-tree
2019-03-20rebase: remove the rebase.useBuiltin settingÆvar Arnfjörð Bjarmason3-10/+19
Remove the rebase.useBuiltin setting, which was added as an escape hatch to disable the builtin version of rebase first released with Git 2.20. See [1] for the initial implementation of rebase.useBuiltin, and [2] and [3] for the documentation and corresponding GIT_TEST_REBASE_USE_BUILTIN option. Carrying the legacy version is a maintenance burden as seen in 7e097e27d3 ("legacy-rebase: backport -C<n> and --whitespace=<option> checks", 2018-11-20) and 9aea5e9286 ("rebase: fix regression in rebase.useBuiltin=false test mode", 2019-02-13). Since the built-in version has been shown to be stable enough let's remove the legacy version. As noted in [3] having use_builtin_rebase() shell out to get its config doesn't make any sense anymore, that was done for the purposes of spawning the legacy rebase without having modified any global state. Let's instead handle this case in rebase_config(). There's still a bunch of references to git-legacy-rebase in po/*.po, but those will be dealt with in time by the i18n effort. Even though this configuration variable only existed two releases let's not entirely delete the entry from the docs, but note its absence. Individual versions of git tend to be around for a while due to distro packaging timelines, so e.g. if we're "lucky" a given version like 2.21 might be installed on say OSX for half a decade. That'll mean some people probably setting this in config, and then when they later wonder if it's needed they can Google search the config option name or check it in git-config. It also allows us to refer to the docs from the warning for details. 1. 55071ea248 ("rebase: start implementing it as a builtin", 2018-08-07) 2. d8d0a546f0 ("rebase doc: document rebase.useBuiltin", 2018-11-14) 3. 62c23938fa ("tests: add a special setup where rebase.useBuiltin is off", 2018-11-14) 3. https://public-inbox.org/git/nycvar.QRO.7.76.6.1903141544110.41@tvgsbejvaqbjf.bet/ Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-18cherry-pick --continue: remember optionsPhillip Wood1-2/+2
Remember --allow-empty, --allow-empty-message and --keep-redundant-commits when cherry-pick stops for a conflict resolution. Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-18cherry-pick: demonstrate option amnesiaPhillip Wood1-0/+24
When cherry-pick stops for a conflict resolution it forgets --allow-empty --allow-empty-message and --keep-redundant-commits. Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-18t4038-diff-combined: quote paths with whitespaceTodd Zullinger1-4/+4
d76ce4f734 ("log,diff-tree: add --combined-all-paths option", 2019-02-07) added tests for files containing tabs. When the tests are run with bash, the lack of quoting during the file setup causes 'ambiguous redirect' errors. Signed-off-by: Todd Zullinger <tmz@pobox.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-18reflog tests: test for the "points nowhere" warningÆvar Arnfjörð Bjarmason1-0/+10
The "git reflog expire" command when given an unknown reference has since 4264dc15e1 ("git reflog expire", 2006-12-19) when this command was implemented emit an error, but this has never been tested for. Let's test for it, also under gc.reflogExpire{Unreachable,}=never in case a future change is tempted to take shortcuts in the presence of such config. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-18reflog tests: make use of "test_config" idiomÆvar Arnfjörð Bjarmason1-8/+5
Change a couple of tests that weren't using the helper to use it. This makes the trailing "--unset" unnecessary. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-18perf-lib.sh: rely on test-lib.sh for --tee handlingJeff King1-23/+11
Since its inception, the perf-lib.sh script has manually handled the "--tee" option (and other options which imply it, like "--valgrind") with a cut-and-pasted block from test-lib.sh. That block has grown stale over the years, and has at least three problems: 1. It uses $SHELL to re-exec the script, whereas the version in test-lib.sh learned to use $TEST_SHELL_PATH. 2. It does an ad-hoc search of the "$*" string, whereas test-lib.sh learned to carefully parse the arguments left to right. 3. It never learned about --verbose-log (which also implies --tee), so it would not trigger for that option. This last one was especially annoying, because t/perf/run uses the GIT_TEST_OPTS from your config.mak to run the perf scripts. So if you've set, say, "-x --verbose-log" there, it will be passed as part of most perf runs. And while this script doesn't recognize the option, the test-lib.sh that we source _does_, and the behavior ends up being much more annoying: - as the comment at the top of the block says, we have to run this tee code early, before we start munging variables (it says GIT_BUILD_DIR, but the problematic variable is actually GIT_TEST_INSTALLED). - since we don't recognize --verbose-log, we don't trigger the block. We go on to munge GIT_TEST_INSTALLED, converting it from a relative to an absolute path. - then we source test-lib.sh, which _does_ recognize --verbose-log. It re-execs the script, which runs again. But this time with an absolute version of GIT_TEST_INSTALLED. - As a result, we copy the absolute version of GIT_TEST_INSTALLED into perf_results_prefix. Instead of writing our results to the expected "test-results/build_1234abcd.p1234-whatever.times", we instead write them to "test-results/_full_path_to_repo_t_perf_build_1234...". The aggregate.perl script doesn't expect this, and so it prints "<missing>" for each result (even though it spent considerable time running the tests!). We can solve all of these in one blow by just deleting our custom handling, and relying on the inclusion of test-lib.sh to handle --tee, --verbose-log, etc. There's one catch, though. We want to handle GIT_TEST_INSTALLED after we've included test-lib.sh, since we want it un-munged in the re-exec'd version of the script. But if we want to convert it from a relative to an absolute path, we must do so before we load test-lib.sh, since it will change our working directory. So we compute the absolute directory first, store it away, then include test-lib.sh, and finally assign to GIT_TEST_INSTALLED as appropriate. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-18pack-objects: default to writing bitmap hash-cacheJeff King3-5/+2
Enabling pack.writebitmaphashcache should always be a performance win. It costs only 4 bytes per object on disk, and the timings in ae4f07fbcc (pack-bitmap: implement optional name_hash cache, 2013-12-21) show it improving fetch and partial-bitmap clone times by 40-50%. The only reason we didn't enable it by default at the time is that early versions of JGit's bitmap reader complained about the presence of optional header bits it didn't understand. But that was changed in JGit's d2fa3987a (Use bitcheck to check for presence of OPT_FULL option, 2013-10-30), which made it into JGit v3.5.0 in late 2014. So let's turn this option on by default. It's backwards-compatible with all versions of Git, and if you are also using JGit on the same repository, you'd only run into problems using a version that's almost 5 years old. We'll drop the manual setting from all of our test scripts, including perf tests. This isn't strictly necessary, but it has two advantages: 1. If the hash-cache ever stops being enabled by default, our perf regression tests will notice. 2. We can use the modified perf tests to show off the behavior of an otherwise unconfigured repo, as shown below. These are the results of a few of a perf tests against linux.git that showed interesting results. You can see the expected speedup in 5310.4, which was noted in ae4f07fbcc. Curiously, 5310.8 did not improve (and actually got slower), despite seeing the opposite in ae4f07fbcc. I don't have an explanation for that. The tests from p5311 did not exist back then, but do show improvements (a smaller pack due to better deltas, which we found in less time). Test HEAD^ HEAD ------------------------------------------------------------------------------------- 5310.4: simulated fetch 7.39(22.70+0.25) 5.64(11.43+0.22) -23.7% 5310.8: clone (partial bitmap) 18.45(24.83+1.19) 19.94(28.40+1.36) +8.1% 5311.31: server (128 days) 0.41(1.13+0.05) 0.34(0.72+0.02) -17.1% 5311.32: size (128 days) 7.4M 7.0M -4.8% 5311.33: client (128 days) 1.33(1.49+0.06) 1.29(1.37+0.12) -3.0% Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-18t5310: correctly remove bitmaps for jgit testJeff King1-1/+1
We use "jgit gc" to generate a pack bitmap file, and then make sure our implementation can read it. To prepare the repo before running jgit, we try to "rm -f" any existing bitmap files. But we got the path wrong; we're in a bare repo, so looking in ".git/" finds nothing. Our "rm" doesn't complain because of the "-f", and when we run "rev-list" there are two bitmap files (ours and jgit's). Our reader implementation will ignore one of the bitmap files, but it's likely non-deterministic which one we will use. We'd prefer the one with the more recent timestamp (just because of the way the packed_git list is sorted), but in most test runs they'd have identical timestamps. So this was probably actually testing something useful about 50% of the time, and other half just testing that we could read our own bitmaps (which is covered elsewhere). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-18repack: enable bitmaps by default on bare reposEric Wong1-1/+18
A typical use case for bare repos is for serving clones and fetches to clients. Enable bitmaps by default on bare repos to make it easier for admins to host git repos in a performant way. Signed-off-by: Eric Wong <e@80x24.org> Helped-by: Jeff King <peff@peff.net> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-18difftool: allow running outside Git worktrees with --no-indexJohannes Schindelin1-0/+10
As far as this developer can tell, the conversion from a Perl script to a built-in caused the regression in the difftool that it no longer runs outside of a Git worktree (with `--no-index`, of course). It is a bit embarrassing that it took over two years after retiring the Perl version to discover this regression, but at least we now know, and can do something, about it. This fixes https://github.com/git-for-windows/git/issues/2123 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-18parse-options: make OPT_ARGUMENT() more usefulJohannes Schindelin1-1/+1
`OPT_ARGUMENT()` is intended to keep the specified long option in `argv` and not to do anything else. However, it would make a lot of sense for the caller to know whether this option was seen at all or not. For example, we want to teach `git difftool` to work outside of any Git worktree, but only when `--no-index` was specified. Note: nothing in Git uses OPT_ARGUMENT(). Even worse, looking through the commit history, one can easily see that nothing even ever used it, apart from the regression test. So not only do we make `OPT_ARGUMENT()` more useful, we are also about to introduce its first real user! Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-14get_oid(): when an object was not found, try harderJohannes Schindelin1-1/+1
It is quite possible that the loose object cache gets stale when new objects are written. In that case, get_oid() would potentially say that it cannot find a given object, even if it should find it. Let's blow away the loose object cache as well as the read packs and try again in that case. Note: this does *not* affect the code path that was introduced to help avoid looking for the same non-existing objects (which made some operations really expensive via NFS): that code path is handled by the `OBJECT_INFO_QUICK` flag (which does not even apply to `get_oid()`, which has no equivalent flag, at least at the time this patch was written). This incidentally fixes the problem identified earlier where an interactive rebase wanted to re-read (and validate) the todo list after an `exec` command modified it. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-14rebase -i: demonstrate obscure loose object cache bugJohannes Schindelin1-0/+22
We specifically support `exec` commands in `git rebase -i`'s todo lists to rewrite the very same todo list. Of course, we need to validate that todo list when re-reading it. It is also totally legitimate to extend the todo list by `pick` lines using short names of commits that were created only after the rebase started. And this is where the loose object cache interferes with this feature: if *some* loose object was read whose hash shares the same first two digits with a commit that was not yet created when that loose object was created, then we fail to find that new commit by its short name in `get_oid()`, and the interactive rebase fails with an obscure error message like: error: invalid line 1: pick 6568fef error: please fix this using 'git rebase --edit-todo'. Let's first demonstrate that this is actually a bug in a new regression test, in a separate commit so that other developers who do not believe me can cherry-pick it to confirm the problem. This new regression test generates two commits whose hashes share the first two hex digits (so that their corresponding loose objects live in the same subdirectory of .git/objects/, and are therefore supposed to be in the same loose object cache bin). It then picks the first, to make sure that the loose object cache is initialized and cached that object directory, then generates the second commit and picks it, too. Since the commit was generated in a different process than the sequencer that wants to pick it, the loose object cache had no chance of being updated in the meantime. Technically, we would need only one `exec` command in this regression test case, but for ease of implementation, it uses a pseudo-recursive call to the same script. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-14t9811-git-p4-label-import: fix pipeline negationSZEDER Gábor1-1/+1
In 't9811-git-p4-label-import.sh' the test 'tag that cannot be exported' runs !(p4 labels | grep GIT_TAG_ON_A_BRANCH) to check that the given string is not printed by 'p4 labels'. This is problematic, because according to POSIX [1]: "If the pipeline begins with the reserved word ! and command1 is a subshell command, the application shall ensure that the ( operator at the beginning of command1 is separated from the ! by one or more <blank> characters. The behavior of the reserved word ! immediately followed by the ( operator is unspecified." While most common shells still interpret this '!' as "negate the exit code of the last command in the pipeline", 'mksh/lksh' don't and interpret it as a negative file name pattern instead. As a result they attempt to run a command made up of the pathnames in the current directory (it contains a single directory called 'main'), which, of course, fails the test. We could fix it simply by adding a space between the '!' and '(', but instead let's fix it by removing the unnecessary subshell. [1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_02 Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-14git p4 test: disable '-x' tracing in the p4d watchdog loopSZEDER Gábor1-1/+1
According to the comments in 't/lib-git-p4.sh', sometimes 'p4d' seems to hang, and to deal with that 'start_p4d' starts a watchdog process to kill it after a long-enough timeout ($P4D_TIMEOUT, defaults to 300s). This watchdog process is implemented as a background subshell loop iterating once every second until the timeout expires, producing a few lines of trace output on each iteration when the test script is run with '-x' tracing enabled. The watchdog loop's trace gets intermixed with the real test output and trace, and makes that harder to read. Send the trace output of this loop to /dev/null to avoid polluting the real test output. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-14git p4 test: simplify timeout handlingSZEDER Gábor1-13/+6
'lib-git-p4.sh' uses timeouts in a watchdog process to kill a potentially stuck 'p4d' process and for certain cleanup operation between tests. It does so by first computing when the timeout should expire, and then repeatedly asking for the current time in seconds until it exceeds the expiration time, and for portability reasons it uses a one-liner Python script to ask for the current time. Replace these timeouts with downcounters, which, though not necessarily shorter, are much simpler, at least in the sense that they don't execute the Python interpreter every second. After this change the helper function with that Python one-liner has no callers left, remove it. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-14git p4 test: clean up the p4d cleanup functionsSZEDER Gábor2-28/+12
Confusingly, the 'git p4' tests used two cleanup functions: - 'kill_p4d' was run in the last test before 'test_done', and it not only killed 'p4d', but it killed the watchdog process, and cleaned up after 'p4d' as well by removing all directories used by the P4 daemon and client. This cleanup is not necessary right before 'test_done', because the whole trash directory is about to get removed anyway, but it is necessary in 't9801-git-p4-branch.sh', which uses 'kill_p4d' to stop 'p4d' before re-starting it in the middle of the test script. - 'cleanup' was run in the trap on EXIT, and it killed 'p4d', but, it didn't kill the watchdog process, and, contrarily to its name, didn't perform any cleanup whatsoever. Make it clearer what's going on by renaming and simplifying the cleanup functions, so in the end we'll have: - 'stop_p4d_and_watchdog' replaces 'cleanup' as it will try to live up to its name and stop both the 'p4d' and the watchdog processes, and as the sole function registered with 'test_atexit' it will be responsible for no leaving any stray processes behind after 'git p4' tests were finished or interrupted. - 'stop_and_cleanup_p4d' replaces 'kill_p4d' as it will stop 'p4d' (and the watchdog) and remove all directories used by the P4 daemon and cliean, so it can be used mid-script to stop and then re-start 'p4d'. Note that while 'cleanup' sent a single SIGKILL to 'p4d', 'kill_p4d' was quite brutal, as it first sent SIGTERM to the daemon repeatedly, either until its pid disappeared or until a given timeout was up, and then it sent SIGKILL repeatedly, for good measure. This is overkill (pardon the pun): a single SIGKILL should be able to take down any process in a sensible state, and if a process were to somehow end up stuck in the dreaded uninterruptible sleep state then even repeated SIGKILLs won't bring immediate help. So ditch all the repeated SIGTERM/SIGKILL parts, and use a single SIGKILL to stop 'p4d', and make sure that there are no races between asynchron signal delivery and subsequent restart of 'p4d' by waiting for it to die. With this change the 'retry_until_fail' helper has no callers left, remove it. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-14git p4 test: use 'test_atexit' to kill p4d and the watchdog processJohannes Schindelin35-139/+9
Use 'test_atexit' to run cleanup commands to stop 'p4d' at the end of the test script or upon interrupt or failure, as it is shorter, simpler, and more robust than registering such cleanup commands in the trap on EXIT in the test scripts. Note that one of the test scripts, 't9801-git-p4-branch.sh', stops and then re-starts 'p4d' twice in the middle of the script; take care that the cleanup functions to stop 'p4d' are only registered once. Note also that 'git p4' tests invoke different functions in the trap on EXIT ('cleanup') and in the last test before 'test_done' ('kill_p4d'). Register both of these functions with 'test_atexit' for now, and a a later patch in this series will then clean up the redundancy. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-14t0301-credential-cache: use 'test_atexit' to stop the credentials helperSZEDER Gábor1-6/+1
Use 'test_atexit' to run cleanup commands to stop the credentials helper at the end of the test script or upon interrupt or failure, as it is shorter, simpler, and more robust than registering such cleanup commands in the trap on EXIT in the test scripts. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-14tests: use 'test_atexit' to stop httpdSZEDER Gábor25-50/+1
Use 'test_atexit' to run cleanup commands to stop httpd at the end of the test script or upon interrupt or failure, as it is shorter, simpler, and more robust than registering such cleanup commands in the trap on EXIT in the test scripts. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-14git-daemon: use 'test_atexit` to stop 'git-daemon'Johannes Schindelin3-7/+9
Use 'test_atexit' to run cleanup commands to stop 'git-daemon' at the end of the test script or upon interrupt or failure, as it is shorter, simpler, and more robust than registering such cleanup commands in the trap on EXIT in the test scripts. Note that in 't5570-git-daemon.sh' the daemon is stopped and then re-started in the middle of the test script; take care that the cleanup functions to stop the daemon are only registered once. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-14test-lib: introduce 'test_atexit'Johannes Schindelin4-0/+89
When running Apache, 'git daemon', or p4d, we want to kill them at the end of the test script, otherwise a leftover daemon process will keep its port open indefinitely, and thus will interfere with subsequent executions of the same test script. So far, we stop these daemon processes "manually", i.e.: - by registering functions or commands in the trap on EXIT to stop the daemon while preserving the last seen exit code before the trap (to deal with a failure when run with '--immediate' or with interrupts by ctrl-C), - and by invoking these functions/commands last thing before 'test_done' (and sometimes restoring the test framework's default trap on EXIT, to prevent the daemons from being killed twice). On one hand, we do this inconsistently, e.g. 'git p4' tests invoke different functions in the trap on EXIT and in the last test before 'test_done', and they neither restore the test framework's default trap on EXIT nor preserve the last seen exit code. On the other hand, this is error prone, because, as shown in a previous patch in this series, any output from the cleanup commands in the trap on EXIT can prevent a proper cleanup when a test script run with '--verbose-log' and certain shells, notably 'dash', is interrupted. Let's introduce 'test_atexit', which is loosely modeled after 'test_when_finished', but has a broader scope: rather than running the commands after the current test case, run them when the test script finishes, and also run them when the test is interrupted, or exits early in case of a failure while the '--immediate' option is in effect. When running the cleanup commands at the end of a successful test, then they will be run in 'test_done' before it removes the trash directory, i.e. the cleanup commands will still be able to access any pidfiles or socket files in there. When running the cleanup commands after an interrupt or failure with '--immediate', then they will be run in the trap on EXIT. In both cases they will be run in 'test_eval_', i.e. both standard error and output of all cleanup commands will go where they should according to the '-v' or '--verbose-log' options, and thus won't cause any troubles when interrupting a test script run with '--verbose-log'. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-14t/lib-git-daemon: make sure to kill the 'git-daemon' processSZEDER Gábor1-2/+4
After 'start_git_daemon' starts 'git daemon' (note the space in the middle) in the background, it saves the background process' PID, so the daemon can be stopped at the end of the test script. However, 'git-daemon' is not a builtin but a dashed external command, which means that the dashless 'git daemon' executes the dashed 'git-daemon' command, and, consequently, the PID recorded is not the PID of the "real" daemon process, but that of the main 'git' wrapper. Now, if a test script involving 'git daemon' is interrupted by ctrl-C, then only the main 'git' process is stopped, but the real daemon process tends to survive somehow, and keeps on running in the background indefinitely, keeping the daemon's port to itself, and thus preventing subsequent runs of the same test script. Work this around by running 'git daemon' with the '--pidfile=...' option to save the PID of the real daemon process, and kill that process in 'stop_git_daemon' as well. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-14test-lib: fix interrupt handling with 'dash' and '--verbose-log -x'SZEDER Gábor1-1/+4
When a test script run with 'dash' and '--verbose-log -x' is interrupted by ctrl-C, SIGTERM, or closing the terminal window, then most of the time the registered EXIT trap actions are not executed. This is an annoying issue with tests involving daemons, because they should run cleanup commands to kill those daemon processes in the trap on EXIT, but since these cleanup commands are not executed, the daemons are left alive and keep their port open, thus interfering with subsequent execution of the same test script. The cause of this issue is the subtle combination of several factors (bear with me, or skip over the indented part): - Even when the test script is interrupted, the cleanup commands are not run in the trap on INT, TERM, or HUP, but in the trap on EXIT after the trap on the signals invokes 'exit' [1]. - According to POSIX [2]: "The environment in which the shell executes a trap on EXIT shall be identical to the environment immediately after the last command executed before the trap on EXIT was taken." Pertinent to the issue at hand is that all open file descriptors and the state of '-x' tracing should be preserved. All shells I've tried [3] preserve '-x'. Unfortunately, however: - 'dash' doesn't conform to this when it comes to open file descriptors: even when standard output and/or error are redirected somewhere when 'exit' is invoked, anything written to them in the trap on EXIT goes to the script's original stdout and stderr [4]. We can't dismiss this with a simple "it doesn't conform to POSIX, so we don't care", because 'dash' is the default /bin/sh in some of the more popular Linux distros. - As far as I can tell, POSIX doesn't explicitly say anything about the environment of trap actions for various signals. In practice it seems that most shells behave sensibly and preserve both open file descriptors and the state of '-x' tracing for the traps on INT, TERM, and HUP, including even 'dash'. The exceptions are 'mksh' and 'lksh': they do preserve '-x', but not the open file descriptors. - When a test script run with '-x' tracing enabled is interrupted, then it's very likely that the signal arrives mid-test, i.e.: - while '-x' tracing is enabled, and, consequently, our trap actions on INT, TERM, HUP, and EXIT will produce trace output as well. - while standard output and error are redirected to a log file, to the test script's original standard output and error, or to /dev/null, depending on whether the test script was run with '--verbose-log', '-v', or neither. According to the above, we can't rely on these redirections still be in effect when running the traps on INT, TERM, HUP, and/or EXIT. - When a test script is run with '--verbose-log', then the test script is re-executed with its standard output and error piped into 'tee', in order to send the "regular" non-verbose test's output both to the terminal and to the log file. When the test is interrupted, then the signal interrupts the downstream 'tee' as well. Putting these together, when a test script run with 'dash' and '--verbose-log -x' is interrupted, then 'dash' tries to write the trace output from the EXIT trap to the script's original standard error, but it very likely can't, because the 'tee' downstream of the pipe is interrupted as well. This causes the shell running the test script to die because of SIGPIPE, without running any of the commands in the EXIT trap. Disable '-x' tracing in the trap on INT, TERM, and HUP to avoid this issue, as it disables tracing in the chained trap on EXIT as well. Wrap it in a '{ ... } 2>/dev/null' block, so the trace of the command disabling the tracing doesn't go to standard error either [5]. Note that it's not only '-x' tracing that can be problematic, but any shell builtin, e.g. 'echo', that writes to standard output or error in the trap on EXIT, while a test running with 'dash' and '--verbose-log' (even without '-x') is interrupted. As far as I can tell, this is not an issue at the moment: - The cleanup commands to stop the credential-helper, Apache, or 'p4d' don't use any such shell builtins. - stop_git_daemon() does use 'say' and 'error', both wrappers around 'echo', but it redirects 'say' to fd 3, i.e. to the log file, and while 'error' does write to standard output, it comes only after the daemon was killed. - The non-builtin commands that actually stop the daemons ('kill', 'apache2 -k stop', 'git credential-cache exit') are silent, so they won't get SIGPIPE before finishing their job. [1] The trap on EXIT must run cleanup commands, because we want to stop any daemons when a test script run with '--immediate' fails and exits early with error. By chaining up the trap on signals to the trap on EXIT we can deal with cleanup commands a bit simpler, because the tests involving daemons only have to set a single trap. [2] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#trap [3] The shells I tried: dash, Bash, ksh, ksh93, mksh, lksh, yash, BusyBox sh, FreeBSD /bin/sh, NetBSD /bin/sh. [4] $ cat trap-output.sh #!/bin/sh trap "echo output; echo error >&2" EXIT { exit; } >OUT 2>ERR $ dash ./trap-output.sh output error $ wc -c OUT ERR 0 OUT 0 ERR On a related note, 'ksh', 'ksh93', and BusyBox sh don't conform to the specs in this respect, either. [5] This '{ set +x; } 2>/dev/null' trick won't help those shells that show trace output for any redirections and don't preserve open file descriptors for the trap on INT, TERM and HUP. The only such shells I'm aware of are 'mksh' and 'lksh'. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-13t4150: remove unused variableAndrei Rybak1-3/+1
In commit 735285b403 ("am: fix signoff when other trailers are present", 2017-08-08) tests using variable $signoff were rewritten and it is no longer used, so just remove it from the test setup. Signed-off-by: Andrei Rybak <rybak.a.v@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-12mingw: respect core.hidedotfiles = false in git-init againJohannes Schindelin1-0/+11
This is a brown paper bag. When adding the tests, we actually failed to verify that the config variable is heeded in git-init at all. And when changing the original patch that marked the .git/ directory as hidden after reading the config, it was lost on this developer that the new code would use the hide_dotfiles variable before the config was read. The fix is obvious: read the (limited, pre-init) config *before* creating the .git/ directory. Please note that we cannot remove the identical-looking `git_config()` call from `create_default_files()`: we create the `.git/` directory between those calls. If we removed it, and if the parent directory is in a Git worktree, and if that worktree's `.git/config` contained any `init.templatedir` setting, we would all of a sudden pick that up. This fixes https://github.com/git-for-windows/git/issues/789 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-11line-log: detect unsupported formatsJeff King1-0/+10
If you use "log -L" with an output format like "--raw" or "--stat", we'll silently ignore the format and just output the normal patch. Let's detect and complain about this, which at least tells the user what's going on. The tests here aren't exhaustive over the set of all formats, but it should at least let us know if somebody breaks the format-checking. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-08t3600: use helpers to replace test -d/f/e/s <path>Rohit Ashiwal1-75/+75
Take advantage of helper functions test_path_is_dir(), test_path_is_missing(), etc. to replace `test -d|f|e|s` since the functions make the code more readable and have better error messages. Signed-off-by: Rohit Ashiwal <rohit.ashiwal265@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-08t3600: modernize styleRohit Ashiwal1-100/+107
The tests in `t3600-rm.sh` were written long time ago, and has a lot of style violations, including the mixed use of tabs and spaces, not having the title and the opening quote of the body on the first line of the tests, and other shell script style violations. Update it to match the CodingGuidelines. Signed-off-by: Rohit Ashiwal <rohit.ashiwal265@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-08test functions: add function `test_file_not_empty`Rohit Ashiwal1-0/+9
Add a helper function to ensure that a given path is a non-empty file, and give an error message when it is not. Signed-off-by: Rohit Ashiwal <rohit.ashiwal265@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-08Make sure refs/rewritten/ is per-worktreeNguyễn Thái Ngọc Duy1-0/+35
a9be29c981 (sequencer: make refs generated by the `label` command worktree-local, 2018-04-25) adds refs/rewritten/ as per-worktree reference space. Unfortunately (my bad) there are a couple places that need update to make sure it's really per-worktree. - add_per_worktree_entries_to_dir() is updated to make sure ref listing look at per-worktree refs/rewritten/ instead of per-repo one [1] - common_list[] is updated so that git_path() returns the correct location. This includes "rev-parse --git-path". This mess is created by me. I started trying to fix it with the introduction of refs/worktree, where all refs will be per-worktree without special treatments. Unfortunate refs/rewritten came before refs/worktree so this is all we can do. This also fixes logs/refs/worktree not being per-worktree. [1] note that ref listing still works sometimes. For example, if you have .git/worktrees/foo/refs/rewritten/bar AND the directory .git/worktrees/refs/rewritten, refs/rewritten/bar will show up. add_per_worktree_entries_to_dir() is only needed when the directory .git/worktrees/refs/rewritten is missing. Reported-by: Phillip Wood <phillip.wood123@gmail.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-08built-in stash: handle :(glob) pathspecs againJohannes Schindelin1-0/+6
When passing a list of pathspecs to, say, `git add`, we need to be careful to use the original form, not the parsed form of the pathspecs. This makes a difference e.g. when calling git stash -- ':(glob)**/*.txt' where the original form includes the `:(glob)` prefix while the parsed form does not. However, in the built-in `git stash`, we passed the parsed (i.e. incorrect) form, and `git add` would fail with the error message: fatal: pathspec '**/*.txt' did not match any files at the stage where `git stash` drops the changes from the worktree, even if `refs/stash` has been actually updated successfully. This fixes https://github.com/git-for-windows/git/issues/2037 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-08line-log: suppress diff output with "-s"Jeff King1-0/+7
When "-L" is in use, we ignore any diff output format that the user provides to us, and just always print a patch (with extra context lines covering the whole area of interest). It's not entirely clear what we should do with all formats (e.g., should "--stat" show just the diffstat of the touched lines, or the stat for the whole file?). But "-s" is pretty clear: the user probably wants to see just the commits that touched those lines, without any diff at all. Let's at least make that work. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07t5552: compensate for v2 filtering ref adv.Jonathan Tan1-1/+4
Protocol v2 filters the ref advertisement, but protocol v0 does not. A test in t5552 uses the ref advertisement, so fix it to use protocol v0. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07tests: fix protocol version for overspecificationsJonathan Tan4-11/+46
These tests are also marked with a NEEDSWORK comment. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07t5700: only run with protocol version 1Jonathan Tan1-0/+3
Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>