aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2024-04-17Merge branch 'icasefs-symlink-confusion'Johannes Schindelin16-57/+559
This topic branch fixes two vulnerabilities: - Recursive clones on case-insensitive filesystems that support symbolic links are susceptible to case confusion that can be exploited to execute just-cloned code during the clone operation. - Repositories can be configured to execute arbitrary code during local clones. To address this, the ownership checks introduced in v2.30.3 are now extended to cover cloning local repositories. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17submodule: require the submodule path to contain directories onlyJohannes Schindelin4-5/+113
Submodules are stored in subdirectories of their superproject. When these subdirectories have been replaced with symlinks by a malicious actor, all kinds of mayhem can be caused. This _should_ not be possible, but many CVEs in the past showed that _when_ possible, it allows attackers to slip in code that gets executed during, say, a `git clone --recursive` operation. Let's add some defense-in-depth to disallow submodule paths to have anything except directories in them. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17clone_submodule: avoid using `access()` on directoriesJohannes Schindelin1-1/+1
In 0060fd1511b (clone --recurse-submodules: prevent name squatting on Windows, 2019-09-12), I introduced code to verify that a git dir either does not exist, or is at least empty, to fend off attacks where an inadvertently (and likely maliciously) pre-populated git dir would be used while cloning submodules recursively. The logic used `access(<path>, X_OK)` to verify that a directory exists before calling `is_empty_dir()` on it. That is a curious way to check for a directory's existence and might well fail for unwanted reasons. Even the original author (it was I ;-) ) struggles to explain why this function was used rather than `stat()`. This code was _almost_ copypastad in the previous commit, but that `access()` call was caught during review. Let's use `stat()` instead also in the code that was almost copied verbatim. Let's not use `lstat()` because in the unlikely event that somebody snuck a symbolic link in, pointing to a crafted directory, we want to verify that that directory is empty. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17submodules: submodule paths must not contain symlinksJohannes Schindelin2-0/+83
When creating a submodule path, we must be careful not to follow symbolic links. Otherwise we may follow a symbolic link pointing to a gitdir (which are valid symbolic links!) e.g. while cloning. On case-insensitive filesystems, however, we blindly replace a directory that has been created as part of the `clone` operation with a symlink when the path to the latter differs only in case from the former's path. Let's simply avoid this situation by expecting not ever having to overwrite any existing file/directory/symlink upon cloning. That way, we won't even replace a directory that we just created. This addresses CVE-2024-32002. Reported-by: Filip Hejsek <filip.hejsek@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17clone: prevent clashing git dirs when cloning submodule in parallelFilip Hejsek3-2/+66
While it is expected to have several git dirs within the `.git/modules/` tree, it is important that they do not interfere with each other. For example, if one submodule was called "captain" and another submodule "captain/hooks", their respective git dirs would clash, as they would be located in `.git/modules/captain/` and `.git/modules/captain/hooks/`, respectively, i.e. the latter's files could clash with the actual Git hooks of the former. To prevent these clashes, and in particular to prevent hooks from being written and then executed as part of a recursive clone, we introduced checks as part of the fix for CVE-2019-1387 in a8dee3ca61 (Disallow dubiously-nested submodule git directories, 2019-10-01). It is currently possible to bypass the check for clashing submodule git dirs in two ways: 1. parallel cloning 2. checkout --recurse-submodules Let's check not only before, but also after parallel cloning (and before checking out the submodule), that the git dir is not clashing with another one, otherwise fail. This addresses the parallel cloning issue. As to the parallel checkout issue: It requires quite a few manual steps to create clashing git dirs because Git itself would refuse to initialize the inner one, as demonstrated by the test case. Nevertheless, let's teach the recursive checkout (namely, the `submodule_move_head()` function that is used by the recursive checkout) to be careful to verify that it does not use a clashing git dir, and if it does, disable it (by deleting the `HEAD` file so that subsequent Git calls won't recognize it as a git dir anymore). Note: The parallel cloning test case contains a `cat err` that proved to be highly useful when analyzing the racy nature of the operation (the operation can fail with three different error messages, depending on timing), and was left on purpose to ease future debugging should the need arise. Signed-off-by: Filip Hejsek <filip.hejsek@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17t7423: add tests for symlinked submodule directoriesFilip Hejsek1-0/+66
Submodule operations must not follow symlinks in working tree, because otherwise files might be written to unintended places, leading to vulnerabilities. Signed-off-by: Filip Hejsek <filip.hejsek@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17has_dir_name(): do not get confused by characters < '/'Filip Hejsek2-53/+47
There is a bug in directory/file ("D/F") conflict checking optimization: It assumes that such a conflict cannot happen if a newly added entry's path is lexicgraphically "greater than" the last already-existing index entry _and_ contains a directory separator that comes strictly after the common prefix (`len > len_eq_offset`). This assumption is incorrect, though: `a-` sorts _between_ `a` and `a/b`, their common prefix is `a`, the slash comes after the common prefix, and there is still a file/directory conflict. Let's re-design this logic, taking these facts into consideration: - It is impossible for a file to sort after another file with whose directory it conflicts because the trailing NUL byte is always smaller than any other character. - Since there are quite a number of ASCII characters that sort before the slash (e.g. `-`, `.`, the space character), looking at the last already-existing index entry is not enough to determine whether there is a D/F conflict when the first character different from the existing last index entry's path is a slash. If it is not a slash, there cannot be a file/directory conflict. And if the existing index entry's first different character is a slash, it also cannot be a file/directory conflict because the optimization requires the newly-added entry's path to sort _after_ the existing entry's, and the conflicting file's path would not. So let's fall back to the regular binary search whenever the newly-added item's path differs in a slash character. If it does not, and it sorts after the last index entry, there is no D/F conflict and the new index entry can be safely appended. This fix also nicely simplifies the logic and makes it much easier to reason about, while the impact on performance should be negligible: After this fix, the optimization will be skipped only when index entry's paths differ in a slash and a space, `!`, `"`, `#`, `$`, `%`, `&`, `'`, | ( `)`, `*`, `+`, `,`, `-`, or `.`, which should be a rare situation. Signed-off-by: Filip Hejsek <filip.hejsek@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17docs: document security issues around untrusted .git dirsJeff King2-0/+46
For a long time our general philosophy has been that it's unsafe to run arbitrary Git commands if you don't trust the hooks or config in .git, but that running upload-pack should be OK. E.g., see 1456b043fc (Remove post-upload-hook, 2009-12-10), or the design of uploadpack.packObjectsHook. But we never really documented this (and even the discussions that led to 1456b043fc were not on the public list!). Let's try to make our approach more clear, but also be realistic that even upload-pack carries some risk. Helped-by: Filip Hejsek <filip.hejsek@gmail.com> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17upload-pack: disable lazy-fetching by defaultJeff King4-0/+46
The upload-pack command tries to avoid trusting the repository in which it's run (e.g., by not running any hooks and not using any config that contains arbitrary commands). But if the server side of a fetch or a clone is a partial clone, then either upload-pack or its child pack-objects may run a lazy "git fetch" under the hood. And it is very easy to convince fetch to run arbitrary commands. The "server" side can be a local repository owned by someone else, who would be able to configure commands that are run during a clone with the current user's permissions. This issue has been designated CVE-2024-32004. The fix in this commit's parent helps in this scenario, as well as in related scenarios using SSH to clone, where the untrusted .git directory is owned by a different user id. But if you received one as a zip file, on a USB stick, etc, it may be owned by your user but still untrusted. This has been designated CVE-2024-32465. To mitigate the issue more completely, let's disable lazy fetching entirely during `upload-pack`. While fetching from a partial repository should be relatively rare, it is certainly not an unreasonable workflow. And thus we need to provide an escape hatch. This commit works by respecting a GIT_NO_LAZY_FETCH environment variable (to skip the lazy-fetch), and setting it in upload-pack, but only when the user has not already done so (which gives us the escape hatch). The name of the variable is specifically chosen to match what has already been added in 'master' via e6d5479e7a (git: extend --no-lazy-fetch to work across subprocesses, 2024-02-27). Since we're building this fix as a backport for older versions, we could cherry-pick that patch and its earlier steps. However, we don't really need the niceties (like a "--no-lazy-fetch" option) that it offers. By using the same name, everything should just work when the two are eventually merged, but here are a few notes: - the blocking of the fetch in e6d5479e7a is incomplete! It sets fetch_if_missing to 0 when we setup the repository variable, but that isn't enough. pack-objects in particular will call prefetch_to_pack() even if that variable is 0. This patch by contrast checks the environment variable at the lowest level before we call the lazy fetch, where we can be sure to catch all code paths. Possibly the setting of fetch_if_missing from e6d5479e7a can be reverted, but it may be useful to have. For example, some code may want to use that flag to change behavior before it gets to the point of trying to start the fetch. At any rate, that's all outside the scope of this patch. - there's documentation for GIT_NO_LAZY_FETCH in e6d5479e7a. We can live without that here, because for the most part the user shouldn't need to set it themselves. The exception is if they do want to override upload-pack's default, and that requires a separate documentation section (which is added here) - it would be nice to use the NO_LAZY_FETCH_ENVIRONMENT macro added by e6d5479e7a, but those definitions have moved from cache.h to environment.h between 2.39.3 and master. I just used the raw string literals, and we can replace them with the macro once this topic is merged to master. At least with respect to CVE-2024-32004, this does render this commit's parent commit somewhat redundant. However, it is worth retaining that commit as defense in depth, and because it may help other issues (e.g., symlink/hardlink TOCTOU races, where zip files are not really an interesting attack vector). The tests in t0411 still pass, but now we have _two_ mechanisms ensuring that the evil command is not run. Let's beef up the existing ones to check that they failed for the expected reason, that we refused to run upload-pack at all with an alternate user id. And add two new ones for the same-user case that both the restriction and its escape hatch. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17fetch/clone: detect dubious ownership of local repositoriesJohannes Schindelin4-3/+38
When cloning from somebody else's repositories, it is possible that, say, the `upload-pack` command is overridden in the repository that is about to be cloned, which would then be run in the user's context who started the clone. To remind the user that this is a potentially unsafe operation, let's extend the ownership checks we have already established for regular gitdir discovery to extend also to local repositories that are about to be cloned. This protection extends also to file:// URLs. The fixes in this commit address CVE-2024-32004. Note: This commit does not touch the `fetch`/`clone` code directly, but instead the function used implicitly by both: `enter_repo()`. This function is also used by `git receive-pack` (i.e. pushes), by `git upload-archive`, by `git daemon` and by `git http-backend`. In setups that want to serve repositories owned by different users than the account running the service, this will require `safe.*` settings to be configured accordingly. Also note: there are tiny time windows where a time-of-check-time-of-use ("TOCTOU") race is possible. The real solution to those would be to work with `fstat()` and `openat()`. However, the latter function is not available on Windows (and would have to be emulated with rather expensive low-level `NtCreateFile()` calls), and the changes would be quite extensive, for my taste too extensive for the little gain given that embargoed releases need to pay extra attention to avoid introducing inadvertent bugs. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17t0411: add tests for cloning from partial repoFilip Hejsek1-0/+60
Cloning from a partial repository must not fetch missing objects into the partial repository, because that can lead to arbitrary code execution. Add a couple of test cases, pretending to the `upload-pack` command (and to that command only) that it is working on a repository owned by someone else. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Filip Hejsek <filip.hejsek@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17Merge branch 'js/github-actions-update'Johannes Schindelin1-10/+10
Update remaining GitHub Actions jobs to avoid warnings against using deprecated version of Node.js. * js/github-actions-update: ci(linux32): add a note about Actions that must not be updated ci: bump remaining outdated Actions versions With this backport, `maint-2.39`'s CI builds are finally healthy again. Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2024-04-17Merge branch 'jc/maint-github-actions-update'Johannes Schindelin2-10/+10
* jc/maint-github-actions-update: GitHub Actions: update to github-script@v7 GitHub Actions: update to checkout@v4 Yet another thing to help `maint-2.39`'s CI builds to become healthy again. Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2024-04-17ci(linux32): add a note about Actions that must not be updatedJohannes Schindelin1-2/+2
The Docker container used by the `linux32` job comes without Node.js, and therefore the `actions/checkout` and `actions/upload-artifact` Actions cannot be upgraded to the latest versions (because they use Node.js). One time too many, I accidentally tried to update them, where `actions/checkout` at least fails immediately, but the `actions/upload-artifact` step is only used when any test fails, and therefore the CI run usually passes even though that Action was updated to a version that is incompatible with the Docker container in which this job runs. So let's add a big fat warning, mainly for my own benefit, to avoid running into the very same issue over and over again. Backported-from: 20e0ff8835 (ci(linux32): add a note about Actions that must not be updated, 2024-02-11) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17GitHub Actions: update to github-script@v7Junio C Hamano1-1/+1
We seem to be getting "Node.js 16 actions are deprecated." warnings for jobs that use github-script@v6. Update to github-script@v7, which is said to use Node.js 20. Backported-from: c4ddbe043e (GitHub Actions: update to github-script@v7, 2024-02-02) Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17ci: bump remaining outdated Actions versionsJohannes Schindelin1-8/+8
After activating automatic Dependabot updates in the git-for-windows/git repository, Dependabot noticed a couple of yet-unaddressed updates. They avoid "Node.js 16 Actions" deprecation messages by bumping the following Actions' versions: - actions/upload-artifact from 3 to 4 - actions/download-artifact from 3 to 4 - actions/cache from 3 to 4 Backported-from: 820a340085 (ci: bump remaining outdated Actions versions, 2024-02-11) Helped-by: Matthias Aßhauer <mha1993@live.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17GitHub Actions: update to checkout@v4Junio C Hamano2-9/+9
We seem to be getting "Node.js 16 actions are deprecated." warnings for jobs that use checkout@v3. Except for the i686 containers job that is kept at checkout@v1 [*], update to checkout@v4, which is said to use Node.js 20. [*] 6cf4d908 (ci(main): upgrade actions/checkout to v3, 2022-12-05) refers to https://github.com/actions/runner/issues/2115 and explains why container jobs are kept at checkout@v1. We may want to check the current status of the issue and move it to the same version as other jobs, but that is outside the scope of this step. Backported-from: e94dec0c1d (GitHub Actions: update to checkout@v4, 2024-02-02) Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17Merge branch 'quicker-asan-lsan'Johannes Schindelin1-0/+4
This patch speeds up the `asan`/`lsan` jobs that are really slow enough already. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17Merge branch 'jk/test-lsan-denoise-output'Johannes Schindelin1-0/+1
Tests with LSan from time to time seem to emit harmless message that makes our tests unnecessarily flakey; we work it around by filtering the uninteresting output. * jk/test-lsan-denoise-output: test-lib: ignore uninteresting LSan output Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-16Merge branch 'js/ci-use-macos-13'Johannes Schindelin4-21/+30
Replace macos-12 used at GitHub CI with macos-13. * js/ci-use-macos-13: ci: upgrade to using macos-13 This is another backport to `maint-2.39` to allow less CI jobs to break. Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2024-04-16Merge branch 'backport/jk/libcurl-8.7-regression-workaround' into maint-2.39Johannes Schindelin4-1/+14
Fix was added to work around a regression in libcURL 8.7.0 (which has already been fixed in their tip of the tree). * jk/libcurl-8.7-regression-workaround: remote-curl: add Transfer-Encoding header only for older curl INSTALL: bump libcurl version to 7.21.3 http: reset POSTFIELDSIZE when clearing curl handle Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2024-04-16Merge branch 'jk/redact-h2h3-headers-fix' into maint-2.42Johannes Schindelin1-5/+31
HTTP Header redaction code has been adjusted for a newer version of cURL library that shows its traces differently from earlier versions. * jk/redact-h2h3-headers-fix: http: update curl http/2 info matching for curl 8.3.0 http: factor out matching of curl http/2 trace lines This backport to `maint-2.39` is needed to bring the following test cases back to a working state in conjunction with recent libcurl versions: - t5559.17 GIT_TRACE_CURL redacts auth details - t5559.18 GIT_CURL_VERBOSE redacts auth details - t5559.38 cookies are redacted by default Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2024-04-16Merge branch 'jk/httpd-test-updates'Johannes Schindelin3-32/+12
Test update. * jk/httpd-test-updates: t/lib-httpd: increase ssl key size to 2048 bits t/lib-httpd: drop SSLMutex config t/lib-httpd: bump required apache version to 2.4 t/lib-httpd: bump required apache version to 2.2 This is a backport onto the `maint-2.39` branch, to improve the CI health of that branch. Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2024-04-16Merge branch 'jk/http-test-fixes'Johannes Schindelin5-114/+122
Various fix-ups on HTTP tests. * jk/http-test-fixes: t5559: make SSL/TLS the default t5559: fix test failures with LIB_HTTPD_SSL t/lib-httpd: enable HTTP/2 "h2" protocol, not just h2c t/lib-httpd: respect $HTTPD_PROTO in expect_askpass() t5551: drop curl trace lines without headers t5551: handle v2 protocol in cookie test t5551: simplify expected cookie file t5551: handle v2 protocol in upload-pack service test t5551: handle v2 protocol when checking curl trace t5551: stop forcing clone to run with v0 protocol t5551: handle HTTP/2 when checking curl trace t5551: lower-case headers in expected curl trace t5551: drop redundant grep for Accept-Language t5541: simplify and move "no empty path components" test t5541: stop marking "used receive-pack service" test as v0 only t5541: run "used receive-pack service" test earlier This is a backport onto the `maint-2.39` branch, starting to take care of making that branch's CI builds healthy again. Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2024-04-16ci(linux-asan/linux-ubsan): let's save some timeJohannes Schindelin1-0/+4
Every once in a while, the `git-p4` tests flake for reasons outside of our control. It typically fails with "Connection refused" e.g. here: https://github.com/git/git/actions/runs/5969707156/job/16196057724 [...] + git p4 clone --dest=/home/runner/work/git/git/t/trash directory.t9807-git-p4-submit/git //depot Initialized empty Git repository in /home/runner/work/git/git/t/trash directory.t9807-git-p4-submit/git/.git/ Perforce client error: Connect to server failed; check $P4PORT. TCP connect to localhost:9807 failed. connect: 127.0.0.1:9807: Connection refused failure accessing depot: could not run p4 Importing from //depot into /home/runner/work/git/git/t/trash directory.t9807-git-p4-submit/git [...] This happens in other jobs, too, but in the `linux-asan`/`linux-ubsan` jobs it hurts the most because those jobs often take an _awfully_ long time to run, therefore re-running a failed `linux-asan`/`linux-ubsan` jobs is _very_ costly. The purpose of the `linux-asan`/`linux-ubsan` jobs is to exercise the C code of Git, anyway, and any part of Git's source code that the `git-p4` tests run and that would benefit from the attention of ASAN/UBSAN are run better in other tests anyway, as debugging C code run via Python scripts can get a bit hairy. In fact, it is not even just `git-p4` that is the problem (even if it flakes often enough to be problematic in the CI builds), but really the part about Python scripts. So let's just skip any Python parts of the tests from being run in that job. For good measure, also skip the Subversion tests because debugging C code run via Perl scripts is as much fun as debugging C code run via Python scripts. And it will reduce the time this very expensive job takes, which is a big benefit. Backported to `maint-2.39` as another step to get that branch's CI builds back to a healthy state. Backported-from: 6ba913629f (ci(linux-asan-ubsan): let's save some time, 2023-08-29) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-16test-lib: ignore uninteresting LSan outputJeff King1-0/+1
When I run the tests in leak-checking mode the same way our CI job does, like: make SANITIZE=leak \ GIT_TEST_PASSING_SANITIZE_LEAK=true \ GIT_TEST_SANITIZE_LEAK_LOG=true \ test then LSan can racily produce useless entries in the log files that look like this: ==git==3034393==Unable to get registers from thread 3034307. I think they're mostly harmless based on the source here: https://github.com/llvm/llvm-project/blob/7e0a52e8e9ef6394bb62e0b56e17fa23e7262411/compiler-rt/lib/lsan/lsan_common.cpp#L414 which reads: PtraceRegistersStatus have_registers = suspended_threads.GetRegistersAndSP(i, &registers, &sp); if (have_registers != REGISTERS_AVAILABLE) { Report("Unable to get registers from thread %llu.\n", os_id); // If unable to get SP, consider the entire stack to be reachable unless // GetRegistersAndSP failed with ESRCH. if (have_registers == REGISTERS_UNAVAILABLE_FATAL) continue; sp = stack_begin; } The program itself still runs fine and LSan doesn't cause us to abort. But test-lib.sh looks for any non-empty LSan logs and marks the test as a failure anyway, under the assumption that we simply missed the failing exit code somehow. I don't think I've ever seen this happen in the CI job, but running locally using clang-14 on an 8-core machine, I can't seem to make it through a full run of the test suite without having at least one failure. And it's a different one every time (though they do seem to often be related to packing tests, which makes sense, since that is one of our biggest users of threaded code). We can hack around this by only counting LSan log files that contain a line that doesn't match our known-uninteresting pattern. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-16ci: upgrade to using macos-13Johannes Schindelin2-7/+5
In April, GitHub announced that the `macos-13` pool is available: https://github.blog/changelog/2023-04-24-github-actions-macos-13-is-now-available/. It is only a matter of time until the `macos-12` pool is going away, therefore we should switch now, without pressure of a looming deadline. Since the `macos-13` runners no longer include Python2, we also drop specifically testing with Python2 and switch uniformly to Python3, see https://github.com/actions/runner-images/blob/HEAD/images/macos/macos-13-Readme.md for details about the software available on the `macos-13` pool's runners. Also, on macOS 13, Homebrew seems to install a `gcc@9` package that no longer comes with a regular `unistd.h` (there seems only to be a `ssp/unistd.h`), and hence builds would fail with: In file included from base85.c:1: git-compat-util.h:223:10: fatal error: unistd.h: No such file or directory 223 | #include <unistd.h> | ^~~~~~~~~~ compilation terminated. The reason why we install GCC v9.x explicitly is historical, and back in the days it was because it was the _newest_ version available via Homebrew: 176441bfb58 (ci: build Git with GCC 9 in the 'osx-gcc' build job, 2019-11-27). To reinstate the spirit of that commit _and_ to fix that build failure, let's switch to the now-newest GCC version: v13.x. Backported-from: 682a868f67 (ci: upgrade to using macos-13, 2023-11-03) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-16Merge branch 'jh/fsmonitor-darwin-modernize'Johannes Schindelin2-14/+25
Stop using deprecated macOS API in fsmonitor. * jh/fsmonitor-darwin-modernize: fsmonitor: eliminate call to deprecated FSEventStream function This backport to `maint-2.39` is needed to be able to build on `macos-13`, which we need to update to as we restore the CI health of that branch. Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2024-04-10remote-curl: add Transfer-Encoding header only for older curlJeff King2-0/+12
As of curl 7.66.0, we don't need to manually specify a "chunked" Transfer-Encoding header. Instead, modern curl deduces the need for it in a POST that has a POSTFIELDSIZE of -1 and uses READFUNCTION rather than POSTFIELDS. That version is recent enough that we can't just drop the header; we need to do so conditionally. Since it's only a single line, it seems like the simplest thing would just be to keep setting it unconditionally (after all, the #ifdefs are much longer than the actual code). But there's another wrinkle: HTTP/2. Curl may choose to use HTTP/2 under the hood if the server supports it. And in that protocol, we do not use the chunked encoding for streaming at all. Most versions of curl handle this just fine by recognizing and removing the header. But there's a regression in curl 8.7.0 and 8.7.1 where it doesn't, and large requests over HTTP/2 are broken (which t5559 notices). That regression has since been fixed upstream, but not yet released. Make the setting of this header conditional, which will let Git work even with those buggy curl versions. And as a bonus, it serves as a reminder that we can eventually clean up the code as we bump the supported curl versions. This is a backport of 92a209bf24 (remote-curl: add Transfer-Encoding header only for older curl, 2024-04-05) into the `maint-2.39` branch. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-10INSTALL: bump libcurl version to 7.21.3Jeff King1-1/+1
Our documentation claims we support curl versions back to 7.19.5. But we can no longer compile with that version since adding an unconditional use of CURLOPT_RESOLVE in 511cfd3bff (http: add custom hostname to IP address resolutions, 2022-05-16). That feature wasn't added to libcurl until 7.21.3. We could add #ifdefs to make this work back to 7.19.5. But given that nobody noticed the compilation failure in the intervening two years, it makes more sense to bump the version in the documentation to 7.21.3 (which is itself over 13 years old). We could perhaps go forward even more (which would let us drop some cruft from git-curl-compat.h), but this should be an obviously safe jump, and we can move forward later. Note that user-visible syntax for CURLOPT_RESOLVE has grown new features in subsequent curl versions. Our documentation mentions "+" and "-" entries, which require more recent versions than 7.21.3. We could perhaps clarify that in our docs, but it's probably not worth cluttering them with restrictions of ancient curl versions. This is a backport of c28ee09503 (INSTALL: bump libcurl version to 7.21.3, 2024-04-02) into the `maint-2.39` branch. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-10http: reset POSTFIELDSIZE when clearing curl handleJeff King1-0/+1
In get_active_slot(), we return a CURL handle that may have been used before (reusing them is good because it lets curl reuse the same connection across many requests). We set a few curl options back to defaults that may have been modified by previous requests. We reset POSTFIELDS to NULL, but do not reset POSTFIELDSIZE (which defaults to "-1"). This usually doesn't matter because most POSTs will set both fields together anyway. But there is one exception: when handling a large request in remote-curl's post_rpc(), we don't set _either_, and instead set a READFUNCTION to stream data into libcurl. This can interact weirdly with a stale POSTFIELDSIZE setting, because curl will assume it should read only some set number of bytes from our READFUNCTION. However, it has worked in practice because we also manually set a "Transfer-Encoding: chunked" header, which libcurl uses as a clue to set the POSTFIELDSIZE to -1 itself. So everything works, but we're better off resetting the size manually for a few reasons: - there was a regression in curl 8.7.0 where the chunked header detection didn't kick in, causing any large HTTP requests made by Git to fail. This has since been fixed (but not yet released). In the issue, curl folks recommended setting it explicitly to -1: https://github.com/curl/curl/issues/13229#issuecomment-2029826058 and it indeed works around the regression. So even though it won't be strictly necessary after the fix there, this will help folks who end up using the affected libcurl versions. - it's consistent with what a new curl handle would look like. Since get_active_slot() may or may not return a used handle, this reduces the possibility of heisenbugs that only appear with certain request patterns. Note that the recommendation in the curl issue is to actually drop the manual Transfer-Encoding header. Modern libcurl will add the header itself when streaming from a READFUNCTION. However, that code wasn't added until 802aa5ae2 (HTTP: use chunked Transfer-Encoding for HTTP_POST if size unknown, 2019-07-22), which is in curl 7.66.0. We claim to support back to 7.19.5, so those older versions still need the manual header. This is a backport of 3242311742 (http: reset POSTFIELDSIZE when clearing curl handle, 2024-04-02) into the `maint-2.39` branch. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-09-15http: update curl http/2 info matching for curl 8.3.0Jeff King1-0/+14
To redact header lines in http/2 curl traces, we have to parse past some prefix bytes that curl sticks in the info lines it passes to us. That changed once already, and we adapted in db30130165 (http: handle both "h2" and "h2h3" in curl info lines, 2023-06-17). Now it has changed again, in curl's fbacb14c4 (http2: cleanup trace messages, 2023-08-04), which was released in curl 8.3.0. Running a build of git linked against that version will fail to redact the trace (and as before, t5559 notices and complains). The format here is a little more complicated than the other ones, as it now includes a "stream id". This is not constant but is always numeric, so we can easily parse past it. We'll continue to match the old versions, of course, since we want to work with many different versions of curl. We can't even select one format at compile time, because the behavior depends on the runtime version of curl we use, not the version we build against. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-09-15http: factor out matching of curl http/2 trace linesJeff King1-6/+17
We have to parse out curl's http/2 trace lines so we can redact their headers. We already match two different types of lines from various vintages of curl. In preparation for adding another (which will be slightly more complex), let's pull the matching into its own function, rather than doing it in the middle of a conditional. While we're doing so, let's expand the comment a bit to describe the two matches. That probably should have been part of db30130165 (http: handle both "h2" and "h2h3" in curl info lines, 2023-06-17), but will become even more important as we add new types. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-17http: handle both "h2" and "h2h3" in curl info linesJeff King1-1/+2
When redacting auth tokens in trace output from curl, we look for http/2 headers of the form "h2h3 [header: value]". This comes from b637a41ebe (http: redact curl h2h3 headers in info, 2022-11-11). But the "h2h3" prefix changed to just "h2" in curl's fc2f1e547 (http2: support HTTP/2 to forward proxies, non-tunneling, 2023-04-14). That's in released version curl 8.1.0; linking against that version means we'll fail to correctly redact the trace. Our t5559.17 notices and fails. We can fix this by matching either prefix, which should handle both old and new versions. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-17Git 2.39.3v2.39.3Johannes Schindelin2-4/+10
Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17Sync with 2.38.5Johannes Schindelin15-13/+191
* maint-2.38: (32 commits) Git 2.38.5 Git 2.37.7 Git 2.36.6 Git 2.35.8 Git 2.34.8 Git 2.33.8 Git 2.32.7 Git 2.31.8 tests: avoid using `test_i18ncmp` Git 2.30.9 gettext: avoid using gettext if the locale dir is not present apply --reject: overwrite existing `.rej` symlink if it exists http.c: clear the 'finished' member once we are done with it clone.c: avoid "exceeds maximum object size" error with GCC v12.x range-diff: use ssize_t for parsed "len" in read_patches() range-diff: handle unterminated lines in read_patches() range-diff: drop useless "offset" variable from read_patches() t5604: GETTEXT_POISON fix, conclusion t5604: GETTEXT_POISON fix, part 1 t5619: GETTEXT_POISON fix t0003: GETTEXT_POISON fix, conclusion ...
2023-04-17Git 2.38.5v2.38.5Johannes Schindelin3-2/+10
Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17Sync with 2.37.7Johannes Schindelin14-13/+183
* maint-2.37: (31 commits) Git 2.37.7 Git 2.36.6 Git 2.35.8 Git 2.34.8 Git 2.33.8 Git 2.32.7 Git 2.31.8 tests: avoid using `test_i18ncmp` Git 2.30.9 gettext: avoid using gettext if the locale dir is not present apply --reject: overwrite existing `.rej` symlink if it exists http.c: clear the 'finished' member once we are done with it clone.c: avoid "exceeds maximum object size" error with GCC v12.x range-diff: use ssize_t for parsed "len" in read_patches() range-diff: handle unterminated lines in read_patches() range-diff: drop useless "offset" variable from read_patches() t5604: GETTEXT_POISON fix, conclusion t5604: GETTEXT_POISON fix, part 1 t5619: GETTEXT_POISON fix t0003: GETTEXT_POISON fix, conclusion t0003: GETTEXT_POISON fix, part 1 ...
2023-04-17Git 2.37.7v2.37.7Johannes Schindelin3-2/+9
Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17Sync with 2.36.6Johannes Schindelin17-28/+189
* maint-2.36: (30 commits) Git 2.36.6 Git 2.35.8 Git 2.34.8 Git 2.33.8 Git 2.32.7 Git 2.31.8 tests: avoid using `test_i18ncmp` Git 2.30.9 gettext: avoid using gettext if the locale dir is not present apply --reject: overwrite existing `.rej` symlink if it exists http.c: clear the 'finished' member once we are done with it clone.c: avoid "exceeds maximum object size" error with GCC v12.x range-diff: use ssize_t for parsed "len" in read_patches() range-diff: handle unterminated lines in read_patches() range-diff: drop useless "offset" variable from read_patches() t5604: GETTEXT_POISON fix, conclusion t5604: GETTEXT_POISON fix, part 1 t5619: GETTEXT_POISON fix t0003: GETTEXT_POISON fix, conclusion t0003: GETTEXT_POISON fix, part 1 t0033: GETTEXT_POISON fix ...
2023-04-17Git 2.36.6v2.36.6Johannes Schindelin3-2/+9
Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17Sync with 2.35.8Johannes Schindelin17-30/+184
* maint-2.35: (29 commits) Git 2.35.8 Git 2.34.8 Git 2.33.8 Git 2.32.7 Git 2.31.8 tests: avoid using `test_i18ncmp` Git 2.30.9 gettext: avoid using gettext if the locale dir is not present apply --reject: overwrite existing `.rej` symlink if it exists http.c: clear the 'finished' member once we are done with it clone.c: avoid "exceeds maximum object size" error with GCC v12.x range-diff: use ssize_t for parsed "len" in read_patches() range-diff: handle unterminated lines in read_patches() range-diff: drop useless "offset" variable from read_patches() t5604: GETTEXT_POISON fix, conclusion t5604: GETTEXT_POISON fix, part 1 t5619: GETTEXT_POISON fix t0003: GETTEXT_POISON fix, conclusion t0003: GETTEXT_POISON fix, part 1 t0033: GETTEXT_POISON fix http: support CURLOPT_PROTOCOLS_STR ...
2023-04-17Git 2.35.8v2.35.8Johannes Schindelin3-2/+9
Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17Sync with 2.34.8Johannes Schindelin19-35/+221
* maint-2.34: (28 commits) Git 2.34.8 Git 2.33.8 Git 2.32.7 Git 2.31.8 tests: avoid using `test_i18ncmp` Git 2.30.9 gettext: avoid using gettext if the locale dir is not present apply --reject: overwrite existing `.rej` symlink if it exists http.c: clear the 'finished' member once we are done with it clone.c: avoid "exceeds maximum object size" error with GCC v12.x range-diff: use ssize_t for parsed "len" in read_patches() range-diff: handle unterminated lines in read_patches() range-diff: drop useless "offset" variable from read_patches() t5604: GETTEXT_POISON fix, conclusion t5604: GETTEXT_POISON fix, part 1 t5619: GETTEXT_POISON fix t0003: GETTEXT_POISON fix, conclusion t0003: GETTEXT_POISON fix, part 1 t0033: GETTEXT_POISON fix http: support CURLOPT_PROTOCOLS_STR http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION ...
2023-04-17Git 2.34.8v2.34.8Johannes Schindelin3-2/+9
Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17Sync with 2.33.8Johannes Schindelin18-32/+211
* maint-2.33: (27 commits) Git 2.33.8 Git 2.32.7 Git 2.31.8 tests: avoid using `test_i18ncmp` Git 2.30.9 gettext: avoid using gettext if the locale dir is not present apply --reject: overwrite existing `.rej` symlink if it exists http.c: clear the 'finished' member once we are done with it clone.c: avoid "exceeds maximum object size" error with GCC v12.x range-diff: use ssize_t for parsed "len" in read_patches() range-diff: handle unterminated lines in read_patches() range-diff: drop useless "offset" variable from read_patches() t5604: GETTEXT_POISON fix, conclusion t5604: GETTEXT_POISON fix, part 1 t5619: GETTEXT_POISON fix t0003: GETTEXT_POISON fix, conclusion t0003: GETTEXT_POISON fix, part 1 t0033: GETTEXT_POISON fix http: support CURLOPT_PROTOCOLS_STR http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT ...
2023-04-17Git 2.33.8v2.33.8Johannes Schindelin3-2/+9
Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17Sync with 2.32.7Johannes Schindelin21-82/+285
* maint-2.32: (26 commits) Git 2.32.7 Git 2.31.8 tests: avoid using `test_i18ncmp` Git 2.30.9 gettext: avoid using gettext if the locale dir is not present apply --reject: overwrite existing `.rej` symlink if it exists http.c: clear the 'finished' member once we are done with it clone.c: avoid "exceeds maximum object size" error with GCC v12.x range-diff: use ssize_t for parsed "len" in read_patches() range-diff: handle unterminated lines in read_patches() range-diff: drop useless "offset" variable from read_patches() t5604: GETTEXT_POISON fix, conclusion t5604: GETTEXT_POISON fix, part 1 t5619: GETTEXT_POISON fix t0003: GETTEXT_POISON fix, conclusion t0003: GETTEXT_POISON fix, part 1 t0033: GETTEXT_POISON fix http: support CURLOPT_PROTOCOLS_STR http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT ci: install python on ubuntu ...
2023-04-17Git 2.32.7v2.32.7Johannes Schindelin3-2/+9
Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17Sync with 2.31.8Johannes Schindelin21-98/+291
* maint-2.31: (25 commits) Git 2.31.8 tests: avoid using `test_i18ncmp` Git 2.30.9 gettext: avoid using gettext if the locale dir is not present apply --reject: overwrite existing `.rej` symlink if it exists http.c: clear the 'finished' member once we are done with it clone.c: avoid "exceeds maximum object size" error with GCC v12.x range-diff: use ssize_t for parsed "len" in read_patches() range-diff: handle unterminated lines in read_patches() range-diff: drop useless "offset" variable from read_patches() t5604: GETTEXT_POISON fix, conclusion t5604: GETTEXT_POISON fix, part 1 t5619: GETTEXT_POISON fix t0003: GETTEXT_POISON fix, conclusion t0003: GETTEXT_POISON fix, part 1 t0033: GETTEXT_POISON fix http: support CURLOPT_PROTOCOLS_STR http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT ci: install python on ubuntu ci: use the same version of p4 on both Linux and macOS ...
2023-04-17Git 2.31.8v2.31.8Johannes Schindelin3-2/+8
Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17tests: avoid using `test_i18ncmp`Johannes Schindelin3-9/+9
Since `test_i18ncmp` was deprecated in v2.31.*, the instances added in v2.30.9 needed to be converted to `test_cmp` calls. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17Sync with 2.30.9Johannes Schindelin21-104/+291
* maint-2.30: (23 commits) Git 2.30.9 gettext: avoid using gettext if the locale dir is not present apply --reject: overwrite existing `.rej` symlink if it exists http.c: clear the 'finished' member once we are done with it clone.c: avoid "exceeds maximum object size" error with GCC v12.x range-diff: use ssize_t for parsed "len" in read_patches() range-diff: handle unterminated lines in read_patches() range-diff: drop useless "offset" variable from read_patches() t5604: GETTEXT_POISON fix, conclusion t5604: GETTEXT_POISON fix, part 1 t5619: GETTEXT_POISON fix t0003: GETTEXT_POISON fix, conclusion t0003: GETTEXT_POISON fix, part 1 t0033: GETTEXT_POISON fix http: support CURLOPT_PROTOCOLS_STR http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT ci: install python on ubuntu ci: use the same version of p4 on both Linux and macOS ci: remove the pipe after "p4 -V" to catch errors github-actions: run gcc-8 on ubuntu-20.04 image ...
2023-04-17Git 2.30.9v2.30.9Taylor Blau3-2/+45
Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17Merge branch 'tb/config-copy-or-rename-in-file-injection'Taylor Blau2-11/+55
Avoids issues with renaming or deleting sections with long lines, where configuration values may be interpreted as sections, leading to configuration injection. Addresses CVE-2023-29007. * tb/config-copy-or-rename-in-file-injection: config.c: disallow overly-long lines in `copy_or_rename_section_in_file()` config.c: avoid integer truncation in `copy_or_rename_section_in_file()` config: avoid fixed-sized buffer when renaming/deleting a section t1300: demonstrate failure when renaming sections with long lines Signed-off-by: Taylor Blau <me@ttaylorr.com>
2023-04-17Merge branch 'avoid-using-uninitialized-gettext'Johannes Schindelin2-1/+10
Avoids the overhead of calling `gettext` when initialization of the translated messages was skipped. Addresses CVE-2023-25815. * avoid-using-uninitialized-gettext: (1 commit) gettext: avoid using gettext if the locale dir is not present
2023-04-17Merge branch 'js/apply-overwrite-rej-symlink-if-exists' into maint-2.30Junio C Hamano2-2/+27
Address CVE-2023-25652 by deleting any existing `.rej` symbolic links instead of following them. * js/apply-overwrite-rej-symlink-if-exists: apply --reject: overwrite existing `.rej` symlink if it exists Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2023-04-17config.c: disallow overly-long lines in `copy_or_rename_section_in_file()`Taylor Blau2-0/+23
As a defense-in-depth measure to guard against any potentially-unknown buffer overflows in `copy_or_rename_section_in_file()`, refuse to work with overly-long lines in a gitconfig. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2023-04-17config.c: avoid integer truncation in `copy_or_rename_section_in_file()`Taylor Blau1-5/+5
There are a couple of spots within `copy_or_rename_section_in_file()` that incorrectly use an `int` to track an offset within a string, which may truncate or wrap around to a negative value. Historically it was impossible to have a line longer than 1024 bytes anyway, since we used fgets() with a fixed-size buffer of exactly that length. But the recent change to use a strbuf permits us to read lines of arbitrary length, so it's possible for a malicious input to cause us to overflow past INT_MAX and do an out-of-bounds array read. Practically speaking, however, this should never happen, since it requires 2GB section names or values, which are unrealistic in non-malicious circumstances. Co-authored-by: Jeff King <peff@peff.net> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2023-04-17config: avoid fixed-sized buffer when renaming/deleting a sectionTaylor Blau2-8/+9
When renaming (or deleting) a section of configuration, Git uses the function `git_config_copy_or_rename_section_in_file()` to rewrite the configuration file after applying the rename or deletion to the given section. To do this, Git repeatedly calls `fgets()` to read the existing configuration data into a fixed size buffer. When the configuration value under `old_name` exceeds the size of the buffer, we will call `fgets()` an additional time even if there is no newline in the configuration file, since our read length is capped at `sizeof(buf)`. If the first character of the buffer (after zero or more characters satisfying `isspace()`) is a '[', Git will incorrectly treat it as beginning a new section when the original section is being removed. In other words, a configuration value satisfying this criteria can incorrectly be considered as a new secftion instead of a variable in the original section. Avoid this issue by using a variable-width buffer in the form of a strbuf rather than a fixed-with region on the stack. A couple of small points worth noting: - Using a strbuf will cause us to allocate arbitrary sizes to match the length of each line. In practice, we don't expect any reasonable configuration files to have lines that long, and a bandaid will be introduced in a later patch to ensure that this is the case. - We are using strbuf_getwholeline() here instead of strbuf_getline() in order to match `fgets()`'s behavior of leaving the trailing LF character on the buffer (as well as a trailing NUL). This could be changed later, but using strbuf_getwholeline() changes the least about this function's implementation, so it is picked as the safest path. - It is temping to want to replace the loop to skip over characters matching isspace() at the beginning of the buffer with a convenience function like `strbuf_ltrim()`. But this is the wrong approach for a couple of reasons: First, it involves a potentially large and expensive `memmove()` which we would like to avoid. Second, and more importantly, we also *do* want to preserve those spaces to avoid changing the output of other sections. In all, this patch is a minimal replacement of the fixed-width buffer in `git_config_copy_or_rename_section_in_file()` to instead use a `struct strbuf`. Reported-by: André Baptista <andre@ethiack.com> Reported-by: Vítor Pinho <vitor@ethiack.com> Helped-by: Patrick Steinhardt <ps@pks.im> Co-authored-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2023-04-17gettext: avoid using gettext if the locale dir is not presentJohannes Schindelin2-1/+10
In cc5e1bf99247 (gettext: avoid initialization if the locale dir is not present, 2018-04-21) Git was taught to avoid a costly gettext start-up when there are not even any localized messages to work with. But we still called `gettext()` and `ngettext()` functions. Which caused a problem in Git for Windows when the libgettext that is consumed from the MSYS2 project stopped using a runtime prefix in https://github.com/msys2/MINGW-packages/pull/10461 Due to that change, we now use an unintialized gettext machinery that might get auto-initialized _using an unintended locale directory_: `C:\mingw64\share\locale`. Let's record the fact when the gettext initialization was skipped, and skip calling the gettext functions accordingly. This addresses CVE-2023-25815. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17t1300: demonstrate failure when renaming sections with long linesTaylor Blau1-0/+20
When renaming a configuration section which has an entry whose length exceeds the size of our buffer in config.c's implementation of `git_config_copy_or_rename_section_in_file()`, Git will incorrectly form a new configuration section with part of the data in the section being removed. In this instance, our first configuration file looks something like: [b] c = d <spaces> [a] e = f [a] g = h Here, we have two configuration values, "b.c", and "a.g". The value "[a] e = f" belongs to the configuration value "b.c", and does not form its own section. However, when renaming the section 'a' to 'xyz', Git will write back "[xyz]\ne = f", but "[xyz]" is still attached to the value of "b.c", which is why "e = f" on its own line becomes a new entry called "b.e". A slightly different example embeds the section being renamed within another section. Demonstrate this failure in a test in t1300, which we will fix in the following commit. Co-authored-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Helped-by: Jeff King <peff@peff.net> Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2023-04-17apply --reject: overwrite existing `.rej` symlink if it existsJohannes Schindelin2-2/+27
The `git apply --reject` is expected to write out `.rej` files in case one or more hunks fail to apply cleanly. Historically, the command overwrites any existing `.rej` files. The idea being that apply/reject/edit cycles are relatively common, and the generated `.rej` files are not considered precious. But the command does not overwrite existing `.rej` symbolic links, and instead follows them. This is unsafe because the same patch could potentially create such a symbolic link and point at arbitrary paths outside the current worktree, and `git apply` would write the contents of the `.rej` file into that location. Therefore, let's make sure that any existing `.rej` file or symbolic link is removed before writing it. Reported-by: RyotaK <ryotak.mail@gmail.com> Helped-by: Taylor Blau <me@ttaylorr.com> Helped-by: Junio C Hamano <gitster@pobox.com> Helped-by: Linus Torvalds <torvalds@linuxfoundation.org> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17Merge branch 'js/gettext-poison-fixes'Johannes Schindelin5-9/+9
The `maint-2.30` branch accumulated quite a few fixes over the past two years. Most of those fixes were originally based on newer versions, and while the patches cherry-picked cleanly, we weren't diligent enough to pay attention to the CI builds and the GETTEXT_POISON job regressed. This topic branch fixes that. * js/gettext-poison-fixes t0033: GETTEXT_POISON fix t0003: GETTEXT_POISON fix, part 1 t0003: GETTEXT_POISON fix, conclusion t5619: GETTEXT_POISON fix t5604: GETTEXT_POISON fix, part 1 t5604: GETTEXT_POISON fix, conclusion
2023-04-17Merge branch 'ds/github-actions-use-newer-ubuntu'Junio C Hamano1-1/+1
Update the version of Ubuntu used for GitHub Actions CI from 18.04 to 22.04. * ds/github-actions-use-newer-ubuntu: ci: update 'static-analysis' to Ubuntu 22.04
2023-04-17ci: update 'static-analysis' to Ubuntu 22.04Derrick Stolee1-1/+1
GitHub Actions scheduled a brownout of Ubuntu 18.04, which canceled all runs of the 'static-analysis' job in our CI runs. Update to 22.04 to avoid this as the brownout later turns into a complete deprecation. The use of 18.04 was set in d051ed77ee6 (.github/workflows/main.yml: run static-analysis on bionic, 2021-02-08) due to the lack of Coccinelle being available on 20.04 (which continues today). Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-23Makefile: force -O0 when compiling with SANITIZE=leakJeff King1-0/+1
Cherry pick commit d3775de0 (Makefile: force -O0 when compiling with SANITIZE=leak, 2022-10-18), as otherwise the leak checker at GitHub Actions CI seems to fail with a false positive. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-03-22Merge branch 'backport/jk/range-diff-fixes'Junio C Hamano1-16/+13
"git range-diff" code clean-up. Needed to pacify modern GCC versions. * jk/range-diff-fixes: range-diff: use ssize_t for parsed "len" in read_patches() range-diff: handle unterminated lines in read_patches() range-diff: drop useless "offset" variable from read_patches()
2023-03-22Merge branch 'backport/jk/curl-avoid-deprecated-api' into maint-2.30Junio C Hamano5-50/+81
Deal with a few deprecation warning from cURL library. * jk/curl-avoid-deprecated-api: http: support CURLOPT_PROTOCOLS_STR http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT
2023-03-22Merge branch 'backport/jx/ci-ubuntu-fix' into maint-2.30Junio C Hamano3-16/+17
Adjust the GitHub CI to newer ubuntu release. * jx/ci-ubuntu-fix: github-actions: run gcc-8 on ubuntu-20.04 image ci: install python on ubuntu ci: use the same version of p4 on both Linux and macOS ci: remove the pipe after "p4 -V" to catch errors
2023-03-22Merge branch 'backport/jc/http-clear-finished-pointer' into maint-2.30Junio C Hamano1-0/+26
Meant to go with js/ci-gcc-12-fixes. source: <xmqq7d68ytj8.fsf_-_@gitster.g> * jc/http-clear-finished-pointer: http.c: clear the 'finished' member once we are done with it
2023-03-22Merge branch 'backport/js/ci-gcc-12-fixes'Junio C Hamano3-1/+11
Fixes real problems noticed by gcc 12 and works around false positives. * js/ci-gcc-12-fixes: nedmalloc: avoid new compile error compat/win32/syslog: fix use-after-realloc
2023-03-22http.c: clear the 'finished' member once we are done with itJunio C Hamano1-0/+26
In http.c, the run_active_slot() function allows the given "slot" to make progress by calling step_active_slots() in a loop repeatedly, and the loop is not left until the request held in the slot completes. Ages ago, we used to use the slot->in_use member to get out of the loop, which misbehaved when the request in "slot" completes (at which time, the result of the request is copied away from the slot, and the in_use member is cleared, making the slot ready to be reused), and the "slot" gets reused to service a different request (at which time, the "slot" becomes in_use again, even though it is for a different request). The loop terminating condition mistakenly thought that the original request has yet to be completed. Today's code, after baa7b67d (HTTP slot reuse fixes, 2006-03-10) fixed this issue, uses a separate "slot->finished" member that is set in run_active_slot() to point to an on-stack variable, and the code that completes the request in finish_active_slot() clears the on-stack variable via the pointer to signal that the particular request held by the slot has completed. It also clears the in_use member (as before that fix), so that the slot itself can safely be reused for an unrelated request. One thing that is not quite clean in this arrangement is that, unless the slot gets reused, at which point the finished member is reset to NULL, the member keeps the value of &finished, which becomes a dangling pointer into the stack when run_active_slot() returns. Clear the finished member before the control leaves the function, which has a side effect of unconfusing compilers like recent GCC 12 that is over-eager to warn against such an assignment. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-22clone.c: avoid "exceeds maximum object size" error with GCC v12.xJohannes Schindelin1-0/+9
Technically, the pointer difference `end - start` _could_ be negative, and when cast to an (unsigned) `size_t` that would cause problems. In this instance, the symptom is: dir.c: In function 'git_url_basename': dir.c:3087:13: error: 'memchr' specified bound [9223372036854775808, 0] exceeds maximum object size 9223372036854775807 [-Werror=stringop-overread] CC ewah/bitmap.o 3087 | if (memchr(start, '/', end - start) == NULL | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ While it is a bit far-fetched to think that `end` (which is defined as `repo + strlen(repo)`) and `start` (which starts at `repo` and never steps beyond the NUL terminator) could result in such a negative difference, GCC has no way of knowing that. See also https://gcc.gnu.org/bugzilla//show_bug.cgi?id=85783. Let's just add a safety check, primarily for GCC's benefit. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-12t5604: GETTEXT_POISON fix, conclusionJohannes Schindelin1-1/+1
In fade728df122 (apply: fix writing behind newly created symbolic links, 2023-02-02), we backported a patch onto v2.30.* that was originally based on a much newer version. The v2.30.* release train still has the GETTEXT_POISON CI job, though, and hence needs `test_i18n*` in its tests. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-03-12t5604: GETTEXT_POISON fix, part 1Johannes Schindelin1-1/+1
In bffc762f87ae (dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS, 2023-01-24), we backported a patch onto v2.30.* that was originally based on a much newer version. The v2.30.* release train still has the GETTEXT_POISON CI job, though, and hence needs `test_i18n*` in its tests. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-03-12t5619: GETTEXT_POISON fixJohannes Schindelin1-1/+1
In cf8f6ce02a13 (clone: delay picking a transport until after get_repo_path(), 2023-01-24), we backported a patch onto v2.30.* that was originally based on a much newer version. The v2.30.* release train still has the GETTEXT_POISON CI job, though, and hence needs `test_i18n*` in its tests. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-03-12range-diff: use ssize_t for parsed "len" in read_patches()Jeff King1-1/+1
As we iterate through the buffer containing git-log output, parsing lines, we use an "int" to store the size of an individual line. This should be a size_t, as we have no guarantee that there is not a malicious 2GB+ commit-message line in the output. Overflowing this integer probably doesn't do anything _too_ terrible. We are not using the value to size a buffer, so the worst case is probably an out-of-bounds read from before the array. But it's easy enough to fix. Note that we have to use ssize_t here, since we also store the length result from parse_git_diff_header(), which may return a negative value for error. That function actually returns an int itself, which has a similar overflow problem, but I'll leave that for another day. Much of the apply.c code uses ints and should be converted as a whole; in the meantime, a negative return from parse_git_diff_header() will be interpreted as an error, and we'll bail (so we can't handle such a case, but given that it's likely to be malicious anyway, the important thing is we don't have any memory errors). Signed-off-by: Jeff King <peff@peff.net> Acked-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-12t0003: GETTEXT_POISON fix, conclusionJohannes Schindelin1-1/+1
In 3c50032ff528 (attr: ignore overly large gitattributes files, 2022-12-01), we backported a patch onto v2.30.* that was originally based on a much newer version. The v2.30.* release train still has the GETTEXT_POISON CI job, though, and hence needs `test_i18n*` in its tests. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-03-12range-diff: handle unterminated lines in read_patches()Jeff King1-14/+11
When parsing our buffer of output from git-log, we have a find_end_of_line() helper that finds the next newline, and gives us the number of bytes to move past it, or the size of the whole remaining buffer if there is no newline. But trying to handle both those cases leads to some oddities: - we try to overwrite the newline with NUL in the caller, by writing over line[len-1]. This is at best redundant, since the helper will already have done so if it saw a newline. But if it didn't see a newline, it's actively wrong; we'll overwrite the byte at the end of the (unterminated) line. We could solve this just dropping the extra NUL assignment in the caller and just letting the helper do the right thing. But... - if we see a "diff --git" line, we'll restore the newline on top of the NUL byte, so we can pass the string to parse_git_diff_header(). But if there was no newline in the first place, we can't do this. There's no place to put it (the current code writes a newline over whatever byte we obliterated earlier). The best we can do is feed the complete remainder of the buffer to the function (which is, in fact, a string, by virtue of being a strbuf). To solve this, the caller needs to know whether we actually found a newline or not. We could modify find_end_of_line() to return that information, but we can further observe that it has only one caller. So let's just inline it in that caller. Nobody seems to have noticed this case, probably because git-log would never produce input that doesn't end with a newline. Arguably we could just return an error as soon as we see that the output does not end in a newline. But the code to do so actually ends up _longer_, mostly because of the cleanup we have to do in handling the error. Signed-off-by: Jeff King <peff@peff.net> Acked-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-12t0003: GETTEXT_POISON fix, part 1Johannes Schindelin1-4/+4
In dfa6b32b5e59 (attr: ignore attribute lines exceeding 2048 bytes, 2022-12-01), we backported a patch onto v2.30.* that was originally based on a much newer version. The v2.30.* release train still has the GETTEXT_POISON CI job, though, and hence needs `test_i18n*` in its tests. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-03-12t0033: GETTEXT_POISON fixJohannes Schindelin1-1/+1
In e47363e5a8bd (t0033: add tests for safe.directory, 2022-04-13), we backported a patch onto v2.30.* that was originally based on a much newer version. The v2.30.* release train still has the GETTEXT_POISON CI job, though, and hence needs `test_i18n*` in its tests. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-03-12http: support CURLOPT_PROTOCOLS_STRJeff King1-13/+44
The CURLOPT_PROTOCOLS (and matching CURLOPT_REDIR_PROTOCOLS) flag was deprecated in curl 7.85.0, and using it generate compiler warnings as of curl 7.87.0. The path forward is to use CURLOPT_PROTOCOLS_STR, but we can't just do so unilaterally, as it was only introduced less than a year ago in 7.85.0. Until that version becomes ubiquitous, we have to either disable the deprecation warning or conditionally use the "STR" variant on newer versions of libcurl. This patch switches to the new variant, which is nice for two reasons: - we don't have to worry that silencing curl's deprecation warnings might cause us to miss other more useful ones - we'd eventually want to move to the new variant anyway, so this gets us set up (albeit with some extra ugly boilerplate for the conditional) There are a lot of ways to split up the two cases. One way would be to abstract the storage type (strbuf versus a long), how to append (strbuf_addstr vs bitwise OR), how to initialize, which CURLOPT to use, and so on. But the resulting code looks pretty magical: GIT_CURL_PROTOCOL_TYPE allowed = GIT_CURL_PROTOCOL_TYPE_INIT; if (...http is allowed...) GIT_CURL_PROTOCOL_APPEND(&allowed, "http", CURLOPT_HTTP); and you end up with more "#define GIT_CURL_PROTOCOL_TYPE" macros than actual code. On the other end of the spectrum, we could just implement two separate functions, one that handles a string list and one that handles bits. But then we end up repeating our list of protocols (http, https, ftp, ftp). This patch takes the middle ground. The run-time code is always there to handle both types, and we just choose which one to feed to curl. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-03-12ci: install python on ubuntuJiang Xin2-4/+4
Python is missing from the default ubuntu-22.04 runner image, which prevents git-p4 from working. To install python on ubuntu, we need to provide the correct package names: * On Ubuntu 18.04 (bionic), "/usr/bin/python2" is provided by the "python" package, and "/usr/bin/python3" is provided by the "python3" package. * On Ubuntu 20.04 (focal) and above, "/usr/bin/python2" is provided by the "python2" package which has a different name from bionic, and "/usr/bin/python3" is provided by "python3". Since the "ubuntu-latest" runner image has a higher version, its safe to use "python2" or "python3" package name. Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-12range-diff: drop useless "offset" variable from read_patches()Jeff King1-2/+2
The "offset" variable was was introduced in 44b67cb62b (range-diff: split lines manually, 2019-07-11), but it has never done anything useful. We use it to count up the number of bytes we've consumed, but we never look at the result. It was probably copied accidentally from an almost-identical loop in apply.c:find_header() (and the point of that commit was to make use of the parse_git_diff_header() function which underlies both). Because the variable was set but not used, most compilers didn't seem to notice, but the upcoming clang-14 does complain about it, via its -Wunused-but-set-variable warning. Signed-off-by: Jeff King <peff@peff.net> Acked-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-12http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTIONJeff King5-36/+36
The IOCTLFUNCTION option has been deprecated, and generates a compiler warning in recent versions of curl. We can switch to using SEEKFUNCTION instead. It was added in 2008 via curl 7.18.0; our INSTALL file already indicates we require at least curl 7.19.4. But there's one catch: curl says we should use CURL_SEEKFUNC_{OK,FAIL}, and those didn't arrive until 7.19.5. One workaround would be to use a bare 0/1 here (or define our own macros). But let's just bump the minimum required version to 7.19.5. That version is only a minor version bump from our existing requirement, and is only a 2 month time bump for versions that are almost 13 years old. So it's not likely that anybody cares about the distinction. Switching means we have to rewrite the ioctl functions into seek functions. In some ways they are simpler (seeking is the only operation), but in some ways more complex (the ioctl allowed only a full rewind, but now we can seek to arbitrary offsets). Curl will only ever use SEEK_SET (per their documentation), so I didn't bother implementing anything else, since it would naturally be completely untested. This seems unlikely to change, but I added an assertion just in case. Likewise, I doubt curl will ever try to seek outside of the buffer sizes we've told it, but I erred on the defensive side here, rather than do an out-of-bounds read. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-03-12http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUTJeff King1-1/+1
The two options do exactly the same thing, but the latter has been deprecated and in recent versions of curl may produce a compiler warning. Since the UPLOAD form is available everywhere (it was introduced in the year 2000 by curl 7.1), we can just switch to it. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-03-12nedmalloc: avoid new compile errorJohannes Schindelin1-1/+0
GCC v12.x complains thusly: compat/nedmalloc/nedmalloc.c: In function 'DestroyCaches': compat/nedmalloc/nedmalloc.c:326:12: error: the comparison will always evaluate as 'true' for the address of 'caches' will never be NULL [-Werror=address] 326 | if(p->caches) | ^ compat/nedmalloc/nedmalloc.c:196:22: note: 'caches' declared here 196 | threadcache *caches[THREADCACHEMAXCACHES]; | ^~~~~~ ... and it is correct, of course. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-12ci: use the same version of p4 on both Linux and macOSJiang Xin2-9/+10
There would be a segmentation fault when running p4 v16.2 on ubuntu 22.04 which is the latest version of ubuntu runner image for github actions. By checking each version from [1], p4d version 21.1 and above can work properly on ubuntu 22.04. But version 22.x will break some p4 test cases. So p4 version 21.x is exactly the version we can use. With this update, the versions of p4 for Linux and macOS happen to be the same. So we can add the version number directly into the "P4WHENCE" variable, and reuse it in p4 installation for macOS. By removing the "LINUX_P4_VERSION" variable from "ci/lib.sh", the comment left above has nothing to do with p4, but still applies to git-lfs. Since we have a fixed version of git-lfs installed on Linux, we may have a different version on macOS. [1]: https://cdist2.perforce.com/perforce/ Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de> Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-12ci: remove the pipe after "p4 -V" to catch errorsJiang Xin1-2/+2
When installing p4 as a dependency, we used to pipe output of "p4 -V" and "p4d -V" to validate the installation and output a condensed version information. But this would hide potential errors of p4 and would stop with an empty output. E.g.: p4d version 16.2 running on ubuntu 22.04 causes sigfaults, even before it produces any output. By removing the pipe after "p4 -V" and "p4d -V", we may get a verbose output, and stop immediately on errors because we have "set -e" in "ci/lib.sh". Since we won't look at these trace logs unless something fails, just including the raw output seems most sensible. Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de> Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-12github-actions: run gcc-8 on ubuntu-20.04 imageJiang Xin1-1/+1
GitHub starts to upgrade its runner image "ubuntu-latest" from version "ubuntu-20.04" to version "ubuntu-22.04". It will fail to find and install "gcc-8" package on the new runner image. Change the runner image of the `linux-gcc` job from "ubuntu-latest" to "ubuntu-20.04" in order to install "gcc-8" as a dependency. Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de> Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-12compat/win32/syslog: fix use-after-reallocJohannes Schindelin1-0/+2
Git for Windows' SDK recently upgraded to GCC v12.x which points out that the `pos` variable might be used even after the corresponding memory was `realloc()`ed and therefore potentially no longer valid. Since a subset of this SDK is used in Git's CI/PR builds, we need to fix this to continue to be able to benefit from the CI/PR runs. Note: This bug has been with us since 2a6b149c64f6 (mingw: avoid using strbuf in syslog, 2011-10-06), and while it looks tempting to replace the hand-rolled string manipulation with a `strbuf`-based one, that commit's message explains why we cannot do that: The `syslog()` function is called as part of the function in `daemon.c` which is set as the `die()` routine, and since `strbuf_grow()` can call that function if it runs out of memory, this would cause a nasty infinite loop that we do not want to re-introduce. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-23t5559: make SSL/TLS the defaultJeff King1-0/+1
The point of t5559 is run the regular t5551 tests with HTTP/2. But it does so with the "h2c" protocol, which uses cleartext upgrades from HTTP/1.1 to HTTP/2 (rather than learning about HTTP/2 support during the TLS negotiation). This has a few problems: - it's not very indicative of the real world. In practice, most servers that support HTTP/2 will also support TLS. - support for upgrading does not seem as robust. In particular, we've run into bugs in some versions of Apache's mod_http2 that trigger only with the upgrade mode. See: https://lore.kernel.org/git/Y8ztIqYgVCPILJlO@coredump.intra.peff.net/ So the upside is that this change makes our HTTP/2 tests more robust and more realistic. The downside is that if we can't set up SSL for any reason, we'll skip the tests (even though you _might_ have been able to run the HTTP/2 tests the old way). We could probably have a conditional fallback, but it would be complicated for little gain, and it's not even clear it would help (i.e., would any test environment even have HTTP/2 but not SSL support?). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-23t5559: fix test failures with LIB_HTTPD_SSLJeff King1-5/+18
One test needs to be tweaked in order for t5559 to pass with SSL/TLS set up. When we make our initial clone, we check that the curl trace of requests is what we expected. But we need to fix two things: - along with ignoring "data" lines from the trace, we need to ignore "SSL data" lines - when TLS is used, the server is able to tell the client (via ALPN) that it supports HTTP/2 before the first HTTP request is made. So rather than request an upgrade using an HTTP header, it can just speak HTTP/2 immediately With this patch, running: LIB_HTTPD_SSL=1 ./t5559-http-fetch-smart-http2.sh works, whereas it did not before. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-23t/lib-httpd: enable HTTP/2 "h2" protocol, not just h2cJeff King1-1/+1
Commit 73c49a4474 (t: run t5551 tests with both HTTP and HTTP/2, 2022-11-11) added Apache config to enable HTTP/2. However, it only enabled the "h2c" protocol, which allows cleartext HTTP/2 (generally based on an upgrade header during an HTTP/1.1 request). This is what t5559 is generally testing, since by default we don't set up SSL/TLS. However, it should be possible to run t5559 with LIB_HTTPD_SSL set. In that case, Apache will advertise support for HTTP/2 via ALPN during the TLS handshake. But we need to tell it support "h2" (the non-cleartext version) to do so. Without that, then curl does not even try to do the HTTP/1.1 upgrade (presumably because after seeing that we did TLS but didn't get the ALPN indicator, it assumes it would be fruitless). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-23t/lib-httpd: respect $HTTPD_PROTO in expect_askpass()Jeff King1-3/+3
When the HTTP tests are run with LIB_HTTPD_SSL in the environment, then we access the test server as https://. This causes expect_askpass to complain, because it tries to blindly match "http://" in the prompt shown to the user. We can adjust this to use $HTTPD_PROTO, which is set during the setup phase. Note that this is enough for t5551 and t5559 to pass when run with https, but there are similar problems in other scripts that will need to be fixed before the whole suite can run with LIB_HTTPD_SSL. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-23t5551: drop curl trace lines without headersJeff King1-1/+1
We pick apart a curl trace, looking for "=> Send header:" and so on, and matching against an expected set of requests and responses. We remove "== Info" lines entirely. However, our parser is fooled when running the test with LIB_HTTPD_SSL on Ubuntu 20.04 (as found in our linux-gcc CI job), as curl hands us an "Info" buffer with a newline, and we get: == Info: successfully set certificate verify locations: == Info: CAfile: /etc/ssl/certs/ca-certificates.crt CApath: /etc/ssl/certs => Send SSL data[...] which results in the "CApath" line ending up in the cleaned-up output, causing the test to fail. Arguably the tracing code should detect this and put it on two separate "== Info" lines. But this is actually a curl bug, fixed by their 80d73bcca (tls: provide the CApath verbose log on its own line, 2020-08-18). It's simpler to just work around it here. Since we are using GIT_TRACE_CURL, every line should just start with one of "<=", "==", or "=>", and we can throw away anything else. In fact, we can just replace the pattern for deleting "*" lines. Those were from the old GIT_CURL_VERBOSE output, but we switched over in 14e24114d9 (t5551-http-fetch-smart.sh: use the GIT_TRACE_CURL environment var, 2016-09-05). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-23t5551: handle v2 protocol in cookie testJeff King1-8/+11
After making a request, we check that it stored the expected cookies. This depends on the protocol version, because the cookies we store depend on the exact requests we made (and for ls-remote, v2 will always hit /git-upload-pack to get the refs, whereas v0 is happy with the initial ref advertisement). As a result, hardly anybody runs this test, as you'd have to manually set GIT_TEST_PROTOCOL_VERSION=0 to do so. Let's teach it to handle both protocol versions. One way to do this would be to make the expectation conditional on the protocol used. But there's a simpler solution. The reason that v0 doesn't hit /git-upload-pack is that ls-remote doesn't fetch any objects. If we instead do a fetch (making sure there's an actual object to grab), then both v0 and v2 will hit the same endpoints and set the same cookies. Note that we do have to clean up our new tag here; otherwise it confuses the later "clone 2,000 tags" test. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-23t5551: simplify expected cookie fileJeff King1-3/+2
After making an HTTP request that should store cookies, we check that the expected values are in the cookie file. We don't want to look at the whole file, because it has noisy comments at the top that we shouldn't depend on. But we strip out the interesting bits using "tail -3", which is brittle. It requires us to put an extra blank line in our expected output, and it would fail to notice any reordering or extra content in the cookie file. Instead, let's just grep for non-blank lines that are not comments, which more directly describes what we're interested in. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-23t5551: handle v2 protocol in upload-pack service testJeff King1-13/+3
We perform a clone and a fetch, and then check that we saw the expected requests in Apache's access log. In the v2 protocol, there will be one extra request to /git-upload-pack for each operation (since the initial /info/refs probe is just used to upgrade the protocol). As a result, this test is a noop unless the use of the v0 protocol is forced. Which means that hardly anybody runs it, since you have to do so manually. Let's update it to handle v2 and run it always. We could do this by just conditionally adding in the extra POST lines. But if we look at the origin of the test in 7da4e2280c (test smart http fetch and push, 2009-10-30), the point is really just to make sure that the smart git-upload-pack service was used at all. So rather than counting up the individual requests, let's just make sure we saw each of the expected types. This is a bit looser, but makes maintenance easier. Since we're now matching with grep, we can also loosen the HTTP/1.1 match, which allows this test to pass when run with HTTP/2 via t5559. That lets: GIT_TEST_PROTOCOL_VERSION=0 ./t5559-http-fetch-smart-http2.sh run to completion, which previously failed (and of course it works if you use v2, as well). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-23t5551: handle v2 protocol when checking curl traceJeff King1-11/+25
After cloning an http repository, we check the curl trace to make sure the expected requests were made. But since the expected trace was never updated to handle v2, it is only run when you ask the test suite to run in v0 mode (which hardly anybody does). Let's update it to handle both protocols. This isn't too hard since v2 just sends an extra header and an extra request. So we can just annotate those extra lines and strip them out for v0 (and drop the annotations for v2). I didn't bother handling v1 here, as it's not really of practical interest (it would drop the extra v2 request, but still have the "git-protocol" lines). There's a similar tweak needed at the end. Since we check the "accept-encoding" value loosely, we grep for it rather than finding it in the verbatim trace. This grep insists that there are exactly 2 matches, but of course in v2 with the extra request there are 3. We could tweak the number, but it's simpler still to just check that we saw at least one match. The verbatim check already confirmed how many instances of the header we have; we're really just checking here that "gzip" is in the value (it's possible, of course, that the headers could have different values, but that seems like an unlikely bug). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-23t5551: stop forcing clone to run with v0 protocolJeff King1-1/+1
In the "clone http repository" test, we check the curl trace to make sure the expected requests were made. This whole script was marked to handle only the v0 protocol in d790ee1707 (tests: fix protocol version for overspecifications, 2019-02-25). That makes sense, since v2 requires an extra request, so tests as specific as this would fail unless modified. Later, in preparation for v2 becoming the default, this was tweaked by 8a1b0978ab (test: request GIT_TEST_PROTOCOL_VERSION=0 when appropriate, 2019-12-23). There we run the trace check only if the user has explicitly asked to test protocol version 0. But it also forced the clone itself to run with the v0 protocol. This makes the check for "can we expect a v0 trace" silly; it will always be v0. But much worse, it means that the clone we are testing is not like the one that normal users would run. They would use the defaults, which are now v2. And since this is supposed to be a basic check of clone-over-http, we should do the same. Let's fix this by dropping the extra v0 override. The test still passes because the trace checking only kicks in if we asked to use v0 explicitly (this is the same as before; even though we were running a v0 clone, unless you specifically set GIT_TEST_PROTOCOL_VERSION=0, the trace check was always skipped). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-23t5551: handle HTTP/2 when checking curl traceJeff King1-3/+11
We check that the curl trace of a clone has the lines we expect, but this won't work when we run the test under t5559, because a few details are different under HTTP/2 (but nobody noticed because it only happens when you manually set GIT_TEST_PROTOCOL_VERSION to "0"). We can handle both HTTP protocols with a few tweaks: - we'll drop the HTTP "101 Switching Protocols" response, as well as various protocol upgrade headers. These details aren't interesting to us. We just want to make sure the correct protocol was used (and we do in the main request/response lines). - successful HTTP/2 responses just say "200" and not "200 OK"; we can normalize these - replace HTTP/1.1 with a variable in the request/response lines. We can use the existing $HTTP_PROTO for this, as it's already set to "HTTP/2" when appropriate. We do need to tweak the fallback value to "HTTP/1.1" to match what curl will write (prior to this patch, the fallback value didn't matter at all; we only checked if it was the literal string "HTTP/2"). Note that several lines still expect HTTP/1.1 unconditionally. The first request does so because the client requests an upgrade during the request. The POST request and response do so because you can't do an upgrade if there is a request body. (This will all be different if we trigger HTTP/2 via ALPN, but the tests aren't yet capable of that). This is enough to let: GIT_TEST_PROTOCOL_VERSION=0 ./t5559-http-fetch-smart-http2.sh pass the "clone http repository" test (but there are some other failures later on). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-23t5551: lower-case headers in expected curl traceJeff King1-25/+30
There's a test in t5551 which checks the curl trace (after simplifying it a bit). It doesn't work with HTTP/2, because in that case curl outputs all of the headers in lower-case. Even though this test is run with HTTP/2 by t5559, nobody has noticed because checking the trace only happens if GIT_TEST_PROTOCOL_VERSION is manually set to "0". Let's fix this by lower-casing all of the header names in the trace, and then checking for those in our expected code (this is easier than making HTTP/2 traces look like HTTP/1.1, since HTTP/1.1 uses title-casing). Sadly, we can't quite do this in our existing sed script. This works if you have GNU sed: s/^\\([><]\\) \\([A-Za-z0-9-]*:\\)/\1 \L\2\E/ but \L is a GNU-ism, and I don't think there's a portable solution. We could just "tr A-Z a-z" on the way in, of course, but that makes the non-header parts harder to read (e.g., lowercase "post" requests). But to paraphrase Baron Munchausen, I have learned from experience that a modicum of Perl can be most efficacious. Note that this doesn't quite get the test passing with t5559; there are more fixes needed on top. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-23t5551: drop redundant grep for Accept-LanguageJeff King1-4/+1
Commit b0c4adcdd7 (remote-curl: send Accept-Language header to server, 2022-07-11) added tests to make sure the header is sent via HTTP. However, it checks in two places: 1. In the expected trace output, we check verbatim for the header and its value. 2. Afterwards, we grep for the header again in the trace file. This (2) is probably cargo-culted from the earlier grep for Accept-Encoding. It is needed for the encoding because we smudge the value of that header when doing the verbatim check; see 1a53e692af (remote-curl: accept all encodings supported by curl, 2018-05-22). But we don't do so for the language header, so any problem that the "grep" would catch in (2) would already have been caught by (1). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-23t5541: simplify and move "no empty path components" testJeff King2-18/+9
Commit 9ee6bcd398 (t5541-http-push: add test for URLs with trailing slash, 2010-04-08) added a test that clones a URL with a trailing slash, and confirms that we don't send a doubled slash (like "$url//info/refs") to the server. But this test makes no sense in t5541, which is about pushing. It should have been added in t5551. Let's move it there. But putting it at the end is tricky, since it checks the entire contents of the Apache access log. We could get around this by clearing the log before our test. But there's an even simpler solution: just make sure no doubled slashes appear in the log (fortunately, "http://" does not appear in the log itself). As a bonus, this also lets us drop the check for the v0 protocol (which is otherwise necessary since v2 makes multiple requests, and check_access_log insists on exactly matching the number of requests, even though we don't care about that here). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-23t5541: stop marking "used receive-pack service" test as v0 onlyJeff King1-6/+1
We have a test which checks to see if a request to git-receive-pack was made. Originally, it was checking the entire set of requests made in the script so far, including clones, and thus it would break when run with the v2 protocol (since that implies an extra request for fetches). Since the previous commit, though, we are only checking the requests made by a single push. And since there is no v2 push protocol, the test now passes no matter what's in GIT_TEST_PROTOCOL_VERSION. We can just run it all the time. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-23t5541: run "used receive-pack service" test earlierJeff King1-26/+18
There's a test in t5541 that confirms that "git push" makes two requests (a GET to /info/refs, and a POST to /git-receive-pack). However, it's a noop unless GIT_TEST_PROTOCOL_VERSION is set to "0", due to 8a1b0978ab (test: request GIT_TEST_PROTOCOL_VERSION=0 when appropriate, 2019-12-23). This means that almost nobody runs it. And indeed, it has been broken since b0c4adcdd7 (remote-curl: send Accept-Language header to server, 2022-07-11). But the fault is not in that commit, but in how brittle the test is. It runs after several operations have been performed, which means that it expects to see the complete set of requests made so far in the script. Commit b0c4adcdd7 added a new test, which means that the "used receive-pack service" test must be updated, too. Let's fix this by making the test less brittle. We'll move it higher in the script, right after the first push has completed. And we'll clear the access log right before doing the push, so we'll see only the requests from that command. This is technically testing less, in that we won't check that all of those other requests also correctly used smart http. But there's no particular reason think that if the first one did, the others wouldn't. After this patch, running: GIT_TEST_PROTOCOL_VERSION=0 ./t5541-http-push-smart.sh passes, whereas it did not before. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-14Prepare for 2.39.3 just in caseJunio C Hamano3-2/+60
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-14Merge branch 'sk/remove-duplicate-includes' into maint-2.39Junio C Hamano3-3/+0
Code clean-up. * sk/remove-duplicate-includes: git: remove duplicate includes
2023-02-14Merge branch 'rs/clarify-error-in-write-loose-object' into maint-2.39Junio C Hamano1-9/+2
Code clean-up. * rs/clarify-error-in-write-loose-object: object-file: inline write_buffer()
2023-02-14Merge branch 'rs/reflog-expiry-cleanup' into maint-2.39Junio C Hamano1-1/+3
Code clean-up. * rs/reflog-expiry-cleanup: reflog: clear leftovers in reflog_expiry_cleanup()
2023-02-14Merge branch 'rs/clear-commit-marks-cleanup' into maint-2.39Junio C Hamano1-2/+4
Code clean-up. * rs/clear-commit-marks-cleanup: commit: skip already cleared parents in clear_commit_marks_1()
2023-02-14Merge branch 'rs/am-parse-options-cleanup' into maint-2.39Junio C Hamano1-1/+11
Code clean-up. * rs/am-parse-options-cleanup: am: don't pass strvec to apply_parse_options()
2023-02-14Merge branch 'jk/server-supports-v2-cleanup' into maint-2.39Junio C Hamano3-16/+20
Code clean-up. * jk/server-supports-v2-cleanup: server_supports_v2(): use a separate function for die_on_error
2023-02-14Merge branch 'jk/unused-post-2.39' into maint-2.39Junio C Hamano30-108/+90
Code clean-up around unused function parameters. * jk/unused-post-2.39: userdiff: mark unused parameter in internal callback list-objects-filter: mark unused parameters in virtual functions diff: mark unused parameters in callbacks xdiff: mark unused parameter in xdl_call_hunk_func() xdiff: drop unused parameter in def_ff() ws: drop unused parameter from ws_blank_line() list-objects: drop process_gitlink() function blob: drop unused parts of parse_blob_buffer() ls-refs: use repository parameter to iterate refs
2023-02-14Merge branch 'rj/branch-copy-and-rename' into maint-2.39Junio C Hamano2-3/+13
Fix a pair of bugs in 'git branch'. * rj/branch-copy-and-rename: branch: force-copy a branch to itself via @{-1} is a no-op
2023-02-14Merge branch 'rs/t3920-crlf-eating-grep-fix' into maint-2.39Junio C Hamano1-1/+1
Test fix. * rs/t3920-crlf-eating-grep-fix: t3920: support CR-eating grep
2023-02-14Merge branch 'js/t3920-shell-and-or-fix' into maint-2.39Junio C Hamano1-1/+1
Test fix. * js/t3920-shell-and-or-fix: t3920: don't ignore errors of more than one command with `|| true`
2023-02-14Merge branch 'ab/t4023-avoid-losing-exit-status-of-diff' into maint-2.39Junio C Hamano1-6/+6
Test fix. * ab/t4023-avoid-losing-exit-status-of-diff: t4023: fix ignored exit codes of git
2023-02-14Merge branch 'ab/t7600-avoid-losing-exit-status-of-git' into maint-2.39Junio C Hamano1-1/+1
Test fix. * ab/t7600-avoid-losing-exit-status-of-git: t7600: don't ignore "rev-parse" exit code in helper
2023-02-14Merge branch 'ab/t5314-avoid-losing-exit-status' into maint-2.39Junio C Hamano1-7/+10
Test fix. * ab/t5314-avoid-losing-exit-status: t5314: check exit code of "git"
2023-02-14Merge branch 'rs/plug-pattern-list-leak-in-lof' into maint-2.39Junio C Hamano1-0/+1
Leak fix. * rs/plug-pattern-list-leak-in-lof: list-objects-filter: plug pattern_list leak
2023-02-14Merge branch 'rs/t4205-do-not-exit-in-test-script' into maint-2.39Junio C Hamano1-1/+1
Test fix. * rs/t4205-do-not-exit-in-test-script: t4205: don't exit test script on failure
2023-02-14Merge branch 'jc/doc-checkout-b' into maint-2.39Junio C Hamano1-6/+8
Clarify how "checkout -b/-B" and "git branch [-f]" are similar but different in the documentation. * jc/doc-checkout-b: checkout: document -b/-B to highlight the differences from "git branch"
2023-02-14Merge branch 'jc/doc-branch-update-checked-out-branch' into maint-2.39Junio C Hamano1-0/+4
Document that "branch -f <branch>" disables only the safety to avoid recreating an existing branch. * jc/doc-branch-update-checked-out-branch: branch: document `-f` and linked worktree behaviour
2023-02-14Merge branch 'rs/ls-tree-path-expansion-fix' into maint-2.39Junio C Hamano2-8/+11
"git ls-tree --format='%(path) %(path)' $tree $path" showed the path three times, which has been corrected. * rs/ls-tree-path-expansion-fix: ls-tree: remove dead store and strbuf for quote_c_style() ls-tree: fix expansion of repeated %(path)
2023-02-14Merge branch 'pb/doc-orig-head' into maint-2.39Junio C Hamano5-4/+14
Document ORIG_HEAD a bit more. * pb/doc-orig-head: git-rebase.txt: add a note about 'ORIG_HEAD' being overwritten revisions.txt: be explicit about commands writing 'ORIG_HEAD' git-merge.txt: mention 'ORIG_HEAD' in the Description git-reset.txt: mention 'ORIG_HEAD' in the Description git-cherry-pick.txt: do not use 'ORIG_HEAD' in example
2023-02-14Merge branch 'es/hooks-and-local-env' into maint-2.39Junio C Hamano1-0/+12
Doc update for environment variables set when hooks are invoked. * es/hooks-and-local-env: githooks: discuss Git operations in foreign repositories
2023-02-14Merge branch 'ws/single-file-cone' into maint-2.39Junio C Hamano2-1/+17
The logic to see if we are using the "cone" mode by checking the sparsity patterns has been tightened to avoid mistaking a pattern that names a single file as specifying a cone. * ws/single-file-cone: dir: check for single file cone patterns
2023-02-14Merge branch 'jk/ext-diff-with-relative' into maint-2.39Junio C Hamano2-17/+42
"git diff --relative" did not mix well with "git diff --ext-diff", which has been corrected. * jk/ext-diff-with-relative: diff: drop "name" parameter from prepare_temp_file() diff: clean up external-diff argv setup diff: use filespec path to set up tempfiles for ext-diff
2023-02-14Merge branch 'ab/bundle-wo-args' into maint-2.39Junio C Hamano2-5/+11
Fix to a small regression in 2.38 days. * ab/bundle-wo-args: bundle <cmd>: have usage_msg_opt() note the missing "<file>" builtin/bundle.c: remove superfluous "newargc" variable bundle: don't segfault on "git bundle <subcmd>"
2023-02-14Merge branch 'ps/fsync-refs-fix' into maint-2.39Junio C Hamano1-1/+2
Fix the sequence to fsync $GIT_DIR/packed-refs file that forgot to flush its output to the disk.. * ps/fsync-refs-fix: refs: fix corruption by not correctly syncing packed-refs to disk
2023-02-14Merge branch 'lk/line-range-parsing-fix' into maint-2.39Junio C Hamano2-3/+26
When given a pattern that matches an empty string at the end of a line, the code to parse the "git diff" line-ranges fell into an infinite loop, which has been corrected. * lk/line-range-parsing-fix: line-range: fix infinite loop bug with '$' regex
2023-02-14Merge branch 'rs/use-enhanced-bre-on-macos' into maint-2.39Junio C Hamano4-0/+24
Newer regex library macOS stopped enabling GNU-like enhanced BRE, where '\(A\|B\)' works as alternation, unless explicitly asked with the REG_ENHANCED flag. "git grep" now can be compiled to do so, to retain the old behaviour. * rs/use-enhanced-bre-on-macos: use enhanced basic regular expressions on macOS
2023-02-14Merge branch 'jk/curl-avoid-deprecated-api' into maint-2.39Junio C Hamano0-0/+0
Deal with a few deprecation warning from cURL library. * jk/curl-avoid-deprecated-api: http: support CURLOPT_PROTOCOLS_STR http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT
2023-02-14Merge branch 'jk/avoid-redef-system-functions' into maint-2.39Junio C Hamano0-0/+0
The jk/avoid-redef-system-functions-2.30 topic pre-merged for more recent codebase. * jk/avoid-redef-system-functions:
2023-02-14Merge branch 'jk/avoid-redef-system-functions-2.30' into maint-2.39Junio C Hamano1-5/+12
Redefining system functions for a few functions did not follow our usual "implement git_foo() and #define foo(args) git_foo(args)" pattern, which has broken build for some folks. * jk/avoid-redef-system-functions-2.30: git-compat-util: undefine system names before redeclaring them git-compat-util: avoid redefining system function names
2023-02-14Merge branch 'tb/ci-concurrency' into maint-2.39Junio C Hamano3-2/+50
Avoid unnecessary builds in CI, with settings configured in ci-config. * tb/ci-concurrency: ci: avoid unnecessary builds
2023-02-14Merge branch 'cw/ci-whitespace' into maint-2.39Junio C Hamano1-11/+46
CI updates. We probably want a clean-up to move the long shell script embedded in yaml file into a separate file, but that can come later. * cw/ci-whitespace: ci (check-whitespace): move to actions/checkout@v3 ci (check-whitespace): add links to job output ci (check-whitespace): suggest fixes for errors
2023-02-14Merge branch 'js/ci-disable-cmake-by-default' into maint-2.39Junio C Hamano1-1/+1
Stop running win+VS build by default. * js/ci-disable-cmake-by-default: ci: only run win+VS build & tests in Git for Windows' fork
2023-02-06Git 2.39.2v2.39.2Johannes Schindelin3-2/+9
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06Sync with 2.38.4Johannes Schindelin23-53/+425
* maint-2.38: Git 2.38.4 Git 2.37.6 Git 2.36.5 Git 2.35.7 Git 2.34.7 http: support CURLOPT_PROTOCOLS_STR http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT Git 2.33.7 Git 2.32.6 Git 2.31.7 Git 2.30.8 apply: fix writing behind newly created symbolic links dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS clone: delay picking a transport until after get_repo_path() t5619: demonstrate clone_local() with ambiguous transport
2023-02-06Git 2.38.4v2.38.4Johannes Schindelin3-2/+9
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06Sync with 2.37.6Johannes Schindelin22-53/+418
* maint-2.37: Git 2.37.6 Git 2.36.5 Git 2.35.7 Git 2.34.7 http: support CURLOPT_PROTOCOLS_STR http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT Git 2.33.7 Git 2.32.6 Git 2.31.7 Git 2.30.8 apply: fix writing behind newly created symbolic links dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS clone: delay picking a transport until after get_repo_path() t5619: demonstrate clone_local() with ambiguous transport
2023-02-06Git 2.37.6v2.37.6Johannes Schindelin3-2/+9
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06Sync with 2.36.5Johannes Schindelin21-53/+411
* maint-2.36: Git 2.36.5 Git 2.35.7 Git 2.34.7 http: support CURLOPT_PROTOCOLS_STR http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT Git 2.33.7 Git 2.32.6 Git 2.31.7 Git 2.30.8 apply: fix writing behind newly created symbolic links dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS clone: delay picking a transport until after get_repo_path() t5619: demonstrate clone_local() with ambiguous transport
2023-02-06Git 2.36.5v2.36.5Johannes Schindelin3-2/+9
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06Sync with 2.35.7Johannes Schindelin20-53/+404
* maint-2.35: Git 2.35.7 Git 2.34.7 http: support CURLOPT_PROTOCOLS_STR http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT Git 2.33.7 Git 2.32.6 Git 2.31.7 Git 2.30.8 apply: fix writing behind newly created symbolic links dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS clone: delay picking a transport until after get_repo_path() t5619: demonstrate clone_local() with ambiguous transport
2023-02-06Git 2.35.7v2.35.7Johannes Schindelin3-2/+9
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06Sync with 2.34.7Johannes Schindelin19-53/+397
* maint-2.34: Git 2.34.7 http: support CURLOPT_PROTOCOLS_STR http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT Git 2.33.7 Git 2.32.6 Git 2.31.7 Git 2.30.8 apply: fix writing behind newly created symbolic links dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS clone: delay picking a transport until after get_repo_path() t5619: demonstrate clone_local() with ambiguous transport
2023-02-06Git 2.34.7v2.34.7Johannes Schindelin3-2/+9
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06Sync with 2.33.7Johannes Schindelin12-9/+309
* maint-2.33: Git 2.33.7 Git 2.32.6 Git 2.31.7 Git 2.30.8 apply: fix writing behind newly created symbolic links dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS clone: delay picking a transport until after get_repo_path() t5619: demonstrate clone_local() with ambiguous transport
2023-02-06Merge branch 'jk/curl-avoid-deprecated-api'Junio C Hamano6-44/+81
Deal with a few deprecation warning from cURL library. * jk/curl-avoid-deprecated-api: http: support CURLOPT_PROTOCOLS_STR http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT
2023-02-06http: support CURLOPT_PROTOCOLS_STRJeff King2-13/+54
The CURLOPT_PROTOCOLS (and matching CURLOPT_REDIR_PROTOCOLS) flag was deprecated in curl 7.85.0, and using it generate compiler warnings as of curl 7.87.0. The path forward is to use CURLOPT_PROTOCOLS_STR, but we can't just do so unilaterally, as it was only introduced less than a year ago in 7.85.0. Until that version becomes ubiquitous, we have to either disable the deprecation warning or conditionally use the "STR" variant on newer versions of libcurl. This patch switches to the new variant, which is nice for two reasons: - we don't have to worry that silencing curl's deprecation warnings might cause us to miss other more useful ones - we'd eventually want to move to the new variant anyway, so this gets us set up (albeit with some extra ugly boilerplate for the conditional) There are a lot of ways to split up the two cases. One way would be to abstract the storage type (strbuf versus a long), how to append (strbuf_addstr vs bitwise OR), how to initialize, which CURLOPT to use, and so on. But the resulting code looks pretty magical: GIT_CURL_PROTOCOL_TYPE allowed = GIT_CURL_PROTOCOL_TYPE_INIT; if (...http is allowed...) GIT_CURL_PROTOCOL_APPEND(&allowed, "http", CURLOPT_HTTP); and you end up with more "#define GIT_CURL_PROTOCOL_TYPE" macros than actual code. On the other end of the spectrum, we could just implement two separate functions, one that handles a string list and one that handles bits. But then we end up repeating our list of protocols (http, https, ftp, ftp). This patch takes the middle ground. The run-time code is always there to handle both types, and we just choose which one to feed to curl. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTIONJeff King5-30/+26
The IOCTLFUNCTION option has been deprecated, and generates a compiler warning in recent versions of curl. We can switch to using SEEKFUNCTION instead. It was added in 2008 via curl 7.18.0; our INSTALL file already indicates we require at least curl 7.19.4. But there's one catch: curl says we should use CURL_SEEKFUNC_{OK,FAIL}, and those didn't arrive until 7.19.5. One workaround would be to use a bare 0/1 here (or define our own macros). But let's just bump the minimum required version to 7.19.5. That version is only a minor version bump from our existing requirement, and is only a 2 month time bump for versions that are almost 13 years old. So it's not likely that anybody cares about the distinction. Switching means we have to rewrite the ioctl functions into seek functions. In some ways they are simpler (seeking is the only operation), but in some ways more complex (the ioctl allowed only a full rewind, but now we can seek to arbitrary offsets). Curl will only ever use SEEK_SET (per their documentation), so I didn't bother implementing anything else, since it would naturally be completely untested. This seems unlikely to change, but I added an assertion just in case. Likewise, I doubt curl will ever try to seek outside of the buffer sizes we've told it, but I erred on the defensive side here, rather than do an out-of-bounds read. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUTJeff King1-1/+1
The two options do exactly the same thing, but the latter has been deprecated and in recent versions of curl may produce a compiler warning. Since the UPLOAD form is available everywhere (it was introduced in the year 2000 by curl 7.1), we can just switch to it. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06Git 2.33.7v2.33.7Johannes Schindelin3-2/+9
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06Sync with 2.32.6Johannes Schindelin11-9/+302
* maint-2.32: Git 2.32.6 Git 2.31.7 Git 2.30.8 apply: fix writing behind newly created symbolic links dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS clone: delay picking a transport until after get_repo_path() t5619: demonstrate clone_local() with ambiguous transport
2023-02-06Git 2.32.6v2.32.6Johannes Schindelin3-2/+8
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06Sync with 2.31.7Johannes Schindelin10-9/+296
* maint-2.31: Git 2.31.7 Git 2.30.8 apply: fix writing behind newly created symbolic links dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS clone: delay picking a transport until after get_repo_path() t5619: demonstrate clone_local() with ambiguous transport
2023-02-06Git 2.31.7v2.31.7Johannes Schindelin3-2/+8
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06Sync with 2.30.8Johannes Schindelin9-9/+290
* maint-2.30: Git 2.30.8 apply: fix writing behind newly created symbolic links dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS clone: delay picking a transport until after get_repo_path() t5619: demonstrate clone_local() with ambiguous transport
2023-02-06Git 2.30.8v2.30.8Junio C Hamano3-2/+54
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-06Merge branch 'ps/apply-beyond-symlink' into maint-2.30Junio C Hamano2-0/+108
Fix a vulnerability (CVE-2023-23946) that allows crafted input to trick `git apply` into writing files outside of the working tree. * ps/apply-beyond-symlink: dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06Merge branch 'tb/clone-local-symlinks' into maint-2.30Taylor Blau6-9/+130
Resolve a security vulnerability (CVE-2023-22490) where `clone_local()` is used in conjunction with non-local transports, leading to arbitrary path exfiltration. * tb/clone-local-symlinks: dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS clone: delay picking a transport until after get_repo_path() t5619: demonstrate clone_local() with ambiguous transport
2023-02-03apply: fix writing behind newly created symbolic linksPatrick Steinhardt2-0/+108
When writing files git-apply(1) initially makes sure that none of the files it is about to create are behind a symlink: ``` $ git init repo Initialized empty Git repository in /tmp/repo/.git/ $ cd repo/ $ ln -s dir symlink $ git apply - <<EOF diff --git a/symlink/file b/symlink/file new file mode 100644 index 0000000..e69de29 EOF error: affected file 'symlink/file' is beyond a symbolic link ``` This safety mechanism is crucial to ensure that we don't write outside of the repository's working directory. It can be fooled though when the patch that is being applied creates the symbolic link in the first place, which can lead to writing files in arbitrary locations. Fix this by checking whether the path we're about to create is beyond a symlink or not. Tightening these checks like this should be fine as we already have these precautions in Git as explained above. Ideally, we should update the check we do up-front before starting to reflect the computed changes to the working tree so that we catch this case as well, but as part of embargoed security work, adding an equivalent check just before we try to write out a file should serve us well as a reasonable first step. Digging back into history shows that this vulnerability has existed since at least Git v2.9.0. As Git v2.8.0 and older don't build on my system anymore I cannot tell whether older versions are affected, as well. Reported-by: Joern Schneeweisz <jschneeweisz@gitlab.com> Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-01t/lib-httpd: increase ssl key size to 2048 bitsJeff King1-1/+1
Recent versions of openssl will refuse to work with 1024-bit RSA keys, as they are considered insecure. I didn't track down the exact version in which the defaults were tightened, but the Debian-package openssl 3.0 on my system yields: $ LIB_HTTPD_SSL=1 ./t5551-http-fetch-smart.sh -v -i [...] SSL Library Error: error:0A00018F:SSL routines::ee key too small 1..0 # SKIP web server setup failed This could probably be overcome with configuration, but that's likely to be a headache (especially if it requires touching /etc/openssl). Let's just pick a key size that's less outrageously out of date. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-01t/lib-httpd: drop SSLMutex configJeff King1-1/+0
The SSL config enabled by setting LIB_HTTPD_SSL does not work with Apache versions greater than 2.2, as more recent versions complain about the SSLMutex directive. According to https://httpd.apache.org/docs/current/upgrading.html: Directives AcceptMutex, LockFile, RewriteLock, SSLMutex, SSLStaplingMutex, and WatchdogMutexPath have been replaced with a single Mutex directive. You will need to evaluate any use of these removed directives in your 2.2 configuration to determine if they can just be deleted or will need to be replaced using Mutex. Deleting this line will just use the system default, which seems sensible. The original came as part of faa4bc35a0 (http-push: add regression tests, 2008-02-27), but no specific reason is given there (or on the mailing list) for its presence. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-01t/lib-httpd: bump required apache version to 2.4Jeff King2-20/+6
Apache 2.4 has been out since early 2012, almost 11 years. And its predecessor, 2.2, has been out of support since its last release in 2017, over 5 years ago. The last mention on the mailing list was from around the same time, in this thread: https://lore.kernel.org/git/20171231023234.21215-1-tmz@pobox.com/ We can probably assume that 2.4 is available everywhere. And the stakes are fairly low, as the worst case is that such a platform would skip the http tests. This lets us clean up a few minor version checks in the config file, but also revert f1f2b45be0 (tests: adjust the configuration for Apache 2.2, 2016-05-09). Its technique isn't _too_ bad, but certainly required a bit more explanation than the 2.4 version it replaced. I manually confirmed that the test in t5551 still behaves as expected (if you replace "cadabra" with "foo", the server correctly rejects the request). It will also help future patches which will no longer have to deal with conditional config for this old version. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-01t/lib-httpd: bump required apache version to 2.2Jeff King2-12/+7
Apache 2.2 was released in 2005, almost 18 years ago. We can probably assume that people are running a version at least that old (and the stakes for removing it are fairly low, as the worst case is that they would not run the http tests against their ancient version). Dropping support for the older versions cleans up the config file a little, and will also enable us to bump the required version further (with more cleanups) in a future patch. Note that the file actually checks for version 2.1. In apache's versioning scheme, odd numbered versions are for development and even numbers are for stable releases. So 2.1 and 2.2 are effectively the same from our perspective. Older versions would just fail to start, which would generally cause us to skip the tests. However, we do have version detection code in lib-httpd.sh which produces a nicer error message, so let's update that, too. I didn't bother handling the case of "3.0", etc. Apache has been on 2.x for 21 years, with no signs of bumping the major version. And if they eventually do, I suspect there will be enough breaking changes that we'd need to update more than just the numeric version check. We can worry about that hypothetical when it happens. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-24dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKSTaylor Blau4-5/+56
When using the dir_iterator API, we first stat(2) the base path, and then use that as a starting point to enumerate the directory's contents. If the directory contains symbolic links, we will immediately die() upon encountering them without the `FOLLOW_SYMLINKS` flag. The same is not true when resolving the top-level directory, though. As explained in a previous commit, this oversight in 6f054f9fb3 (builtin/clone.c: disallow `--local` clones with symlinks, 2022-07-28) can be used as an attack vector to include arbitrary files on a victim's filesystem from outside of the repository. Prevent resolving top-level symlinks unless the FOLLOW_SYMLINKS flag is given, which will cause clones of a repository with a symlink'd "$GIT_DIR/objects" directory to fail. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-24clone: delay picking a transport until after get_repo_path()Taylor Blau2-8/+15
In the previous commit, t5619 demonstrates an issue where two calls to `get_repo_path()` could trick Git into using its local clone mechanism in conjunction with a non-local transport. That sequence is: - the starting state is that the local path https:/example.com/foo is a symlink that points to ../../../.git/modules/foo. So it's dangling. - get_repo_path() sees that no such path exists (because it's dangling), and thus we do not canonicalize it into an absolute path - because we're using --separate-git-dir, we create .git/modules/foo. Now our symlink is no longer dangling! - we pass the url to transport_get(), which sees it as an https URL. - we call get_repo_path() again, on the url. This second call was introduced by f38aa83f9a (use local cloning if insteadOf makes a local URL, 2014-07-17). The idea is that we want to pull the url fresh from the remote.c API, because it will apply any aliases. And of course now it sees that there is a local file, which is a mismatch with the transport we already selected. The issue in the above sequence is calling `transport_get()` before deciding whether or not the repository is indeed local, and not passing in an absolute path if it is local. This is reminiscent of a similar bug report in [1], where it was suggested to perform the `insteadOf` lookup earlier. Taking that approach may not be as straightforward, since the intent is to store the original URL in the config, but to actually fetch from the insteadOf one, so conflating the two early on is a non-starter. Note: we pass the path returned by `get_repo_path(remote->url[0])`, which should be the same as `repo_name` (aside from any `insteadOf` rewrites). We *could* pass `absolute_pathdup()` of the same argument, which 86521acaca (Bring local clone's origin URL in line with that of a remote clone, 2008-09-01) indicates may differ depending on the presence of ".git/" for a non-bare repo. That matters for forming relative submodule paths, but doesn't matter for the second call, since we're just feeding it to the transport code, which is fine either way. [1]: https://lore.kernel.org/git/CAMoD=Bi41mB3QRn3JdZL-FGHs4w3C2jGpnJB-CqSndO7FMtfzA@mail.gmail.com/ Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-24t5619: demonstrate clone_local() with ambiguous transportTaylor Blau1-0/+63
When cloning a repository, Git must determine (a) what transport mechanism to use, and (b) whether or not the clone is local. Since f38aa83f9a (use local cloning if insteadOf makes a local URL, 2014-07-17), the latter check happens after the remote has been initialized, and references the remote's URL instead of the local path. This is done to make it possible for a `url.<base>.insteadOf` rule to convert a remote URL into a local one, in which case the `clone_local()` mechanism should be used. However, with a specially crafted repository, Git can be tricked into using a non-local transport while still setting `is_local` to "1" and using the `clone_local()` optimization. The below test case demonstrates such an instance, and shows that it can be used to include arbitrary (known) paths in the working copy of a cloned repository on a victim's machine[^1], even if local file clones are forbidden by `protocol.file.allow`. This happens in a few parts: 1. We first call `get_repo_path()` to see if the remote is a local path. If it is, we replace the repo name with its absolute path. 2. We then call `transport_get()` on the repo name and decide how to access it. If it was turned into an absolute path in the previous step, then we should always treat it like a file. 3. We use `get_repo_path()` again, and set `is_local` as appropriate. But it's already too late to rewrite the repo name as an absolute path, since we've already fed it to the transport code. The attack works by including a submodule whose URL corresponds to a path on disk. In the below example, the repository "sub" is reachable via the dumb HTTP protocol at (something like): http://127.0.0.1:NNNN/dumb/sub.git However, the path "http:/127.0.0.1:NNNN/dumb" (that is, a top-level directory called "http:", then nested directories "127.0.0.1:NNNN", and "dumb") exists within the repository, too. To determine this, it first picks the appropriate transport, which is dumb HTTP. It then uses the remote's URL in order to determine whether the repository exists locally on disk. However, the malicious repository also contains an embedded stub repository which is the target of a symbolic link at the local path corresponding to the "sub" repository on disk (i.e., there is a symbolic link at "http:/127.0.0.1/dumb/sub.git", pointing to the stub repository via ".git/modules/sub/../../../repo"). This stub repository fools Git into thinking that a local repository exists at that URL and thus can be cloned locally. The affected call is in `get_repo_path()`, which in turn calls `get_repo_path_1()`, which locates a valid repository at that target. This then causes Git to set the `is_local` variable to "1", and in turn instructs Git to clone the repository using its local clone optimization via the `clone_local()` function. The exploit comes into play because the stub repository's top-level "$GIT_DIR/objects" directory is a symbolic link which can point to an arbitrary path on the victim's machine. `clone_local()` resolves the top-level "objects" directory through a `stat(2)` call, meaning that we read through the symbolic link and copy or hardlink the directory contents at the destination of the link. In other words, we can get steps (1) and (3) to disagree by leveraging the dangling symlink to pick a non-local transport in the first step, and then set is_local to "1" in the third step when cloning with `--separate-git-dir`, which makes the symlink non-dangling. This can result in data-exfiltration on the victim's machine when sensitive data is at a known path (e.g., "/home/$USER/.ssh"). The appropriate fix is two-fold: - Resolve the transport later on (to avoid using the local clone optimization with a non-local transport). - Avoid reading through the top-level "objects" directory when (correctly) using the clone_local() optimization. This patch merely demonstrates the issue. The following two patches will implement each part of the above fix, respectively. [^1]: Provided that any target directory does not contain symbolic links, in which case the changes from 6f054f9fb3 (builtin/clone.c: disallow `--local` clones with symlinks, 2022-07-28) will abort the clone. Reported-by: yvvdwf <yvvdwf@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-19Sync with maint-2.38Junio C Hamano0-0/+0
* maint-2.38: attr: adjust a mismatched data type
2023-01-19Sync with maint-2.37Junio C Hamano1-1/+1
* maint-2.37: attr: adjust a mismatched data type
2023-01-19Sync with maint-2.36Junio C Hamano1-1/+1
* maint-2.36: attr: adjust a mismatched data type
2023-01-19Sync with maint-2.35Junio C Hamano1-1/+1
* maint-2.35: attr: adjust a mismatched data type
2023-01-19Sync with maint-2.34Junio C Hamano1-1/+1
* maint-2.34: attr: adjust a mismatched data type
2023-01-19Sync with maint-2.33Junio C Hamano1-1/+1
* maint-2.33: attr: adjust a mismatched data type
2023-01-19Sync with maint-2.32Junio C Hamano1-1/+1
* maint-2.32: attr: adjust a mismatched data type
2023-01-19Sync with maint-2.31Junio C Hamano1-1/+1
* maint-2.31: attr: adjust a mismatched data type
2023-01-19Sync with maint-2.30Junio C Hamano1-1/+1
* maint-2.30: attr: adjust a mismatched data type
2023-01-19attr: adjust a mismatched data typeJohannes Schindelin1-1/+1
On platforms where `size_t` does not have the same width as `unsigned long`, passing a pointer to the former when a pointer to the latter is expected can lead to problems. Windows and 32-bit Linux are among the affected platforms. In this instance, we want to store the size of the blob that was read in that variable. However, `read_blob_data_from_index()` passes that pointer to `read_object_file()` which expects an `unsigned long *`. Which means that on affected platforms, the variable is not fully populated and part of its value is left uninitialized. (On Big-Endian platforms, this problem would be even worse.) The consequence is that depending on the uninitialized memory's contents, we may erroneously reject perfectly fine attributes. Let's address this by passing a pointer to a variable of the expected data type. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-19checkout: document -b/-B to highlight the differences from "git branch"Junio C Hamano1-6/+8
The existing text read as if "git checkout -b/-B name" were equivalent to "git branch [-f] name", which clearly was not what we wanted to say. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-18branch: document `-f` and linked worktree behaviourJunio C Hamano1-0/+4
"git branch -f name start" forces to recreate the named branch, but the forcing does not defeat the "do not touch a branch that is checked out elsewhere" safety valve. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-17http: support CURLOPT_PROTOCOLS_STRJeff King2-13/+54
The CURLOPT_PROTOCOLS (and matching CURLOPT_REDIR_PROTOCOLS) flag was deprecated in curl 7.85.0, and using it generate compiler warnings as of curl 7.87.0. The path forward is to use CURLOPT_PROTOCOLS_STR, but we can't just do so unilaterally, as it was only introduced less than a year ago in 7.85.0. Until that version becomes ubiquitous, we have to either disable the deprecation warning or conditionally use the "STR" variant on newer versions of libcurl. This patch switches to the new variant, which is nice for two reasons: - we don't have to worry that silencing curl's deprecation warnings might cause us to miss other more useful ones - we'd eventually want to move to the new variant anyway, so this gets us set up (albeit with some extra ugly boilerplate for the conditional) There are a lot of ways to split up the two cases. One way would be to abstract the storage type (strbuf versus a long), how to append (strbuf_addstr vs bitwise OR), how to initialize, which CURLOPT to use, and so on. But the resulting code looks pretty magical: GIT_CURL_PROTOCOL_TYPE allowed = GIT_CURL_PROTOCOL_TYPE_INIT; if (...http is allowed...) GIT_CURL_PROTOCOL_APPEND(&allowed, "http", CURLOPT_HTTP); and you end up with more "#define GIT_CURL_PROTOCOL_TYPE" macros than actual code. On the other end of the spectrum, we could just implement two separate functions, one that handles a string list and one that handles bits. But then we end up repeating our list of protocols (http, https, ftp, ftp). This patch takes the middle ground. The run-time code is always there to handle both types, and we just choose which one to feed to curl. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-17http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTIONJeff King5-30/+26
The IOCTLFUNCTION option has been deprecated, and generates a compiler warning in recent versions of curl. We can switch to using SEEKFUNCTION instead. It was added in 2008 via curl 7.18.0; our INSTALL file already indicates we require at least curl 7.19.4. But there's one catch: curl says we should use CURL_SEEKFUNC_{OK,FAIL}, and those didn't arrive until 7.19.5. One workaround would be to use a bare 0/1 here (or define our own macros). But let's just bump the minimum required version to 7.19.5. That version is only a minor version bump from our existing requirement, and is only a 2 month time bump for versions that are almost 13 years old. So it's not likely that anybody cares about the distinction. Switching means we have to rewrite the ioctl functions into seek functions. In some ways they are simpler (seeking is the only operation), but in some ways more complex (the ioctl allowed only a full rewind, but now we can seek to arbitrary offsets). Curl will only ever use SEEK_SET (per their documentation), so I didn't bother implementing anything else, since it would naturally be completely untested. This seems unlikely to change, but I added an assertion just in case. Likewise, I doubt curl will ever try to seek outside of the buffer sizes we've told it, but I erred on the defensive side here, rather than do an out-of-bounds read. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-17http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUTJeff King1-1/+1
The two options do exactly the same thing, but the latter has been deprecated and in recent versions of curl may produce a compiler warning. Since the UPLOAD form is available everywhere (it was introduced in the year 2000 by curl 7.1), we can just switch to it. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-17attr: adjust a mismatched data typeJohannes Schindelin1-1/+1
On platforms where `size_t` does not have the same width as `unsigned long`, passing a pointer to the former when a pointer to the latter is expected can lead to problems. Windows and 32-bit Linux are among the affected platforms. In this instance, we want to store the size of the blob that was read in that variable. However, `read_blob_data_from_index()` passes that pointer to `read_object_file()` which expects an `unsigned long *`. Which means that on affected platforms, the variable is not fully populated and part of its value is left uninitialized. (On Big-Endian platforms, this problem would be even worse.) The consequence is that depending on the uninitialized memory's contents, we may erroneously reject perfectly fine attributes. Let's address this by passing a pointer to a variable of the expected data type. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-16fsck: document the new `gitattributes` message IDsJohannes Schindelin1-0/+12
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-14ls-tree: remove dead store and strbuf for quote_c_style()René Scharfe1-5/+2
Stop initializing "name" because it is set again before use. Let quote_c_style() write directly to "sb" instead of taking a detour through "quoted". This avoids an allocation and a string copy. The result is the same because the function only appends. Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-14ls-tree: fix expansion of repeated %(path)René Scharfe2-3/+9
expand_show_tree() borrows the base strbuf given to us by read_tree() to build the full path of the current entry when handling %(path). Only its indirect caller, show_tree_fmt(), removes the added entry name. That works fine as long as %(path) is only included once in the format string, but accumulates duplicates if it's repeated: $ git ls-tree --format='%(path) %(path) %(path)' HEAD M* Makefile MakefileMakefile MakefileMakefileMakefile Reset the length after each use to get the same expansion every time; here's the behavior with this patch: $ ./git ls-tree --format='%(path) %(path) %(path)' HEAD M* Makefile Makefile Makefile Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-13githooks: discuss Git operations in foreign repositoriesEric Sunshine1-0/+12
Hook authors are periodically caught off-guard by difficult-to-diagnose errors when their hook invokes Git commands in a repository other than the local one. In particular, Git environment variables, such as GIT_DIR and GIT_WORK_TREE, which reference the local repository cause the Git commands to operate on the local repository rather than on the repository which the author intended. This is true whether the environment variables have been set manually by the user or automatically by Git itself. The same problem crops up when a hook invokes Git commands in a different worktree of the same repository, as well. Recommended best-practice[1,2,3,4,5,6] for avoiding this problem is for the hook to ensure that Git variables are unset before invoking Git commands in foreign repositories or other worktrees: unset $(git rev-parse --local-env-vars) However, this advice is not documented anywhere. Rectify this shortcoming by mentioning it in githooks.txt documentation. [1]: https://lore.kernel.org/git/YFuHd1MMlJAvtdzb@coredump.intra.peff.net/ [2]: https://lore.kernel.org/git/20200228190218.GC1408759@coredump.intra.peff.net/ [3]: https://lore.kernel.org/git/20190516221702.GA11784@sigill.intra.peff.net/ [4]: https://lore.kernel.org/git/20190422162127.GC9680@sigill.intra.peff.net/ [5]: https://lore.kernel.org/git/20180716183942.GB22298@sigill.intra.peff.net/ [6]: https://lore.kernel.org/git/20150203163235.GA9325@peff.net/ Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-13git-rebase.txt: add a note about 'ORIG_HEAD' being overwrittenPhilippe Blain1-0/+7
'ORIG_HEAD' is written at the start of the rebase, but is not guaranteed to still point to the original branch tip at the end of the rebase. Indeed, using other commands that write 'ORIG_HEAD' during the rebase, like splitting a commit using 'git reset HEAD^', will lead to 'ORIG_HEAD' being overwritten. This causes confusion for some users [1]. Add a note about that in the 'Description' section, and mention the more robust alternative of using the branch's reflog. [1] https://lore.kernel.org/git/28ebf03b-e8bb-3769-556b-c9db17e43dbb@gmail.com/T/#m827179c5adcfb504d67f76d03c8e6942b55e5ed0 Reported-by: Erik Cervin Edin <erik@cervined.in> Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Acked-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-13revisions.txt: be explicit about commands writing 'ORIG_HEAD'Philippe Blain1-1/+2
When mentioning 'ORIG_HEAD', be explicit about which command write that pseudo-ref, namely 'git am', 'git merge', 'git rebase' and 'git reset'. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Acked-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-13git-merge.txt: mention 'ORIG_HEAD' in the DescriptionPhilippe Blain1-1/+2
The fact that 'git merge' writes 'ORIG_HEAD' before performing the merge is missing from the documentation of the command. Mention it in the 'Description' section. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Acked-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-13git-reset.txt: mention 'ORIG_HEAD' in the DescriptionPhilippe Blain1-1/+2
The fact that 'git reset' writes 'ORIG_HEAD' before changing HEAD is mentioned in an example, but is missing from the 'Description' section. Mention it in the discussion of the "'git reset' [<mode>] [<commit>]" form of the command. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Acked-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-13git-cherry-pick.txt: do not use 'ORIG_HEAD' in examplePhilippe Blain1-1/+1
Commit 67ac1e1d57 (cherry-pick/revert: add support for -X/--strategy-option, 2010-12-10) added an example to the documentation of 'git cherry-pick'. This example mentions how to abort a failed cherry-pick and retry with an additional merge strategy option. The command used in the example to abort the cherry-pick is 'git reset --merge ORIG_HEAD', but cherry-pick does not write 'ORIG_HEAD' before starting its operation. So this command would checkout a commit unrelated to what was at HEAD when the user invoked cherry-pick. Use 'git cherry-pick --abort' instead. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Acked-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-08use enhanced basic regular expressions on macOSRené Scharfe4-0/+24
When 1819ad327b (grep: fix multibyte regex handling under macOS, 2022-08-26) started to use the native regex library instead of Git's own (compat/regex/), it lost support for alternation in basic regular expressions. Bring it back by enabling the flag REG_ENHANCED on macOS when compiling basic regular expressions. Reported-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com> Suggested-by: Jeff King <peff@peff.net> Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>