aboutsummaryrefslogtreecommitdiffstats
path: root/git-compat-util.h
AgeCommit message (Collapse)AuthorFilesLines
2019-02-22trace2: create new combined trace facilityJeff Hostetler1-0/+7
Create a new unified tracing facility for git. The eventual intent is to replace the current trace_printf* and trace_performance* routines with a unified set of git_trace2* routines. In addition to the usual printf-style API, trace2 provides higer-level event verbs with fixed-fields allowing structured data to be written. This makes post-processing and analysis easier for external tools. Trace2 defines 3 output targets. These are set using the environment variables "GIT_TR2", "GIT_TR2_PERF", and "GIT_TR2_EVENT". These may be set to "1" or to an absolute pathname (just like the current GIT_TRACE). * GIT_TR2 is intended to be a replacement for GIT_TRACE and logs command summary data. * GIT_TR2_PERF is intended as a replacement for GIT_TRACE_PERFORMANCE. It extends the output with columns for the command process, thread, repo, absolute and relative elapsed times. It reports events for child process start/stop, thread start/stop, and per-thread function nesting. * GIT_TR2_EVENT is a new structured format. It writes event data as a series of JSON records. Calls to trace2 functions log to any of the 3 output targets enabled without the need to call different trace_printf* or trace_performance* routines. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-13Merge branch 'nd/fileno-may-be-macro'Junio C Hamano1-0/+8
* nd/fileno-may-be-macro: git-compat-util: work around fileno(fp) that is a macro
2019-02-12git-compat-util: work around fileno(fp) that is a macroDuy Nguyen1-0/+8
On various BSD's, fileno(fp) is implemented as a macro that directly accesses the fields in the FILE * object, which breaks a function that accepts a "void *fp" parameter and calls fileno(fp) and expect it to work. Work it around by adding a compile-time knob FILENO_IS_A_MACRO that inserts a real helper function in the middle of the callchain. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-18Merge branch 'rb/hpe'Junio C Hamano1-0/+13
Portability updates for the HPE NonStop platform. * rb/hpe: compat/regex/regcomp.c: define intptr_t and uintptr_t on NonStop git-compat-util.h: add FLOSS headers for HPE NonStop config.mak.uname: support for modern HPE NonStop config. transport-helper: drop read/write errno checks transport-helper: use xread instead of read
2019-01-14Merge branch 'nd/indentation-fix'Junio C Hamano1-1/+1
Code cleanup. * nd/indentation-fix: Indent code with TABs
2019-01-14Merge branch 'tb/use-common-win32-pathfuncs-on-cygwin'Junio C Hamano1-1/+2
Cygwin update. * tb/use-common-win32-pathfuncs-on-cygwin: git clone <url> C:\cygwin\home\USER\repo' is working (again)
2019-01-03git-compat-util.h: add FLOSS headers for HPE NonStopRandall S. Becker1-0/+13
The HPE NonStop (a.k.a. __TANDEM) platform cannot build git without using the FLOSS package supplied by HPE. The convenient location for including the relevant headers is in this file. The NSIG define is also not defined on __TANDEM, so we define it here as 100 if it is not defined only for __TANDEM builds. Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-12-26git clone <url> C:\cygwin\home\USER\repo' is working (again)Torsten Bögershausen1-1/+2
A regression for cygwin users was introduced with commit 05b458c, "real_path: resolve symlinks by hand". In the the commit message we read: The current implementation of real_path uses chdir() in order to resolve symlinks. Unfortunately this isn't thread-safe as chdir() affects a process as a whole... The old (and non-thread-save) OS calls chdir()/pwd() had been replaced by a string operation. The cygwin layer "knows" that "C:\cygwin" is an absolute path, but the new string operation does not. "git clone <url> C:\cygwin\home\USER\repo" fails like this: fatal: Invalid path '/home/USER/repo/C:\cygwin\home\USER\repo' The solution is to implement has_dos_drive_prefix(), skip_dos_drive_prefix() is_dir_sep(), offset_1st_component() and convert_slashes() for cygwin in the same way as it is done in 'Git for Windows' in compat/mingw.[ch] Extract the needed code into compat/win32/path-utils.[ch] and use it for cygwin as well. Reported-by: Steven Penny <svnpenn@gmail.com> Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-12-09Indent code with TABsNguyễn Thái Ngọc Duy1-1/+1
We indent with TABs and sometimes for fine alignment, TABs followed by spaces, but never all spaces (unless the indentation is less than 8 columns). Indenting with spaces slips through in some places. Fix them. Imported code and compat/ are left alone on purpose. The former should remain as close as upstream as possible. The latter pretty much has separate maintainers, it's up to them to decide. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-21Merge branch 'cc/delta-islands'Junio C Hamano1-0/+1
A few issues in the implementation of "delta-islands" feature has been corrected. * cc/delta-islands: pack-objects: fix off-by-one in delta-island tree-depth computation pack-objects: zero-initialize tree_depth/layer arrays pack-objects: fix tree_depth and layer invariants
2018-11-21pack-objects: zero-initialize tree_depth/layer arraysJeff King1-0/+1
Commit 108f530385 (pack-objects: move tree_depth into 'struct packing_data', 2018-08-16) started maintaining a tree_depth array that matches the "objects" array. We extend the array when: 1. The objects array is extended, in which case we use realloc to extend the tree_depth array. 2. A caller asks to store a tree_depth for object N, and this is the first such request; we create the array from scratch and store the value for N. In the latter case, though, we use regular xmalloc(), and the depth values for any objects besides N is undefined. This happens to not trigger a bug with the current code, but the reasons are quite subtle: - we never ask about the depth for any object with index i < N. This is because we store the depth immediately for all trees and blobs. So any such "i" must be a non-tree, and therefore we will never need to care about its depth (in fact, we really only care about the depth of trees). - there are no objects at this point with index i > N, because we always fill in the depth for a tree immediately after its object entry is created (we may still allocate uninitialized depth entries, but they'll be initialized by packlist_alloc() when it initializes the entry in the "objects" array). So it works, but only by chance. To be defensive, let's zero the array, which matches the "unset" values which would be handed out by oe_tree_depth() already. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-14git-compat-util: prefer poll.h to sys/poll.hĐoàn Trần Công Danh1-1/+4
POSIX specifies that <poll.h> is the correct header for poll(2) whereas <sys/poll.h> is only needed for some old libc. Let's follow the POSIX way by default. This effectively eliminates musl's warning: warning redirecting incorrect #include <sys/poll.h> to <poll.h> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-31config: allow for platform-specific core.* config settingsJohannes Schindelin1-0/+8
In the Git for Windows project, we have ample precendent for config settings that apply to Windows, and to Windows only. Let's formalize this concept by introducing a platform_core_config() function that can be #define'd in a platform-specific manner. This will allow us to contain platform-specific code better, as the corresponding variables no longer need to be exported so that they can be defined in environment.c and be set in config.c Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-30Merge branch 'cb/khash-maybe-unused-function'Junio C Hamano1-0/+2
Build fix. * cb/khash-maybe-unused-function: khash: silence -Wunused-function for delta-islands commit-slabs: move MAYBE_UNUSED out
2018-10-26Merge branch 'js/mingw-default-ident'Junio C Hamano1-0/+4
The logic to select the default user name and e-mail on Windows has been improved. * js/mingw-default-ident: mingw: use domain information for default email getpwuid(mingw): provide a better default for the user name getpwuid(mingw): initialize the structure only once
2018-10-24commit-slabs: move MAYBE_UNUSED outCarlo Marcelo Arenas Belón1-0/+2
after 36da893114 ("config.mak.dev: enable -Wunused-function", 2018-10-18) it is expected to be used to prevent -Wunused-function warnings for code that was macro generated Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-16mingw: use domain information for default emailJohannes Schindelin1-0/+4
When a user is registered in a Windows domain, it is really easy to obtain the email address. So let's do that. Suggested by Lutz Roeder. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-04mingw: bump the minimum Windows version to VistaJohannes Schindelin1-1/+1
Quite some time ago, a last plea to the XP users out there who want to see Windows XP support in Git for Windows, asking them to get engaged and help, vanished into the depths of the universe. We tried for a long time to play nice with the last remaining XP users who somehow manage to build Git from source, but a recent update of mingw-w64 (7.0.0.5233.e0c09544 -> 7.0.0.5245.edf66197) finally dropped the last sign of XP support, and Git for Windows' SDK is no longer able to build core Git's `master` branch as a consequence. (Git for Windows' `master` branch already bumped the minimum Windows version to Vista a while ago, so it is fine.) It is time to require Windows Vista or later to build Git from source. This, incidentally, lets us use quite a few nice new APIs. It also means that we no longer need the inet_pton() and inet_ntop() emulation, which is nice. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-04mingw: set _WIN32_WINNT explicitly for Git for WindowsJohannes Schindelin1-1/+1
Previously, we only ever declared a target Windows version if compiling with Visual C. Which meant that we were relying on the MinGW headers to guess which Windows version we want to target... Let's be explicit about it, in particular because we actually want to bump the target Windows version to Vista (which we will do in the next commit). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-20Merge branch 'js/typofixes'Junio C Hamano1-1/+1
Comment update. * js/typofixes: remote-curl: remove spurious period git-compat-util.h: fix typo
2018-08-08git-compat-util.h: fix typoJohannes Schindelin1-1/+1
The words "save" and "safe" are both very wonderful words, each with their own set of meanings. Let's not confuse them with one another save on occasion of a pun. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-26automatically ban strcpy()Jeff King1-0/+6
There are a few standard C functions (like strcpy) which are easy to misuse. E.g.: char path[PATH_MAX]; strcpy(path, arg); may overflow the "path" buffer. Sometimes there's an earlier constraint on the size of "arg", but even in such a case it's hard to verify that the code is correct. If the size really is unbounded, you're better off using a dynamic helper like strbuf: struct strbuf path = STRBUF_INIT; strbuf_addstr(path, arg); or if it really is bounded, then use xsnprintf to show your expectation (and get a run-time assertion): char path[PATH_MAX]; xsnprintf(path, sizeof(path), "%s", arg); which makes further auditing easier. We'd usually catch undesirable code like this in a review, but there's no automated enforcement. Adding that enforcement can help us be more consistent and save effort (and a round-trip) during review. This patch teaches the compiler to report an error when it sees strcpy (and will become a model for banning a few other functions). This has a few advantages over a separate linting tool: 1. We know it's run as part of a build cycle, so it's hard to ignore. Whereas an external linter is an extra step the developer needs to remember to do. 2. Likewise, it's basically free since the compiler is parsing the code anyway. 3. We know it's robust against false positives (unlike a grep-based linter). The two big disadvantages are: 1. We'll only check code that is actually compiled, so it may miss code that isn't triggered on your particular system. But since presumably people don't add new code without compiling it (and if they do, the banned function list is the least of their worries), we really only care about failing to clean up old code when adding new functions to the list. And that's easy enough to address with a manual audit when adding a new function (which is what I did for the functions here). 2. If this ends up generating false positives, it's going to be harder to disable (as opposed to a separate linter, which may have mechanisms for overriding a particular case). But the intent is to only ban functions which are obviously bad, and for which we accept using an alternative even when this particular use isn't buggy (e.g., the xsnprintf alternative above). The implementation here is simple: we'll define a macro for the banned function which replaces it with a reference to a descriptively named but undeclared identifier. Replacing it with any invalid code would work (since we just want to break compilation). But ideally we'd meet these goals: - it should be portable; ideally this would trigger everywhere, and does not need to be part of a DEVELOPER=1 setup (because unlike warnings which may depend on the compiler or system, this is a clear indicator of something wrong in the code). - it should generate a readable error that gives the developer a clue what happened - it should avoid generating too much other cruft that makes it hard to see the actual error - it should mention the original callsite in the error The output with this patch looks like this (using gcc 7, on a checkout with 022d2ac1f3 reverted, which removed the final strcpy from blame.c): CC builtin/blame.o In file included from ./git-compat-util.h:1246, from ./cache.h:4, from builtin/blame.c:8: builtin/blame.c: In function ‘cmd_blame’: ./banned.h:11:22: error: ‘sorry_strcpy_is_a_banned_function’ undeclared (first use in this function) #define BANNED(func) sorry_##func##_is_a_banned_function ^~~~~~ ./banned.h:14:21: note: in expansion of macro ‘BANNED’ #define strcpy(x,y) BANNED(strcpy) ^~~~~~ builtin/blame.c:1074:4: note: in expansion of macro ‘strcpy’ strcpy(repeated_meta_color, GIT_COLOR_CYAN); ^~~~~~ ./banned.h:11:22: note: each undeclared identifier is reported only once for each function it appears in #define BANNED(func) sorry_##func##_is_a_banned_function ^~~~~~ ./banned.h:14:21: note: in expansion of macro ‘BANNED’ #define strcpy(x,y) BANNED(strcpy) ^~~~~~ builtin/blame.c:1074:4: note: in expansion of macro ‘strcpy’ strcpy(repeated_meta_color, GIT_COLOR_CYAN); ^~~~~~ This prominently shows the phrase "strcpy is a banned function", along with the original callsite in blame.c and the location of the ban code in banned.h. Which should be enough to get even a developer seeing this for the first time pointed in the right direction. This doesn't match our ideals perfectly, but it's a pretty good balance. A few alternatives I tried: 1. Instead of using an undeclared variable, using an undeclared function. This shortens the message, because the "each undeclared identifier" message is not needed (and as you can see above, it triggers a separate mention of each of the expansion points). But it doesn't actually stop compilation unless you use -Werror=implicit-function-declaration in your CFLAGS. This is the case for DEVELOPER=1, but not for a default build (on the other hand, we'd eventually produce a link error pointing to the correct source line with the descriptive name). 2. The linux kernel uses a similar mechanism in its BUILD_BUG_ON_MSG(), where they actually declare the function but do so with gcc's error attribute. But that's not portable to other compilers (and it also runs afoul of our error() macro). We could make a gcc-specific technique and fallback on other compilers, but it's probably not worth the complexity. It also isn't significantly shorter than the error message shown above. 3. We could drop the BANNED() macro, which would shorten the number of lines in the error. But curiously, removing it (and just expanding strcpy directly to the bogus identifier) causes gcc _not_ to report the original line of code. So this strategy seems to be an acceptable mix of information, portability, simplicity, and robustness, without _too_ much extra clutter. I also tested it with clang, and it looks as good (actually, slightly less cluttered than with gcc). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-30Merge branch 'js/use-bug-macro'Junio C Hamano1-1/+4
Developer support update, by using BUG() macro instead of die() to mark codepaths that should not happen more clearly. * js/use-bug-macro: BUG_exit_code: fix sparse "symbol not declared" warning Convert remaining die*(BUG) messages Replace all die("BUG: ...") calls by BUG() ones run-command: use BUG() to report bugs, not die() test-tool: help verifying BUG() code paths
2018-05-29Sync with Git 2.17.1Junio C Hamano1-0/+17
* maint: (25 commits) Git 2.17.1 Git 2.16.4 Git 2.15.2 Git 2.14.4 Git 2.13.7 fsck: complain when .gitmodules is a symlink index-pack: check .gitmodules files with --strict unpack-objects: call fsck_finish() after fscking objects fsck: call fsck_finish() after fscking objects fsck: check .gitmodules content fsck: handle promisor objects in .gitmodules check fsck: detect gitmodules files fsck: actually fsck blob data fsck: simplify ".git" check index-pack: make fsck error message more specific verify_path: disallow symlinks in .gitmodules update-index: stat updated files earlier verify_dotfile: mention case-insensitivity in comment verify_path: drop clever fallthrough skip_prefix: add case-insensitive variant ...
2018-05-23Merge branch 'nd/repack-keep-pack'Junio C Hamano1-0/+4
"git gc" in a large repository takes a lot of time as it considers to repack all objects into one pack by default. The command has been taught to pretend as if the largest existing packfile is marked with ".keep" so that it is left untouched while objects in other packs and loose ones are repacked. * nd/repack-keep-pack: pack-objects: show some progress when counting kept objects gc --auto: exclude base pack if not enough mem to "repack -ad" gc: handle a corner case in gc.bigPackThreshold gc: add gc.bigPackThreshold config gc: add --keep-largest-pack option repack: add --keep-pack option t7700: have closing quote of a test at the beginning of line
2018-05-22Sync with Git 2.16.4Junio C Hamano1-0/+17
* maint-2.16: Git 2.16.4 Git 2.15.2 Git 2.14.4 Git 2.13.7 verify_path: disallow symlinks in .gitmodules update-index: stat updated files earlier verify_dotfile: mention case-insensitivity in comment verify_path: drop clever fallthrough skip_prefix: add case-insensitive variant is_{hfs,ntfs}_dotgitmodules: add tests is_ntfs_dotgit: match other .git files is_hfs_dotgit: match other .git files is_ntfs_dotgit: use a size_t for traversing string submodule-config: verify submodule names as paths
2018-05-22Sync with Git 2.15.2Junio C Hamano1-0/+17
* maint-2.15: Git 2.15.2 Git 2.14.4 Git 2.13.7 verify_path: disallow symlinks in .gitmodules update-index: stat updated files earlier verify_dotfile: mention case-insensitivity in comment verify_path: drop clever fallthrough skip_prefix: add case-insensitive variant is_{hfs,ntfs}_dotgitmodules: add tests is_ntfs_dotgit: match other .git files is_hfs_dotgit: match other .git files is_ntfs_dotgit: use a size_t for traversing string submodule-config: verify submodule names as paths
2018-05-22Sync with Git 2.14.4Junio C Hamano1-0/+17
* maint-2.14: Git 2.14.4 Git 2.13.7 verify_path: disallow symlinks in .gitmodules update-index: stat updated files earlier verify_dotfile: mention case-insensitivity in comment verify_path: drop clever fallthrough skip_prefix: add case-insensitive variant is_{hfs,ntfs}_dotgitmodules: add tests is_ntfs_dotgit: match other .git files is_hfs_dotgit: match other .git files is_ntfs_dotgit: use a size_t for traversing string submodule-config: verify submodule names as paths
2018-05-22Sync with Git 2.13.7Junio C Hamano1-0/+17
* maint-2.13: Git 2.13.7 verify_path: disallow symlinks in .gitmodules update-index: stat updated files earlier verify_dotfile: mention case-insensitivity in comment verify_path: drop clever fallthrough skip_prefix: add case-insensitive variant is_{hfs,ntfs}_dotgitmodules: add tests is_ntfs_dotgit: match other .git files is_hfs_dotgit: match other .git files is_ntfs_dotgit: use a size_t for traversing string submodule-config: verify submodule names as paths
2018-05-21skip_prefix: add case-insensitive variantJeff King1-0/+17
We have the convenient skip_prefix() helper, but if you want to do case-insensitive matching, you're stuck doing it by hand. We could add an extra parameter to the function to let callers ask for this, but the function is small and somewhat performance-critical. Let's just re-implement it for the case-insensitive version. Signed-off-by: Jeff King <peff@peff.net>
2018-05-10BUG_exit_code: fix sparse "symbol not declared" warningRamsay Jones1-0/+3
Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-08Merge branch 'ls/checkout-encoding'Junio C Hamano1-0/+1
The new "checkout-encoding" attribute can ask Git to convert the contents to the specified encoding when checking out to the working tree (and the other way around when checking in). * ls/checkout-encoding: convert: add round trip check based on 'core.checkRoundtripEncoding' convert: add tracing for 'working-tree-encoding' attribute convert: check for detectable errors in UTF encodings convert: add 'working-tree-encoding' attribute utf8: add function to detect a missing UTF-16/32 BOM utf8: add function to detect prohibited UTF-16/32 BOM utf8: teach same_encoding() alternative UTF encoding names strbuf: add a case insensitive starts_with() strbuf: add xstrdup_toupper() strbuf: remove unnecessary NUL assignment in xstrdup_tolower()
2018-05-06Convert remaining die*(BUG) messagesJohannes Schindelin1-1/+1
These were not caught by the previous commit, as they did not match the regular expression. While at it, remove the localization from one instance: we never want BUG() messages to be translated, as they target Git developers, not the end user (hence it would be quite unhelpful to not only burden the translators, but then even end up with a bug report in a language that no core Git contributor understands). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-16gc --auto: exclude base pack if not enough mem to "repack -ad"Nguyễn Thái Ngọc Duy1-0/+4
pack-objects could be a big memory hog especially on large repos, everybody knows that. The suggestion to stick a .keep file on the giant base pack to avoid this problem is also known for a long time. Recent patches add an option to do just this, but it has to be either configured or activated manually. This patch lets `git gc --auto` activate this mode automatically when it thinks `repack -ad` will use a lot of memory and start affecting the system due to swapping or flushing OS cache. gc --auto decides to do this based on an estimation of pack-objects memory usage, which is quite accurate at least for the heap part, and whether that fits in half of system memory (the assumption here is for desktop environment where there are many other applications running). This mechanism only kicks in if gc.bigBasePackThreshold is not configured. If it is, it is assumed that the user already knows what they want. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-09strbuf: add a case insensitive starts_with()Lars Schneider1-0/+1
Check in a case insensitive manner if one string is a prefix of another string. This function is used in a subsequent commit. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-22wrapper: rename 'template' variablesBrandon Williams1-2/+2
Rename C++ keyword in order to bring the codebase closer to being able to be compiled with a C++ compiler. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-11git-compat-util: introduce skip_to_optional_arg()Christian Couder1-0/+23
We often accept both a "--key" option and a "--key=<val>" option. These options currently are parsed using something like: if (!strcmp(arg, "--key")) { /* do something */ } else if (skip_prefix(arg, "--key=", &arg)) { /* do something with arg */ } which is a bit cumbersome compared to just: if (skip_to_optional_arg(arg, "--key", &arg)) { /* do something with arg */ } This also introduces skip_to_optional_arg_default() for the few cases where something different should be done when the first argument is exactly "--key" than when it is exactly "--key=". In general it is better for UI consistency and simplicity if "--key" and "--key=" do the same thing though, so that using skip_to_optional_arg() should be encouraged compared to skip_to_optional_arg_default(). Note that these functions can be used to parse any "key=value" string where "key" is also considered as valid, not just command line options. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-29Merge branch 'rj/no-sign-compare'Junio C Hamano1-2/+4
Many codepaths have been updated to squelch -Wsign-compare warnings. * rj/no-sign-compare: ALLOC_GROW: avoid -Wsign-compare warnings cache.h: hex2chr() - avoid -Wsign-compare warnings commit-slab.h: avoid -Wsign-compare warnings git-compat-util.h: xsize_t() - avoid -Wsign-compare warnings
2017-09-22git-compat-util.h: xsize_t() - avoid -Wsign-compare warningsRamsay Jones1-2/+4
Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-20git-compat-util: make UNLEAK less error-proneJonathan Tan1-2/+2
Commit 0e5bba5 ("add UNLEAK annotation for reducing leak false positives", 2017-09-08) introduced an UNLEAK macro to be used as "UNLEAK(var);", but its existing definitions leave semicolons that act as empty statements, which will lead to syntax errors, e.g. if (condition) UNLEAK(var); else something_else(var); would be broken with two statements between if (condition) and else. Lose the excess semicolon from the end of the macro replacement text. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-08add UNLEAK annotation for reducing leak false positivesJeff King1-0/+20
It's a common pattern in git commands to allocate some memory that should last for the lifetime of the program and then not bother to free it, relying on the OS to throw it away. This keeps the code simple, and it's fast (we don't waste time traversing structures or calling free at the end of the program). But it also triggers warnings from memory-leak checkers like valgrind or LSAN. They know that the memory was still allocated at program exit, but they don't know _when_ the leaked memory stopped being useful. If it was early in the program, then it's probably a real and important leak. But if it was used right up until program exit, it's not an interesting leak and we'd like to suppress it so that we can see the real leaks. This patch introduces an UNLEAK() macro that lets us do so. To understand its design, let's first look at some of the alternatives. Unfortunately the suppression systems offered by leak-checking tools don't quite do what we want. A leak-checker basically knows two things: 1. Which blocks were allocated via malloc, and the callstack during the allocation. 2. Which blocks were left un-freed at the end of the program (and which are unreachable, but more on that later). Their suppressions work by mentioning the function or callstack of a particular allocation, and marking it as OK to leak. So imagine you have code like this: int cmd_foo(...) { /* this allocates some memory */ char *p = some_function(); printf("%s", p); return 0; } You can say "ignore allocations from some_function(), they're not leaks". But that's not right. That function may be called elsewhere, too, and we would potentially want to know about those leaks. So you can say "ignore the callstack when main calls some_function". That works, but your annotations are brittle. In this case it's only two functions, but you can imagine that the actual allocation is much deeper. If any of the intermediate code changes, you have to update the suppression. What we _really_ want to say is that "the value assigned to p at the end of the function is not a real leak". But leak-checkers can't understand that; they don't know about "p" in the first place. However, we can do something a little bit tricky if we make some assumptions about how leak-checkers work. They generally don't just report all un-freed blocks. That would report even globals which are still accessible when the leak-check is run. Instead they take some set of memory (like BSS) as a root and mark it as "reachable". Then they scan the reachable blocks for anything that looks like a pointer to a malloc'd block, and consider that block reachable. And then they scan those blocks, and so on, transitively marking anything reachable from a global as "not leaked" (or at least leaked in a different category). So we can mark the value of "p" as reachable by putting it into a variable with program lifetime. One way to do that is to just mark "p" as static. But that actually affects the run-time behavior if the function is called twice (you aren't likely to call main() twice, but some of our cmd_*() functions are called from other commands). Instead, we can trick the leak-checker by putting the value into _any_ reachable bytes. This patch keeps a global linked-list of bytes copied from "unleaked" variables. That list is reachable even at program exit, which confers recursive reachability on whatever values we unleak. In other words, you can do: int cmd_foo(...) { char *p = some_function(); printf("%s", p); UNLEAK(p); return 0; } to annotate "p" and suppress the leak report. But wait, couldn't we just say "free(p)"? In this toy example, yes. But UNLEAK()'s byte-copying strategy has several advantages over actually freeing the memory: 1. It's recursive across structures. In many cases our "p" is not just a pointer, but a complex struct whose fields may have been allocated by a sub-function. And in some cases (e.g., dir_struct) we don't even have a function which knows how to free all of the struct members. By marking the struct itself as reachable, that confers reachability on any pointers it contains (including those found in embedded structs, or reachable by walking heap blocks recursively. 2. It works on cases where we're not sure if the value is allocated or not. For example: char *p = argc > 1 ? argv[1] : some_function(); It's safe to use UNLEAK(p) here, because it's not freeing any memory. In the case that we're pointing to argv here, the reachability checker will just ignore our bytes. 3. Likewise, it works even if the variable has _already_ been freed. We're just copying the pointer bytes. If the block has been freed, the leak-checker will skip over those bytes as uninteresting. 4. Because it's not actually freeing memory, you can UNLEAK() before we are finished accessing the variable. This is helpful in cases like this: char *p = some_function(); return another_function(p); Writing this with free() requires: int ret; char *p = some_function(); ret = another_function(p); free(p); return ret; But with unleak we can just write: char *p = some_function(); UNLEAK(p); return another_function(p); This patch adds the UNLEAK() macro and enables it automatically when Git is compiled with SANITIZE=leak. In normal builds it's a noop, so we pay no runtime cost. It also adds some UNLEAK() annotations to show off how the feature works. On top of other recent leak fixes, these are enough to get t0000 and t0001 to pass when compiled with LSAN. Note the case in commit.c which actually converts a strbuf_release() into an UNLEAK. This code was already non-leaky, but the free didn't do anything useful, since we're exiting. Converting it to an annotation means that non-leak-checking builds pay no runtime cost. The cost is minimal enough that it's probably not worth going on a crusade to convert these kinds of frees to UNLEAKS. I did it here for consistency with the "sb" leak (though it would have been equally correct to go the other way, and turn them both into strbuf_release() calls). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-23pack: move release_pack_memory()Jonathan Tan1-2/+0
The function unuse_one_window() needs to be temporarily made global. Its scope will be restored to static in a subsequent commit. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-23Merge branch 'rs/move-array' into maintJunio C Hamano1-0/+8
Code clean-up. * rs/move-array: ls-files: don't try to prune an empty index apply: use COPY_ARRAY and MOVE_ARRAY in update_image() use MOVE_ARRAY add MOVE_ARRAY
2017-08-11Merge branch 'rs/move-array'Junio C Hamano1-0/+8
Code clean-up. * rs/move-array: ls-files: don't try to prune an empty index apply: use COPY_ARRAY and MOVE_ARRAY in update_image() use MOVE_ARRAY add MOVE_ARRAY
2017-07-18Merge branch 'tb/push-to-cygwin-unc-path'Junio C Hamano1-0/+3
On Cygwin, similar to Windows, "git push //server/share/repository" ought to mean a repository on a network share that can be accessed locally, but this did not work correctly due to stripping the double slashes at the beginning. This may need to be heavily tested before it gets unleashed to the wild, as the change is at a fairly low-level code and would affect not just the code to decide if the push destination is local. There may be unexpected fallouts in the path normalization. * tb/push-to-cygwin-unc-path: cygwin: allow pushing to UNC paths
2017-07-17add MOVE_ARRAYRené Scharfe1-0/+8
Similar to COPY_ARRAY (introduced in 60566cbb58), add a safe and convenient helper for moving potentially overlapping ranges of array entries. It infers the element size, multiplies automatically and safely to get the size in bytes, does a basic type safety check by comparing element sizes and unlike memmove(3) it supports NULL pointers iff 0 elements are to be moved. Also add a semantic patch to demonstrate the helper's intended usage. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-05cygwin: allow pushing to UNC pathsTorsten Bögershausen1-0/+3
cygwin can use an UNC path like //server/share/repo $ cd //server/share/dir $ mkdir test $ cd test $ git init --bare However, when we try to push from a local Git repository to this repo, there is a problem: Git converts the leading "//" into a single "/". As cygwin handles an UNC path so well, Git can support them better: - Introduce cygwin_offset_1st_component() which keeps the leading "//", similar to what Git for Windows does. - Move CYGWIN out of the POSIX in the tests for path normalization in t0060 Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-15git-compat-util: add a FREE_AND_NULL() wrapper around free(ptr); ptr = NULLÆvar Arnfjörð Bjarmason1-0/+6
Add a FREE_AND_NULL() wrapper marco for the common pattern of freeing a pointer and assigning NULL to it right afterwards. The implementation is similar to the (currently unused) XDL_PTRFREE macro in xdiff/xmacros.h added in commit 3443546f6e ("Use a *real* built-in diff generator", 2006-03-24). The only difference is that free() is called unconditionally, see [1]. See [2] for a suggested alternative which does this via a function instead of a macro. As covered in replies to that message, while it's a viable approach, it would introduce caveats which this approach doesn't have, so that potential change is left to a future follow-up change. This merely allows us to translate exactly what we're doing now to a less verbose & idiomatic form using a macro, while guaranteeing that we don't introduce any functional changes. 1. <alpine.DEB.2.20.1608301948310.129229@virtualbox> (http://public-inbox.org/git/alpine.DEB.2.20.1608301948310.129229@virtualbox/) 2. <20170610032143.GA7880@starla> (https://public-inbox.org/git/20170610032143.GA7880@starla/) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-13Merge branch 'nd/fopen-errors'Junio C Hamano1-6/+9
We often try to open a file for reading whose existence is optional, and silently ignore errors from open/fopen; report such errors if they are not due to missing files. * nd/fopen-errors: mingw_fopen: report ENOENT for invalid file names mingw: verify that paths are not mistaken for remote nicknames log: fix memory leak in open_next_file() rerere.c: move error_errno() closer to the source system call print errno when reporting a system call error wrapper.c: make warn_on_inaccessible() static wrapper.c: add and use fopen_or_warn() wrapper.c: add and use warn_on_fopen_errors() config.mak.uname: set FREAD_READS_DIRECTORIES for Darwin, too config.mak.uname: set FREAD_READS_DIRECTORIES for Linux and FreeBSD clone: use xfopen() instead of fopen() use xfopen() in more places git_fopen: fix a sparse 'not declared' warning
2017-06-13Merge branch 'jc/noent-notdir'Junio C Hamano1-0/+15
Our code often opens a path to an optional file, to work on its contents when we can successfully open it. We can ignore a failure to open if such an optional file does not exist, but we do want to report a failure in opening for other reasons (e.g. we got an I/O error, or the file is there, but we lack the permission to open). The exact errors we need to ignore are ENOENT (obviously) and ENOTDIR (less obvious). Instead of repeating comparison of errno with these two constants, introduce a helper function to do so. * jc/noent-notdir: treewide: use is_missing_file_error() where ENOENT and ENOTDIR are checked compat-util: is_missing_file_error()
2017-06-13Merge branch 'bw/forking-and-threading' into maintJunio C Hamano1-1/+0
The "run-command" API implementation has been made more robust against dead-locking in a threaded environment. * bw/forking-and-threading: usage.c: drop set_error_handle() run-command: restrict PATH search to executable files run-command: expose is_executable function run-command: block signals between fork and execve run-command: add note about forking and threading run-command: handle dup2 and close errors in child run-command: eliminate calls to error handling functions in child run-command: don't die in child when duping /dev/null run-command: prepare child environment before forking string-list: add string_list_remove function run-command: use the async-signal-safe execv instead of execvp run-command: prepare command before forking t0061: run_command executes scripts without a #! line t5550: use write_script to generate post-update hook
2017-05-30Merge branch 'bw/forking-and-threading'Junio C Hamano1-1/+0
The "run-command" API implementation has been made more robust against dead-locking in a threaded environment. * bw/forking-and-threading: usage.c: drop set_error_handle() run-command: restrict PATH search to executable files run-command: expose is_executable function run-command: block signals between fork and execve run-command: add note about forking and threading run-command: handle dup2 and close errors in child run-command: eliminate calls to error handling functions in child run-command: don't die in child when duping /dev/null run-command: prepare child environment before forking string-list: add string_list_remove function run-command: use the async-signal-safe execv instead of execvp run-command: prepare command before forking t0061: run_command executes scripts without a #! line t5550: use write_script to generate post-update hook
2017-05-30compat-util: is_missing_file_error()Junio C Hamano1-0/+15
Our code often opens a path to an optional file, to work on its contents when we can successfully open it. We can ignore a failure to open if such an optional file does not exist, but we do want to report a failure in opening for other reasons (e.g. we got an I/O error, or the file is there, but we lack the permission to open). The exact errors we need to ignore are ENOENT (obviously) and ENOTDIR (less obvious). Instead of repeating comparison of errno with these two constants, introduce a helper function to do so. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-29Merge branch 'jk/bug-to-abort'Junio C Hamano1-0/+9
Introduce the BUG() macro to improve die("BUG: ..."). * jk/bug-to-abort: usage: add NORETURN to BUG() function definitions config: complain about --local outside of a git repo setup_git_env: convert die("BUG") to BUG() usage.c: add BUG() function
2017-05-26wrapper.c: make warn_on_inaccessible() staticNguyễn Thái Ngọc Duy1-2/+0
After the last patch, this function is not used outside anymore. Keep it static. Noticed-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26wrapper.c: add and use fopen_or_warn()Nguyễn Thái Ngọc Duy1-0/+1
When fopen() returns NULL, it could be because the given path does not exist, but it could also be some other errors and the caller has to check. Add a wrapper so we don't have to repeat the same error check everywhere. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26wrapper.c: add and use warn_on_fopen_errors()Nguyễn Thái Ngọc Duy1-0/+2
In many places, Git warns about an inaccessible file after a fopen() failed. To discern these cases from other cases where we want to warn about inaccessible files, introduce a new helper specifically to test whether fopen() failed because the current user lacks the permission to open file in question. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26git_fopen: fix a sparse 'not declared' warningRamsay Jones1-4/+6
If git is built with the FREAD_READS_DIRECTORIES build variable set, this would cause sparse to issue a 'not declared, should it be static?' warning on Linux. This is a result of the method employed by 'compat/fopen.c' to suppress the (possible) redefinition of the (system) fopen macro, which also removes the extern declaration of the git_fopen function. In order to suppress the warning, introduce a new macro to suppress the definition (or possibly the re-definition) of the fopen symbol as a macro override. This new macro (SUPPRESS_FOPEN_REDEFINITION) is only defined in 'compat/fopen.c', just prior to the inclusion of the 'git-compat-util.h' header file. Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-16Merge branch 'js/larger-timestamps'Junio C Hamano1-0/+5
Some platforms have ulong that is smaller than time_t, and our historical use of ulong for timestamp would mean they cannot represent some timestamp that the platform allows. Invent a separate and dedicated timestamp_t (so that we can distingiuish timestamps and a vanilla ulongs, which along is already a good move), and then declare uintmax_t is the type to be used as the timestamp_t. * js/larger-timestamps: archive-tar: fix a sparse 'constant too large' warning use uintmax_t for timestamps date.c: abort if the system time cannot handle one of our timestamps timestamp_t: a new data type for timestamps PRItime: introduce a new "printf format" for timestamps parse_timestamp(): specify explicitly where we parse timestamps t0006 & t5000: skip "far in the future" test when time_t is too limited t0006 & t5000: prepare for 64-bit timestamps ref-filter: avoid using `unsigned long` for catch-all data type
2017-05-16Merge branch 'dt/raise-core-packed-git-limit'Junio C Hamano1-1/+1
The default packed-git limit value has been raised on larger platforms to save "git fetch" from a (recoverable) failure while "gc" is running in parallel. * dt/raise-core-packed-git-limit: Increase core.packedGitLimit
2017-05-15usage.c: drop set_error_handle()Jeff King1-1/+0
The set_error_handle() function was introduced by 3b331e926 (vreportf: report to arbitrary filehandles, 2015-08-11) so that run-command could send post-fork, pre-exec errors to the parent's original stderr. That use went away in 79319b194 (run-command: eliminate calls to error handling functions in child, 2017-04-19), which pushes all of the error reporting to the parent. This leaves no callers of set_error_handle(). As we're not likely to add any new ones, let's drop it. Signed-off-by: Jeff King <peff@peff.net> Acked-by: Brandon Williams <bmwill@google.com> Reviewed-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-15usage.c: add BUG() functionJeff King1-0/+9
There's a convention in Git's code base to write assertions as: if (...some_bad_thing...) die("BUG: the terrible thing happened"); with the idea that users should never see a "BUG:" message (but if they, it at least gives a clue what happened). We use die() here because it's convenient, but there are a few draw-backs: 1. Without parsing the messages, it's hard for callers to distinguish BUG assertions from regular errors. For instance, it would be nice if the test suite could check that we don't hit any assertions, but test_must_fail will pass BUG deaths as OK. 2. It would be useful to add more debugging features to BUG assertions, like file/line numbers or dumping core. 3. The die() handler can be replaced, and might not actually exit the whole program (e.g., it may just pthread_exit()). This is convenient for normal errors, but for an assertion failure (which is supposed to never happen), we're probably better off taking down the whole process as quickly and cleanly as possible. We could address these by checking in die() whether the error message starts with "BUG", and behaving appropriately. But there's little advantage at that point to sharing the die() code, and only downsides (e.g., we can't change the BUG() interface independently). Moreover, converting all of the existing BUG calls reveals that the test suite does indeed trigger a few of them. Instead, this patch introduces a new BUG() function, which prints an error before dying via SIGABRT. This gives us test suite checking and core dumps. The function is actually a macro (when supported) so that we can show the file/line number. We can convert die("BUG") invocations to BUG() in further patches, dealing with any test fallouts individually. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-27use uintmax_t for timestampsJohannes Schindelin1-4/+4
Previously, we used `unsigned long` for timestamps. This was only a good choice on Linux, where we know implicitly that `unsigned long` is what is used for `time_t`. However, we want to use a different data type for timestamps for two reasons: - there is nothing that says that `unsigned long` should be the same data type as `time_t`, and indeed, on 64-bit Windows for example, it is not: `unsigned long` is 32-bit but `time_t` is 64-bit. - even on 32-bit Linux, where `unsigned long` (and thereby `time_t`) is 32-bit, we *want* to be able to encode timestamps in Git that are currently absurdly far in the future, *even if* the system library is not able to format those timestamps into date strings. So let's just switch to the maximal integer type available, which should be at least 64-bit for all practical purposes these days. It certainly cannot be worse than `unsigned long`, so... Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-27timestamp_t: a new data type for timestampsJohannes Schindelin1-0/+2
Git's source code assumes that unsigned long is at least as precise as time_t. Which is incorrect, and causes a lot of problems, in particular where unsigned long is only 32-bit (notably on Windows, even in 64-bit versions). So let's just use a more appropriate data type instead. In preparation for this, we introduce the new `timestamp_t` data type. By necessity, this is a very, very large patch, as it has to replace all timestamps' data type in one go. As we will use a data type that is not necessarily identical to `time_t`, we need to be very careful to use `time_t` whenever we interact with the system functions, and `timestamp_t` everywhere else. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-23Merge branch 'dt/xgethostname-nul-termination'Junio C Hamano1-0/+6
gethostname(2) may not NUL terminate the buffer if hostname does not fit; unfortunately there is no easy way to see if our buffer was too small, but at least this will make sure we will not end up using garbage past the end of the buffer. * dt/xgethostname-nul-termination: xgethostname: handle long hostnames use HOST_NAME_MAX to size buffers for gethostname(2)
2017-04-23PRItime: introduce a new "printf format" for timestampsJohannes Schindelin1-0/+1
Currently, Git's source code treats all timestamps as if they were unsigned longs. Therefore, it is okay to write "%lu" when printing them. There is a substantial problem with that, though: at least on Windows, time_t is *larger* than unsigned long, and hence we will want to switch away from the ill-specified `unsigned long` data type. So let's introduce the pseudo format "PRItime" (currently simply being defined to "lu") to make it easier to change the data type used for timestamps. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-23parse_timestamp(): specify explicitly where we parse timestampsJohannes Schindelin1-0/+2
Currently, Git's source code represents all timestamps as `unsigned long`. In preparation for using a more appropriate data type, let's introduce a symbol `parse_timestamp` (currently being defined to `strtoul`) where appropriate, so that we can later easily switch to, say, use `strtoull()` instead. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-20Increase core.packedGitLimitDavid Turner1-1/+1
When core.packedGitLimit is exceeded, git will close packs. If there is a repack operation going on in parallel with a fetch, the fetch might open a pack, and then be forced to close it due to packedGitLimit being hit. The repack could then delete the pack out from under the fetch, causing the fetch to fail. Increase core.packedGitLimit's default value to prevent this. On current 64-bit x86_64 machines, 48 bits of address space are available. It appears that 64-bit ARM machines have no standard amount of address space (that is, it varies by manufacturer), and IA64 and POWER machines have the full 64 bits. So 48 bits is the only limit that we can reasonably care about. We reserve a few bits of the 48-bit address space for the kernel's use (this is not strictly necessary, but it's better to be safe), and use up to the remaining 45. No git repository will be anywhere near this large any time soon, so this should prevent the failure. Helped-by: Jeff King <peff@peff.net> Signed-off-by: David Turner <dturner@twosigma.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-18xgethostname: handle long hostnamesDavid Turner1-0/+2
If the full hostname doesn't fit in the buffer supplied to gethostname, POSIX does not specify whether the buffer will be null-terminated, so to be safe, we should do it ourselves. Introduce new function, xgethostname, which ensures that there is always a \0 at the end of the buffer. Signed-off-by: David Turner <dturner@twosigma.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-18use HOST_NAME_MAX to size buffers for gethostname(2)René Scharfe1-0/+4
POSIX limits the length of host names to HOST_NAME_MAX. Export the fallback definition from daemon.c and use this constant to make all buffers used with gethostname(2) big enough for any possible result and a terminating NUL. Inspired-by: David Turner <dturner@twosigma.com> Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: David Turner <dturner@twosigma.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-28Merge branch 'jk/pack-name-cleanups' into maintJunio C Hamano1-2/+0
Code clean-up. * jk/pack-name-cleanups: index-pack: make pointer-alias fallbacks safer replace snprintf with odb_pack_name() odb_pack_keep(): stop generating keepfile name sha1_file.c: make pack-name helper globally accessible move odb_* declarations out of git-compat-util.h
2017-03-21Merge branch 'jk/pack-name-cleanups'Junio C Hamano1-2/+0
Code clean-up. * jk/pack-name-cleanups: index-pack: make pointer-alias fallbacks safer replace snprintf with odb_pack_name() odb_pack_keep(): stop generating keepfile name sha1_file.c: make pack-name helper globally accessible move odb_* declarations out of git-compat-util.h
2017-03-16move odb_* declarations out of git-compat-util.hJeff King1-2/+0
These functions were originally conceived as wrapper functions similar to xmkstemp(). They were later moved by 463db9b10 (wrapper: move odb_* to environment.c, 2010-11-06). The more appropriate place for a declaration is in cache.h. While we're at it, let's add some basic docstrings. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-28wrapper.c: remove unused gitmkstemps() functionRamsay Jones1-5/+0
The last call to the mkstemps() function was removed in commit 659488326 ("wrapper.c: delete dead function git_mkstemps()", 22-04-2016). In order to support platforms without mkstemps(), this functionality was provided, along with a Makefile build variable (NO_MKSTEMPS), by the gitmkstemps() function. Remove the dead code, along with the defunct build machinery. Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-15Merge branch 'rs/swap'Junio C Hamano1-0/+10
Code clean-up. * rs/swap: graph: use SWAP macro diff: use SWAP macro use SWAP macro apply: use SWAP macro add SWAP macro
2017-01-30add SWAP macroRené Scharfe1-0/+10
Add a macro for exchanging the values of variables. It allows users to avoid repetition and takes care of the temporary variable for them. It also makes sure that the storage sizes of its two parameters are the same. Its memcpy(1) calls are optimized away by current compilers. Also add a conservative semantic patch for transforming only swaps of variables of the same type. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-23add QSORT_SRené Scharfe1-0/+5
Add the macro QSORT_S, a convenient wrapper for qsort_s() that infers the size of the array elements and dies on error. Basically all possible errors are programming mistakes (passing NULL as base of a non-empty array, passing NULL as comparison function, out-of-bounds accesses), so terminating the program should be acceptable for most callers. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-23compat: add qsort_s()René Scharfe1-0/+6
The function qsort_s() was introduced with C11 Annex K; it provides the ability to pass a context pointer to the comparison function, supports the convention of using a NULL pointer for an empty array and performs a few safety checks. Add an implementation based on compat/qsort.c for platforms that lack a native standards-compliant qsort_s() (i.e. basically everyone). It doesn't perform the full range of possible checks: It uses size_t instead of rsize_t and doesn't check nmemb and size against RSIZE_MAX because we probably don't have the restricted size type defined. For the same reason it returns int instead of errno_t. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-05Merge branch 'jk/common-main' into maint-2.10Junio C Hamano1-2/+2
* jk/common-main: common-main: stop munging argv[0] path git-compat-util: move content inside ifdef/endif guards
2016-10-31Merge branch 'jk/common-main'Junio C Hamano1-2/+2
A trivial clean-up to a recently graduated topic. * jk/common-main: git-compat-util: move content inside ifdef/endif guards
2016-10-28Merge branch 'jk/tighten-alloc' into maintJunio C Hamano1-11/+6
Protect our code from over-eager compilers. * jk/tighten-alloc: inline xalloc_flex() into FLEXPTR_ALLOC_MEM avoid pointer arithmetic involving NULL in FLEX_ALLOC_MEM
2016-10-27git-compat-util: move content inside ifdef/endif guardsJeff King1-2/+2
Commit 3f2e2297b9 (add an extra level of indirection to main(), 2016-07-01) added a declaration to git-compat-util.h, but it was accidentally placed after the final #endif that guards against multiple inclusions. This doesn't have any actual impact on the code, since it's not incorrect to repeat a function declaration in C. But it's a bad habit, and makes it more likely for somebody else to make the same mistake. It also defeats gcc's optimization to avoid opening header files whose contents are completely guarded. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-26Merge branch 'jk/tighten-alloc'Junio C Hamano1-11/+6
Protect our code from over-eager compilers. * jk/tighten-alloc: inline xalloc_flex() into FLEXPTR_ALLOC_MEM avoid pointer arithmetic involving NULL in FLEX_ALLOC_MEM
2016-10-17inline xalloc_flex() into FLEXPTR_ALLOC_MEMRené Scharfe1-9/+3
Allocate and copy directly in FLEXPTR_ALLOC_MEM and remove the now unused helper function xalloc_flex(). The resulting code is shorter and the offset arithmetic is a bit simpler. Suggested-by: Jeff King <peff@peff.net> Signed-off-by: Rene Scharfe <l.s.r@web.de> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-17avoid pointer arithmetic involving NULL in FLEX_ALLOC_MEMRené Scharfe1-2/+3
Calculating offsets involving a NULL pointer is undefined. It works in practice (for now?), but we should not rely on it. Allocate first and then simply refer to the flexible array member by its name instead of performing pointer arithmetic up front. The resulting code is slightly shorter, easier to read and doesn't rely on undefined behaviour. NB: The cast to a (non-const) void pointer is necessary to keep support for flexible array members declared as const. Signed-off-by: Rene Scharfe <l.s.r@web.de> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-11Merge branch 'rs/copy-array' into maintJunio C Hamano1-0/+8
Code cleanup. * rs/copy-array: use COPY_ARRAY add COPY_ARRAY
2016-10-10Merge branch 'rs/qsort'Junio C Hamano1-0/+8
We call "qsort(array, nelem, sizeof(array[0]), fn)", and most of the time third parameter is redundant. A new QSORT() macro lets us omit it. * rs/qsort: show-branch: use QSORT use QSORT, part 2 coccicheck: use --all-includes by default remove unnecessary check before QSORT use QSORT add QSORT
2016-10-03Merge branch 'rs/copy-array'Junio C Hamano1-0/+8
Code cleanup. * rs/copy-array: use COPY_ARRAY add COPY_ARRAY
2016-09-29Merge branch 'js/regexec-buf' into maintJunio C Hamano1-0/+13
Some codepaths in "git diff" used regexec(3) on a buffer that was mmap(2)ed, which may not have a terminating NUL, leading to a read beyond the end of the mapped region. This was fixed by introducing a regexec_buf() helper that takes a <ptr,len> pair with REG_STARTEND extension. * js/regexec-buf: regex: use regexec_buf() regex: add regexec_buf() that can work on a non NUL-terminated string regex: -G<pattern> feeds a non NUL-terminated string to regexec() and fails
2016-09-29add QSORTRené Scharfe1-0/+8
Add the macro QSORT, a convenient wrapper for qsort(3) that infers the size of the array elements and supports the convention of initializing empty arrays with a NULL pointer, which we use in some places. Calling qsort(3) directly with a NULL pointer is undefined -- even with an element count of zero -- and allows the compiler to optimize away any following NULL checks. Using the macro avoids such surprises. Add a semantic patch as well to demonstrate the macro's usage and to automate the transformation of trivial cases. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-26Merge branch 'js/regexec-buf'Junio C Hamano1-0/+13
Some codepaths in "git diff" used regexec(3) on a buffer that was mmap(2)ed, which may not have a terminating NUL, leading to a read beyond the end of the mapped region. This was fixed by introducing a regexec_buf() helper that takes a <ptr,len> pair with REG_STARTEND extension. * js/regexec-buf: regex: use regexec_buf() regex: add regexec_buf() that can work on a non NUL-terminated string regex: -G<pattern> feeds a non NUL-terminated string to regexec() and fails
2016-09-25add COPY_ARRAYRené Scharfe1-0/+8
Add COPY_ARRAY, a safe and convenient helper for copying arrays, complementing ALLOC_ARRAY and REALLOC_ARRAY. Users just specify source, destination and the number of elements; the size of an element is inferred automatically. It checks if the multiplication of size and element count overflows. The inferred size is passed first to st_mult, which allows the division there to be done at compilation time. As a basic type safety check it makes sure the sizes of source and destination elements are the same. That's evaluated at compilation time as well. COPY_ARRAY is safe to use with NULL as source pointer iff 0 elements are to be copied. That convention is used in some cases for initializing arrays. Raw memcpy(3) does not support it -- compilers are allowed to assume that only valid pointers are passed to it and can optimize away NULL checks after such a call. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-21regex: add regexec_buf() that can work on a non NUL-terminated stringJohannes Schindelin1-0/+13
We just introduced a test that demonstrates that our sloppy use of regexec() on a mmap()ed area can result in incorrect results or even hard crashes. So what we need to fix this is a function that calls regexec() on a length-delimited, rather than a NUL-terminated, string. Happily, there is an extension to regexec() introduced by the NetBSD project and present in all major regex implementation including Linux', MacOSX' and the one Git includes in compat/regex/: by using the (non-POSIX) REG_STARTEND flag, it is possible to tell the regexec() function that it should only look at the offsets between pmatch[0].rm_so and pmatch[0].rm_eo. That is exactly what we need. Since support for REG_STARTEND is so widespread by now, let's just introduce a helper function that always uses it, and tell people on a platform whose regex library does not support it to use the one from our compat/regex/ directory. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-19Merge branch 'rs/compat-strdup' into maintJunio C Hamano1-0/+8
Code cleanup. * rs/compat-strdup: compat: move strdup(3) replacement to its own file
2016-09-19Merge branch 'jk/squelch-false-warning-from-gcc-o3' into maintJunio C Hamano1-0/+1
Compilation fix. * jk/squelch-false-warning-from-gcc-o3: color_parse_mem: initialize "struct color" temporary error_errno: use constant return similar to error()
2016-09-19Merge branch 'cc/apply-am'Junio C Hamano1-0/+3
"git am" has been taught to make an internal call to "git apply"'s innards without spawning the latter as a separate process. * cc/apply-am: (41 commits) builtin/am: use apply API in run_apply() apply: learn to use a different index file apply: pass apply state to build_fake_ancestor() apply: refactor `git apply` option parsing apply: change error_routine when silent usage: add get_error_routine() and get_warn_routine() usage: add set_warn_routine() apply: don't print on stdout in verbosity_silent mode apply: make it possible to silently apply apply: use error_errno() where possible apply: make some parsing functions static again apply: move libified code from builtin/apply.c to apply.{c,h} apply: rename and move opt constants to apply.h builtin/apply: rename option parsing functions builtin/apply: make create_one_file() return -1 on error builtin/apply: make try_create_file() return -1 on error builtin/apply: make write_out_results() return -1 on error builtin/apply: make write_out_one_result() return -1 on error builtin/apply: make create_file() return -1 on error builtin/apply: make add_index_file() return -1 on error ...
2016-09-12Merge branch 'rs/compat-strdup'Junio C Hamano1-0/+8
* rs/compat-strdup: compat: move strdup(3) replacement to its own file
2016-09-12Merge branch 'jk/squelch-false-warning-from-gcc-o3'Junio C Hamano1-0/+1
* jk/squelch-false-warning-from-gcc-o3: color_parse_mem: initialize "struct color" temporary error_errno: use constant return similar to error()
2016-09-07usage: add get_error_routine() and get_warn_routine()Christian Couder1-0/+2
Let's make it possible to get the current error_routine and warn_routine, so that we can store them before using set_error_routine() or set_warn_routine() to use new ones. This way we will be able put back the original routines, when we are done with using new ones. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-07usage: add set_warn_routine()Christian Couder1-0/+1
There are already set_die_routine() and set_error_routine(), so let's add set_warn_routine() as this will be needed in a following commit. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-07compat: move strdup(3) replacement to its own fileRené Scharfe1-0/+8
Move our implementation of strdup(3) out of compat/nedmalloc/ and allow it to be used independently from USE_NED_ALLOCATOR. The original nedmalloc doesn't come with strdup() and doesn't need it. Only _users_ of nedmalloc need it, which was added when we imported it to our compat/ hierarchy. This reduces the difference of our copy of nedmalloc from the original, making it easier to update, and allows for easier testing and reusing of our version of strdup(). Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-31error_errno: use constant return similar to error()Jeff King1-0/+1
Commit e208f9c (make error()'s constant return value more visible, 2012-12-15) introduced some macro trickery to make the constant return from error() more visible to callers, which in turn can help gcc produce better warnings (and possibly even better code). Later, fd1d672 (usage.c: add warning_errno() and error_errno(), 2016-05-08) introduced another variant, and subsequent commits converted some uses of error() to error_errno(), losing the magic from e208f9c for those sites. As a result, compiling vcs-svn/svndiff.c with "gcc -O3" produces -Wmaybe-uninitialized false positives (at least with gcc 6.2.0). Let's give error_errno() the same treatment, which silences these warnings. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-25Merge branch 'bw/mingw-avoid-inheriting-fd-to-lockfile'Junio C Hamano1-0/+4
The tempfile (hence its user lockfile) API lets the caller to open a file descriptor to a temporary file, write into it and then finalize it by first closing the filehandle and then either removing or renaming the temporary file. When the process spawns a subprocess after obtaining the file descriptor, and if the subprocess has not exited when the attempt to remove or rename is made, the last step fails on Windows, because the subprocess has the file descriptor still open. Open tempfile with O_CLOEXEC flag to avoid this (on Windows, this is mapped to O_NOINHERIT). * bw/mingw-avoid-inheriting-fd-to-lockfile: mingw: ensure temporary file handles are not inherited by child processes t6026-merge-attr: child processes must not inherit index.lock handles
2016-08-23mingw: ensure temporary file handles are not inherited by child processesBen Wijen1-0/+4
When the index is locked and child processes inherit the handle to said lock and the parent process wants to remove the lock before the child process exits, on Windows there is a problem: it won't work because files cannot be deleted if a process holds a handle on them. The symptom: Rename from 'xxx/.git/index.lock' to 'xxx/.git/index' failed. Should I try again? (y/n) Spawning child processes with bInheritHandles==FALSE would not work because no file handles would be inherited, not even the hStdXxx handles in STARTUPINFO (stdin/stdout/stderr). Opening every file with O_NOINHERIT does not work, either, as e.g. git-upload-pack expects inherited file handles. This leaves us with the only way out: creating temp files with the O_NOINHERIT flag. This flag is Windows-specific, however. For our purposes, it is equivalent to O_CLOEXEC (which does not exist on Windows), so let's just open temporary files with the O_CLOEXEC flag and map that flag to O_NOINHERIT on Windows. As Eric Wong pointed out, we need to be careful to handle the case where the Linux headers used to compile Git support O_CLOEXEC but the Linux kernel used to run Git does not: it returns an EINVAL. This fixes the test that we just introduced to demonstrate the problem. Signed-off-by: Ben Wijen <ben@wijen.net> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-17Merge branch 'jk/tighten-alloc'Junio C Hamano1-1/+1
Small code and comment clean-up. * jk/tighten-alloc: receive-pack: use FLEX_ALLOC_MEM in queue_command() correct FLEXPTR_* example in comment
2016-08-13correct FLEXPTR_* example in commentRené Scharfe1-1/+1
This section is about "The FLEXPTR_* variants", so use FLEXPTR_ALLOC_STR in the example. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-19Merge branch 'jk/common-main'Junio C Hamano1-0/+2
There are certain house-keeping tasks that need to be performed at the very beginning of any Git program, and programs that are not built-in commands had to do them exactly the same way as "git" potty does. It was easy to make mistakes in one-off standalone programs (like test helpers). A common "main()" function that calls cmd_main() of individual program has been introduced to make it harder to make mistakes. * jk/common-main: mingw: declare main()'s argv as const common-main: call git_setup_gettext() common-main: call restore_sigpipe_to_default() common-main: call sanitize_stdfds() common-main: call git_extract_argv0_path() add an extra level of indirection to main()
2016-07-11Merge branch 'jk/ansi-color'Junio C Hamano1-0/+17
The output coloring scheme learned two new attributes, italic and strike, in addition to existing bold, reverse, etc. * jk/ansi-color: color: support strike-through attribute color: support "italic" attribute color: allow "no-" for negating attributes color: refactor parse_attr add skip_prefix_mem helper doc: refactor description of color format color: fix max-size comment
2016-07-06Merge branch 'jk/common-main-2.8' into jk/common-mainJunio C Hamano1-0/+2
* jk/common-main-2.8: mingw: declare main()'s argv as const common-main: call git_setup_gettext() common-main: call restore_sigpipe_to_default() common-main: call sanitize_stdfds() common-main: call git_extract_argv0_path() add an extra level of indirection to main()
2016-07-01add an extra level of indirection to main()Jeff King1-0/+2
There are certain startup tasks that we expect every git process to do. In some cases this is just to improve the quality of the program (e.g., setting up gettext()). In others it is a requirement for using certain functions in libgit.a (e.g., system_path() expects that you have called git_extract_argv0_path()). Most commands are builtins and are covered by the git.c version of main(). However, there are still a few external commands that use their own main(). Each of these has to remember to include the correct startup sequence, and we are not always consistent. Rather than just fix the inconsistencies, let's make this harder to get wrong by providing a common main() that can run this standard startup. We basically have two options to do this: - the compat/mingw.h file already does something like this by adding a #define that replaces the definition of main with a wrapper that calls mingw_startup(). The upside is that the code in each program doesn't need to be changed at all; it's rewritten on the fly by the preprocessor. The downside is that it may make debugging of the startup sequence a bit more confusing, as the preprocessor is quietly inserting new code. - the builtin functions are all of the form cmd_foo(), and git.c's main() calls them. This is much more explicit, which may make things more obvious to somebody reading the code. It's also more flexible (because of course we have to figure out _which_ cmd_foo() to call). The downside is that each of the builtins must define cmd_foo(), instead of just main(). This patch chooses the latter option, preferring the more explicit approach, even though it is more invasive. We introduce a new file common-main.c, with the "real" main. It expects to call cmd_main() from whatever other objects it is linked against. We link common-main.o against anything that links against libgit.a, since we know that such programs will need to do this setup. Note that common-main.o can't actually go inside libgit.a, as the linker would not pick up its main() function automatically (it has no callers). The rest of the patch is just adjusting all of the various external programs (mostly in t/helper) to use cmd_main(). I've provided a global declaration for cmd_main(), which means that all of the programs also need to match its signature. In particular, many functions need to switch to "const char **" instead of "char **" for argv. This effect ripples out to a few other variables and functions, as well. This makes the patch even more invasive, but the end result is much better. We should be treating argv strings as const anyway, and now all programs conform to the same signature (which also matches the way builtins are defined). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-23add skip_prefix_mem helperJeff King1-0/+17
The skip_prefix function has been very useful for simplifying pointer arithmetic and avoiding repeated magic numbers, but we have no equivalent for length-limited buffers. So we're stuck with: if (3 <= len && skip_prefix(buf, "foo", &buf)) len -= 3; That's not that complicated, but it needs to use magic numbers for the length of the prefix (or else write out strlen("foo"), repeating the string). By using a helper, we can get the string length behind the scenes (and often at compile time for string literals). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-17Merge branch 'nd/error-errno'Junio C Hamano1-0/+2
The code for warning_errno/die_errno has been refactored and a new error_errno() reporting helper is introduced. * nd/error-errno: (41 commits) wrapper.c: use warning_errno() vcs-svn: use error_errno() upload-pack.c: use error_errno() unpack-trees.c: use error_errno() transport-helper.c: use error_errno() sha1_file.c: use {error,die,warning}_errno() server-info.c: use error_errno() sequencer.c: use error_errno() run-command.c: use error_errno() rerere.c: use error_errno() and warning_errno() reachable.c: use error_errno() mailmap.c: use error_errno() ident.c: use warning_errno() http.c: use error_errno() and warning_errno() grep.c: use error_errno() gpg-interface.c: use error_errno() fast-import.c: use error_errno() entry.c: use error_errno() editor.c: use error_errno() diff-no-index.c: use error_errno() ...
2016-05-09usage.c: add warning_errno() and error_errno()Nguyễn Thái Ngọc Duy1-0/+2
Similar to die_errno(), these functions will append strerror() automatically. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-06Merge branch 'ky/imap-send-openssl-1.1.0' into maintJunio C Hamano1-3/+0
Upcoming OpenSSL 1.1.0 will break compilation b updating a few APIs we use in imap-send, which has been adjusted for the change. * ky/imap-send-openssl-1.1.0: configure: remove checking for HMAC_CTX_cleanup imap-send: avoid deprecated TLSv1_method() imap-send: check NULL return of SSL_CTX_new() imap-send: use HMAC() function provided by OpenSSL
2016-04-22Merge branch 'ky/imap-send-openssl-1.1.0'Junio C Hamano1-3/+0
Upcoming OpenSSL 1.1.0 will break compilation b updating a few APIs we use in imap-send, which has been adjusted for the change. * ky/imap-send-openssl-1.1.0: configure: remove checking for HMAC_CTX_cleanup imap-send: avoid deprecated TLSv1_method() imap-send: check NULL return of SSL_CTX_new() imap-send: use HMAC() function provided by OpenSSL
2016-04-08configure: remove checking for HMAC_CTX_cleanupKazuki Yamaguchi1-3/+0
We don't need it, as we no longer use HMAC_CTX_cleanup() directly. Signed-off-by: Kazuki Yamaguchi <k@rhe.jp> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-21Merge branch 'es/st-add4-gcc-4.2-workaround' into maintJunio C Hamano1-2/+2
* es/st-add4-gcc-4.2-workaround: git-compat-util: st_add4: work around gcc 4.2.x compiler crash
2016-03-21git-compat-util: st_add4: work around gcc 4.2.x compiler crashEric Sunshine1-2/+2
Although changes by 5b442c4 (tree-diff: catch integer overflow in combine_diff_path allocation, 2016-02-19) are perfectly valid, they unfortunately trigger an internal compiler error in gcc 4.2.x: combine-diff.c: In function 'diff_tree_combined': combine-diff.c:1391: internal compiler error: Segmentation fault: 11 Experimentation reveals that changing st_add4()'s argument evaluation order is sufficient to sidestep this problem. Although st_add3() does not trigger the compiler bug, for style consistency, change its argument evaluation order to match. Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-17Merge branch 'maint-2.5' into maint-2.6Junio C Hamano1-0/+34
* maint-2.5: Git 2.5.5 Git 2.4.11 list-objects: pass full pathname to callbacks list-objects: drop name_path entirely list-objects: convert name_path to a strbuf show_object_with_name: simplify by using path_name() http-push: stop using name_path tree-diff: catch integer overflow in combine_diff_path allocation add helpers for detecting size_t overflow
2016-03-17Merge branch 'maint-2.4' into maint-2.5Junio C Hamano1-0/+34
* maint-2.4: Git 2.4.11 list-objects: pass full pathname to callbacks list-objects: drop name_path entirely list-objects: convert name_path to a strbuf show_object_with_name: simplify by using path_name() http-push: stop using name_path tree-diff: catch integer overflow in combine_diff_path allocation add helpers for detecting size_t overflow
2016-03-16add helpers for detecting size_t overflowJeff King1-0/+34
Performing computations on size_t variables that we feed to xmalloc and friends can be dangerous, as an integer overflow can cause us to allocate a much smaller chunk than we realized. We already have unsigned_add_overflows(), but let's add unsigned_mult_overflows() to that. Furthermore, rather than have each site manually check and die on overflow, we can provide some helpers that will: - promote the arguments to size_t, so that we know we are doing our computation in the same size of integer that will ultimately be fed to xmalloc - check and die on overflow - return the result so that computations can be done in the parameter list of xmalloc. These functions are a lot uglier to use than normal arithmetic operators (you have to do "st_add(foo, bar)" instead of "foo + bar"). To at least limit the damage, we also provide multi-valued versions. So rather than: st_add(st_add(a, b), st_add(c, d)); you can write: st_add4(a, b, c, d); This isn't nearly as elegant as a varargs function, but it's a lot harder to get it wrong. You don't have to remember to add a sentinel value at the end, and the compiler will complain if you get the number of arguments wrong. This patch adds only the numbered variants required to convert the current code base; we can easily add more later if needed. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-10Merge branch 'ak/git-strip-extension-from-dashed-command' into maintJunio C Hamano1-4/+0
Code simplification. * ak/git-strip-extension-from-dashed-command: git.c: simplify stripping extension of a file in handle_builtin()
2016-02-22git-compat-util: drop mempcpy compat codeJeff King1-9/+0
There are no callers of this left, as the last one was dropped in the previous patch. And there are not likely to be new ones, as the function has been around since 2010 without gaining any new callers. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-22add helpers for allocating flex-array structsJeff King1-0/+62
Allocating a struct with a flex array is pretty simple in practice: you over-allocate the struct, then copy some data into the over-allocation. But it can be a slight pain to make sure you're allocating and copying the right amounts. This patch adds a few helpers to turn simple cases of flex-array struct allocation into a one-liner that properly checks for overflow. See the embedded documentation for details. Ideally we could provide a more flexible version that could handle multiple strings, like: FLEX_ALLOC_FMT(ref, name, "%s%s", prefix, name); But we have to implement this as a macro (because of the offset calculation of the flex member), which means we would need all compilers to support variadic macros. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-22harden REALLOC_ARRAY and xcalloc against size_t overflowJeff King1-1/+2
REALLOC_ARRAY inherently involves a multiplication which can overflow size_t, resulting in a much smaller buffer than we think we've allocated. We can easily harden it by using st_mult() to check for overflow. Likewise, we can add ALLOC_ARRAY to do the same thing for xmalloc calls. xcalloc() should already be fine, because it takes the two factors separately, assuming the system calloc actually checks for overflow. However, before we even hit the system calloc(), we do our memory_limit_check, which involves a multiplication. Let's check for overflow ourselves so that this limit cannot be bypassed. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-21git.c: simplify stripping extension of a file in handle_builtin()Alexander Kuleshov1-4/+0
The handle_builtin() starts from stripping of command extension if STRIP_EXTENSION is enabled. Actually STRIP_EXTENSION does not used anywhere else. This patch introduces strip_extension() helper to strip STRIP_EXTENSION extension from argv[0] with the strip_suffix() instead of manually stripping. Signed-off-by: Alexander Kuleshov <kuleshovmail@gmail.com> Helped-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-19add helpers for detecting size_t overflowJeff King1-0/+34
Performing computations on size_t variables that we feed to xmalloc and friends can be dangerous, as an integer overflow can cause us to allocate a much smaller chunk than we realized. We already have unsigned_add_overflows(), but let's add unsigned_mult_overflows() to that. Furthermore, rather than have each site manually check and die on overflow, we can provide some helpers that will: - promote the arguments to size_t, so that we know we are doing our computation in the same size of integer that will ultimately be fed to xmalloc - check and die on overflow - return the result so that computations can be done in the parameter list of xmalloc. These functions are a lot uglier to use than normal arithmetic operators (you have to do "st_add(foo, bar)" instead of "foo + bar"). To at least limit the damage, we also provide multi-valued versions. So rather than: st_add(st_add(a, b), st_add(c, d)); you can write: st_add4(a, b, c, d); This isn't nearly as elegant as a varargs function, but it's a lot harder to get it wrong. You don't have to remember to add a sentinel value at the end, and the compiler will complain if you get the number of arguments wrong. This patch adds only the numbered variants required to convert the current code base; we can easily add more later if needed. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-05Merge branch 'js/dirname-basename' into maintJunio C Hamano1-0/+10
dirname() emulation has been added, as Msys2 lacks it. * js/dirname-basename: mingw: avoid linking to the C library's isalpha() t0060: loosen overly strict expectations t0060: verify that basename() and dirname() work as expected compat/basename.c: provide a dirname() compatibility function compat/basename: make basename() conform to POSIX Refactor skipping DOS drive prefixes
2016-02-05Merge branch 'js/fopen-harder' into maintJunio C Hamano1-0/+1
Some codepaths used fopen(3) when opening a fixed path in $GIT_DIR (e.g. COMMIT_EDITMSG) that is meant to be left after the command is done. This however did not work well if the repository is set to be shared with core.sharedRepository and the umask of the previous user is tighter. They have been made to work better by calling unlink(2) and retrying after fopen(3) fails with EPERM. * js/fopen-harder: Handle more file writes correctly in shared repos commit: allow editing the commit message even in shared repos
2016-01-12compat/basename.c: provide a dirname() compatibility functionJohannes Schindelin1-0/+2
When there is no `libgen.h` to our disposal, we miss the `dirname()` function. Earlier we added basename() compatibility function for the same reason at e1c06886 (compat: add a basename() compatibility function, 2009-05-31). So far, we only had one user of that function: credential-cache--daemon (which was only compiled when Unix sockets are available, anyway). But now we also have `builtin/am.c` as user, so we need it. Since `dirname()` is a sibling of `basename()`, we simply put our very own `gitdirname()` implementation next to `gitbasename()` and use it if `NO_LIBGEN_H` has been set. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-12Refactor skipping DOS drive prefixesJohannes Schindelin1-0/+8
Junio noticed that there is an implicit assumption in pretty much all the code calling has_dos_drive_prefix(): it forces all of its callsites to hardcode the knowledge that the DOS drive prefix is always two bytes long. While this assumption is pretty safe, we can still make the code more readable and less error-prone by introducing a function that skips the DOS drive prefix safely. While at it, we change the has_dos_drive_prefix() return value: it now returns the number of bytes to be skipped if there is a DOS drive prefix. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-07commit: allow editing the commit message even in shared reposJohannes Schindelin1-0/+1
It was pointed out by Yaroslav Halchenko that the file containing the commit message is writable only by the owner, which means that we have to rewrite it from scratch in a shared repository. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-21Merge branch 'jk/ident-loosen-getpwuid'Junio C Hamano1-3/+0
When getpwuid() on the system returned NULL (e.g. the user is not in the /etc/passwd file or other uid-to-name mappings), the codepath to find who the user is to record it in the reflog barfed and died. Loosen the check in this codepath, which already accepts questionable ident string (e.g. host part of the e-mail address is obviously bogus), and in general when we operate fmt_ident() function in non-strict mode. * jk/ident-loosen-getpwuid: ident: loosen getpwuid error in non-strict mode ident: keep a flag for bogus default_email ident: make xgetpwuid_self() a static local helper
2015-12-10ident: make xgetpwuid_self() a static local helperJeff King1-3/+0
This function is defined in wrapper.c, but nobody besides ident.c uses it. And nobody is likely to in the future, either, as anything that cares about the user's name should be going through the ident code. Moving it here is a cleanup of the global namespace, but it will also enable further cleanups inside ident.c. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-11-05Merge branch 'js/misc-fixes' into maintJunio C Hamano1-1/+5
Various compilation fixes and squelching of warnings. * js/misc-fixes: Correct fscanf formatting string for I64u values Silence GCC's "cast of pointer to integer of a different size" warning Squelch warning about an integer overflow
2015-10-30Merge branch 'js/misc-fixes'Junio C Hamano1-1/+5
Various compilation fixes and squelching of warnings. * js/misc-fixes: Correct fscanf formatting string for I64u values Silence GCC's "cast of pointer to integer of a different size" warning Squelch warning about an integer overflow
2015-10-26Correct fscanf formatting string for I64u valuesWaldek Maleska1-0/+4
This fix is probably purely cosmetic because PRIuMAX is likely identical to SCNuMAX. Nevertheless, when using a function of the scanf() family, the correct interpolation to use is the latter, not the former. Signed-off-by: Waldek Maleska <w.maleska@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-10-26Squelch warning about an integer overflowJohannes Schindelin1-1/+1
We cannot rely on long integers to have more than 32 bits. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-10-20Merge branch 'jk/war-on-sprintf'Junio C Hamano1-1/+4
Many allocations that is manually counted (correctly) that are followed by strcpy/sprintf have been replaced with a less error prone constructs such as xstrfmt. Macintosh-specific breakage was noticed and corrected in this reroll. * jk/war-on-sprintf: (70 commits) name-rev: use strip_suffix to avoid magic numbers use strbuf_complete to conditionally append slash fsck: use for_each_loose_file_in_objdir Makefile: drop D_INO_IN_DIRENT build knob fsck: drop inode-sorting code convert strncpy to memcpy notes: document length of fanout path with a constant color: add color_set helper for copying raw colors prefer memcpy to strcpy help: clean up kfmclient munging receive-pack: simplify keep_arg computation avoid sprintf and strcpy with flex arrays use alloc_ref rather than hand-allocating "struct ref" color: add overflow checks for parsing colors drop strcpy in favor of raw sha1_to_hex use sha1_to_hex_r() instead of strcpy daemon: use cld->env_array when re-spawning stat_tracking_info: convert to argv_array http-push: use an argv_array for setup_revisions fetch-pack: use argv_array for index-pack / unpack-objects ...
2015-10-05Merge branch 'kn/for-each-tag'Junio C Hamano1-0/+3
The "ref-filter" code was taught about many parts of what "tag -l" does and then "tag -l" is being reimplemented in terms of "ref-filter". * kn/for-each-tag: tag.c: implement '--merged' and '--no-merged' options tag.c: implement '--format' option tag.c: use 'ref-filter' APIs tag.c: use 'ref-filter' data structures ref-filter: add option to match literal pattern ref-filter: add support to sort by version ref-filter: add support for %(contents:lines=X) ref-filter: add option to filter out tags, branches and remotes ref-filter: implement an `align` atom ref-filter: introduce match_atom_name() ref-filter: introduce handler function for each atom utf8: add function to align a string into given strbuf ref-filter: introduce ref_formatting_state and ref_formatting_stack ref-filter: move `struct atom_value` to ref-filter.c strtoul_ui: reject negative values
2015-10-05probe_utf8_pathname_composition: use internal strbufJeff King1-1/+1
When we are initializing a .git directory, we may call probe_utf8_pathname_composition to detect utf8 mangling. We pass in a path buffer for it to use, and it blindly strcpy()s into it, not knowing whether the buffer is large enough to hold the result or not. In practice this isn't a big deal, because the buffer we pass in already contains "$GIT_DIR/config", and we append only a few extra bytes to it. But we can easily do the right thing just by calling git_path_buf ourselves. Technically this results in a different pathname (before we appended our utf8 characters to the "config" path, and now they get their own files in $GIT_DIR), but that should not matter for our purposes. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25add xsnprintf helper functionJeff King1-0/+3
There are a number of places in the code where we call sprintf(), with the assumption that the output will fit into the buffer. In many cases this is true (e.g., formatting a number into a large buffer), but it is hard to tell immediately from looking at the code. It would be nice if we had some run-time check to make sure that our assumption is correct (and to communicate to readers of the code that we are not blindly calling sprintf, but have actually thought about this case). This patch introduces xsnprintf, which behaves just like snprintf, except that it dies whenever the output is truncated. This acts as a sort of assert() for these cases, which can help find places where the assumption is violated (as opposed to truncating and proceeding, which may just silently give a wrong answer). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-17strtoul_ui: reject negative valuesMatthieu Moy1-0/+3
strtoul_ui uses strtoul to get a long unsigned, then checks that casting to unsigned does not lose information and return the casted value. On 64 bits architecture, checking that the cast does not change the value catches most errors, but when sizeof(int) == sizeof(long) (e.g. i386), the check does nothing. Unfortunately, strtoul silently accepts negative values, and as a result strtoul_ui("-1", ...) raised no error. This patch catches negative values before it's too late, i.e. before calling strtoul. Reported-by: Max Kirillov <max@max630.net> Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-25Merge branch 'jk/long-error-messages'Junio C Hamano1-1/+1
The codepath to produce error messages had a hard-coded limit to the size of the message, primarily to avoid memory allocation while calling die(). * jk/long-error-messages: vreportf: avoid intermediate buffer vreportf: report to arbitrary filehandles
2015-08-11vreportf: report to arbitrary filehandlesJeff King1-1/+1
The vreportf function always goes to stderr, but run-command wants child errors to go to the parent's original stderr. To solve this, commit a5487dd duplicates the stderr fd and installs die and error handlers to direct the output appropriately (which later turned into the vwritef function). This has two downsides, though: - we make multiple calls to write(), which contradicts the "write at once" logic from d048a96 (print warning/error/fatal messages in one shot, 2007-11-09). - the custom handlers basically duplicate the normal handlers. They're only a few lines of code, but we should not have to repeat the magic "exit(128)", for example. We can solve the first by using fdopen() on the duplicated descriptor. We can't pass this to vreportf, but we could introduce a new vreportf_to to handle it. However, to fix the second problem, we instead introduce a new "set_error_handle" function, which lets the normal vreportf calls output to a handle besides stderr. Thus we can get rid of our custom handlers entirely, and just ask the regular handlers to output to our new descriptor. And as vwritef has no more callers, it can just go away. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-04wrapper: implement xfopen()Paul Tan1-0/+1
A common usage pattern of fopen() is to check if it succeeded, and die() if it failed: FILE *fp = fopen(path, "w"); if (!fp) die_errno(_("could not open '%s' for writing"), path); Implement a wrapper function xfopen() for the above, so that we can save a few lines of code and make the die() messages consistent. Helped-by: Jeff King <peff@peff.net> Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Paul Tan <pyokagan@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-04wrapper: implement xopen()Paul Tan1-0/+1
A common usage pattern of open() is to check if it was successful, and die() if it was not: int fd = open(path, O_WRONLY | O_CREAT, 0777); if (fd < 0) die_errno(_("Could not open '%s' for writing."), path); Implement a wrapper function xopen() that does the above so that we can save a few lines of code, and make the die() messages consistent. Helped-by: Torsten Bögershausen <tboegi@web.de> Helped-by: Jeff King <peff@peff.net> Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Paul Tan <pyokagan@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-06-25Merge branch 'cb/array-size' into maintJunio C Hamano1-5/+3
* cb/array-size: Fix definition of ARRAY_SIZE for non-gcc builds
2015-06-25Merge branch 'cb/array-size'Junio C Hamano1-5/+3
* cb/array-size: Fix definition of ARRAY_SIZE for non-gcc builds
2015-06-25Merge branch 'jk/diagnose-config-mmap-failure' into maintJunio C Hamano1-0/+1
The configuration reader/writer uses mmap(2) interface to access the files; when we find a directory, it barfed with "Out of memory?". * jk/diagnose-config-mmap-failure: xmmap(): drop "Out of memory?" config.c: rewrite ENODEV into EISDIR when mmap fails config.c: avoid xmmap error messages config.c: fix mmap leak when writing config read-cache.c: drop PROT_WRITE from mmap of index
2015-06-24Fix definition of ARRAY_SIZE for non-gcc buildsCharles Bailey1-5/+3
The improved ARRAY_SIZE macro uses BARF_UNLESS_AN_ARRAY which expands to a valid check for recent gcc versions and to 0 for older gcc versions but is not defined on non-gcc builds. Non-gcc builds need this macro to expand to 0 as well. The current outer test (defined(__GNUC__) && (__GNUC__ >= 3)) is a strictly weaker condition than the inner test (GIT_GNUC_PREREQ(3, 1)) so we can omit the outer test and cause the BARF_UNLESS_AN_ARRAY macro to be defined correctly on non-gcc builds as well as gcc builds with older versions. Signed-off-by: Charles Bailey <cbailey32@bloomberg.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-06-16Merge branch 'ep/do-not-feed-a-pointer-to-array-size' into maintJunio C Hamano1-1/+53
Catch a programmer mistake to feed a pointer not an array to ARRAY_SIZE() macro, by using a couple of GCC extensions. * ep/do-not-feed-a-pointer-to-array-size: git-compat-util.h: implement a different ARRAY_SIZE macro for for safely deriving the size of array
2015-06-11Merge branch 'jk/diagnose-config-mmap-failure'Junio C Hamano1-0/+1
The configuration reader/writer uses mmap(2) interface to access the files; when we find a directory, it barfed with "Out of memory?". * jk/diagnose-config-mmap-failure: xmmap(): drop "Out of memory?" config.c: rewrite ENODEV into EISDIR when mmap fails config.c: avoid xmmap error messages config.c: fix mmap leak when writing config read-cache.c: drop PROT_WRITE from mmap of index
2015-05-28config.c: avoid xmmap error messagesJeff King1-0/+1
The config-writing code uses xmmap to map the existing config file, which will die if the map fails. This has two downsides: 1. The error message is not very helpful, as it lacks any context about the file we are mapping: $ mkdir foo $ git config --file=foo some.key value fatal: Out of memory? mmap failed: No such device 2. We normally do not die in this code path; instead, we'd rather report the error and return an appropriate exit status (which is part of the public interface documented in git-config.1). This patch introduces a "gentle" form of xmmap which lets us produce our own error message. We do not want to use mmap directly, because we would like to use the other compatibility elements of xmmap (e.g., handling 0-length maps portably). The end result is: $ git.compile config --file=foo some.key value error: unable to mmap 'foo': No such device $ echo $? 3 Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-05-26Merge branch 'nd/untracked-cache'Junio C Hamano1-0/+1
Teach the index to optionally remember already seen untracked files to speed up "git status" in a working tree with tons of cruft. * nd/untracked-cache: (24 commits) git-status.txt: advertisement for untracked cache untracked cache: guard and disable on system changes mingw32: add uname() t7063: tests for untracked cache update-index: test the system before enabling untracked cache update-index: manually enable or disable untracked cache status: enable untracked cache untracked-cache: temporarily disable with $GIT_DISABLE_UNTRACKED_CACHE untracked cache: mark index dirty if untracked cache is updated untracked cache: print stats with $GIT_TRACE_UNTRACKED_STATS untracked cache: avoid racy timestamps read-cache.c: split racy stat test to a separate function untracked cache: invalidate at index addition or removal untracked cache: load from UNTR index extension untracked cache: save to an index extension ewah: add convenient wrapper ewah_serialize_strbuf() untracked cache: don't open non-existent .gitignore untracked cache: mark what dirs should be recursed/saved untracked cache: record/validate dir mtime and reuse cached output untracked cache: make a wrapper around {open,read,close}dir() ...
2015-05-11Merge branch 'ep/do-not-feed-a-pointer-to-array-size'Junio C Hamano1-1/+53
Catch a programmer mistake to feed a pointer not an array to ARRAY_SIZE() macro, by using a couple of GCC extensions. * ep/do-not-feed-a-pointer-to-array-size: git-compat-util.h: implement a different ARRAY_SIZE macro for for safely deriving the size of array
2015-05-05git-compat-util.h: implement a different ARRAY_SIZE macro for for safely ↵Elia Pinto1-1/+53
deriving the size of array To get number of elements in an array git use the ARRAY_SIZE macro defined as: #define ARRAY_SIZE(x) (sizeof(x)/sizeof((x)[0])) The problem with it is a possibility of mistakenly passing to it a pointer instead an array. The ARRAY_SIZE macro as conventionally defined does not provide good type-safety and the open-coded approach is more fragile, more verbose and provides no improvement in type-safety. Use instead a different but compatible ARRAY_SIZE() macro, which will also break compile if you try to use it on a pointer. This implemention revert to the original code if the compiler doesn't know the typeof and __builtin_types_compatible_p GCC extensions. This can ensure our code is robust to changes, without needing a gratuitous macro or constant. A similar ARRAY_SIZE implementation also exists in the linux kernel. Credits to Rusty Russell and his ccan library. Signed-off-by: Elia Pinto <gitter.spiros@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16git-compat-util: add fallbacks for unlocked stdioJeff King1-0/+6
POSIX.1-2001 specifies some functions for optimizing the locking out of tight getc() loops. Not all systems are POSIX, though, and even not all POSIX systems are required to implement these functions. We can check for the feature-test macro to see if they are available, and if not, provide a noop implementation. There's no Makefile knob here, because we should just detect this automatically. If there are very bizarre systems, we may need to add one, but it's not clear yet in which direction: 1. If a system defines _POSIX_THREAD_SAFE_FUNCTIONS but these functions are missing or broken, we would want a knob to manually turn them off. 2. If a system has these functions but does not define _POSIX_THREAD_SAFE_FUNCTIONS, we would want a knob to manually turn them on. We can add such a knob when we find a real-world system that matches this. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-20Merge branch 'km/bsd-sysctl'Junio C Hamano1-0/+3
We now detect number of CPUs on older BSD-derived systems. * km/bsd-sysctl: thread-utils.c: detect CPU count on older BSD-like systems configure: support HAVE_BSD_SYSCTL option
2015-03-20Merge branch 'km/bsd-shells'Junio C Hamano1-0/+4
Portability fixes and workarounds for shell scripts have been added to help BSD-derived systems. * km/bsd-shells: t5528: do not fail with FreeBSD shell help.c: use SHELL_PATH instead of hard-coded "/bin/sh" git-compat-util.h: move SHELL_PATH default into header git-instaweb: use @SHELL_PATH@ instead of /bin/sh git-instaweb: allow running in a working tree subdirectory
2015-03-12untracked cache: guard and disable on system changesNguyễn Thái Ngọc Duy1-0/+1
If the user enables untracked cache, then - move worktree to an unsupported filesystem - or simply upgrade OS - or move the whole (portable) disk from one machine to another - or access a shared fs from another machine there's no guarantee that untracked cache can still function properly. Record the worktree location and OS footprint in the cache. If it changes, err on the safe side and disable the cache. The user can 'update-index --untracked-cache' again to make sure all conditions are met. This adds a new requirement that setup_git_directory* must be called before read_cache() because we need worktree location by then, or the cache is dropped. This change does not cover all bases, you can fool it if you try hard. The point is to stop accidents. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: brian m. carlson <sandals@crustytoothpaste.net> Helped-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-10configure: support HAVE_BSD_SYSCTL optionKyle J. McKay1-0/+3
On BSD-compatible systems some information such as the number of available CPUs may only be available via the sysctl function. Add support for a HAVE_BSD_SYSCTL option complete with autoconf support and include the sys/syctl.h header when the option is enabled to make the sysctl function available. Signed-off-by: Kyle J. McKay <mackyle@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-10git-compat-util.h: move SHELL_PATH default into headerKyle J. McKay1-0/+4
If SHELL_PATH is not defined we use "/bin/sh". However, run-command.c is not the only file that needs to use the default value so move it into a common header. Signed-off-by: Kyle J. McKay <mackyle@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-06Merge branch 'bw/kwset-use-unsigned'Junio C Hamano1-1/+1
The borrowed code in kwset API did not follow our usual convention to use "unsigned char" to store values that range from 0-255. * bw/kwset-use-unsigned: kwset: use unsigned char to store values with high-bit set
2015-03-06Merge branch 'rj/no-xopen-source-for-cygwin' into maintJunio C Hamano1-6/+0
Code cleanups. * rj/no-xopen-source-for-cygwin: git-compat-util.h: remove redundant code
2015-03-05Merge branch 'es/squelch-openssl-warnings-on-macosx' into maintJunio C Hamano1-3/+6
An earlier workaround to squelch unhelpful deprecation warnings from the complier on Mac OSX unnecessarily set minimum required version of the OS, which the user might want to raise (or lower) for other reasons. * es/squelch-openssl-warnings-on-macosx: git-compat-util: do not step on MAC_OS_X_VERSION_MIN_REQUIRED
2015-03-05Merge branch 'rj/no-xopen-source-for-cygwin'Junio C Hamano1-6/+0
Code cleanups. * rj/no-xopen-source-for-cygwin: git-compat-util.h: remove redundant code
2015-03-02kwset: use unsigned char to store values with high-bit setBen Walton1-1/+1
Sun Studio on Solaris issues warnings about improper initialization values being used when defining tolower_trans_tbl[] in ctype.c. The array wants to store values with high-bit set and treat them as values between 128 to 255. Unlike the rest of the Git codebase where we explicitly specify 'unsigned char' for such variables and arrays, however, kwset code we borrowed from elsewhere uses 'char' for this and other variables. Fix the declarations to explicitly use 'unsigned char' where necessary to bring it in line with the rest of the Git. Signed-off-by: Ben Walton <bdwalton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-02-24Merge branch 'jk/blame-commit-label' into maintJunio C Hamano1-0/+5
"git blame HEAD -- missing" failed to correctly say "HEAD" when it tried to say "No such path 'missing' in HEAD". * jk/blame-commit-label: blame.c: fix garbled error message use xstrdup_or_null to replace ternary conditionals builtin/commit.c: use xstrdup_or_null instead of envdup builtin/apply.c: use xstrdup_or_null instead of null_strdup git-compat-util: add xstrdup_or_null helper
2015-02-22git-compat-util.h: remove redundant codeRamsay Jones1-6/+0
Since commit 3a0a3a89 ("git-compat-util.h: don't define _XOPEN_SOURCE on cygwin", 23-11-2014) removed the definition of _XOPEN_SOURCE on cygwin, the code within a pre-processor conditional further down the file became redundant. Remove the redundant code. This effectively reverts commit 41b20017 ("Fix an "implicit function definition" warning", 03-03-2007). Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-02-22Merge branch 'es/squelch-openssl-warnings-on-macosx'Junio C Hamano1-3/+6
An earlier workaround to squelch unhelpful deprecation warnings from the complier on Mac OSX unnecessarily set minimum required version of the OS, which the user might want to raise (or lower) for other reasons. * es/squelch-openssl-warnings-on-macosx: git-compat-util: do not step on MAC_OS_X_VERSION_MIN_REQUIRED
2015-02-11Merge branch 'km/gettext-n'Junio C Hamano1-0/+4
* km/gettext-n: gettext.h: add parentheses around N_ expansion if supported
2015-02-11Merge branch 'jk/blame-commit-label'Junio C Hamano1-0/+5
"git blame HEAD -- missing" failed to correctly say "HEAD" when it tried to say "No such path 'missing' in HEAD". * jk/blame-commit-label: blame.c: fix garbled error message use xstrdup_or_null to replace ternary conditionals builtin/commit.c: use xstrdup_or_null instead of envdup builtin/apply.c: use xstrdup_or_null instead of null_strdup git-compat-util: add xstrdup_or_null helper
2015-02-09git-compat-util: do not step on MAC_OS_X_VERSION_MIN_REQUIREDKyle J. McKay1-3/+6
MAC_OS_X_VERSION_MIN_REQUIRED may be defined by the builder to a specific version in order to produce compatible binaries for a particular system. Blindly defining it to MAC_OS_X_VERSION_10_6 is bad. Additionally MAC_OS_X_VERSION_10_6 will not be defined on older systems and should AvailabilityMacros.h be included on such as system an error will result. However, using the explicit value of 1060 (which is what MAC_OS_X_VERSION_10_6 is defined to) does not solve the problem. The changes that introduced stepping on MAC_OS_X_VERSION_MIN were made in b195aa00 (git-compat-util: suppress unavoidable Apple-specific deprecation warnings) to avoid deprecation warnings. Instead of blindly setting MAC_OS_X_VERSION_MIN to 1060 change the definition of DEPRECATED_ATTRIBUTE to empty to avoid the warnings. This preserves any MAC_OS_X_VERSION_MIN_REQUIRED setting while avoiding the warnings as intended by b195aa00. Signed-off-by: Kyle J. McKay <mackyle@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-01-14Merge branch 'rh/autoconf-rhel3'Junio C Hamano1-0/+3
Build update for older RHEL. * rh/autoconf-rhel3: configure.ac: check for HMAC_CTX_cleanup configure.ac: check for clock_gettime and CLOCK_MONOTONIC configure.ac: check 'tv_nsec' field in 'struct stat'
2015-01-13git-compat-util: add xstrdup_or_null helperJeff King1-0/+5
It's a common idiom to duplicate a string if it is non-NULL, or pass a literal NULL through. This is already a one-liner in C, but you do have to repeat the name of the string twice. So if there's a function call, you must write: const char *x = some_fun(...); return x ? xstrdup(x) : NULL; instead of (with this patch) just: return xstrdup_or_null(some_fun(...)); Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-01-12gettext.h: add parentheses around N_ expansion if supportedKyle J. McKay1-0/+4
The gettext N_ macro is used to mark strings for translation without actually translating them. At runtime the string is expected to be passed to the gettext API for translation. If two N_ macro invocations appear next to each other with only whitespace (or nothing at all) between them, the two separate strings will be marked for translation, but the preprocessor will then silently combine the strings into one and at runtime the string passed to gettext will not match the strings that were translated so no translation will actually occur. Avoid this by adding parentheses around the expansion of the N_ macro so that instead of ending up with two adjacent strings that are then combined by the preprocessor, two adjacent strings surrounded by parentheses result instead which causes a compile error so the mistake can be quickly found and corrected. However, since these string literals are typically assigned to static variables and not all compilers support parenthesized string literal assignments, allow this to be controlled by the Makefile with the default only enabled when the compiler is known to support the syntax. For now only __GNUC__ enables this by default which covers both gcc and clang which should result in early detection of any adjacent N_ macros. Although the necessary tests make the affected files a bit less elegant, the benefit of avoiding propagation of a translation- marking error to all the translation teams thus creating extra work for them when the error is eventually detected and fixed would seem to outweigh the minor inelegance the additional configuration tests introduce. Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Kyle J. McKay <mackyle@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-01-09configure.ac: check for HMAC_CTX_cleanupReuben Hawkins1-0/+3
OpenSSL version 0.9.6b and before defined the function HMAC_cleanup. Newer versions define HMAC_CTX_cleanup. Check for HMAC_CTX_cleanup and fall back to HMAC_cleanup when the newer function is missing. Signed-off-by: Reuben Hawkins <reubenhwk@gmail.com> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-01-07Merge branch 'es/squelch-openssl-warnings-on-macosx'Junio C Hamano1-0/+4
Squelch useless compiler warnings on Mac OS X. * es/squelch-openssl-warnings-on-macosx: git-compat-util: suppress unavoidable Apple-specific deprecation warnings
2014-12-22Merge branch 'dm/compat-s-ifmt-for-zos'Junio C Hamano1-0/+34
Long overdue departure from the assumption that S_IFMT is shared by everybody made in 2005. * dm/compat-s-ifmt-for-zos: compat: convert modes to use portable file type values
2014-12-22Merge branch 'rj/no-xopen-source-for-cygwin'Junio C Hamano1-1/+2
Avoid compilation warnings on recent gcc toolchain on Cygwin. * rj/no-xopen-source-for-cygwin: git-compat-util.h: don't define _XOPEN_SOURCE on cygwin
2014-12-18git-compat-util: suppress unavoidable Apple-specific deprecation warningsEric Sunshine1-0/+4
With the release of Mac OS X 10.7 in July 2011, Apple deprecated all openssl.h functionality due to OpenSSL ABI (application binary interface) instability, resulting in an explosion of compilation warnings about deprecated SSL, SHA1, and X509 functions (among others). 61067954ce (cache.h: eliminate SHA-1 deprecation warnings on Mac OS X; 2013-05-19) and be4c828b76 (imap-send: eliminate HMAC deprecation warnings on Mac OS X; 2013-05-19) attempted to ameliorate the situation by taking advantage of drop-in replacement functionality provided by Apple's (ABI-stable) CommonCrypto facility, however CommonCrypto supplies only a subset of deprecated OpenSSL functionality, thus a host of warnings remain. Despite this shortcoming, it was hoped that Apple would ultimately provide CommonCrypto replacements for all deprecated OpenSSL functionality, and that the effort started by 61067954ce and be4c828b76 would be continued and eventually eliminate all deprecation warnings. However, now 3.5 years later, and with Mac OS X at 10.10, the hoped-for CommonCrypto replacements have not yet materialized, nor is there any indication that they will be forthcoming. These Apple-specific warnings are pure noise: they don't tell us anything useful and we have no control over them, nor is Apple likely to provide replacements any time soon. Such noise may obscure other legitimate warnings, therefore silence them. Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-12-04compat: convert modes to use portable file type valuesDavid Michael1-0/+34
This adds simple wrapper functions around calls to stat(), fstat(), and lstat() that translate the operating system's native file type bits to those used by most operating systems. It also rewrites the S_IF* macros to the common values, so all file type processing is performed using the translated modes. This makes projects portable across operating systems that use different file type definitions. Only the file type bits may be affected by these compatibility functions; the file permission bits are assumed to be 07777 and are passed through unchanged. Signed-off-by: David Michael <fedora.dm0@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-11-24git-compat-util.h: don't define _XOPEN_SOURCE on cygwinRamsay Jones1-1/+2
A recent update to the gcc compiler (v4.8.3-5 x86_64) on 64-bit cygwin leads to several new warnings about the implicit declaration of the memmem(), strlcpy() and strcasestr() functions. For example: CC archive.o archive.c: In function 'format_subst': archive.c:44:3: warning: implicit declaration of function 'memmem' \ [-Wimplicit-function-declaration] b = memmem(src, len, "$Format:", 8); ^ archive.c:44:5: warning: assignment makes pointer from integer \ without a cast [enabled by default] b = memmem(src, len, "$Format:", 8); ^ This is because <string.h> on Cygwin used to always declare the above functions, but a recent version of it no longer make them visible when _XOPEN_SOURCE is set (even if _GNU_SOURCE and _BSD_SOURCE is set). In order to suppress the warnings, don't define the _XOPEN_SOURCE macro on cygwin. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-29Merge branch 'dm/port2zos'Junio C Hamano1-1/+1
z/OS port * dm/port2zos: compat/bswap.h: detect endianness from XL C compiler macros Makefile: reorder linker flags in the git executable rule git-compat-util.h: support variadic macros with the XL C compiler
2014-10-29Merge branch 'jk/prune-mtime'Junio C Hamano1-1/+1
Tighten the logic to decide that an unreachable cruft is sufficiently old by covering corner cases such as an ancient object becoming reachable and then going unreachable again, in which case its retention period should be prolonged. * jk/prune-mtime: (28 commits) drop add_object_array_with_mode revision: remove definition of unused 'add_object' function pack-objects: double-check options before discarding objects repack: pack objects mentioned by the index pack-objects: use argv_array reachable: use revision machinery's --indexed-objects code rev-list: add --indexed-objects option rev-list: document --reflog option t5516: test pushing a tag of an otherwise unreferenced blob traverse_commit_list: support pending blobs/trees with paths make add_object_array_with_context interface more sane write_sha1_file: freshen existing objects pack-objects: match prune logic for discarding objects pack-objects: refactor unpack-unreachable expiration check prune: keep objects reachable from recent objects sha1_file: add for_each iterators for loose and packed objects count-objects: use for_each_loose_file_in_objdir count-objects: do not use xsize_t when counting object size prune-packed: use for_each_loose_file_in_objdir reachable: mark index blobs as SEEN ...
2014-10-27git-compat-util.h: support variadic macros with the XL C compilerDavid Michael1-1/+1
When the XL C compiler is run with an appropriate language level or suboption, it defines a feature test macro to indicate support for variadic macros by defining __C99_MACRO_WITH_VA_ARGS C preprocessor macro. This was tested on z/OS, but it should also work on AIX according to IBM documentation. Signed-off-by: David Michael <fedora.dm0@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-24Merge branch 'eb/no-pthreads'Junio C Hamano1-0/+5
Allow us build with NO_PTHREADS=NoThanks compilation option. * eb/no-pthreads: Handle atexit list internaly for unthreaded builds pack-objects: set number of threads before checking and warning index-pack: fix compilation with NO_PTHREADS
2014-10-19Handle atexit list internaly for unthreaded buildsEtienne Buira1-0/+5
Wrap atexit()s calls on unthreaded builds to handle callback list internally. This is needed because on unthreaded builds, asyncs inherits parent's atexit() list, that gets run as soon as the async exit()s (and again at the end of async's parent process). That led to remove temporary files too early. Also remove a by-atexit-callback guard against this kind of issue in clone.c, as this patch makes it redundant. Fixes test 5537 (temporary shallow file vanished before unpack-objects could open it) BTW remove an unused variable in shallow.c. Helped-by: Duy Nguyen <pclouds@gmail.com> Helped-by: Andreas Schwab <schwab@linux-m68k.org> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Etienne Buira <etienne.buira@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-16isxdigit: cast input to unsigned charJeff King1-1/+1
Otherwise, callers must do so or risk triggering warnings -Wchar-subscript (and rightfully so; a signed char might cause us to use a bogus negative index into the hexval_table). While we are dropping the now-unnecessary casts from the caller in urlmatch.c, we can get rid of similar casts in actually parsing the hex by using the hexval() helper, which implicitly casts to unsigned (but note that we cannot implement isxdigit in terms of hexval(), as it also casts its return value to unsigned). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-15wrapper.c: add a new function unlink_or_msgRonnie Sahlberg1-0/+9
This behaves like unlink_or_warn except that on failure it writes the message to its 'err' argument, which the caller can display in an appropriate way or ignore. Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-15wrapper.c: remove/unlink_or_warn: simplify, treat ENOENT as successRonnie Sahlberg1-2/+5
Simplify the function warn_if_unremovable slightly. Additionally, change behaviour slightly. If we failed to remove the object because the object does not exist, we can still return success back to the caller since none of the callers depend on "fail if the file did not exist". Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-29Merge branch 'jt/itimer-autoconf'Junio C Hamano1-1/+1
setitmer(2) and related API elements can be configured from Makefile but autoconf did not know about it. * jt/itimer-autoconf: autoconf: check for setitimer() autoconf: check for struct itimerval git-compat-util.h: add missing semicolon after struct itimerval
2014-09-26Merge branch 'rs/realloc-array'Junio C Hamano1-0/+2
Code cleanup. * rs/realloc-array: use REALLOC_ARRAY for changing the allocation size of arrays add macro REALLOC_ARRAY
2014-09-19Merge branch 'ss/compat-default-source-for-newer-gnu'Junio C Hamano1-0/+1
* ss/compat-default-source-for-newer-gnu: compat-util: add _DEFAULT_SOURCE define
2014-09-18add macro REALLOC_ARRAYRené Scharfe1-0/+2
The macro ALLOC_GROW manages several aspects of dynamic memory allocations for arrays: It performs overprovisioning in order to avoid reallocations in future calls, updates the allocation size variable, multiplies the item size and thus allows users to simply specify the item count, performs the reallocation and updates the array pointer. Sometimes this is too much. Add the macro REALLOC_ARRAY, which only takes care of the latter three points and allows users to specfiy the number of items the array can store. It can increase and also decrease the size. Using the macro avoid duplicating the variable name and takes care of the item sizes automatically. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-15compat-util: add _DEFAULT_SOURCE defineSergey Senozhatsky1-0/+1
glibc has deprecated the use of _BSD_SOURCE define warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE" To make it easier to maintain a cross platform source code, that warning can be suppressed by _DEFAULT_SOURCE. Define both _BSD_SOURCE and _DEFAULT_SOURCE to clean-up the build. Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-11Merge branch 'nd/large-blobs'Junio C Hamano1-0/+1
Teach a few codepaths to punt (instead of dying) when large blobs that would not fit in core are involved in the operation. * nd/large-blobs: diff: shortcut for diff'ing two binary SHA-1 objects diff --stat: mark any file larger than core.bigfilethreshold binary diff.c: allow to pass more flags to diff_populate_filespec sha1_file.c: do not die failing to malloc in unpack_compressed_entry wrapper.c: introduce gentle xmallocz that does not die()
2014-09-09Merge branch 'rs/inline-compat-path-macros'Junio C Hamano1-6/+22
* rs/inline-compat-path-macros: turn path macros into inline function
2014-09-02Merge branch 'rs/strbuf-getcwd'Junio C Hamano1-0/+1
Reduce the use of fixed sized buffer passed to getcwd() calls by introducing xgetcwd() helper. * rs/strbuf-getcwd: use strbuf_add_absolute_path() to add absolute paths abspath: convert absolute_path() to strbuf use xgetcwd() to set $GIT_DIR use xgetcwd() to get the current directory or die wrapper: add xgetcwd() abspath: convert real_path_internal() to strbuf abspath: use strbuf_getcwd() to remember original working directory setup: convert setup_git_directory_gently_1 et al. to strbuf unix-sockets: use strbuf_getcwd() strbuf: add strbuf_getcwd()