aboutsummaryrefslogtreecommitdiffstats
path: root/builtin/update-index.c
AgeCommit message (Collapse)AuthorFilesLines
2017-11-21Merge branch 'bp/fsmonitor'Junio C Hamano1-1/+37
We learned to talk to watchman to speed up "git status" and other operations that need to see which paths have been modified. * bp/fsmonitor: fsmonitor: preserve utf8 filenames in fsmonitor-watchman log fsmonitor: read entirety of watchman output fsmonitor: MINGW support for watchman integration fsmonitor: add a performance test fsmonitor: add a sample integration script for Watchman fsmonitor: add test cases for fsmonitor extension split-index: disable the fsmonitor extension when running the split index test fsmonitor: add a test tool to dump the index extension update-index: add fsmonitor support to update-index ls-files: Add support in ls-files to display the fsmonitor valid bit fsmonitor: add documentation for the fsmonitor extension. fsmonitor: teach git to optionally utilize a file system monitor to speed up detecting new or changed files. update-index: add a new --force-write-index option preload-index: add override to enable testing preload-index bswap: add 64 bit endianness helper get_be64
2017-10-16refs: convert resolve_gitlink_ref to struct object_idbrian m. carlson1-2/+2
Convert the declaration and definition of resolve_gitlink_ref to use struct object_id and apply the following semantic patch: @@ expression E1, E2, E3; @@ - resolve_gitlink_ref(E1, E2, E3.hash) + resolve_gitlink_ref(E1, E2, &E3) @@ expression E1, E2, E3; @@ - resolve_gitlink_ref(E1, E2, E3->hash) + resolve_gitlink_ref(E1, E2, E3) Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-16refs: convert read_ref and read_ref_full to object_idbrian m. carlson1-3/+3
All but two of the call sites already have parameters using the hash parameter of struct object_id, so convert them to take a pointer to the struct directly. Also convert refs_read_refs_full, the underlying implementation. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-01update-index: add fsmonitor support to update-indexBen Peart1-1/+32
Add support in update-index to manually add/remove the fsmonitor extension via --[no-]fsmonitor flags. Add support in update-index to manually set/clear the fsmonitor valid bit via --[no-]fsmonitor-valid flags. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-01fsmonitor: teach git to optionally utilize a file system monitor to speed up ↵Ben Peart1-0/+2
detecting new or changed files. When the index is read from disk, the fsmonitor index extension is used to flag the last known potentially dirty index entries. The registered core.fsmonitor command is called with the time the index was last updated and returns the list of files changed since that time. This list is used to flag any additional dirty cache entries and untracked cache directories. We can then use this valid state to speed up preload_index(), ie_match_stat(), and refresh_cache_ent() as they do not need to lstat() files to detect potential changes for those entries marked CE_FSMONITOR_VALID. In addition, if the untracked cache is turned on valid_cached_dir() can skip checking directories for new or changed files as fsmonitor will invalidate the cache only for those directories that have been identified as having potential changes. To keep the CE_FSMONITOR_VALID state accurate during git operations; when git updates a cache entry to match the current state on disk, it will now set the CE_FSMONITOR_VALID bit. Inversely, anytime git changes a cache entry, the CE_FSMONITOR_VALID bit is cleared and the corresponding untracked cache directory is marked invalid. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-24update-index: add a new --force-write-index optionBen Peart1-1/+4
At times, it makes sense to avoid the cost of writing out the index when the only changes can easily be recomputed on demand. This causes problems when trying to write test cases to verify that state as they can't guarantee the state has been persisted to disk. Add a new option (--force-write-index) to update-index that will ensure the index is written out even if the cache_changed flag is not set. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-19Merge branch 'jk/leak-checkers'Junio C Hamano1-1/+3
Many of our programs consider that it is OK to release dynamic storage that is used throughout the life of the program by simply exiting, but this makes it harder to leak detection tools to avoid reporting false positives. Plug many existing leaks and introduce a mechanism for developers to mark that the region of memory pointed by a pointer is not lost/leaking to help these tools. * jk/leak-checkers: add UNLEAK annotation for reducing leak false positives set_git_dir: handle feeding gitdir to itself repository: free fields before overwriting them reset: free allocated tree buffers reset: make tree counting less confusing config: plug user_config leak update-index: fix cache entry leak in add_one_file() add: free leaked pathspec after add_files_to_cache() test-lib: set LSAN_OPTIONS to abort by default test-lib: --valgrind should not override --verbose-log
2017-09-06update-index: fix cache entry leak in add_one_file()Jeff King1-1/+3
When we fail to add the cache entry to the index, we end up just leaking the struct. We should follow the pattern of the early-return above and free it. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-06stop leaking lock structs in some simple casesJeff King1-7/+4
Now that it's safe to declare a "struct lock_file" on the stack, we can do so (and avoid an intentional leak). These leaks were found by running t0000 and t0001 under valgrind (though certainly other similar leaks exist and just don't happen to be exercised by those tests). Initializing the lock_file's inner tempfile with NULL is not strictly necessary in these cases, but it's a good practice to model. It means that if we were to call a function like rollback_lock_file() on a lock that was never taken in the first place, it becomes a quiet noop (rather than undefined behavior). Likewise, it's always safe to rollback_lock_file() on a file that has already been committed or deleted, since that operation is a noop on an inactive lockfile (and that's why the case in config.c can drop the "if (lock)" check as we move away from using a pointer). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-20sha1_file: convert index_path to struct object_idPatryk Obara1-1/+1
Convert all remaining callers as well. Signed-off-by: Patryk Obara <patryk.obara@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-24Merge branch 'bw/config-h'Junio C Hamano1-0/+1
Fix configuration codepath to pay proper attention to commondir that is used in multi-worktree situation, and isolate config API into its own header file. * bw/config-h: config: don't implicitly use gitdir or commondir config: respect commondir setup: teach discover_git_directory to respect the commondir config: don't include config.h by default config: remove git_config_iter config: create config.h
2017-06-15config: don't include config.h by defaultBrandon Williams1-0/+1
Stop including config.h by default in cache.h. Instead only include config.h in those files which require use of the config system. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-13Merge branch 'jc/noent-notdir'Junio C Hamano1-1/+1
Our code often opens a path to an optional file, to work on its contents when we can successfully open it. We can ignore a failure to open if such an optional file does not exist, but we do want to report a failure in opening for other reasons (e.g. we got an I/O error, or the file is there, but we lack the permission to open). The exact errors we need to ignore are ENOENT (obviously) and ENOTDIR (less obvious). Instead of repeating comparison of errno with these two constants, introduce a helper function to do so. * jc/noent-notdir: treewide: use is_missing_file_error() where ENOENT and ENOTDIR are checked compat-util: is_missing_file_error()
2017-05-30treewide: use is_missing_file_error() where ENOENT and ENOTDIR are checkedJunio C Hamano1-1/+1
Using the is_missing_file_error() helper introduced in the previous step, update all hits from $ git grep -e ENOENT --and -e ENOTDIR There are codepaths that only check ENOENT, and it is possible that some of them should be checking both. Updating them is kept out of this step deliberately, as we do not want to change behaviour in this step. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-11Merge branch 'cc/untracked'Junio C Hamano1-1/+5
Code cleanup. * cc/untracked: update-index: fix xgetcwd() related memory leak
2017-03-30update-index: fix xgetcwd() related memory leakChristian Couder1-1/+5
As xgetcwd() returns an allocated buffer, we should free this buffer when we don't need it any more. This was found by Coverity. Reported-by: Jeff King <peff@peff.net> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-01update-index: warn in case of split-index incoherencyChristian Couder1-1/+10
When users are using `git update-index --(no-)split-index`, they may expect the split-index feature to be used or not according to the option they just used, but this might not be the case if the new "core.splitIndex" config variable has been set. In this case let's warn about what will happen and why. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-01split-index: add {add,remove}_split_index() functionsChristian Couder1-12/+6
Also use the functions in cmd_update_index() in builtin/update-index.c. These functions will be used in a following commit to tweak our use of the split-index feature depending on the setting of a configuration variable. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-07hold_locked_index(): align error handling with hold_lockfile_for_update()Junio C Hamano1-0/+1
Callers of the hold_locked_index() function pass 0 when they want to prepare to write a new version of the index file without wishing to die or emit an error message when the request fails (e.g. somebody else already held the lock), and pass 1 when they want the call to die upon failure. This option is called LOCK_DIE_ON_ERROR by the underlying lockfile API, and the hold_locked_index() function translates the paramter to LOCK_DIE_ON_ERROR when calling the hold_lock_file_for_update(). Replace these hardcoded '1' with LOCK_DIE_ON_ERROR and stop translating. Callers other than the ones that are replaced with this change pass '0' to the function; no behaviour change is intended with this patch. Signed-off-by: Junio C Hamano <gitster@pobox.com> --- Among the callers of hold_locked_index() that passes 0: - diff.c::refresh_index_quietly() at the end of "git diff" is an opportunistic update; it leaks the lockfile structure but it is just before the program exits and nobody should care. - builtin/describe.c::cmd_describe(), builtin/commit.c::cmd_status(), sequencer.c::read_and_refresh_cache() are all opportunistic updates and they are OK. - builtin/update-index.c::cmd_update_index() takes a lock upfront but we may end up not needing to update the index (i.e. the entries may be fully up-to-date), in which case we do not need to issue an error upon failure to acquire the lock. We do diagnose and die if we indeed need to update, so it is OK. - wt-status.c::require_clean_work_tree() IS BUGGY. It asks silence, does not check the returned value. Compare with callsites like cmd_describe() and cmd_status() to notice that it is wrong to call update_index_if_able() unconditionally.
2016-09-26Merge branch 'tg/add-chmod+x-fix'Junio C Hamano1-15/+3
"git add --chmod=+x <pathspec>" added recently only toggled the executable bit for paths that are either new or modified. This has been corrected to flip the executable bit for all paths that match the given pathspec. * tg/add-chmod+x-fix: t3700-add: do not check working tree file mode without POSIXPERM t3700-add: create subdirectory gently add: modify already added files when --chmod is given read-cache: introduce chmod_index_entry update-index: add test for chmod flags
2016-09-21Merge branch 'va/i18n'Junio C Hamano1-6/+6
More i18n. * va/i18n: i18n: update-index: mark warnings for translation i18n: show-branch: mark plural strings for translation i18n: show-branch: mark error messages for translation i18n: receive-pack: mark messages for translation notes: spell first word of error messages in lowercase i18n: notes: mark error messages for translation i18n: merge-recursive: mark verbose message for translation i18n: merge-recursive: mark error messages for translation i18n: config: mark error message for translation i18n: branch: mark option description for translation i18n: blame: mark error messages for translation
2016-09-15i18n: update-index: mark warnings for translationVasco Almeida1-6/+6
Signed-off-by: Vasco Almeida <vascomalmeida@sapo.pt> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-15read-cache: introduce chmod_index_entryThomas Gummerer1-14/+2
As there are chmod options for both add and update-index, introduce a new chmod_index_entry function to do the work. Use it in update-index, while it will be used in add in the next patch. Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-14update-index: add test for chmod flagsThomas Gummerer1-1/+1
Currently there is no test checking the expected behaviour when multiple chmod flags with different arguments are passed. As argument handling is not in line with other git commands it's easy to miss and accidentally change the current behaviour. While there, fix the argument type of chmod_path, which takes an int, but had a char passed in. Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-07builtin/update-index: convert file to struct object_idbrian m. carlson1-30/+31
Convert all functions to use struct object_id, and replace instances of hardcoded 40, 41, and 42 with appropriate references to GIT_SHA1_HEXSZ. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-07cache: convert struct cache_entry to use struct object_idbrian m. carlson1-5/+5
Convert struct cache_entry to use struct object_id by applying the following semantic patch and the object_id transforms from contrib, plus the actual change to the struct: @@ struct cache_entry E1; @@ - E1.sha1 + E1.oid.hash @@ struct cache_entry *E1; @@ - E1->sha1 + E1->oid.hash Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-26die("bug"): report bugs consistentlyJohannes Schindelin1-1/+1
The vast majority of error messages in Git's source code which report a bug use the convention to prefix the message with "BUG:". As part of cleaning up merge-recursive to stop die()ing except in case of detected bugs, let's just make the remainder of the bug reports consistent with the de facto rule. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-02pathspec: rename free_pathspec() to clear_pathspec()Junio C Hamano1-1/+1
The function takes a pointer to a pathspec structure, and releases the resources held by it, but does not free() the structure itself. Such a function should be called "clear", not "free". Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-09builtin/update-index.c: prefer "err" to "errno" in process_lstat_errorNguyễn Thái Ngọc Duy1-1/+1
"errno" is already passed in as "err". Here we should use err instead of errno. errno is probably a copy/paste mistake in e011054 (Teach git-update-index about gitlinks - 2007-04-12) Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-10Merge branch 'jk/options-cleanup'Junio C Hamano1-5/+6
Various clean-ups to the command line option parsing. * jk/options-cleanup: apply, ls-files: simplify "-z" parsing checkout-index: disallow "--no-stage" option checkout-index: handle "--no-index" option checkout-index: handle "--no-prefix" option checkout-index: simplify "-z" option parsing give "nbuf" strbuf a more meaningful name
2016-02-10Merge branch 'cc/untracked'Junio C Hamano1-23/+39
Update the untracked cache subsystem and change its primary UI from "git update-index" to "git config". * cc/untracked: t7063: add tests for core.untrackedCache test-dump-untracked-cache: don't modify the untracked cache config: add core.untrackedCache dir: simplify untracked cache "ident" field dir: add remove_untracked_cache() dir: add {new,add}_untracked_cache() update-index: move 'uc' var declaration update-index: add untracked cache notifications update-index: add --test-untracked-cache update-index: use enum for untracked cache options dir: free untracked cache when removing it
2016-02-01give "nbuf" strbuf a more meaningful nameJeff King1-5/+6
It's a common pattern in our code to read paths from stdin, separated either by newlines or NULs, and unquote as necessary. In each of these five cases we use "nbuf" to temporarily store the unquoted value. Let's give it the more meaningful name "unquoted", which makes it easier to understand the purpose of the variable. While we're at it, let's also static-initialize all of our strbufs. It's not wrong to call strbuf_init, but it increases the cognitive load on the reader, who might wonder "do we sometimes avoid initializing them? why?". Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-27config: add core.untrackedCacheChristian Couder1-11/+24
When we know that mtime on directory as given by the environment is usable for the purpose of untracked cache, we may want the untracked cache to be always used without any mtime test or kernel name check being performed. Also when we know that mtime is not usable for the purpose of untracked cache, for example because the repo is shared over a network file system, we may want the untracked-cache to be automatically removed from the index. Allow the user to express such preference by setting the 'core.untrackedCache' configuration variable, which can take 'keep', 'false', or 'true' and default to 'keep'. When read_index_from() is called, it now adds or removes the untracked cache in the index to respect the value of this variable. So it does nothing if the value is `keep` or if the variable is unset; it adds the untracked cache if the value is `true`; and it removes the cache if the value is `false`. `git update-index --[no-|force-]untracked-cache` still adds the untracked cache to, or removes it, from the index, but this shows a warning if it goes against the value of core.untrackedCache, because the next time the index is read the untracked cache will be added or removed if the configuration is set to do so. Also `--untracked-cache` used to check that the underlying operating system and file system change `st_mtime` field of a directory if files are added or deleted in that directory. But because those tests take a long time, `--untracked-cache` no longer performs them. Instead, there is now `--test-untracked-cache` to perform the tests. This change makes `--untracked-cache` the same as `--force-untracked-cache`. This last change is backward incompatible and should be mentioned in the release notes. Helped-by: Duy Nguyen <pclouds@gmail.com> Helped-by: Torsten Bögershausen <tboegi@web.de> Helped-by: Stefan Beller <sbeller@google.com> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> read-cache: Duy'sfixup Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-25dir: add remove_untracked_cache()Christian Couder1-5/+1
Factor out code into remove_untracked_cache(), which will be used in a later commit. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-25dir: add {new,add}_untracked_cache()Christian Couder1-10/+1
Factor out code into new_untracked_cache() and add_untracked_cache(), which will be used in later commits. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-25update-index: move 'uc' var declarationChristian Couder1-3/+1
Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-25update-index: add untracked cache notificationsChristian Couder1-5/+9
Attempting to flip the untracked-cache feature on for a random index file with cd /random/unrelated/place git --git-dir=/somewhere/else/.git update-index --untracked-cache would not work as you might expect. Because flipping the feature on in the index also records the location of the corresponding working tree (/random/unrelated/place in the above example), when the index is subsequently used to keep track of files in the working tree in /somewhere/else, the feature is disabled. With this patch "git update-index --[test-]untracked-cache" tells the user in which directory tests are performed. This makes it easy to spot any problem. Also in verbose mode, let's tell the user when the cache is enabled or disabled. Helped-by: Duy Nguyen <pclouds@gmail.com> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-25update-index: add --test-untracked-cacheChristian Couder1-0/+5
It is nice to just be able to test if untracked cache is supported without enabling it. Helped-by: David Turner <dturner@twopensource.com> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-25update-index: use enum for untracked cache optionsChristian Couder1-5/+13
Helped-by: Duy Nguyen <pclouds@gmail.com> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-15update-index: there are only two possible line terminationsJunio C Hamano1-11/+16
The program by default reads LF terminated lines, with an option to use NUL terminated records. Instead of pretending that there can be other useful values for line_termination, use a boolean variable, nul_term_line, to tell if NUL terminated records are used, and switch between strbuf_getline_{lf,nul} based on it. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-29dir: free untracked cache when removing itChristian Couder1-0/+1
Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-05-26Merge branch 'nd/untracked-cache'Junio C Hamano1-0/+188
Teach the index to optionally remember already seen untracked files to speed up "git status" in a working tree with tons of cruft. * nd/untracked-cache: (24 commits) git-status.txt: advertisement for untracked cache untracked cache: guard and disable on system changes mingw32: add uname() t7063: tests for untracked cache update-index: test the system before enabling untracked cache update-index: manually enable or disable untracked cache status: enable untracked cache untracked-cache: temporarily disable with $GIT_DISABLE_UNTRACKED_CACHE untracked cache: mark index dirty if untracked cache is updated untracked cache: print stats with $GIT_TRACE_UNTRACKED_STATS untracked cache: avoid racy timestamps read-cache.c: split racy stat test to a separate function untracked cache: invalidate at index addition or removal untracked cache: load from UNTR index extension untracked cache: save to an index extension ewah: add convenient wrapper ewah_serialize_strbuf() untracked cache: don't open non-existent .gitignore untracked cache: mark what dirs should be recursed/saved untracked cache: record/validate dir mtime and reuse cached output untracked cache: make a wrapper around {open,read,close}dir() ...
2015-05-05prefix_path(): unconditionally free results in the callersStefan Beller1-7/+6
As of d089ebaa (setup: sanitize absolute and funny paths in get_pathspec(), 2008-01-28), prefix_path() always returns a newly allocated string, so callers should free its result. Additionally, drop the const from variables to which the result of the prefix_path() is assigned, so they can be free()'d without having to cast-away the constness. Signed-off-by: Stefan Beller <sbeller@google.com> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-22update-index: fix a memleakStefan Beller1-0/+1
`old` is not used outside the loop and would get lost once we reach the goto. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-12untracked cache: guard and disable on system changesNguyễn Thái Ngọc Duy1-6/+10
If the user enables untracked cache, then - move worktree to an unsupported filesystem - or simply upgrade OS - or move the whole (portable) disk from one machine to another - or access a shared fs from another machine there's no guarantee that untracked cache can still function properly. Record the worktree location and OS footprint in the cache. If it changes, err on the safe side and disable the cache. The user can 'update-index --untracked-cache' again to make sure all conditions are met. This adds a new requirement that setup_git_directory* must be called before read_cache() because we need worktree location by then, or the cache is dropped. This change does not cover all bases, you can fool it if you try hard. The point is to stop accidents. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: brian m. carlson <sandals@crustytoothpaste.net> Helped-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-12update-index: test the system before enabling untracked cacheNguyễn Thái Ngọc Duy1-0/+168
Helped-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-12update-index: manually enable or disable untracked cacheNguyễn Thái Ngọc Duy1-0/+16
Overall time saving on "git status" is about 40% in the best case scenario, removing ..collect_untracked() as the most time consuming function. read and refresh index operations are now at the top (which should drop when index-helper and/or watchman support is added). More numbers and analysis below. webkit.git ========== 169k files. 6k dirs. Lots of test data (i.e. not touched most of the time) Base status ----------- Index version 4 in split index mode and cache-tree populated. No untracked cache. It shows how time is consumed by "git status". The same settings are used for other repos below. 18:28:10.199679 builtin/commit.c:1394 performance: 0.000000451 s: cmd_status:setup 18:28:10.474847 read-cache.c:1407 performance: 0.274873831 s: read_index 18:28:10.475295 read-cache.c:1407 performance: 0.000000656 s: read_index 18:28:10.728443 preload-index.c:131 performance: 0.253147487 s: read_index_preload 18:28:10.741422 read-cache.c:1254 performance: 0.012868340 s: refresh_index 18:28:10.752300 wt-status.c:623 performance: 0.010421357 s: wt_status_collect_changes_worktree 18:28:10.762069 wt-status.c:629 performance: 0.009644748 s: wt_status_collect_changes_index 18:28:11.601019 wt-status.c:632 performance: 0.838859547 s: wt_status_collect_untracked 18:28:11.605939 builtin/commit.c:1421 performance: 0.004835004 s: cmd_status:update_index 18:28:11.606580 trace.c:415 performance: 1.407878388 s: git command: 'git' 'status' Populating status ----------------- This is after enabling untracked cache and the cache is still empty. We see a slight increase in .._collect_untracked() and update_index (because new cache has to be written to $GIT_DIR/index). 18:28:18.915213 builtin/commit.c:1394 performance: 0.000000326 s: cmd_status:setup 18:28:19.197364 read-cache.c:1407 performance: 0.281901416 s: read_index 18:28:19.197754 read-cache.c:1407 performance: 0.000000546 s: read_index 18:28:19.451355 preload-index.c:131 performance: 0.253599607 s: read_index_preload 18:28:19.464400 read-cache.c:1254 performance: 0.012935336 s: refresh_index 18:28:19.475115 wt-status.c:623 performance: 0.010236920 s: wt_status_collect_changes_worktree 18:28:19.486022 wt-status.c:629 performance: 0.010801685 s: wt_status_collect_changes_index 18:28:20.362660 wt-status.c:632 performance: 0.876551366 s: wt_status_collect_untracked 18:28:20.396199 builtin/commit.c:1421 performance: 0.033447969 s: cmd_status:update_index 18:28:20.396939 trace.c:415 performance: 1.482695902 s: git command: 'git' 'status' Populated status ---------------- After the cache is populated, wt_status_collect_untracked() drops 82% from 0.838s to 0.144s. Overall time drops 45%. Top offenders are now read_index() and read_index_preload(). 18:28:20.408605 builtin/commit.c:1394 performance: 0.000000457 s: cmd_status:setup 18:28:20.692864 read-cache.c:1407 performance: 0.283980458 s: read_index 18:28:20.693273 read-cache.c:1407 performance: 0.000000661 s: read_index 18:28:20.958814 preload-index.c:131 performance: 0.265540254 s: read_index_preload 18:28:20.972375 read-cache.c:1254 performance: 0.013437429 s: refresh_index 18:28:20.983959 wt-status.c:623 performance: 0.011146646 s: wt_status_collect_changes_worktree 18:28:20.993948 wt-status.c:629 performance: 0.009879094 s: wt_status_collect_changes_index 18:28:21.138125 wt-status.c:632 performance: 0.144084737 s: wt_status_collect_untracked 18:28:21.173678 builtin/commit.c:1421 performance: 0.035463949 s: cmd_status:update_index 18:28:21.174251 trace.c:415 performance: 0.766707355 s: git command: 'git' 'status' gentoo-x86.git ============== This repository is a strange one with a balanced, wide and shallow worktree (about 100k files and 23k dirs) and no .gitignore in worktree. .._collect_untracked() time drops 88%, total time drops 56%. Base status ----------- 18:20:40.828642 builtin/commit.c:1394 performance: 0.000000496 s: cmd_status:setup 18:20:41.027233 read-cache.c:1407 performance: 0.198130532 s: read_index 18:20:41.027670 read-cache.c:1407 performance: 0.000000581 s: read_index 18:20:41.171716 preload-index.c:131 performance: 0.144045594 s: read_index_preload 18:20:41.179171 read-cache.c:1254 performance: 0.007320424 s: refresh_index 18:20:41.185785 wt-status.c:623 performance: 0.006144638 s: wt_status_collect_changes_worktree 18:20:41.192701 wt-status.c:629 performance: 0.006780184 s: wt_status_collect_changes_index 18:20:41.991723 wt-status.c:632 performance: 0.798927029 s: wt_status_collect_untracked 18:20:41.994664 builtin/commit.c:1421 performance: 0.002852772 s: cmd_status:update_index 18:20:41.995458 trace.c:415 performance: 1.168427502 s: git command: 'git' 'status' Populating status ----------------- 18:20:48.968848 builtin/commit.c:1394 performance: 0.000000380 s: cmd_status:setup 18:20:49.172918 read-cache.c:1407 performance: 0.203734214 s: read_index 18:20:49.173341 read-cache.c:1407 performance: 0.000000562 s: read_index 18:20:49.320013 preload-index.c:131 performance: 0.146671391 s: read_index_preload 18:20:49.328039 read-cache.c:1254 performance: 0.007921957 s: refresh_index 18:20:49.334680 wt-status.c:623 performance: 0.006172020 s: wt_status_collect_changes_worktree 18:20:49.342526 wt-status.c:629 performance: 0.007731746 s: wt_status_collect_changes_index 18:20:50.257510 wt-status.c:632 performance: 0.914864222 s: wt_status_collect_untracked 18:20:50.338371 builtin/commit.c:1421 performance: 0.080776477 s: cmd_status:update_index 18:20:50.338900 trace.c:415 performance: 1.371462446 s: git command: 'git' 'status' Populated status ---------------- 18:20:50.351160 builtin/commit.c:1394 performance: 0.000000571 s: cmd_status:setup 18:20:50.577358 read-cache.c:1407 performance: 0.225917338 s: read_index 18:20:50.577794 read-cache.c:1407 performance: 0.000000617 s: read_index 18:20:50.734140 preload-index.c:131 performance: 0.156345564 s: read_index_preload 18:20:50.745717 read-cache.c:1254 performance: 0.011463075 s: refresh_index 18:20:50.755176 wt-status.c:623 performance: 0.008877929 s: wt_status_collect_changes_worktree 18:20:50.763768 wt-status.c:629 performance: 0.008471633 s: wt_status_collect_changes_index 18:20:50.854885 wt-status.c:632 performance: 0.090988721 s: wt_status_collect_untracked 18:20:50.857765 builtin/commit.c:1421 performance: 0.002789097 s: cmd_status:update_index 18:20:50.858411 trace.c:415 performance: 0.508647673 s: git command: 'git' 'status' linux-2.6 ========= Reference repo. Not too big. .._collect_status() drops 84%. Total time drops 42%. Base status ----------- 18:34:09.870122 builtin/commit.c:1394 performance: 0.000000385 s: cmd_status:setup 18:34:09.943218 read-cache.c:1407 performance: 0.072871177 s: read_index 18:34:09.943614 read-cache.c:1407 performance: 0.000000491 s: read_index 18:34:10.004364 preload-index.c:131 performance: 0.060748102 s: read_index_preload 18:34:10.008190 read-cache.c:1254 performance: 0.003714285 s: refresh_index 18:34:10.012087 wt-status.c:623 performance: 0.002775446 s: wt_status_collect_changes_worktree 18:34:10.016054 wt-status.c:629 performance: 0.003862140 s: wt_status_collect_changes_index 18:34:10.214747 wt-status.c:632 performance: 0.198604837 s: wt_status_collect_untracked 18:34:10.216102 builtin/commit.c:1421 performance: 0.001244166 s: cmd_status:update_index 18:34:10.216817 trace.c:415 performance: 0.347670735 s: git command: 'git' 'status' Populating status ----------------- 18:34:16.595102 builtin/commit.c:1394 performance: 0.000000456 s: cmd_status:setup 18:34:16.666600 read-cache.c:1407 performance: 0.070992413 s: read_index 18:34:16.667012 read-cache.c:1407 performance: 0.000000606 s: read_index 18:34:16.729375 preload-index.c:131 performance: 0.062362492 s: read_index_preload 18:34:16.732565 read-cache.c:1254 performance: 0.003075517 s: refresh_index 18:34:16.736148 wt-status.c:623 performance: 0.002422201 s: wt_status_collect_changes_worktree 18:34:16.739990 wt-status.c:629 performance: 0.003746618 s: wt_status_collect_changes_index 18:34:16.948505 wt-status.c:632 performance: 0.208426710 s: wt_status_collect_untracked 18:34:16.961744 builtin/commit.c:1421 performance: 0.013151887 s: cmd_status:update_index 18:34:16.962233 trace.c:415 performance: 0.368537535 s: git command: 'git' 'status' Populated status ---------------- 18:34:16.970026 builtin/commit.c:1394 performance: 0.000000631 s: cmd_status:setup 18:34:17.046235 read-cache.c:1407 performance: 0.075904673 s: read_index 18:34:17.046644 read-cache.c:1407 performance: 0.000000681 s: read_index 18:34:17.113564 preload-index.c:131 performance: 0.066920253 s: read_index_preload 18:34:17.117281 read-cache.c:1254 performance: 0.003604055 s: refresh_index 18:34:17.121115 wt-status.c:623 performance: 0.002508345 s: wt_status_collect_changes_worktree 18:34:17.125089 wt-status.c:629 performance: 0.003871636 s: wt_status_collect_changes_index 18:34:17.156089 wt-status.c:632 performance: 0.030895703 s: wt_status_collect_untracked 18:34:17.169861 builtin/commit.c:1421 performance: 0.013686404 s: cmd_status:update_index 18:34:17.170391 trace.c:415 performance: 0.201474531 s: git command: 'git' 'status' Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-01-14standardize usage info string formatAlex Henrie1-1/+1
This patch puts the usage info strings that were not already in docopt- like format into docopt-like format, which will be a litle easier for end users and a lot easier for translators. Changes include: - Placing angle brackets around fill-in-the-blank parameters - Putting dashes in multiword parameter names - Adding spaces to [-f|--foobar] to make [-f | --foobar] - Replacing <foobar>* with [<foobar>...] Signed-off-by: Alex Henrie <alexhenrie24@gmail.com> Reviewed-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-01lockfile.h: extract new header file for the functions in lockfile.cMichael Haggerty1-0/+1
Move the interface declaration for the functions in lockfile.c from cache.h to a new file, lockfile.h. Add #includes where necessary (and remove some redundant includes of cache.h by files that already include builtin.h). Move the documentation of the lock_file state diagram from lockfile.c to the new header file. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-01unable_to_lock_die(): rename function from unable_to_lock_index_die()Michael Haggerty1-1/+1
This function is used for other things besides the index, so rename it accordingly. Suggested-by: Jeff King <peff@peff.net> Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Reviewed-by: Ronnie Sahlberg <sahlberg@google.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-07-16Merge branch 'nd/split-index'Junio C Hamano1-7/+26
An experiment to use two files (the base file and incremental changes relative to it) to represent the index to reduce I/O cost of rewriting a large index when only small part of the working tree changes. * nd/split-index: (32 commits) t1700: new tests for split-index mode t2104: make sure split index mode is off for the version test read-cache: force split index mode with GIT_TEST_SPLIT_INDEX read-tree: note about dropping split-index mode or index version read-tree: force split-index mode off on --index-output rev-parse: add --shared-index-path to get shared index path update-index --split-index: do not split if $GIT_DIR is read only update-index: new options to enable/disable split index mode split-index: strip pathname of on-disk replaced entries split-index: do not invalidate cache-tree at read time split-index: the reading part split-index: the writing part read-cache: mark updated entries for split index read-cache: save deleted entries in split index read-cache: mark new entries for split index read-cache: split-index mode read-cache: save index SHA-1 after reading entry.c: update cache_changed if refresh_cache is set in checkout_entry() cache-tree: mark istate->cache_changed on prime_cache_tree() cache-tree: mark istate->cache_changed on cache tree update ...
2014-06-13update-index: new options to enable/disable split index modeNguyễn Thái Ngọc Duy1-0/+18
If you have a large work tree but only make changes in a subset, then $GIT_DIR/index's size should be stable after a while. If you change branches that touch something else, $GIT_DIR/index's size may grow large that it becomes as slow as the unified index. Do --split-index again occasionally to force all changes back to the shared index and keep $GIT_DIR/index small. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-13read-cache: mark updated entries for split indexNguyễn Thái Ngọc Duy1-0/+2
The large part of this patch just follows CE_ENTRY_CHANGED marks. replace_index_entry() is updated to update split_index->base->cache[] as well so base->cache[] does not reference to a freed entry. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-13cache-tree: mark istate->cache_changed on cache tree invalidationNguyễn Thái Ngọc Duy1-2/+2
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-13update-index: be specific what part of the index has changedNguyễn Thái Ngọc Duy1-3/+3
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-13read-cache: be specific what part of the index has changedNguyễn Thái Ngọc Duy1-3/+3
cache entry additions, removals and modifications are separated out. The rest of changes are still in the catch-all flag SOMETHING_CHANGED, which would be more specific later. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-13read-cache: new API write_locked_index instead of write_index/write_cacheNguyễn Thái Ngọc Duy1-2/+1
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-04update-index: fix segfault with missing --cacheinfo argumentJeff King1-0/+3
Running "git update-index --cacheinfo" without any further arguments results in a segfault rather than an error message. Commit ec160ae (update-index: teach --cacheinfo a new syntax "mode,sha1,path", 2014-03-23) added code to examine the format of the argument, but forgot to handle the NULL case. Returning an error from the parser is enough, since we then treat it as an old-style "--cacheinfo <mode> <sha1> <path>", and complain that we have less than 3 arguments to read. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-24update-index: teach --cacheinfo a new syntax "mode,sha1,path"Junio C Hamano1-3/+31
The "--cacheinfo" option is unusual in that it takes three option parameters. An option with an optional parameter is bad enough. An option with multiple parameters is simply insane. Introduce a new syntax that takes these three things concatenated together with a comma, which makes the command line syntax more uniform across subcommands, while retaining the traditional syntax for backward compatiblity. If we were designing the "update-index" subcommand from scratch today, it may probably have made sense to make this option (and possibly others) a command mode option that does not take any option parameter (hence no need for arg-help). But we do not live in such an ideal world, and as far as I can tell, the command still supports (and must support) mixed command modes in a single invocation, e.g. $ git update-index path1 --add path2 \ --cacheinfo 100644 $(git hash-object --stdin -w <path3) path3 \ path4 must make sure path1 is already in the index and update all of these four paths. So this is probably as far as we can go to fix this issue without risking to break people's existing scripts. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-02-27Merge branch 'nd/submodule-pathspec-ending-with-slash'Junio C Hamano1-1/+2
Allow "git cmd path/", when the 'path' is where a submodule is bound to the top-level working tree, to match 'path', despite the extra and unnecessary trailing slash. * nd/submodule-pathspec-ending-with-slash: clean: use cache_name_is_other() clean: replace match_pathspec() with dir_path_match() pathspec: pass directory indicator to match_pathspec_item() match_pathspec: match pathspec "foo/" against directory "foo" dir.c: prepare match_pathspec_item for taking more flags pathspec: rename match_pathspec_depth() to match_pathspec() pathspec: convert some match_pathspec_depth() to dir_path_match() pathspec: convert some match_pathspec_depth() to ce_path_match()
2014-02-24pathspec: convert some match_pathspec_depth() to ce_path_match()Nguyễn Thái Ngọc Duy1-1/+2
This helps reduce the number of match_pathspec_depth() call sites and show how match_pathspec_depth() is used. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-11-18read-cache.c: fix memory leaks caused by removed cache entriesKarsten Blees1-1/+4
When cache_entry structs are removed from index_state.cache, they are not properly freed. Freeing those entries wasn't possible before because we couldn't remove them from index_state.name_hash. Now that we _do_ remove the entries from name_hash, we can also free them. Add 'free(cache_entry)' to all call sites of name-hash.c::remove_name_hash in read-cache.c (we could free() directly in remove_name_hash(), but name-hash.c isn't concerned with cache_entry allocation at all). Accessing a cache_entry after removing it from the index is now no longer allowed, as the memory has been freed. The following functions need minor fixes (typically by copying ce->name before use): - builtin/rm.c::cmd_rm - builtin/update-index.c::do_reupdate - read-cache.c::read_index_unmerged - resolve-undo.c::unmerge_index_entry_at Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-11-18builtin/update-index.c: cleanup update_oneKarsten Blees1-21/+15
do_reupdate calls update_one with a cache_entry.name, there's no need for the extra sanitation / normalization that happens in prefix_path. cmd_update_index calls update_one with an already prefixed path, no need to prefix_path twice. Remove the extra prefix_path from update_one. Also remove the now unused 'prefix' and 'prefix_length' parameters. As of d089eba "setup: sanitize absolute and funny paths in get_pathspec()", prefix_path uncoditionally returns a copy, even if the passed in path isn't changed. Lets unconditionally free() the result. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-11-18fix 'git update-index --verbose --again' outputKarsten Blees1-1/+1
'git update-index --verbose' consistently reports paths relative to the work-tree root. The only exception is the '--again' option, which reports paths relative to the current working directory. Change do_reupdate to use non-prefixed paths. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-09-09Merge branch 'jl/submodule-mv'Junio C Hamano1-2/+4
"git mv A B" when moving a submodule A does "the right thing", inclusing relocating its working tree and adjusting the paths in the .gitmodules file. * jl/submodule-mv: (53 commits) rm: delete .gitmodules entry of submodules removed from the work tree mv: update the path entry in .gitmodules for moved submodules submodule.c: add .gitmodules staging helper functions mv: move submodules using a gitfile mv: move submodules together with their work trees rm: do not set a variable twice without intermediate reading. t6131 - skip tests if on case-insensitive file system parse_pathspec: accept :(icase)path syntax pathspec: support :(glob) syntax pathspec: make --literal-pathspecs disable pathspec magic pathspec: support :(literal) syntax for noglob pathspec kill limit_pathspec_to_literal() as it's only used by parse_pathspec() parse_pathspec: preserve prefix length via PATHSPEC_PREFIX_ORIGIN parse_pathspec: make sure the prefix part is wildcard-free rename field "raw" to "_raw" in struct pathspec tree-diff: remove the use of pathspec's raw[] in follow-rename codepath remove match_pathspec() in favor of match_pathspec_depth() remove init_pathspec() in favor of parse_pathspec() remove diff_tree_{setup,release}_paths convert common_prefix() to use struct pathspec ...
2013-07-15convert some get_pathspec() calls to parse_pathspec()Nguyễn Thái Ngọc Duy1-2/+3
These call sites follow the pattern: paths = get_pathspec(prefix, argv); init_pathspec(&pathspec, paths); which can be converted into a single parse_pathspec() call. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-07-15move struct pathspec and related functions to pathspec.[ch]Nguyễn Thái Ngọc Duy1-0/+1
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-07-09Convert "struct cache_entry *" to "const ..." wherever possibleNguyễn Thái Ngọc Duy1-7/+7
I attempted to make index_state->cache[] a "const struct cache_entry **" to find out how existing entries in index are modified and where. The question I have is what do we do if we really need to keep track of on-disk changes in the index. The result is - diff-lib.c: setting CE_UPTODATE - name-hash.c: setting CE_HASHED - preload-index.c, read-cache.c, unpack-trees.c and builtin/update-index: obvious - entry.c: write_entry() may refresh the checked out entry via fill_stat_cache_info(). This causes "non-const struct cache_entry *" in builtin/apply.c, builtin/checkout-index.c and builtin/checkout.c - builtin/ls-files.c: --with-tree changes stagemask and may set CE_UPDATE Of these, write_entry() and its call sites are probably most interesting because it modifies on-disk info. But this is stat info and can be retrieved via refresh, at least for porcelain commands. Other just uses ce_flags for local purposes. So, keeping track of "dirty" entries is just a matter of setting a flag in index modification functions exposed by read-cache.c. Except unpack-trees, the rest of the code base does not do anything funny behind read-cache's back. The actual patch is less valueable than the summary above. But if anyone wants to re-identify the above sites. Applying this patch, then this: diff --git a/cache.h b/cache.h index 430d021..1692891 100644 --- a/cache.h +++ b/cache.h @@ -267,7 +267,7 @@ static inline unsigned int canon_mode(unsigned int mode) #define cache_entry_size(len) (offsetof(struct cache_entry,name) + (len) + 1) struct index_state { - struct cache_entry **cache; + const struct cache_entry **cache; unsigned int version; unsigned int cache_nr, cache_alloc, cache_changed; struct string_list *resolve_undo; will help quickly identify them without bogus warnings. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-02-23update-index: allow "-h" to also display optionsAntoine Pelisse1-1/+1
Even though "git update-index" was updated to use parse-options infrastracture some time ago to make it possible to show list of options with usage_with_options(), "git update-index -h" only shows the usage. Detect this case and call usage_with_options() to show the list of options as well. Signed-off-by: Antoine Pelisse <apelisse@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-11-20Merge branch 'kb/preload-index-more'Junio C Hamano1-0/+1
Use preloadindex in more places, which has a nice speedup on systems with slow stat calls (and even on Linux). * kb/preload-index-more: update-index/diff-index: use core.preloadindex to improve performance
2012-11-02update-index/diff-index: use core.preloadindex to improve performanceKarsten Blees1-0/+1
'update-index --refresh' and 'diff-index' (without --cached) don't honor the core.preloadindex setting yet. Porcelain commands using these (such as git [svn] rebase) suffer from this, especially on Windows. Use read_cache_preload to improve performance. Additionally, in builtin/diff.c, don't preload index status if we don't access the working copy (--cached). Results with msysgit on WebKit repo (2GB in 200k files): | update-index | diff-index | rebase ----------------+--------------+------------+--------- msysgit-v1.8.0 | 9.157s | 10.536s | 42.791s + preloadindex | 9.157s | 10.536s | 28.725s + this patch | 2.329s | 2.752s | 15.152s + fscache [1] | 0.731s | 1.171s | 8.877s [1] https://github.com/kblees/git/tree/kb/fscache-v3 Thanks-to: Albert Krawczyk <pro-logic@optusnet.com.au> Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Jeff King <peff@peff.net>
2012-08-22i18n: update-index: mark parseopt strings for translationNguyễn Thái Ngọc Duy1-28/+28
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-11Strip namelen out of ce_flags into a ce_namelen fieldThomas Gummerer1-3/+6
Strip the name length from the ce_flags field and move it into its own ce_namelen field in struct cache_entry. This will both give us a tiny bit of a performance enhancement when working with long pathnames and is a refactoring for more readability of the code. It enhances readability, by making it more clear what is a flag, and where the length is stored and make it clear which functions use stages in comparisions and which only use the length. It also makes CE_NAMEMASK private, so that users don't mistakenly write the name length in the flags. Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-06-28Merge branch 'hv/submodule-update-nuke-submodules'Junio C Hamano1-6/+0
"git add" allows adding a regular file to the path where a submodule used to exist, but "git update-index" does not allow an equivalent operation to Porcelain writers.
2012-06-11update-index: allow overwriting existing submodule index entriesHeiko Voigt1-6/+0
In commit e01105 Linus introduced gitlinks to update-index. He explains that he thinks it is not the right thing to replace a gitlink with something else. That commit is from the very first beginnings of submodule support. Since then we have gotten a lot closer to being able to remove a submodule without losing its history. This check prevents such a use case, so I think this assumption has changed. Additionally in the git add codepath we do not have such a check, so for consistency reasons I think removing this check is the correct thing to do. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-04update-index: upgrade/downgrade on-disk index versionJunio C Hamano1-0/+14
With the "--index-version <n>" parameter, write the index out in the specified version. With this, an index file that is written in newer format (say v4) can be downgraded to be read by older versions of Git. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-06-20plug a few coverity-spotted leaksJim Meyering1-1/+3
Signed-off-by: Jim Meyering <meyering@redhat.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-05-09index_fd(): turn write_object and format_check arguments into one flagJunio C Hamano1-1/+2
The "format_check" parameter tucked after the existing parameters is too ugly an afterthought to live in any reasonable API. Combine it with the other boolean parameter "write_object" into a single "flags" parameter. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-02-03Convert ce_path_match() to use struct pathspecNguyễn Thái Ngọc Duy1-2/+6
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-12-12Merge branch 'jn/parse-options-extra'Junio C Hamano1-150/+244
* jn/parse-options-extra: update-index: migrate to parse-options API setup: save prefix (original cwd relative to toplevel) in startup_info parse-options: make resuming easier after PARSE_OPT_STOP_AT_NON_OPTION parse-options: allow git commands to invent new option types parse-options: never suppress arghelp if LITERAL_ARGHELP is set parse-options: do not infer PARSE_OPT_NOARG from option type parse-options: sanity check PARSE_OPT_NOARG flag parse-options: move NODASH sanity checks to parse_options_check parse-options: clearer reporting of API misuse parse-options: Don't call parse_options_check() so much
2010-12-12Merge branch 'jn/git-cmd-h-bypass-setup'Junio C Hamano1-0/+3
* jn/git-cmd-h-bypass-setup: update-index -h: show usage even with corrupt index merge -h: show usage even with corrupt index ls-files -h: show usage even with corrupt index gc -h: show usage even with broken configuration commit/status -h: show usage even with broken configuration checkout-index -h: show usage even in an invalid repository branch -h: show usage even in an invalid repository Conflicts: builtin/merge.c
2010-12-07update-index: migrate to parse-options APIJonathan Nieder1-149/+243
--refresh and --really-refresh accept flags (like -q) and modify an error indicator. It might make sense to make the error indicator global, but just pass the flags and a pointer to the error indicator in a struct instead. --cacheinfo wants 3 arguments. Use the OPTION_LOWLEVEL_CALLBACK extension to grab them and PARSE_OPT_NOARG to disallow the "sticked" --cacheinfo=foo form. (The resulting message $ git update-index --cacheinfo=foo error: option `cacheinfo' takes no value is unfortunately incorrect.) --assume-unchanged and --no-assume-unchanged probably should use the OPT_UYN feature; but use a callback for now so the existing MARK_FLAG and UNMARK_FLAG values can be used. --stdin and --index-info are still constrained to be the last argument (implemented using the OPTION_LOWLEVEL_CALLBACK extension). --unresolve and --again consume all arguments that come after them (also using OPTION_LOWLEVEL_CALLBACK). The order of options matters. Each path on the command line is affected only by the options that come before it. A custom argument-parsing loop with parse_options_step() brings that about. In exchange for all the fuss, we get the usual perks: support for un-sticked options, better usage error messages, more useful -h output, and argument parsing code that should be easier to tweak in the future. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-10-22update-index -h: show usage even with corrupt indexNguyễn Thái Ngọc Duy1-0/+3
When trying to fix up a corrupt repository, one might prefer that "update-index -h" print an accurate usage message and exit rather than reading the repository and complaining about the corruption. [jn: with rewritten log message and tests] Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-10-08Fix {update,checkout}-index usage stringsŠtěpán Němec1-1/+1
The `<file>' argument is optional in both cases (the man pages are already correct). Signed-off-by: Štěpán Němec <stepnem@gmail.com> Acked-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-02-22Move 'builtin-*' into a 'builtin/' subdirectoryLinus Torvalds1-0/+788
This shrinks the top-level directory a bit, and makes it much more pleasant to use auto-completion on the thing. Instead of [torvalds@nehalem git]$ em buil<tab> Display all 180 possibilities? (y or n) [torvalds@nehalem git]$ em builtin-sh builtin-shortlog.c builtin-show-branch.c builtin-show-ref.c builtin-shortlog.o builtin-show-branch.o builtin-show-ref.o [torvalds@nehalem git]$ em builtin-shor<tab> builtin-shortlog.c builtin-shortlog.o [torvalds@nehalem git]$ em builtin-shortlog.c you get [torvalds@nehalem git]$ em buil<tab> [type] builtin/ builtin.h [torvalds@nehalem git]$ em builtin [auto-completes to] [torvalds@nehalem git]$ em builtin/sh<tab> [type] shortlog.c shortlog.o show-branch.c show-branch.o show-ref.c show-ref.o [torvalds@nehalem git]$ em builtin/sho [auto-completes to] [torvalds@nehalem git]$ em builtin/shor<tab> [type] shortlog.c shortlog.o [torvalds@nehalem git]$ em builtin/shortlog.c which doesn't seem all that different, but not having that annoying break in "Display all 180 possibilities?" is quite a relief. NOTE! If you do this in a clean tree (no object files etc), or using an editor that has auto-completion rules that ignores '*.o' files, you won't see that annoying 'Display all 180 possibilities?' message - it will just show the choices instead. I think bash has some cut-off around 100 choices or something. So the reason I see this is that I'm using an odd editory, and thus don't have the rules to cut down on auto-completion. But you can simulate that by using 'ls' instead, or something similar. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>