mirror of
https://github.com/git/git.git
synced 2024-06-10 04:06:11 +02:00
f32dde8c12
The previous changes to the line-log machinery focused on making the
first result appear faster. This was achieved by no longer walking the
entire commit history before returning the early results. There is still
another way to improve the performance: walk most commits much faster.
Let's use the changed-path Bloom filters to reduce time spent computing
diffs.
Since the line-log computation requires opening blobs and checking the
content-diff, there is still a lot of necessary computation that cannot
be replaced with changed-path Bloom filters. The part that we can reduce
is most effective when checking the history of a file that is deep in
several directories and those directories are modified frequently. In
this case, the computation to check if a commit is TREESAME to its first
parent takes a large fraction of the time. That is ripe for improvement
with changed-path Bloom filters.
We must ensure that prepare_to_use_bloom_filters() is called in
revision.c so that the bloom_filter_settings are loaded into the struct
rev_info from the commit-graph. Of course, some cases are still
forbidden, but in the line-log case the pathspec is provided in a
different way than normal.
Since multiple paths and segments could be requested, we compute the
struct bloom_key data dynamically during the commit walk. This could
likely be improved, but adds code complexity that is not valuable at
this time.
There are two cases to care about: merge commits and "ordinary" commits.
Merge commits have multiple parents, but if we are TREESAME to our first
parent in every range, then pass the blame for all ranges to the first
parent. Ordinary commits have the same condition, but each is done
slightly differently in the process_ranges_[merge|ordinary]_commit()
methods. By checking if the changed-path Bloom filter can guarantee
TREESAME, we can avoid that tree-diff cost. If the filter says "probably
changed", then we need to run the tree-diff and then the blob-diff if
there was a real edit.
The Linux kernel repository is a good testing ground for the performance
improvements claimed here. There are two different cases to test. The
first is the "entire history" case, where we output the entire history
to /dev/null to see how long it would take to compute the full line-log
history. The second is the "first result" case, where we find how long
it takes to show the first value, which is an indicator of how quickly a
user would see responses when waiting at a terminal.
To test, I selected the paths that were changed most frequently in the
top 10,000 commits using this command (stolen from StackOverflow [1]):
git log --pretty=format: --name-only -n 10000 | sort | \
uniq -c | sort -rg | head -10
which results in
121 MAINTAINERS
63 fs/namei.c
60 arch/x86/kvm/cpuid.c
59 fs/io_uring.c
58 arch/x86/kvm/vmx/vmx.c
51 arch/x86/kvm/x86.c
45 arch/x86/kvm/svm.c
42 fs/btrfs/disk-io.c
42 Documentation/scsi/index.rst
(along with a bogus first result). It appears that the path
arch/x86/kvm/svm.c was renamed, so we ignore that entry. This leaves the
following results for the real command time:
| | Entire History | First Result |
| Path | Before | After | Before | After |
|------------------------------|--------|--------|--------|--------|
| MAINTAINERS | 4.26 s | 3.87 s | 0.41 s | 0.39 s |
| fs/namei.c | 1.99 s | 0.99 s | 0.42 s | 0.21 s |
| arch/x86/kvm/cpuid.c | 5.28 s | 1.12 s | 0.16 s | 0.09 s |
| fs/io_uring.c | 4.34 s | 0.99 s | 0.94 s | 0.27 s |
| arch/x86/kvm/vmx/vmx.c | 5.01 s | 1.34 s | 0.21 s | 0.12 s |
| arch/x86/kvm/x86.c | 2.24 s | 1.18 s | 0.21 s | 0.14 s |
| fs/btrfs/disk-io.c | 1.82 s | 1.01 s | 0.06 s | 0.05 s |
| Documentation/scsi/index.rst | 3.30 s | 0.89 s | 1.46 s | 0.03 s |
It is worth noting that the least speedup comes for the MAINTAINERS file
which is
* edited frequently,
* low in the directory heirarchy, and
* quite a large file.
All of those points lead to spending more time doing the blob diff and
less time doing the tree diff. Still, we see some improvement in that
case and significant improvement in other cases. A 2-4x speedup is
likely the more typical case as opposed to the small 5% change for that
file.
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
||
|---|---|---|
| .github | ||
| Documentation | ||
| block-sha1 | ||
| builtin | ||
| ci | ||
| compat | ||
| contrib | ||
| ewah | ||
| git-gui | ||
| gitk-git | ||
| gitweb | ||
| mergetools | ||
| negotiator | ||
| perl | ||
| po | ||
| ppc | ||
| refs | ||
| sha1collisiondetection@855827c583 | ||
| sha1dc | ||
| sha256 | ||
| t | ||
| templates | ||
| trace2 | ||
| vcs-svn | ||
| xdiff | ||
| .cirrus.yml | ||
| .clang-format | ||
| .editorconfig | ||
| .gitattributes | ||
| .gitignore | ||
| .gitmodules | ||
| .mailmap | ||
| .travis.yml | ||
| .tsan-suppressions | ||
| CODE_OF_CONDUCT.md | ||
| COPYING | ||
| GIT-VERSION-GEN | ||
| INSTALL | ||
| LGPL-2.1 | ||
| Makefile | ||
| README.md | ||
| RelNotes | ||
| abspath.c | ||
| aclocal.m4 | ||
| add-interactive.c | ||
| add-interactive.h | ||
| add-patch.c | ||
| advice.c | ||
| advice.h | ||
| alias.c | ||
| alias.h | ||
| alloc.c | ||
| alloc.h | ||
| apply.c | ||
| apply.h | ||
| archive-tar.c | ||
| archive-zip.c | ||
| archive.c | ||
| archive.h | ||
| argv-array.c | ||
| argv-array.h | ||
| attr.c | ||
| attr.h | ||
| azure-pipelines.yml | ||
| banned.h | ||
| base85.c | ||
| bisect.c | ||
| bisect.h | ||
| blame.c | ||
| blame.h | ||
| blob.c | ||
| blob.h | ||
| bloom.c | ||
| bloom.h | ||
| branch.c | ||
| branch.h | ||
| builtin.h | ||
| bulk-checkin.c | ||
| bulk-checkin.h | ||
| bundle.c | ||
| bundle.h | ||
| cache-tree.c | ||
| cache-tree.h | ||
| cache.h | ||
| chdir-notify.c | ||
| chdir-notify.h | ||
| check-builtins.sh | ||
| check_bindir | ||
| checkout.c | ||
| checkout.h | ||
| color.c | ||
| color.h | ||
| column.c | ||
| column.h | ||
| combine-diff.c | ||
| command-list.txt | ||
| commit-graph.c | ||
| commit-graph.h | ||
| commit-reach.c | ||
| commit-reach.h | ||
| commit-slab-decl.h | ||
| commit-slab-impl.h | ||
| commit-slab.h | ||
| commit.c | ||
| commit.h | ||
| common-main.c | ||
| config.c | ||
| config.h | ||
| config.mak.dev | ||
| config.mak.in | ||
| config.mak.uname | ||
| configure.ac | ||
| connect.c | ||
| connect.h | ||
| connected.c | ||
| connected.h | ||
| convert.c | ||
| convert.h | ||
| copy.c | ||
| credential-cache--daemon.c | ||
| credential-cache.c | ||
| credential-store.c | ||
| credential.c | ||
| credential.h | ||
| csum-file.c | ||
| csum-file.h | ||
| ctype.c | ||
| daemon.c | ||
| date.c | ||
| decorate.c | ||
| decorate.h | ||
| delta-islands.c | ||
| delta-islands.h | ||
| delta.h | ||
| detect-compiler | ||
| diff-delta.c | ||
| diff-lib.c | ||
| diff-no-index.c | ||
| diff.c | ||
| diff.h | ||
| diffcore-break.c | ||
| diffcore-delta.c | ||
| diffcore-order.c | ||
| diffcore-pickaxe.c | ||
| diffcore-rename.c | ||
| diffcore.h | ||
| dir-iterator.c | ||
| dir-iterator.h | ||
| dir.c | ||
| dir.h | ||
| editor.c | ||
| entry.c | ||
| environment.c | ||
| exec-cmd.c | ||
| exec-cmd.h | ||
| fast-import.c | ||
| fetch-negotiator.c | ||
| fetch-negotiator.h | ||
| fetch-pack.c | ||
| fetch-pack.h | ||
| fmt-merge-msg.h | ||
| fsck.c | ||
| fsck.h | ||
| fsmonitor.c | ||
| fsmonitor.h | ||
| fuzz-commit-graph.c | ||
| fuzz-pack-headers.c | ||
| fuzz-pack-idx.c | ||
| generate-cmdlist.sh | ||
| gettext.c | ||
| gettext.h | ||
| git-add--interactive.perl | ||
| git-archimport.perl | ||
| git-bisect.sh | ||
| git-compat-util.h | ||
| git-cvsexportcommit.perl | ||
| git-cvsimport.perl | ||
| git-cvsserver.perl | ||
| git-difftool--helper.sh | ||
| git-filter-branch.sh | ||
| git-instaweb.sh | ||
| git-merge-octopus.sh | ||
| git-merge-one-file.sh | ||
| git-merge-resolve.sh | ||
| git-mergetool--lib.sh | ||
| git-mergetool.sh | ||
| git-p4.py | ||
| git-parse-remote.sh | ||
| git-quiltimport.sh | ||
| git-rebase--preserve-merges.sh | ||
| git-request-pull.sh | ||
| git-send-email.perl | ||
| git-sh-i18n.sh | ||
| git-sh-setup.sh | ||
| git-submodule.sh | ||
| git-svn.perl | ||
| git-web--browse.sh | ||
| git.c | ||
| git.rc | ||
| gpg-interface.c | ||
| gpg-interface.h | ||
| graph.c | ||
| graph.h | ||
| grep.c | ||
| grep.h | ||
| hash.h | ||
| hashmap.c | ||
| hashmap.h | ||
| help.c | ||
| help.h | ||
| hex.c | ||
| http-backend.c | ||
| http-fetch.c | ||
| http-push.c | ||
| http-walker.c | ||
| http.c | ||
| http.h | ||
| ident.c | ||
| imap-send.c | ||
| interdiff.c | ||
| interdiff.h | ||
| iterator.h | ||
| json-writer.c | ||
| json-writer.h | ||
| khash.h | ||
| kwset.c | ||
| kwset.h | ||
| levenshtein.c | ||
| levenshtein.h | ||
| line-log.c | ||
| line-log.h | ||
| line-range.c | ||
| line-range.h | ||
| linear-assignment.c | ||
| linear-assignment.h | ||
| list-objects-filter-options.c | ||
| list-objects-filter-options.h | ||
| list-objects-filter.c | ||
| list-objects-filter.h | ||
| list-objects.c | ||
| list-objects.h | ||
| list.h | ||
| ll-merge.c | ||
| ll-merge.h | ||
| lockfile.c | ||
| lockfile.h | ||
| log-tree.c | ||
| log-tree.h | ||
| ls-refs.c | ||
| ls-refs.h | ||
| mailinfo.c | ||
| mailinfo.h | ||
| mailmap.c | ||
| mailmap.h | ||
| match-trees.c | ||
| mem-pool.c | ||
| mem-pool.h | ||
| merge-blobs.c | ||
| merge-blobs.h | ||
| merge-recursive.c | ||
| merge-recursive.h | ||
| merge.c | ||
| mergesort.c | ||
| mergesort.h | ||
| midx.c | ||
| midx.h | ||
| name-hash.c | ||
| notes-cache.c | ||
| notes-cache.h | ||
| notes-merge.c | ||
| notes-merge.h | ||
| notes-utils.c | ||
| notes-utils.h | ||
| notes.c | ||
| notes.h | ||
| object-store.h | ||
| object.c | ||
| object.h | ||
| oidmap.c | ||
| oidmap.h | ||
| oidset.c | ||
| oidset.h | ||
| pack-bitmap-write.c | ||
| pack-bitmap.c | ||
| pack-bitmap.h | ||
| pack-check.c | ||
| pack-objects.c | ||
| pack-objects.h | ||
| pack-revindex.c | ||
| pack-revindex.h | ||
| pack-write.c | ||
| pack.h | ||
| packfile.c | ||
| packfile.h | ||
| pager.c | ||
| parse-options-cb.c | ||
| parse-options.c | ||
| parse-options.h | ||
| patch-delta.c | ||
| patch-ids.c | ||
| patch-ids.h | ||
| path.c | ||
| path.h | ||
| pathspec.c | ||
| pathspec.h | ||
| pkt-line.c | ||
| pkt-line.h | ||
| preload-index.c | ||
| pretty.c | ||
| pretty.h | ||
| prio-queue.c | ||
| prio-queue.h | ||
| progress.c | ||
| progress.h | ||
| promisor-remote.c | ||
| promisor-remote.h | ||
| prompt.c | ||
| prompt.h | ||
| protocol.c | ||
| protocol.h | ||
| quote.c | ||
| quote.h | ||
| range-diff.c | ||
| range-diff.h | ||
| reachable.c | ||
| reachable.h | ||
| read-cache.c | ||
| rebase-interactive.c | ||
| rebase-interactive.h | ||
| rebase.c | ||
| rebase.h | ||
| ref-filter.c | ||
| ref-filter.h | ||
| reflog-walk.c | ||
| reflog-walk.h | ||
| refs.c | ||
| refs.h | ||
| refspec.c | ||
| refspec.h | ||
| remote-curl.c | ||
| remote-testsvn.c | ||
| remote.c | ||
| remote.h | ||
| replace-object.c | ||
| replace-object.h | ||
| repo-settings.c | ||
| repository.c | ||
| repository.h | ||
| rerere.c | ||
| rerere.h | ||
| resolve-undo.c | ||
| resolve-undo.h | ||
| revision.c | ||
| revision.h | ||
| run-command.c | ||
| run-command.h | ||
| send-pack.c | ||
| send-pack.h | ||
| sequencer.c | ||
| sequencer.h | ||
| serve.c | ||
| serve.h | ||
| server-info.c | ||
| setup.c | ||
| sh-i18n--envsubst.c | ||
| sha1-array.c | ||
| sha1-array.h | ||
| sha1-file.c | ||
| sha1-lookup.c | ||
| sha1-lookup.h | ||
| sha1-name.c | ||
| sha1dc_git.c | ||
| sha1dc_git.h | ||
| shallow.c | ||
| shell.c | ||
| shortlog.h | ||
| sideband.c | ||
| sideband.h | ||
| sigchain.c | ||
| sigchain.h | ||
| split-index.c | ||
| split-index.h | ||
| stable-qsort.c | ||
| strbuf.c | ||
| strbuf.h | ||
| streaming.c | ||
| streaming.h | ||
| string-list.c | ||
| string-list.h | ||
| sub-process.c | ||
| sub-process.h | ||
| submodule-config.c | ||
| submodule-config.h | ||
| submodule.c | ||
| submodule.h | ||
| symlinks.c | ||
| tag.c | ||
| tag.h | ||
| tar.h | ||
| tempfile.c | ||
| tempfile.h | ||
| thread-utils.c | ||
| thread-utils.h | ||
| tmp-objdir.c | ||
| tmp-objdir.h | ||
| trace.c | ||
| trace.h | ||
| trace2.c | ||
| trace2.h | ||
| trailer.c | ||
| trailer.h | ||
| transport-helper.c | ||
| transport-internal.h | ||
| transport.c | ||
| transport.h | ||
| tree-diff.c | ||
| tree-walk.c | ||
| tree-walk.h | ||
| tree.c | ||
| tree.h | ||
| unicode-width.h | ||
| unimplemented.sh | ||
| unix-socket.c | ||
| unix-socket.h | ||
| unpack-trees.c | ||
| unpack-trees.h | ||
| upload-pack.c | ||
| upload-pack.h | ||
| url.c | ||
| url.h | ||
| urlmatch.c | ||
| urlmatch.h | ||
| usage.c | ||
| userdiff.c | ||
| userdiff.h | ||
| utf8.c | ||
| utf8.h | ||
| varint.c | ||
| varint.h | ||
| version.c | ||
| version.h | ||
| versioncmp.c | ||
| walker.c | ||
| walker.h | ||
| wildmatch.c | ||
| wildmatch.h | ||
| worktree.c | ||
| worktree.h | ||
| wrap-for-bin.sh | ||
| wrapper.c | ||
| write-or-die.c | ||
| ws.c | ||
| wt-status.c | ||
| wt-status.h | ||
| xdiff-interface.c | ||
| xdiff-interface.h | ||
| zlib.c | ||
Git - fast, scalable, distributed revision control system
Git is a fast, scalable, distributed revision control system with an unusually rich command set that provides both high-level operations and full access to internals.
Git is an Open Source project covered by the GNU General Public License version 2 (some parts of it are under different licenses, compatible with the GPLv2). It was originally written by Linus Torvalds with help of a group of hackers around the net.
Please read the file INSTALL for installation instructions.
Many Git online resources are accessible from https://git-scm.com/ including full documentation and Git related tools.
See Documentation/gittutorial.txt to get started, then see
Documentation/giteveryday.txt for a useful minimum set of commands, and
Documentation/git-<commandname>.txt for documentation of each command.
If git has been correctly installed, then the tutorial can also be
read with man gittutorial or git help tutorial, and the
documentation of each command with man git-<commandname> or git help <commandname>.
CVS users may also want to read Documentation/gitcvs-migration.txt
(man gitcvs-migration or git help cvs-migration if git is
installed).
The user discussion and development of Git take place on the Git mailing list -- everyone is welcome to post bug reports, feature requests, comments and patches to git@vger.kernel.org (read Documentation/SubmittingPatches for instructions on patch submission). To subscribe to the list, send an email with just "subscribe git" in the body to majordomo@vger.kernel.org. The mailing list archives are available at https://lore.kernel.org/git/, http://marc.info/?l=git and other archival sites.
Issues which are security relevant should be disclosed privately to the Git Security mailing list git-security@googlegroups.com.
The maintainer frequently sends the "What's cooking" reports that list the current status of various development topics to the mailing list. The discussion following them give a good reference for project status, development direction and remaining tasks.
The name "git" was given by Linus Torvalds when he wrote the very first version. He described the tool as "the stupid content tracker" and the name as (depending on your mood):
- random three-letter combination that is pronounceable, and not actually used by any common UNIX command. The fact that it is a mispronunciation of "get" may or may not be relevant.
- stupid. contemptible and despicable. simple. Take your pick from the dictionary of slang.
- "global information tracker": you're in a good mood, and it actually works for you. Angels sing, and a light suddenly fills the room.
- "goddamn idiotic truckload of sh*t": when it breaks