1
0
Fork 0
mirror of https://github.com/git/git.git synced 2024-05-24 06:06:14 +02:00
Commit Graph

210 Commits

Author SHA1 Message Date
Taylor Blau b31e2680c4 ref-filter.c: find disjoint pattern prefixes
Since cfe004a5a9 (ref-filter: limit traversal to prefix, 2017-05-22),
the ref-filter code has sought to limit the traversals to a prefix of
the given patterns.

That code stopped short of handling more than one pattern, because it
means invoking 'for_each_ref_in' multiple times. If we're not careful
about which patterns overlap, we will output the same refs multiple
times.

For instance, consider the set of patterns 'refs/heads/a/*',
'refs/heads/a/b/c', and 'refs/tags/v1.0.0'. If we naïvely ran:

  for_each_ref_in("refs/heads/a/*", ...);
  for_each_ref_in("refs/heads/a/b/c", ...);
  for_each_ref_in("refs/tags/v1.0.0", ...);

we would see 'refs/heads/a/b/c' (and everything underneath it) twice.

Instead, we want to partition the patterns into disjoint sets, where we
know that no ref will be matched by any two patterns in different sets.
In the above, these are:

  - {'refs/heads/a/*', 'refs/heads/a/b/c'}, and
  - {'refs/tags/v1.0.0'}

Given one of these disjoint sets, what is a suitable pattern to pass to
'for_each_ref_in'? One approach is to compute the longest common prefix
over all elements in that disjoint set, and let the caller cull out the
refs they didn't want. Computing the longest prefix means that in most
cases, we won't match too many things the caller would like to ignore.

The longest common prefixes of the above are:

  - {'refs/heads/a/*', 'refs/heads/a/b/c'} -> refs/heads/a/*
  - {'refs/tags/v1.0.0'}                   -> refs/tags/v1.0.0

We instead invoke:

  for_each_ref_in("refs/heads/a/*", ...);
  for_each_ref_in("refs/tags/v1.0.0", ...);

Which provides us with the refs we were looking for with a minimal
amount of extra cruft, but never a duplicate of the ref we asked for.

Implemented here is an algorithm which accomplishes the above, which
works as follows:

  1. Lexicographically sort the given list of patterns.

  2. Initialize 'prefix' to the empty string, where our goal is to
     build each element in the above set of longest common prefixes.

  3. Consider each pattern in the given set, and emit 'prefix' if it
     reaches the end of a pattern, or touches a wildcard character. The
     end of a string is treated as if it precedes a wildcard. (Note that
     there is some room for future work to detect that, e.g., 'a?b' and
     'abc' are disjoint).

  4. Otherwise, recurse on step (3) with the slice of the list
     corresponding to our current prefix (i.e., the subset of patterns
     that have our prefix as a literal string prefix.)

This algorithm is 'O(kn + n log(n))', where 'k' is max(len(pattern)) for
each pattern in the list, and 'n' is len(patterns).

By discovering this set of interesting patterns, we reduce the runtime
of multi-pattern 'git for-each-ref' (and other ref traversals) from
O(N) to O(n log(N)), where 'N' is the total number of packed references.

Running 'git for-each-ref refs/tags/a refs/tags/b' on a repository with
10,000,000 refs in 'refs/tags/huge-N', my best-of-five times drop from:

  real    0m5.805s
  user    0m5.188s
  sys     0m0.468s

to:

  real    0m0.001s
  user    0m0.000s
  sys     0m0.000s

On linux.git, the times to dig out two of the latest -rc tags drops from
0.002s to 0.001s, so the change on repositories with fewer tags is much
less noticeable.

Co-authored-by: Jeff King <peff@peff.net>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-27 13:14:06 -07:00
Junio C Hamano f560a4d159 Merge branch 'dr/ref-filter-push-track-fix'
%(push:track) token used in the --format option to "git
for-each-ref" and friends was not showing the right branch, which
has been fixed.

* dr/ref-filter-push-track-fix:
  ref-filter: use correct branch for %(push:track)
2019-05-09 00:37:26 +09:00
Damien Robert c646d0934e ref-filter: use correct branch for %(push:track)
In ref-filter.c, when processing the atom %(push:track), the
ahead/behind values are computed using `stat_tracking_info` which refers
to the upstream branch.

Fix that by introducing a new flag `for_push` in `stat_tracking_info`
in remote.c, which does the same thing but for the push branch.
Update the few callers of `stat_tracking_info` to handle this flag. This
ensure that whenever we use this function in the future, we are careful
to specify is this should apply to the upstream or the push branch.

This bug was not detected in t/t6300-for-each-ref.sh because in the test
for push:track, both the upstream and the push branches were behind by 1
from the local branch. Change the test so that the upstream branch is
behind by 1 while the push branch is ahead by 1. This allows us to test
that %(push:track) refers to the correct branch.

This changes the expected value of some following tests (by introducing
new references), so update them too.

Signed-off-by: Damien Robert <damien.olivier.robert+git@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18 09:17:41 +09:00
Jeff King 95be717cd5 parse_opt_ref_sorting: always use with NONEG flag
The "--sort" parameter of for-each-ref, etc, does not handle negation,
and instead returns an error to the parse-options code. But neither
piece of code prints anything for the user, which may leave them
confused:

  $ git for-each-ref --no-sort
  $ echo $?
  129

As the comment in the callback function notes, this probably should
clear the list, which would make it consistent with other list-like
options (i.e., anything that uses OPT_STRING_LIST currently).
Unfortunately that's a bit tricky due to the way the ref-filter code
works. But in the meantime, let's at least make the error a little less
confusing:

  - switch to using PARSE_OPT_NONEG in the option definition, which will
    cause the options code to produce a useful message

  - since this was cut-and-pasted to four different spots, let's define
    a single OPT_REF_SORT() macro that we can use everywhere

  - the callback can use BUG_ON_OPT_NEG() to make sure the correct flags
    are used (incidentally, this also satisfies -Wunused-parameters,
    since we're now looking at "unset")

  - expand the comment into a NEEDSWORK to make it clear that the
    direction is right, but the details need to be worked out

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-21 12:03:35 +09:00
Junio C Hamano b0e7fb2e5c Merge branch 'nd/completion-more-parameters'
The command line completion (in contrib/) has been taught to
complete more subcommand parameters.

* nd/completion-more-parameters:
  completion: add more parameter value completion
2019-03-07 09:59:58 +09:00
Nguyễn Thái Ngọc Duy 5a59a2301f completion: add more parameter value completion
This adds value completion for a couple more paramters. To make it
easier to maintain these hard coded lists, add a comment at the original
list/code to remind people to update git-completion.bash too.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-20 12:31:56 -08:00
Jeff King 5c326d1252 ref-filter: drop unused "sz" parameters
Many of our grab_* functions, which parse the object content, take a
buf/sz pair of the object bytes. However, the functions which actually
parse the buffers (like find_wholine() and find_subpos()) never look at
"sz", and instead use functions like strchr() and strchrnul() that
assume the result is NUL-terminated.

This is OK in practice (and common for Git's parsing code), since we
always allocate an extra NUL when loading an object into memory (and
likewise, we are OK with stopping parsing if a commit or tag contains an
embedded NUL).

Let's drop these extra "sz" parameters, as they are misleading about how
the functions intend to access the buffer. We can drop from both the
functions mentioned above, which in turn lets us drop from their
callers, cascading all the way up to the top-level grab_values().

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-14 15:26:15 -08:00
Jeff King 25051cac80 ref-filter: drop unused "obj" parameters
The grab_person() and grab_sub_body_contents() functions take both an
object struct and a buf/sz pair of the object bytes. However, they use
only the latter, since "struct object" does not contain the parsed ident
(nor the whole commit message, of course).

Let's get rid of these misleading "struct object" parameters. It's
possible we may want them in the future (e.g., to generate error
messages that mention the object id), but since these are static
functions, we can easily add them back in later (and if we do want that
information, it's likely we'd pass it through a more generalized
"parsing context" struct anyway).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-14 15:26:15 -08:00
Jeff King e0329199a7 ref-filter: drop unused buf/sz pairs
The grab_tag_values() and grab_commit_values() functions take both the
"struct object" as well as the buf/sz pair for the actual object bytes.
However, neither function uses the latter, as they pull the data
directly from the parsed object struct.

Let's get rid of these misleading parameters.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-14 15:26:15 -08:00
Junio C Hamano 55574bd04a Merge branch 'ot/ref-filter-object-info'
The "--format=<placeholder>" option of for-each-ref, branch and tag
learned to show a few more traits of objects that can be learned by
the object_info API.

* ot/ref-filter-object-info:
  ref-filter: give uintmax_t to format with %PRIuMAX
  ref-filter: add docs for new options
  ref-filter: add tests for deltabase
  ref-filter: add deltabase option
  ref-filter: add tests for objectsize:disk
  ref-filter: add check for negative file size
  ref-filter: add objectsize:disk option
2019-01-18 13:49:56 -08:00
Junio C Hamano f2ddd9e53f ref-filter: give uintmax_t to format with %PRIuMAX
As long as we are casting to a wider type, we should cast to the one
with the correct signed-ness.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-10 10:15:49 -08:00
Junio C Hamano cde555480b Merge branch 'nd/the-index'
More codepaths become aware of working with in-core repository
instance other than the default "the_repository".

* nd/the-index: (22 commits)
  rebase-interactive.c: remove the_repository references
  rerere.c: remove the_repository references
  pack-*.c: remove the_repository references
  pack-check.c: remove the_repository references
  notes-cache.c: remove the_repository references
  line-log.c: remove the_repository reference
  diff-lib.c: remove the_repository references
  delta-islands.c: remove the_repository references
  cache-tree.c: remove the_repository references
  bundle.c: remove the_repository references
  branch.c: remove the_repository reference
  bisect.c: remove the_repository reference
  blame.c: remove implicit dependency the_repository
  sequencer.c: remove implicit dependency on the_repository
  sequencer.c: remove implicit dependency on the_index
  transport.c: remove implicit dependency on the_index
  notes-merge.c: remove implicit dependency the_repository
  notes-merge.c: remove implicit dependency on the_index
  list-objects.c: reduce the_repository references
  list-objects-filter.c: remove implicit dependency on the_index
  ...
2019-01-04 13:33:33 -08:00
Junio C Hamano 3813a89fae Merge branch 'nd/i18n'
More _("i18n") markings.

* nd/i18n:
  fsck: mark strings for translation
  fsck: reduce word legos to help i18n
  parse-options.c: mark more strings for translation
  parse-options.c: turn some die() to BUG()
  parse-options: replace opterror() with optname()
  repack: mark more strings for translation
  remote.c: mark messages for translation
  remote.c: turn some error() or die() to BUG()
  reflog: mark strings for translation
  read-cache.c: add missing colon separators
  read-cache.c: mark more strings for translation
  read-cache.c: turn die("internal error") to BUG()
  attr.c: mark more string for translation
  archive.c: mark more strings for translation
  alias.c: mark split_cmdline_strerror() strings for translation
  git.c: mark more strings for translation
2019-01-04 13:33:31 -08:00
Olga Telezhnaya 33311fa1ad ref-filter: add deltabase option
Add new formatting option: deltabase.
If the object is stored as a delta on-disk, this expands
to the 40-hex sha1 of the delta base object.
Otherwise, expands to the null sha1 (40 zeroes).
We have same option in cat-file command.
Hopefully, in the end I will remove formatting code from
cat-file and reuse formatting parts from ref-filter.

Signed-off-by: Olga Telezhnaia <olyatelezhnaya@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-12-28 10:08:11 -08:00
Olga Telezhnaya 5305a55348 ref-filter: add check for negative file size
If we have negative file size, we are doing something wrong.

Signed-off-by: Olga Telezhnaia <olyatelezhnaya@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-12-28 10:07:34 -08:00
Olga Telezhnaya 1867ce6cbe ref-filter: add objectsize:disk option
Add new formatting option objectsize:disk to know
exact size that object takes up on disk.

Signed-off-by: Olga Telezhnaia <olyatelezhnaya@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-12-28 10:07:34 -08:00
Junio C Hamano f5f0f68d61 Merge branch 'tb/print-size-t-with-uintmax-format'
Code preparation to replace ulong vars with size_t vars where
appropriate.

* tb/print-size-t-with-uintmax-format:
  Upcast size_t variables to uintmax_t when printing
2018-11-19 16:24:41 +09:00
Junio C Hamano 7bdebaa646 Merge branch 'sg/ref-filter-wo-repository'
"git ls-remote --sort=<thing>" can feed an object that is not yet
available into the comparison machinery and segfault, which has
been corrected to check such a request upfront and reject it.

* sg/ref-filter-wo-repository:
  ref-filter: don't look for objects when outside of a repository
2018-11-18 18:23:59 +09:00
SZEDER Gábor 47bd3d0c14 ref-filter: don't look for objects when outside of a repository
The command 'git ls-remote --sort=authordate <remote>' segfaults when
run outside of a repository, ever since the introduction of its
'--sort' option in 1fb20dfd8e (ls-remote: create '--sort' option,
2018-04-09).

While in general the 'git ls-remote' command can be run outside of a
repository just fine, its '--sort=<key>' option with certain keys does
require access to the referenced objects.  This sorting is implemented
using the generic ref-filter sorting facility, which already handles
missing objects gracefully with the appropriate 'missing object
deadbeef for HEAD' message.  However, being generic means that it
checks replace refs while trying to retrieve an object, and while
doing so it accesses the 'git_replace_ref_base' variable, which has
not been initialized and is still a NULL pointer when outside of a
repository, thus causing the segfault.

Make ref-filter more careful upfront while parsing the format string,
and make it error out when encountering a format atom requiring object
access when we are not in a repository.  Also add a test to ensure
that 'git ls-remote --sort' fails gracefully when executed outside of
a repository.

Reported-by: H.Merijn Brand <h.m.brand@xs4all.nl>
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-16 13:49:08 +09:00
Torsten Bögershausen ca473cef91 Upcast size_t variables to uintmax_t when printing
When printing variables which contain a size, today "unsigned long"
is used at many places.
In order to be able to change the type from "unsigned long" into size_t
some day in the future, we need to have a way to print 64 bit variables
on a system that has "unsigned long" defined to be 32 bit, like Win64.

Upcast all those variables into uintmax_t before they are printed.
This is to prepare for a bigger change, when "unsigned long"
will be converted into size_t for variables which may be > 4Gib.

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-12 16:43:52 +09:00
Nguyễn Thái Ngọc Duy 788454576f wt-status.c: remove implicit dependency the_repository
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-12 14:50:05 +09:00
Nguyễn Thái Ngọc Duy 9440b831ad parse-options: replace opterror() with optname()
Introduce optname() that does the early half of original opterror() to
come up with the name of the option reported back to the user, and use
it to kill opterror().  The callers of opterror() now directly call
error() using the string returned by opterror() instead.

There are a few issues with opterror()

- it tries to assemble an English sentence from pieces. This is not
  great for translators because we give them pieces instead of a full
  sentence.

- It's a wrapper around error() and needs some hack to let the
  compiler know it always returns -1.

- Since it takes a string instead of printf format, one call site has
  to assemble the string manually before passing to it.

Using error() directly solves the second and third problems.

It kind helps the first problem as well because "%s does foo" does
give a translator a full sentence in a sense and let them reorder if
needed. But it has limitations, if the subject part has to change
based on the rest of the sentence, that language is screwed. This is
also why I try to avoid calling optname() when 'flags' is known in
advance.

Mark of these strings for translation as well while at there.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-12 14:47:09 +09:00
Jeff King 517fe807d6 assert NOARG/NONEG behavior of parse-options callbacks
When we define a parse-options callback, the flags we put in the option
struct must match what the callback expects. For example, a callback
which does not handle the "unset" parameter should only be used with
PARSE_OPT_NONEG. But since the callback and the option struct are not
defined next to each other, it's easy to get this wrong (as earlier
patches in this series show).

Fortunately, the compiler can help us here: compiling with
-Wunused-parameters can show us which callbacks ignore their "unset"
parameters (and likewise, ones that ignore "arg" expect to be triggered
with PARSE_OPT_NOARG).

But after we've inspected a callback and determined that all of its
callers use the right flags, what do we do next? We'd like to silence
the compiler warning, but do so in a way that will catch any wrong calls
in the future.

We can do that by actually checking those variables and asserting that
they match our expectations. Because this is such a common pattern,
we'll introduce some helper macros. The resulting messages aren't
as descriptive as we could make them, but the file/line information from
BUG() is enough to identify the problem (and anyway, the point is that
these should never be seen).

Each of the annotated callbacks in this patch triggers
-Wunused-parameters, and was manually inspected to make sure all callers
use the correct options (so none of these BUGs should be triggerable).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-06 12:56:29 +09:00
Junio C Hamano 9d00100c38 Merge branch 'ot/ref-filter-plug-leaks'
Plugging a handful of memory leaks in the ref-filter codepath.

* ot/ref-filter-plug-leaks:
  ref-filter: free item->value and item->value->s
  ls-remote: release memory instead of UNLEAK
  ref-filter: free memory from used_atom
2018-10-30 15:43:45 +09:00
Junio C Hamano 11877b9ebe Merge branch 'nd/the-index'
Various codepaths in the core-ish part learn to work on an
arbitrary in-core index structure, not necessarily the default
instance "the_index".

* nd/the-index: (23 commits)
  revision.c: reduce implicit dependency the_repository
  revision.c: remove implicit dependency on the_index
  ws.c: remove implicit dependency on the_index
  tree-diff.c: remove implicit dependency on the_index
  submodule.c: remove implicit dependency on the_index
  line-range.c: remove implicit dependency on the_index
  userdiff.c: remove implicit dependency on the_index
  rerere.c: remove implicit dependency on the_index
  sha1-file.c: remove implicit dependency on the_index
  patch-ids.c: remove implicit dependency on the_index
  merge.c: remove implicit dependency on the_index
  merge-blobs.c: remove implicit dependency on the_index
  ll-merge.c: remove implicit dependency on the_index
  diff-lib.c: remove implicit dependency on the_index
  read-cache.c: remove implicit dependency on the_index
  diff.c: remove implicit dependency on the_index
  grep.c: remove implicit dependency on the_index
  diff.c: remove the_index dependency in textconv() functions
  blame.c: rename "repo" argument to "r"
  combine-diff.c: remove implicit dependency on the_index
  ...
2018-10-19 13:34:02 +09:00
Olga Telezhnaya f0062d3b74 ref-filter: free item->value and item->value->s
Release item->value.
Initialize item->value->s dynamically and then release its resources.
Release some local variables.

Final goal of this patch is to reduce number of memory leaks.

Signed-off-by: Olga Telezhnaia <olyatelezhnaya@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-19 11:28:13 +09:00
Olga Telezhnaya 23941dd7b8 ref-filter: free memory from used_atom
Release memory from used_atom variable for reducing number of memory
leaks.

Signed-off-by: Olga Telezhnaia <olyatelezhnaya@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-19 11:28:10 +09:00
Nguyễn Thái Ngọc Duy 2abf350385 revision.c: remove implicit dependency on the_index
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-21 09:51:19 -07:00
Junio C Hamano fba9654364 Merge branch 'jk/trailer-fixes'
"git interpret-trailers" and its underlying machinery had a buggy
code that attempted to ignore patch text after commit log message,
which triggered in various codepaths that will always get the log
message alone and never get such an input.

* jk/trailer-fixes:
  append_signoff: use size_t for string offsets
  sequencer: ignore "---" divider when parsing trailers
  pretty, ref-filter: format %(trailers) with no_divider option
  interpret-trailers: allow suppressing "---" divider
  interpret-trailers: tighten check for "---" patch boundary
  trailer: pass process_trailer_opts to trailer_info_get()
  trailer: use size_t for iterating trailer list
  trailer: use size_t for string offsets
2018-09-17 13:53:54 -07:00
Junio C Hamano 1b7a91da71 Merge branch 'ds/reachable'
The code for computing history reachability has been shuffled,
obtained a bunch of new tests to cover them, and then being
improved.

* ds/reachable:
  commit-reach: correct accidental #include of C file
  commit-reach: use can_all_from_reach
  commit-reach: make can_all_from_reach... linear
  commit-reach: replace ref_newer logic
  test-reach: test commit_contains
  test-reach: test can_all_from_reach_with_flags
  test-reach: test reduce_heads
  test-reach: test get_merge_bases_many
  test-reach: test is_descendant_of
  test-reach: test in_merge_bases
  test-reach: create new test tool for ref_newer
  commit-reach: move can_all_from_reach_with_flags
  upload-pack: generalize commit date cutoff
  upload-pack: refactor ok_to_give_up()
  upload-pack: make reachable() more generic
  commit-reach: move commit_contains from ref-filter
  commit-reach: move ref_newer from remote.c
  commit.h: remove method declarations
  commit-reach: move walk methods from commit.c
2018-09-17 13:53:52 -07:00
Jeff King e5fba5d558 pretty, ref-filter: format %(trailers) with no_divider option
In both of these cases we know that we are feeding the
trailer-parsing code a pure commit message. We should tell
it so, which avoids false positives for a commit message
that contains a "---" line.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-23 10:08:51 -07:00
Junio C Hamano c83149ace6 Merge branch 'ot/ref-filter-object-info'
A few atoms like %(objecttype) and %(objectsize) in the format
specifier of "for-each-ref --format=<format>" can be filled without
getting the full contents of the object, but just with the object
header.  These cases have been optimized by calling
oid_object_info() API (instead of reading and inspecting the data).

* ot/ref-filter-object-info:
  ref-filter: use oid_object_info() to get object
  ref-filter: merge get_obj and get_object
  ref-filter: initialize eaten variable
  ref-filter: fill empty fields with empty values
  ref-filter: add info_source to valid_atom
2018-08-17 13:09:57 -07:00
Junio C Hamano 78a72ad4f8 Merge branch 'jt/commit-graph-per-object-store'
The singleton commit-graph in-core instance is made per in-core
repository instance.

* jt/commit-graph-per-object-store:
  commit-graph: add repo arg to graph readers
  commit-graph: store graph in struct object_store
  commit-graph: add free_commit_graph
  commit-graph: add missing forward declaration
  object-store: add missing include
  commit-graph: refactor preparing commit graph
2018-08-02 15:30:47 -07:00
Junio C Hamano 3a2a1dc170 Merge branch 'sb/object-store-lookup'
lookup_commit_reference() and friends have been updated to find
in-core object for a specific in-core repository instance.

* sb/object-store-lookup: (32 commits)
  commit.c: allow lookup_commit_reference to handle arbitrary repositories
  commit.c: allow lookup_commit_reference_gently to handle arbitrary repositories
  tag.c: allow deref_tag to handle arbitrary repositories
  object.c: allow parse_object to handle arbitrary repositories
  object.c: allow parse_object_buffer to handle arbitrary repositories
  commit.c: allow get_cached_commit_buffer to handle arbitrary repositories
  commit.c: allow set_commit_buffer to handle arbitrary repositories
  commit.c: migrate the commit buffer to the parsed object store
  commit-slabs: remove realloc counter outside of slab struct
  commit.c: allow parse_commit_buffer to handle arbitrary repositories
  tag: allow parse_tag_buffer to handle arbitrary repositories
  tag: allow lookup_tag to handle arbitrary repositories
  commit: allow lookup_commit to handle arbitrary repositories
  tree: allow lookup_tree to handle arbitrary repositories
  blob: allow lookup_blob to handle arbitrary repositories
  object: allow lookup_object to handle arbitrary repositories
  object: allow object_as_type to handle arbitrary repositories
  tag: add repository argument to deref_tag
  tag: add repository argument to parse_tag_buffer
  tag: add repository argument to lookup_tag
  ...
2018-08-02 15:30:42 -07:00
Junio C Hamano 4301330588 Merge branch 'jk/for-each-ref-icase'
The "--ignore-case" option of "git for-each-ref" (and its friends)
did not work correctly, which has been fixed.

* jk/for-each-ref-icase:
  ref-filter: avoid backend filtering with --ignore-case
  for-each-ref: consistently pass WM_IGNORECASE flag
  t6300: add a test for --ignore-case
2018-07-24 14:50:44 -07:00
Derrick Stolee 920f93ca1c commit-reach: move commit_contains from ref-filter
There are several commit walks in the codebase. Group them together into
a new commit-reach.c file and corresponding header. After we group these
walks into one place, we can reduce duplicate logic by calling
equivalent methods.

All methods are direct moves, except we also make the commit_contains()
method public so its consumers in ref-filter.c can still call it. We can
also test this method in a test-tool in a later commit.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-20 15:38:54 -07:00
Derrick Stolee 6404355657 commit.h: remove method declarations
These methods are now declared in commit-reach.h. Remove them from
commit.h and add new include statements in all files that require these
declarations.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-20 15:38:54 -07:00
Junio C Hamano 00624d608c Merge branch 'sb/object-store-grafts'
The conversion to pass "the_repository" and then "a_repository"
throughout the object access API continues.

* sb/object-store-grafts:
  commit: allow lookup_commit_graft to handle arbitrary repositories
  commit: allow prepare_commit_graft to handle arbitrary repositories
  shallow: migrate shallow information into the object parser
  path.c: migrate global git_path_* to take a repository argument
  cache: convert get_graft_file to handle arbitrary repositories
  commit: convert read_graft_file to handle arbitrary repositories
  commit: convert register_commit_graft to handle arbitrary repositories
  commit: convert commit_graft_pos() to handle arbitrary repositories
  shallow: add repository argument to is_repository_shallow
  shallow: add repository argument to check_shallow_file_for_update
  shallow: add repository argument to register_shallow
  shallow: add repository argument to set_alternate_shallow_file
  commit: add repository argument to lookup_commit_graft
  commit: add repository argument to prepare_commit_graft
  commit: add repository argument to read_graft_file
  commit: add repository argument to register_commit_graft
  commit: add repository argument to commit_graft_pos
  object: move grafts to object parser
  object-store: move object access functions to object-store.h
2018-07-18 12:20:28 -07:00
Jonathan Tan dade47c06c commit-graph: add repo arg to graph readers
Add a struct repository argument to the functions in commit-graph.h that
read the commit graph. (This commit does not affect functions that write
commit graphs.)

Because the commit graph functions can now read the commit graph of any
repository, the global variable core_commit_graph has been removed.
Instead, the config option core.commitGraph is now read on the first
time in a repository that a commit is attempted to be parsed using its
commit graph.

This commit includes a test that exercises the functionality on an
arbitrary repository that is not the_repository.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-17 15:47:48 -07:00
Olga Telezhnaya aa46a0da30 ref-filter: use oid_object_info() to get object
Use oid_object_info_extended() to get object info instead of
read_object_file().
It will help to handle some requests faster (e.g., we do not need to
parse whole object if we need to know %(objectsize)).
It could also help us to add new atoms such as %(objectsize:disk)
and %(deltabase).

Signed-off-by: Olga Telezhnaia <olyatelezhnaya@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-17 15:04:33 -07:00
Olga Telezhnaya e2255179f6 ref-filter: merge get_obj and get_object
Inline get_obj(): it would be easier to edit the code
without this split.

Signed-off-by: Olga Telezhnaia <olyatelezhnaya@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-17 15:04:31 -07:00
Olga Telezhnaya 04f6ee1a58 ref-filter: initialize eaten variable
Initialize variable `eaten` before its using. We may initialize it in
parse_object_buffer(), but there are cases when we do not reach this
invocation.

Signed-off-by: Olga Telezhnaia <olyatelezhnaya@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-17 15:04:29 -07:00
Olga Telezhnaya 20a9c15619 ref-filter: fill empty fields with empty values
Atoms like "align" or "end" do not have string representation.
Earlier we had to go and parse whole object with a hope that we
could fill their string representations. It's easier to fill them
with an empty string before we start to work with whole object.

It is important to mention that we fill only these atoms that must
contain nothing. So, if we could not fill the atom because, for example,
the object is missing, we leave it with NULL.

Signed-off-by: Olga Telezhnaia <olyatelezhnaya@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-17 15:04:27 -07:00
Olga Telezhnaya a8e7e385cd ref-filter: add info_source to valid_atom
Add the source of object data to prevent parsing of unneeded data.
The goal is to improve performance by avoiding calling expensive
functions when we don't need the information they provide
or when we could get it by using a cheaper function.

It is stored in valid_atoms because it depends on the atoms we are
interested in.

Signed-off-by: Olga Telezhnaia <olyatelezhnaya@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-17 15:03:59 -07:00
Jeff King e674eb2528 ref-filter: avoid backend filtering with --ignore-case
When for-each-ref is used with --ignore-case, we expect
match_name_as_path() to do a case-insensitive match. But
there's an extra layer of filtering that happens before we
even get there. Since commit cfe004a5a9 (ref-filter: limit
traversal to prefix, 2017-05-22), we feed the prefix to the
ref backend so that it can optimize the ref iteration.

There's no mechanism for us to tell the backend we're matching
case-insensitively.  Nor is there likely to be one anytime soon,
since the packed backend relies on binary-searching the sorted list
of refs. Let's just punt on this case. The extra filtering is an
optimization that we simply can't do. We'll still give the correct
answer via the filtering in match_name_as_path().

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-03 14:49:37 -07:00
Aleksandr Makarov 639ab5efa1 for-each-ref: consistently pass WM_IGNORECASE flag
The match_name_as_path() function learned to set
WM_IGNORECASE in the "flags" field when the user passed
--ignore-case. But it forgot to actually pass the flags to
wildmatch()!

As a result, the --ignore-case feature has been broken since
it was added in 3bb16a8bf2 (tag, branch, for-each-ref: add
--ignore-case for sorting and filtering, 2016-12-04). We
didn't notice because we added tests only for git-branch and
git-tag. Whereas git-for-each-ref has slightly different
matching rules, and thus uses a different function (the
related function match_pattern() does it correctly).

Incidentally, this also caused clang's scan-build to
complain about the code; the assignment to "flags" was dead
code.

Note that we can't flip the test in t6300 to expect_success
yet. There's another bug, which will be dealt with in the
next patch.

Commit-message-by: Jeff King <peff@peff.net>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-03 14:49:15 -07:00
Stefan Beller 21e1ee8f4f commit: add repository argument to lookup_commit_reference_gently
Add a repository argument to allow callers of
lookup_commit_reference_gently to be more specific about which
repository to handle. This is a small mechanical change; it doesn't
change the implementation to handle repositories other than
the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-29 10:43:39 -07:00
Stefan Beller 1ec5bfd24e object: add repository argument to parse_object_buffer
Add a repository argument to allow the callers of parse_object_buffer
to be more specific about which repository to act on. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-29 10:43:38 -07:00
Stefan Beller 109cd76dd3 object: add repository argument to parse_object
Add a repository argument to allow the callers of parse_object
to be more specific about which repository to act on. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-29 10:43:38 -07:00
Junio C Hamano b16b60f71b Merge branch 'sb/object-store-grafts' into sb/object-store-lookup
* sb/object-store-grafts:
  commit: allow lookup_commit_graft to handle arbitrary repositories
  commit: allow prepare_commit_graft to handle arbitrary repositories
  shallow: migrate shallow information into the object parser
  path.c: migrate global git_path_* to take a repository argument
  cache: convert get_graft_file to handle arbitrary repositories
  commit: convert read_graft_file to handle arbitrary repositories
  commit: convert register_commit_graft to handle arbitrary repositories
  commit: convert commit_graft_pos() to handle arbitrary repositories
  shallow: add repository argument to is_repository_shallow
  shallow: add repository argument to check_shallow_file_for_update
  shallow: add repository argument to register_shallow
  shallow: add repository argument to set_alternate_shallow_file
  commit: add repository argument to lookup_commit_graft
  commit: add repository argument to prepare_commit_graft
  commit: add repository argument to read_graft_file
  commit: add repository argument to register_commit_graft
  commit: add repository argument to commit_graft_pos
  object: move grafts to object parser
  object-store: move object access functions to object-store.h
2018-06-29 10:43:28 -07:00