mirror of
https://github.com/git/git.git
synced 2024-05-06 14:36:18 +02:00
Merge branch 'ds/partial-bundles'
Bundle file format gets extended to allow a partial bundle, filtered by similar criteria you would give when making a partial/lazy clone. * ds/partial-bundles: clone: fail gracefully when cloning filtered bundle bundle: unbundle promisor packs bundle: create filtered bundles rev-list: move --filter parsing into revision.c bundle: parse filter capability list-objects: handle NULL function pointers MyFirstObjectWalk: update recommended usage list-objects: consolidate traverse_commit_list[_filtered] pack-bitmap: drop filter in prepare_bitmap_walk() pack-objects: use rev.filter when possible revision: put object filter into struct rev_info list-objects-filter-options: create copy helper index-pack: document and test the --promisor option
This commit is contained in:
Junio C Hamano
commit
7391ecd338
|
|
@ -522,24 +522,25 @@ function shows that the all-object walk is being performed by
|
|||
`traverse_commit_list()` or `traverse_commit_list_filtered()`. Those two
|
||||
functions reside in `list-objects.c`; examining the source shows that, despite
|
||||
the name, these functions traverse all kinds of objects. Let's have a look at
|
||||
the arguments to `traverse_commit_list_filtered()`, which are a superset of the
|
||||
arguments to the unfiltered version.
|
||||
the arguments to `traverse_commit_list()`.
|
||||
|
||||
- `struct list_objects_filter_options *filter_options`: This is a struct which
|
||||
stores a filter-spec as outlined in `Documentation/rev-list-options.txt`.
|
||||
- `struct rev_info *revs`: This is the `rev_info` used for the walk.
|
||||
- `struct rev_info *revs`: This is the `rev_info` used for the walk. If
|
||||
its `filter` member is not `NULL`, then `filter` contains information for
|
||||
how to filter the object list.
|
||||
- `show_commit_fn show_commit`: A callback which will be used to handle each
|
||||
individual commit object.
|
||||
- `show_object_fn show_object`: A callback which will be used to handle each
|
||||
non-commit object (so each blob, tree, or tag).
|
||||
- `void *show_data`: A context buffer which is passed in turn to `show_commit`
|
||||
and `show_object`.
|
||||
|
||||
In addition, `traverse_commit_list_filtered()` has an additional paramter:
|
||||
|
||||
- `struct oidset *omitted`: A linked-list of object IDs which the provided
|
||||
filter caused to be omitted.
|
||||
|
||||
It looks like this `traverse_commit_list_filtered()` uses callbacks we provide
|
||||
instead of needing us to call it repeatedly ourselves. Cool! Let's add the
|
||||
callbacks first.
|
||||
It looks like these methods use callbacks we provide instead of needing us
|
||||
to call it repeatedly ourselves. Cool! Let's add the callbacks first.
|
||||
|
||||
For the sake of this tutorial, we'll simply keep track of how many of each kind
|
||||
of object we find. At file scope in `builtin/walken.c` add the following
|
||||
|
|
@ -712,20 +713,9 @@ help understand. In our case, that means we omit trees and blobs not directly
|
|||
referenced by `HEAD` or `HEAD`'s history, because we begin the walk with only
|
||||
`HEAD` in the `pending` list.)
|
||||
|
||||
First, we'll need to `#include "list-objects-filter-options.h"` and set up the
|
||||
`struct list_objects_filter_options` at the top of the function.
|
||||
|
||||
----
|
||||
static void walken_object_walk(struct rev_info *rev)
|
||||
{
|
||||
struct list_objects_filter_options filter_options = { 0 };
|
||||
|
||||
...
|
||||
----
|
||||
|
||||
For now, we are not going to track the omitted objects, so we'll replace those
|
||||
parameters with `NULL`. For the sake of simplicity, we'll add a simple
|
||||
build-time branch to use our filter or not. Replace the line calling
|
||||
build-time branch to use our filter or not. Preface the line calling
|
||||
`traverse_commit_list()` with the following, which will remind us which kind of
|
||||
walk we've just performed:
|
||||
|
||||
|
|
@ -733,19 +723,17 @@ walk we've just performed:
|
|||
if (0) {
|
||||
/* Unfiltered: */
|
||||
trace_printf(_("Unfiltered object walk.\n"));
|
||||
traverse_commit_list(rev, walken_show_commit,
|
||||
walken_show_object, NULL);
|
||||
} else {
|
||||
trace_printf(
|
||||
_("Filtered object walk with filterspec 'tree:1'.\n"));
|
||||
parse_list_objects_filter(&filter_options, "tree:1");
|
||||
|
||||
traverse_commit_list_filtered(&filter_options, rev,
|
||||
walken_show_commit, walken_show_object, NULL, NULL);
|
||||
CALLOC_ARRAY(rev->filter, 1);
|
||||
parse_list_objects_filter(rev->filter, "tree:1");
|
||||
}
|
||||
traverse_commit_list(rev, walken_show_commit,
|
||||
walken_show_object, NULL);
|
||||
----
|
||||
|
||||
`struct list_objects_filter_options` is usually built directly from a command
|
||||
The `rev->filter` member is usually built directly from a command
|
||||
line argument, so the module provides an easy way to build one from a string.
|
||||
Even though we aren't taking user input right now, we can still build one with
|
||||
a hardcoded string using `parse_list_objects_filter()`.
|
||||
|
|
@ -784,7 +772,7 @@ object:
|
|||
----
|
||||
...
|
||||
|
||||
traverse_commit_list_filtered(&filter_options, rev,
|
||||
traverse_commit_list_filtered(rev,
|
||||
walken_show_commit, walken_show_object, NULL, &omitted);
|
||||
|
||||
...
|
||||
|
|
|
|||
|
|
@ -75,8 +75,11 @@ verify <file>::
|
|||
cleanly to the current repository. This includes checks on the
|
||||
bundle format itself as well as checking that the prerequisite
|
||||
commits exist and are fully linked in the current repository.
|
||||
'git bundle' prints a list of missing commits, if any, and exits
|
||||
with a non-zero status.
|
||||
Information about additional capabilities, such as "object filter",
|
||||
is printed. See "Capabilities" in link:technical/bundle-format.html
|
||||
for more information. Finally, 'git bundle' prints a list of
|
||||
missing commits, if any. The exit code is zero for success, but
|
||||
will be nonzero if the bundle file is invalid.
|
||||
|
||||
list-heads <file>::
|
||||
Lists the references defined in the bundle. If followed by a
|
||||
|
|
|
|||
|
|
@ -122,6 +122,14 @@ This option cannot be used with --stdin.
|
|||
+
|
||||
include::object-format-disclaimer.txt[]
|
||||
|
||||
--promisor[=<message>]::
|
||||
Before committing the pack-index, create a .promisor file for this
|
||||
pack. Particularly helpful when writing a promisor pack with --fix-thin
|
||||
since the name of the pack is not final until the pack has been fully
|
||||
written. If a `<message>` is provided, then that content will be
|
||||
written to the .promisor file for future reference. See
|
||||
link:technical/partial-clone.html[partial clone] for more information.
|
||||
|
||||
NOTES
|
||||
-----
|
||||
|
||||
|
|
|
|||
|
|
@ -71,6 +71,11 @@ and the Git bundle v2 format cannot represent a shallow clone repository.
|
|||
== Capabilities
|
||||
|
||||
Because there is no opportunity for negotiation, unknown capabilities cause 'git
|
||||
bundle' to abort. The only known capability is `object-format`, which specifies
|
||||
the hash algorithm in use, and can take the same values as the
|
||||
`extensions.objectFormat` configuration value.
|
||||
bundle' to abort.
|
||||
|
||||
* `object-format` specifies the hash algorithm in use, and can take the same
|
||||
values as the `extensions.objectFormat` configuration value.
|
||||
|
||||
* `filter` specifies an object filter as in the `--filter` option in
|
||||
linkgit:git-rev-list[1]. The resulting pack-file must be marked as a
|
||||
`.promisor` pack-file after it is unbundled.
|
||||
|
|
|
|||
|
|
@ -33,6 +33,7 @@
|
|||
#include "packfile.h"
|
||||
#include "list-objects-filter-options.h"
|
||||
#include "hook.h"
|
||||
#include "bundle.h"
|
||||
|
||||
/*
|
||||
* Overall FIXMEs:
|
||||
|
|
@ -1172,6 +1173,18 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
|
|||
warning(_("--local is ignored"));
|
||||
transport->cloning = 1;
|
||||
|
||||
if (is_bundle) {
|
||||
struct bundle_header header = BUNDLE_HEADER_INIT;
|
||||
int fd = read_bundle_header(path, &header);
|
||||
int has_filter = header.filter.choice != LOFC_DISABLED;
|
||||
|
||||
if (fd > 0)
|
||||
close(fd);
|
||||
bundle_header_release(&header);
|
||||
if (has_filter)
|
||||
die(_("cannot clone from filtered bundle"));
|
||||
}
|
||||
|
||||
transport_set_option(transport, TRANS_OPT_KEEP, "yes");
|
||||
|
||||
if (reject_shallow)
|
||||
|
|
|
|||
|
|
@ -3651,7 +3651,7 @@ static int pack_options_allow_reuse(void)
|
|||
|
||||
static int get_object_list_from_bitmap(struct rev_info *revs)
|
||||
{
|
||||
if (!(bitmap_git = prepare_bitmap_walk(revs, &filter_options, 0)))
|
||||
if (!(bitmap_git = prepare_bitmap_walk(revs, 0)))
|
||||
return -1;
|
||||
|
||||
if (pack_options_allow_reuse() &&
|
||||
|
|
@ -3727,6 +3727,7 @@ static void get_object_list(int ac, const char **av)
|
|||
repo_init_revisions(the_repository, &revs, NULL);
|
||||
save_commit_buffer = 0;
|
||||
setup_revisions(ac, av, &revs, &s_r_opt);
|
||||
list_objects_filter_copy(&revs.filter, &filter_options);
|
||||
|
||||
/* make sure shallows are read */
|
||||
is_repository_shallow(the_repository);
|
||||
|
|
@ -3777,9 +3778,9 @@ static void get_object_list(int ac, const char **av)
|
|||
|
||||
if (!fn_show_object)
|
||||
fn_show_object = show_object;
|
||||
traverse_commit_list_filtered(&filter_options, &revs,
|
||||
show_commit, fn_show_object, NULL,
|
||||
NULL);
|
||||
traverse_commit_list(&revs,
|
||||
show_commit, fn_show_object,
|
||||
NULL);
|
||||
|
||||
if (unpack_unreachable_expiration) {
|
||||
revs.ignore_missing_links = 1;
|
||||
|
|
|
|||
|
|
@ -62,7 +62,6 @@ static const char rev_list_usage[] =
|
|||
static struct progress *progress;
|
||||
static unsigned progress_counter;
|
||||
|
||||
static struct list_objects_filter_options filter_options;
|
||||
static struct oidset omitted_objects;
|
||||
static int arg_print_omitted; /* print objects omitted by filter */
|
||||
|
||||
|
|
@ -400,7 +399,6 @@ static inline int parse_missing_action_value(const char *value)
|
|||
}
|
||||
|
||||
static int try_bitmap_count(struct rev_info *revs,
|
||||
struct list_objects_filter_options *filter,
|
||||
int filter_provided_objects)
|
||||
{
|
||||
uint32_t commit_count = 0,
|
||||
|
|
@ -436,7 +434,7 @@ static int try_bitmap_count(struct rev_info *revs,
|
|||
*/
|
||||
max_count = revs->max_count;
|
||||
|
||||
bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_objects);
|
||||
bitmap_git = prepare_bitmap_walk(revs, filter_provided_objects);
|
||||
if (!bitmap_git)
|
||||
return -1;
|
||||
|
||||
|
|
@ -453,7 +451,6 @@ static int try_bitmap_count(struct rev_info *revs,
|
|||
}
|
||||
|
||||
static int try_bitmap_traversal(struct rev_info *revs,
|
||||
struct list_objects_filter_options *filter,
|
||||
int filter_provided_objects)
|
||||
{
|
||||
struct bitmap_index *bitmap_git;
|
||||
|
|
@ -465,7 +462,7 @@ static int try_bitmap_traversal(struct rev_info *revs,
|
|||
if (revs->max_count >= 0)
|
||||
return -1;
|
||||
|
||||
bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_objects);
|
||||
bitmap_git = prepare_bitmap_walk(revs, filter_provided_objects);
|
||||
if (!bitmap_git)
|
||||
return -1;
|
||||
|
||||
|
|
@ -475,7 +472,6 @@ static int try_bitmap_traversal(struct rev_info *revs,
|
|||
}
|
||||
|
||||
static int try_bitmap_disk_usage(struct rev_info *revs,
|
||||
struct list_objects_filter_options *filter,
|
||||
int filter_provided_objects)
|
||||
{
|
||||
struct bitmap_index *bitmap_git;
|
||||
|
|
@ -483,7 +479,7 @@ static int try_bitmap_disk_usage(struct rev_info *revs,
|
|||
if (!show_disk_usage)
|
||||
return -1;
|
||||
|
||||
bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_objects);
|
||||
bitmap_git = prepare_bitmap_walk(revs, filter_provided_objects);
|
||||
if (!bitmap_git)
|
||||
return -1;
|
||||
|
||||
|
|
@ -595,17 +591,6 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
|
|||
show_progress = arg;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (skip_prefix(arg, ("--" CL_ARG__FILTER "="), &arg)) {
|
||||
parse_list_objects_filter(&filter_options, arg);
|
||||
if (filter_options.choice && !revs.blob_objects)
|
||||
die(_("object filtering requires --objects"));
|
||||
continue;
|
||||
}
|
||||
if (!strcmp(arg, ("--no-" CL_ARG__FILTER))) {
|
||||
list_objects_filter_set_no_filter(&filter_options);
|
||||
continue;
|
||||
}
|
||||
if (!strcmp(arg, "--filter-provided-objects")) {
|
||||
filter_provided_objects = 1;
|
||||
continue;
|
||||
|
|
@ -688,11 +673,11 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
|
|||
progress = start_delayed_progress(show_progress, 0);
|
||||
|
||||
if (use_bitmap_index) {
|
||||
if (!try_bitmap_count(&revs, &filter_options, filter_provided_objects))
|
||||
if (!try_bitmap_count(&revs, filter_provided_objects))
|
||||
return 0;
|
||||
if (!try_bitmap_disk_usage(&revs, &filter_options, filter_provided_objects))
|
||||
if (!try_bitmap_disk_usage(&revs, filter_provided_objects))
|
||||
return 0;
|
||||
if (!try_bitmap_traversal(&revs, &filter_options, filter_provided_objects))
|
||||
if (!try_bitmap_traversal(&revs, filter_provided_objects))
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
@ -733,7 +718,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
|
|||
oidset_init(&missing_objects, DEFAULT_OIDSET_SIZE);
|
||||
|
||||
traverse_commit_list_filtered(
|
||||
&filter_options, &revs, show_commit, show_object, &info,
|
||||
&revs, show_commit, show_object, &info,
|
||||
(arg_print_omitted ? &omitted_objects : NULL));
|
||||
|
||||
if (arg_print_omitted) {
|
||||
|
|
|
|||
74
bundle.c
74
bundle.c
|
|
@ -11,7 +11,7 @@
|
|||
#include "run-command.h"
|
||||
#include "refs.h"
|
||||
#include "strvec.h"
|
||||
|
||||
#include "list-objects-filter-options.h"
|
||||
|
||||
static const char v2_bundle_signature[] = "# v2 git bundle\n";
|
||||
static const char v3_bundle_signature[] = "# v3 git bundle\n";
|
||||
|
|
@ -33,6 +33,7 @@ void bundle_header_release(struct bundle_header *header)
|
|||
{
|
||||
string_list_clear(&header->prerequisites, 1);
|
||||
string_list_clear(&header->references, 1);
|
||||
list_objects_filter_release(&header->filter);
|
||||
}
|
||||
|
||||
static int parse_capability(struct bundle_header *header, const char *capability)
|
||||
|
|
@ -45,6 +46,10 @@ static int parse_capability(struct bundle_header *header, const char *capability
|
|||
header->hash_algo = &hash_algos[algo];
|
||||
return 0;
|
||||
}
|
||||
if (skip_prefix(capability, "filter=", &arg)) {
|
||||
parse_list_objects_filter(&header->filter, arg);
|
||||
return 0;
|
||||
}
|
||||
return error(_("unknown capability '%s'"), capability);
|
||||
}
|
||||
|
||||
|
|
@ -220,6 +225,8 @@ int verify_bundle(struct repository *r,
|
|||
req_nr = revs.pending.nr;
|
||||
setup_revisions(2, argv, &revs, NULL);
|
||||
|
||||
list_objects_filter_copy(&revs.filter, &header->filter);
|
||||
|
||||
if (prepare_revision_walk(&revs))
|
||||
die(_("revision walk setup failed"));
|
||||
|
||||
|
|
@ -259,6 +266,12 @@ int verify_bundle(struct repository *r,
|
|||
r->nr),
|
||||
(uintmax_t)r->nr);
|
||||
list_refs(r, 0, NULL);
|
||||
|
||||
if (header->filter.choice) {
|
||||
printf_ln("The bundle uses this filter: %s",
|
||||
list_objects_filter_spec(&header->filter));
|
||||
}
|
||||
|
||||
r = &header->prerequisites;
|
||||
if (!r->nr) {
|
||||
printf_ln(_("The bundle records a complete history."));
|
||||
|
|
@ -319,6 +332,9 @@ static int write_pack_data(int bundle_fd, struct rev_info *revs, struct strvec *
|
|||
"--stdout", "--thin", "--delta-base-offset",
|
||||
NULL);
|
||||
strvec_pushv(&pack_objects.args, pack_options->v);
|
||||
if (revs->filter.choice)
|
||||
strvec_pushf(&pack_objects.args, "--filter=%s",
|
||||
list_objects_filter_spec(&revs->filter));
|
||||
pack_objects.in = -1;
|
||||
pack_objects.out = bundle_fd;
|
||||
pack_objects.git_cmd = 1;
|
||||
|
|
@ -486,10 +502,37 @@ int create_bundle(struct repository *r, const char *path,
|
|||
int bundle_to_stdout;
|
||||
int ref_count = 0;
|
||||
struct rev_info revs, revs_copy;
|
||||
int min_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3;
|
||||
int min_version = 2;
|
||||
struct bundle_prerequisites_info bpi;
|
||||
int i;
|
||||
|
||||
/* init revs to list objects for pack-objects later */
|
||||
save_commit_buffer = 0;
|
||||
repo_init_revisions(r, &revs, NULL);
|
||||
|
||||
/*
|
||||
* Pre-initialize the '--objects' flag so we can parse a
|
||||
* --filter option successfully.
|
||||
*/
|
||||
revs.tree_objects = revs.blob_objects = 1;
|
||||
|
||||
argc = setup_revisions(argc, argv, &revs, NULL);
|
||||
|
||||
/*
|
||||
* Reasons to require version 3:
|
||||
*
|
||||
* 1. @object-format is required because our hash algorithm is not
|
||||
* SHA1.
|
||||
* 2. @filter is required because we parsed an object filter.
|
||||
*/
|
||||
if (the_hash_algo != &hash_algos[GIT_HASH_SHA1] || revs.filter.choice)
|
||||
min_version = 3;
|
||||
|
||||
if (argc > 1) {
|
||||
error(_("unrecognized argument: %s"), argv[1]);
|
||||
goto err;
|
||||
}
|
||||
|
||||
bundle_to_stdout = !strcmp(path, "-");
|
||||
if (bundle_to_stdout)
|
||||
bundle_fd = 1;
|
||||
|
|
@ -512,17 +555,14 @@ int create_bundle(struct repository *r, const char *path,
|
|||
write_or_die(bundle_fd, capability, strlen(capability));
|
||||
write_or_die(bundle_fd, the_hash_algo->name, strlen(the_hash_algo->name));
|
||||
write_or_die(bundle_fd, "\n", 1);
|
||||
}
|
||||
|
||||
/* init revs to list objects for pack-objects later */
|
||||
save_commit_buffer = 0;
|
||||
repo_init_revisions(r, &revs, NULL);
|
||||
|
||||
argc = setup_revisions(argc, argv, &revs, NULL);
|
||||
|
||||
if (argc > 1) {
|
||||
error(_("unrecognized argument: %s"), argv[1]);
|
||||
goto err;
|
||||
if (revs.filter.choice) {
|
||||
const char *value = expand_list_objects_filter_spec(&revs.filter);
|
||||
capability = "@filter=";
|
||||
write_or_die(bundle_fd, capability, strlen(capability));
|
||||
write_or_die(bundle_fd, value, strlen(value));
|
||||
write_or_die(bundle_fd, "\n", 1);
|
||||
}
|
||||
}
|
||||
|
||||
/* save revs.pending in revs_copy for later use */
|
||||
|
|
@ -544,6 +584,12 @@ int create_bundle(struct repository *r, const char *path,
|
|||
die("revision walk setup failed");
|
||||
bpi.fd = bundle_fd;
|
||||
bpi.pending = &revs_copy.pending;
|
||||
|
||||
/*
|
||||
* Remove any object walking here. We only care about commits and
|
||||
* tags here. The revs_copy has the right instances of these values.
|
||||
*/
|
||||
revs.blob_objects = revs.tree_objects = 0;
|
||||
traverse_commit_list(&revs, write_bundle_prerequisites, NULL, &bpi);
|
||||
object_array_remove_duplicates(&revs_copy.pending);
|
||||
|
||||
|
|
@ -574,6 +620,10 @@ int unbundle(struct repository *r, struct bundle_header *header,
|
|||
struct child_process ip = CHILD_PROCESS_INIT;
|
||||
strvec_pushl(&ip.args, "index-pack", "--fix-thin", "--stdin", NULL);
|
||||
|
||||
/* If there is a filter, then we need to create the promisor pack. */
|
||||
if (header->filter.choice)
|
||||
strvec_push(&ip.args, "--promisor=from-bundle");
|
||||
|
||||
if (extra_index_pack_args) {
|
||||
strvec_pushv(&ip.args, extra_index_pack_args->v);
|
||||
strvec_clear(extra_index_pack_args);
|
||||
|
|
|
|||
2
bundle.h
2
bundle.h
|
|
@ -4,12 +4,14 @@
|
|||
#include "strvec.h"
|
||||
#include "cache.h"
|
||||
#include "string-list.h"
|
||||
#include "list-objects-filter-options.h"
|
||||
|
||||
struct bundle_header {
|
||||
unsigned version;
|
||||
struct string_list prerequisites;
|
||||
struct string_list references;
|
||||
const struct git_hash_algo *hash_algo;
|
||||
struct list_objects_filter_options filter;
|
||||
};
|
||||
|
||||
#define BUNDLE_HEADER_INIT \
|
||||
|
|
|
|||
|
|
@ -40,22 +40,7 @@ const char *list_object_filter_config_name(enum list_objects_filter_choice c)
|
|||
BUG("list_object_filter_config_name: invalid argument '%d'", c);
|
||||
}
|
||||
|
||||
/*
|
||||
* Parse value of the argument to the "filter" keyword.
|
||||
* On the command line this looks like:
|
||||
* --filter=<arg>
|
||||
* and in the pack protocol as:
|
||||
* "filter" SP <arg>
|
||||
*
|
||||
* The filter keyword will be used by many commands.
|
||||
* See Documentation/rev-list-options.txt for allowed values for <arg>.
|
||||
*
|
||||
* Capture the given arg as the "filter_spec". This can be forwarded to
|
||||
* subordinate commands when necessary (although it's better to pass it through
|
||||
* expand_list_objects_filter_spec() first). We also "intern" the arg for the
|
||||
* convenience of the current command.
|
||||
*/
|
||||
static int gently_parse_list_objects_filter(
|
||||
int gently_parse_list_objects_filter(
|
||||
struct list_objects_filter_options *filter_options,
|
||||
const char *arg,
|
||||
struct strbuf *errbuf)
|
||||
|
|
@ -415,3 +400,22 @@ void partial_clone_get_default_filter_spec(
|
|||
&errbuf);
|
||||
strbuf_release(&errbuf);
|
||||
}
|
||||
|
||||
void list_objects_filter_copy(
|
||||
struct list_objects_filter_options *dest,
|
||||
const struct list_objects_filter_options *src)
|
||||
{
|
||||
int i;
|
||||
struct string_list_item *item;
|
||||
|
||||
/* Copy everything. We will overwrite the pointers shortly. */
|
||||
memcpy(dest, src, sizeof(struct list_objects_filter_options));
|
||||
|
||||
string_list_init_dup(&dest->filter_spec);
|
||||
for_each_string_list_item(item, &src->filter_spec)
|
||||
string_list_append(&dest->filter_spec, item->string);
|
||||
|
||||
ALLOC_ARRAY(dest->sub, dest->sub_alloc);
|
||||
for (i = 0; i < src->sub_nr; i++)
|
||||
list_objects_filter_copy(&dest->sub[i], &src->sub[i]);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -72,6 +72,26 @@ struct list_objects_filter_options {
|
|||
/* Normalized command line arguments */
|
||||
#define CL_ARG__FILTER "filter"
|
||||
|
||||
/*
|
||||
* Parse value of the argument to the "filter" keyword.
|
||||
* On the command line this looks like:
|
||||
* --filter=<arg>
|
||||
* and in the pack protocol as:
|
||||
* "filter" SP <arg>
|
||||
*
|
||||
* The filter keyword will be used by many commands.
|
||||
* See Documentation/rev-list-options.txt for allowed values for <arg>.
|
||||
*
|
||||
* Capture the given arg as the "filter_spec". This can be forwarded to
|
||||
* subordinate commands when necessary (although it's better to pass it through
|
||||
* expand_list_objects_filter_spec() first). We also "intern" the arg for the
|
||||
* convenience of the current command.
|
||||
*/
|
||||
int gently_parse_list_objects_filter(
|
||||
struct list_objects_filter_options *filter_options,
|
||||
const char *arg,
|
||||
struct strbuf *errbuf);
|
||||
|
||||
void list_objects_filter_die_if_populated(
|
||||
struct list_objects_filter_options *filter_options);
|
||||
|
||||
|
|
@ -132,4 +152,8 @@ void partial_clone_get_default_filter_spec(
|
|||
struct list_objects_filter_options *filter_options,
|
||||
const char *remote);
|
||||
|
||||
void list_objects_filter_copy(
|
||||
struct list_objects_filter_options *dest,
|
||||
const struct list_objects_filter_options *src);
|
||||
|
||||
#endif /* LIST_OBJECTS_FILTER_OPTIONS_H */
|
||||
|
|
|
|||
|
|
@ -21,6 +21,23 @@ struct traversal_context {
|
|||
struct filter *filter;
|
||||
};
|
||||
|
||||
static void show_commit(struct traversal_context *ctx,
|
||||
struct commit *commit)
|
||||
{
|
||||
if (!ctx->show_commit)
|
||||
return;
|
||||
ctx->show_commit(commit, ctx->show_data);
|
||||
}
|
||||
|
||||
static void show_object(struct traversal_context *ctx,
|
||||
struct object *object,
|
||||
const char *name)
|
||||
{
|
||||
if (!ctx->show_object)
|
||||
return;
|
||||
ctx->show_object(object, name, ctx->show_data);
|
||||
}
|
||||
|
||||
static void process_blob(struct traversal_context *ctx,
|
||||
struct blob *blob,
|
||||
struct strbuf *path,
|
||||
|
|
@ -60,7 +77,7 @@ static void process_blob(struct traversal_context *ctx,
|
|||
if (r & LOFR_MARK_SEEN)
|
||||
obj->flags |= SEEN;
|
||||
if (r & LOFR_DO_SHOW)
|
||||
ctx->show_object(obj, path->buf, ctx->show_data);
|
||||
show_object(ctx, obj, path->buf);
|
||||
strbuf_setlen(path, pathlen);
|
||||
}
|
||||
|
||||
|
|
@ -194,7 +211,7 @@ static void process_tree(struct traversal_context *ctx,
|
|||
if (r & LOFR_MARK_SEEN)
|
||||
obj->flags |= SEEN;
|
||||
if (r & LOFR_DO_SHOW)
|
||||
ctx->show_object(obj, base->buf, ctx->show_data);
|
||||
show_object(ctx, obj, base->buf);
|
||||
if (base->len)
|
||||
strbuf_addch(base, '/');
|
||||
|
||||
|
|
@ -210,7 +227,7 @@ static void process_tree(struct traversal_context *ctx,
|
|||
if (r & LOFR_MARK_SEEN)
|
||||
obj->flags |= SEEN;
|
||||
if (r & LOFR_DO_SHOW)
|
||||
ctx->show_object(obj, base->buf, ctx->show_data);
|
||||
show_object(ctx, obj, base->buf);
|
||||
|
||||
strbuf_setlen(base, baselen);
|
||||
free_tree_buffer(tree);
|
||||
|
|
@ -228,7 +245,7 @@ static void process_tag(struct traversal_context *ctx,
|
|||
if (r & LOFR_MARK_SEEN)
|
||||
tag->object.flags |= SEEN;
|
||||
if (r & LOFR_DO_SHOW)
|
||||
ctx->show_object(&tag->object, name, ctx->show_data);
|
||||
show_object(ctx, &tag->object, name);
|
||||
}
|
||||
|
||||
static void mark_edge_parents_uninteresting(struct commit *commit,
|
||||
|
|
@ -402,7 +419,7 @@ static void do_traverse(struct traversal_context *ctx)
|
|||
if (r & LOFR_MARK_SEEN)
|
||||
commit->object.flags |= SEEN;
|
||||
if (r & LOFR_DO_SHOW)
|
||||
ctx->show_commit(commit, ctx->show_data);
|
||||
show_commit(ctx, commit);
|
||||
|
||||
if (ctx->revs->tree_blobs_in_commit_order)
|
||||
/*
|
||||
|
|
@ -416,35 +433,25 @@ static void do_traverse(struct traversal_context *ctx)
|
|||
strbuf_release(&csp);
|
||||
}
|
||||
|
||||
void traverse_commit_list(struct rev_info *revs,
|
||||
show_commit_fn show_commit,
|
||||
show_object_fn show_object,
|
||||
void *show_data)
|
||||
{
|
||||
struct traversal_context ctx;
|
||||
ctx.revs = revs;
|
||||
ctx.show_commit = show_commit;
|
||||
ctx.show_object = show_object;
|
||||
ctx.show_data = show_data;
|
||||
ctx.filter = NULL;
|
||||
do_traverse(&ctx);
|
||||
}
|
||||
|
||||
void traverse_commit_list_filtered(
|
||||
struct list_objects_filter_options *filter_options,
|
||||
struct rev_info *revs,
|
||||
show_commit_fn show_commit,
|
||||
show_object_fn show_object,
|
||||
void *show_data,
|
||||
struct oidset *omitted)
|
||||
{
|
||||
struct traversal_context ctx;
|
||||
struct traversal_context ctx = {
|
||||
.revs = revs,
|
||||
.show_object = show_object,
|
||||
.show_commit = show_commit,
|
||||
.show_data = show_data,
|
||||
};
|
||||
|
||||
if (revs->filter.choice)
|
||||
ctx.filter = list_objects_filter__init(omitted, &revs->filter);
|
||||
|
||||
ctx.revs = revs;
|
||||
ctx.show_object = show_object;
|
||||
ctx.show_commit = show_commit;
|
||||
ctx.show_data = show_data;
|
||||
ctx.filter = list_objects_filter__init(omitted, filter_options);
|
||||
do_traverse(&ctx);
|
||||
list_objects_filter__free(ctx.filter);
|
||||
|
||||
if (ctx.filter)
|
||||
list_objects_filter__free(ctx.filter);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -7,7 +7,6 @@ struct rev_info;
|
|||
|
||||
typedef void (*show_commit_fn)(struct commit *, void *);
|
||||
typedef void (*show_object_fn)(struct object *, const char *, void *);
|
||||
void traverse_commit_list(struct rev_info *, show_commit_fn, show_object_fn, void *);
|
||||
|
||||
typedef void (*show_edge_fn)(struct commit *);
|
||||
void mark_edges_uninteresting(struct rev_info *revs,
|
||||
|
|
@ -18,11 +17,20 @@ struct oidset;
|
|||
struct list_objects_filter_options;
|
||||
|
||||
void traverse_commit_list_filtered(
|
||||
struct list_objects_filter_options *filter_options,
|
||||
struct rev_info *revs,
|
||||
show_commit_fn show_commit,
|
||||
show_object_fn show_object,
|
||||
void *show_data,
|
||||
struct oidset *omitted);
|
||||
|
||||
static inline void traverse_commit_list(
|
||||
struct rev_info *revs,
|
||||
show_commit_fn show_commit,
|
||||
show_object_fn show_object,
|
||||
void *show_data)
|
||||
{
|
||||
traverse_commit_list_filtered(revs, show_commit,
|
||||
show_object, show_data, NULL);
|
||||
}
|
||||
|
||||
#endif /* LIST_OBJECTS_H */
|
||||
|
|
|
|||
|
|
@ -739,8 +739,7 @@ static int add_commit_to_bitmap(struct bitmap_index *bitmap_git,
|
|||
static struct bitmap *find_objects(struct bitmap_index *bitmap_git,
|
||||
struct rev_info *revs,
|
||||
struct object_list *roots,
|
||||
struct bitmap *seen,
|
||||
struct list_objects_filter_options *filter)
|
||||
struct bitmap *seen)
|
||||
{
|
||||
struct bitmap *base = NULL;
|
||||
int needs_walk = 0;
|
||||
|
|
@ -823,9 +822,9 @@ static struct bitmap *find_objects(struct bitmap_index *bitmap_git,
|
|||
show_data.bitmap_git = bitmap_git;
|
||||
show_data.base = base;
|
||||
|
||||
traverse_commit_list_filtered(filter, revs,
|
||||
show_commit, show_object,
|
||||
&show_data, NULL);
|
||||
traverse_commit_list(revs,
|
||||
show_commit, show_object,
|
||||
&show_data);
|
||||
|
||||
revs->include_check = NULL;
|
||||
revs->include_check_obj = NULL;
|
||||
|
|
@ -1219,7 +1218,6 @@ static int can_filter_bitmap(struct list_objects_filter_options *filter)
|
|||
}
|
||||
|
||||
struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs,
|
||||
struct list_objects_filter_options *filter,
|
||||
int filter_provided_objects)
|
||||
{
|
||||
unsigned int i;
|
||||
|
|
@ -1240,7 +1238,7 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs,
|
|||
if (revs->prune)
|
||||
return NULL;
|
||||
|
||||
if (!can_filter_bitmap(filter))
|
||||
if (!can_filter_bitmap(&revs->filter))
|
||||
return NULL;
|
||||
|
||||
/* try to open a bitmapped pack, but don't parse it yet
|
||||
|
|
@ -1297,8 +1295,7 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs,
|
|||
|
||||
if (haves) {
|
||||
revs->ignore_missing_links = 1;
|
||||
haves_bitmap = find_objects(bitmap_git, revs, haves, NULL,
|
||||
filter);
|
||||
haves_bitmap = find_objects(bitmap_git, revs, haves, NULL);
|
||||
reset_revision_walk();
|
||||
revs->ignore_missing_links = 0;
|
||||
|
||||
|
|
@ -1306,8 +1303,7 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs,
|
|||
BUG("failed to perform bitmap walk");
|
||||
}
|
||||
|
||||
wants_bitmap = find_objects(bitmap_git, revs, wants, haves_bitmap,
|
||||
filter);
|
||||
wants_bitmap = find_objects(bitmap_git, revs, wants, haves_bitmap);
|
||||
|
||||
if (!wants_bitmap)
|
||||
BUG("failed to perform bitmap walk");
|
||||
|
|
@ -1315,8 +1311,10 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs,
|
|||
if (haves_bitmap)
|
||||
bitmap_and_not(wants_bitmap, haves_bitmap);
|
||||
|
||||
filter_bitmap(bitmap_git, (filter && filter_provided_objects) ? NULL : wants,
|
||||
wants_bitmap, filter);
|
||||
filter_bitmap(bitmap_git,
|
||||
(revs->filter.choice && filter_provided_objects) ? NULL : wants,
|
||||
wants_bitmap,
|
||||
&revs->filter);
|
||||
|
||||
bitmap_git->result = wants_bitmap;
|
||||
bitmap_git->haves = haves_bitmap;
|
||||
|
|
|
|||
|
|
@ -10,7 +10,6 @@
|
|||
struct commit;
|
||||
struct repository;
|
||||
struct rev_info;
|
||||
struct list_objects_filter_options;
|
||||
|
||||
static const char BITMAP_IDX_SIGNATURE[] = {'B', 'I', 'T', 'M'};
|
||||
|
||||
|
|
@ -54,7 +53,6 @@ void test_bitmap_walk(struct rev_info *revs);
|
|||
int test_bitmap_commits(struct repository *r);
|
||||
int test_bitmap_hashes(struct repository *r);
|
||||
struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs,
|
||||
struct list_objects_filter_options *filter,
|
||||
int filter_provided_objects);
|
||||
uint32_t midx_preferred_pack(struct bitmap_index *bitmap_git);
|
||||
int reuse_partial_packfile_from_bitmap(struct bitmap_index *,
|
||||
|
|
|
|||
|
|
@ -205,7 +205,7 @@ void mark_reachable_objects(struct rev_info *revs, int mark_reflog,
|
|||
cp.progress = progress;
|
||||
cp.count = 0;
|
||||
|
||||
bitmap_git = prepare_bitmap_walk(revs, NULL, 0);
|
||||
bitmap_git = prepare_bitmap_walk(revs, 0);
|
||||
if (bitmap_git) {
|
||||
traverse_bitmap_commit_list(bitmap_git, revs, mark_object_seen);
|
||||
free_bitmap_index(bitmap_git);
|
||||
|
|
|
|||
|
|
@ -32,6 +32,7 @@
|
|||
#include "utf8.h"
|
||||
#include "bloom.h"
|
||||
#include "json-writer.h"
|
||||
#include "list-objects-filter-options.h"
|
||||
|
||||
volatile show_early_output_fn_t show_early_output;
|
||||
|
||||
|
|
@ -2690,6 +2691,10 @@ static int handle_revision_pseudo_opt(struct rev_info *revs,
|
|||
revs->no_walk = 0;
|
||||
} else if (!strcmp(arg, "--single-worktree")) {
|
||||
revs->single_worktree = 1;
|
||||
} else if (skip_prefix(arg, ("--" CL_ARG__FILTER "="), &arg)) {
|
||||
parse_list_objects_filter(&revs->filter, arg);
|
||||
} else if (!strcmp(arg, ("--no-" CL_ARG__FILTER))) {
|
||||
list_objects_filter_set_no_filter(&revs->filter);
|
||||
} else {
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -2892,6 +2897,8 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s
|
|||
die("cannot combine --walk-reflogs with history-limiting options");
|
||||
if (revs->rewrite_parents && revs->children.name)
|
||||
die(_("options '%s' and '%s' cannot be used together"), "--parents", "--children");
|
||||
if (revs->filter.choice && !revs->blob_objects)
|
||||
die(_("object filtering requires --objects"));
|
||||
|
||||
/*
|
||||
* Limitations on the graph functionality
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@
|
|||
#include "pretty.h"
|
||||
#include "diff.h"
|
||||
#include "commit-slab-decl.h"
|
||||
#include "list-objects-filter-options.h"
|
||||
|
||||
/**
|
||||
* The revision walking API offers functions to build a list of revisions
|
||||
|
|
@ -94,6 +95,12 @@ struct rev_info {
|
|||
/* The end-points specified by the end user */
|
||||
struct rev_cmdline_info cmdline;
|
||||
|
||||
/*
|
||||
* Object filter options. No filtering is specified
|
||||
* if and only if filter.choice is zero.
|
||||
*/
|
||||
struct list_objects_filter_options filter;
|
||||
|
||||
/* excluding from --branches, --refs, etc. expansion */
|
||||
struct string_list *ref_excludes;
|
||||
|
||||
|
|
|
|||
|
|
@ -315,8 +315,10 @@ test_expect_success \
|
|||
git index-pack -o tmp.idx test-3.pack &&
|
||||
cmp tmp.idx test-1-${packname_1}.idx &&
|
||||
|
||||
git index-pack test-3.pack &&
|
||||
git index-pack --promisor=message test-3.pack &&
|
||||
cmp test-3.idx test-1-${packname_1}.idx &&
|
||||
echo message >expect &&
|
||||
test_cmp expect test-3.promisor &&
|
||||
|
||||
cat test-2-${packname_2}.pack >test-3.pack &&
|
||||
git index-pack -o tmp.idx test-2-${packname_2}.pack &&
|
||||
|
|
|
|||
|
|
@ -475,4 +475,78 @@ test_expect_success 'clone from bundle' '
|
|||
test_cmp expect actual
|
||||
'
|
||||
|
||||
test_expect_success 'unfiltered bundle with --objects' '
|
||||
git bundle create all-objects.bdl \
|
||||
--all --objects &&
|
||||
git bundle create all.bdl \
|
||||
--all &&
|
||||
|
||||
# Compare the headers of these files.
|
||||
sed -n -e "/^$/q" -e "p" all.bdl >expect &&
|
||||
sed -n -e "/^$/q" -e "p" all-objects.bdl >actual &&
|
||||
test_cmp expect actual
|
||||
'
|
||||
|
||||
for filter in "blob:none" "tree:0" "tree:1" "blob:limit=100"
|
||||
do
|
||||
test_expect_success "filtered bundle: $filter" '
|
||||
test_when_finished rm -rf .git/objects/pack cloned unbundled &&
|
||||
git bundle create partial.bdl \
|
||||
--all \
|
||||
--filter=$filter &&
|
||||
|
||||
git bundle verify partial.bdl >unfiltered &&
|
||||
make_user_friendly_and_stable_output <unfiltered >actual &&
|
||||
|
||||
cat >expect <<-EOF &&
|
||||
The bundle contains these 10 refs:
|
||||
<COMMIT-P> refs/heads/main
|
||||
<COMMIT-N> refs/heads/release
|
||||
<COMMIT-D> refs/heads/topic/1
|
||||
<COMMIT-H> refs/heads/topic/2
|
||||
<COMMIT-D> refs/pull/1/head
|
||||
<COMMIT-G> refs/pull/2/head
|
||||
<TAG-1> refs/tags/v1
|
||||
<TAG-2> refs/tags/v2
|
||||
<TAG-3> refs/tags/v3
|
||||
<COMMIT-P> HEAD
|
||||
The bundle uses this filter: $filter
|
||||
The bundle records a complete history.
|
||||
EOF
|
||||
test_cmp expect actual &&
|
||||
|
||||
test_config uploadpack.allowfilter 1 &&
|
||||
test_config uploadpack.allowanysha1inwant 1 &&
|
||||
git clone --no-local --filter=$filter --bare "file://$(pwd)" cloned &&
|
||||
|
||||
git init unbundled &&
|
||||
git -C unbundled bundle unbundle ../partial.bdl >ref-list.txt &&
|
||||
ls unbundled/.git/objects/pack/pack-*.promisor >promisor &&
|
||||
test_line_count = 1 promisor &&
|
||||
|
||||
# Count the same number of reachable objects.
|
||||
reflist=$(git for-each-ref --format="%(objectname)") &&
|
||||
git rev-list --objects --filter=$filter --missing=allow-any \
|
||||
$reflist >expect &&
|
||||
for repo in cloned unbundled
|
||||
do
|
||||
git -C $repo rev-list --objects --missing=allow-any \
|
||||
$reflist >actual &&
|
||||
test_cmp expect actual || return 1
|
||||
done
|
||||
'
|
||||
done
|
||||
|
||||
# NEEDSWORK: 'git clone --bare' should be able to clone from a filtered
|
||||
# bundle, but that requires a change to promisor/filter config options.
|
||||
# For now, we fail gracefully with a helpful error. This behavior can be
|
||||
# changed in the future to succeed as much as possible.
|
||||
test_expect_success 'cloning from filtered bundle has useful error' '
|
||||
git bundle create partial.bdl \
|
||||
--all \
|
||||
--filter=blob:none &&
|
||||
test_must_fail git clone --bare partial.bdl partial 2>err &&
|
||||
grep "cannot clone from filtered bundle" err
|
||||
'
|
||||
|
||||
test_done
|
||||
|
|
|
|||
Loading…
Reference in New Issue