1
0
Fork 0
mirror of https://github.com/git/git.git synced 2024-05-10 07:16:09 +02:00

Merge branch 'ak/corrected-commit-date'

The commit-graph learned to use corrected commit dates instead of
the generation number to help topological revision traversal.

* ak/corrected-commit-date:
  doc: add corrected commit date info
  commit-reach: use corrected commit dates in paint_down_to_common()
  commit-graph: use generation v2 only if entire chain does
  commit-graph: implement generation data chunk
  commit-graph: implement corrected commit date
  commit-graph: return 64-bit generation number
  commit-graph: add a slab to store topological levels
  t6600-test-reach: generalize *_three_modes
  commit-graph: consolidate fill_commit_graph_info
  revision: parse parent in indegree_walk_step()
  commit-graph: fix regression when computing Bloom filters
This commit is contained in:
Junio C Hamano 2021-02-17 17:21:40 -08:00
commit 8b4701ae4f
19 changed files with 668 additions and 155 deletions

View File

@ -4,11 +4,7 @@ Git commit graph format
The Git commit graph stores a list of commit OIDs and some associated
metadata, including:
- The generation number of the commit. Commits with no parents have
generation number 1; commits with parents have generation number
one more than the maximum generation number of its parents. We
reserve zero as special, and can be used to mark a generation
number invalid or as "not computed".
- The generation number of the commit.
- The root tree OID.
@ -86,13 +82,33 @@ CHUNK DATA:
position. If there are more than two parents, the second value
has its most-significant bit on and the other bits store an array
position into the Extra Edge List chunk.
* The next 8 bytes store the generation number of the commit and
* The next 8 bytes store the topological level (generation number v1)
of the commit and
the commit time in seconds since EPOCH. The generation number
uses the higher 30 bits of the first 4 bytes, while the commit
time uses the 32 bits of the second 4 bytes, along with the lowest
2 bits of the lowest byte, storing the 33rd and 34th bit of the
commit time.
Generation Data (ID: {'G', 'D', 'A', 'T' }) (N * 4 bytes) [Optional]
* This list of 4-byte values store corrected commit date offsets for the
commits, arranged in the same order as commit data chunk.
* If the corrected commit date offset cannot be stored within 31 bits,
the value has its most-significant bit on and the other bits store
the position of corrected commit date into the Generation Data Overflow
chunk.
* Generation Data chunk is present only when commit-graph file is written
by compatible versions of Git and in case of split commit-graph chains,
the topmost layer also has Generation Data chunk.
Generation Data Overflow (ID: {'G', 'D', 'O', 'V' }) [Optional]
* This list of 8-byte values stores the corrected commit date offsets
for commits with corrected commit date offsets that cannot be
stored within 31 bits.
* Generation Data Overflow chunk is present only when Generation Data
chunk is present and atleast one corrected commit date offset cannot
be stored within 31 bits.
Extra Edge List (ID: {'E', 'D', 'G', 'E'}) [Optional]
This list of 4-byte values store the second through nth parents for
all octopus merges. The second parent value in the commit data stores

View File

@ -38,14 +38,31 @@ A consumer may load the following info for a commit from the graph:
Values 1-4 satisfy the requirements of parse_commit_gently().
Define the "generation number" of a commit recursively as follows:
There are two definitions of generation number:
1. Corrected committer dates (generation number v2)
2. Topological levels (generation nummber v1)
* A commit with no parents (a root commit) has generation number one.
Define "corrected committer date" of a commit recursively as follows:
* A commit with at least one parent has generation number one more than
the largest generation number among its parents.
* A commit with no parents (a root commit) has corrected committer date
equal to its committer date.
Equivalently, the generation number of a commit A is one more than the
* A commit with at least one parent has corrected committer date equal to
the maximum of its commiter date and one more than the largest corrected
committer date among its parents.
* As a special case, a root commit with timestamp zero has corrected commit
date of 1, to be able to distinguish it from GENERATION_NUMBER_ZERO
(that is, an uncomputed corrected commit date).
Define the "topological level" of a commit recursively as follows:
* A commit with no parents (a root commit) has topological level of one.
* A commit with at least one parent has topological level one more than
the largest topological level among its parents.
Equivalently, the topological level of a commit A is one more than the
length of a longest path from A to a root commit. The recursive definition
is easier to use for computation and observing the following property:
@ -60,6 +77,9 @@ is easier to use for computation and observing the following property:
generation numbers, then we always expand the boundary commit with highest
generation number and can easily detect the stopping condition.
The property applies to both versions of generation number, that is both
corrected committer dates and topological levels.
This property can be used to significantly reduce the time it takes to
walk commits and determine topological relationships. Without generation
numbers, the general heuristic is the following:
@ -67,7 +87,9 @@ numbers, the general heuristic is the following:
If A and B are commits with commit time X and Y, respectively, and
X < Y, then A _probably_ cannot reach B.
This heuristic is currently used whenever the computation is allowed to
In absence of corrected commit dates (for example, old versions of Git or
mixed generation graph chains),
this heuristic is currently used whenever the computation is allowed to
violate topological relationships due to clock skew (such as "git log"
with default order), but is not used when the topological order is
required (such as merge base calculations, "git log --graph").
@ -77,7 +99,7 @@ in the commit graph. We can treat these commits as having "infinite"
generation number and walk until reaching commits with known generation
number.
We use the macro GENERATION_NUMBER_INFINITY = 0xFFFFFFFF to mark commits not
We use the macro GENERATION_NUMBER_INFINITY to mark commits not
in the commit-graph file. If a commit-graph file was written by a version
of Git that did not compute generation numbers, then those commits will
have generation number represented by the macro GENERATION_NUMBER_ZERO = 0.
@ -93,12 +115,12 @@ fully-computed generation numbers. Using strict inequality may result in
walking a few extra commits, but the simplicity in dealing with commits
with generation number *_INFINITY or *_ZERO is valuable.
We use the macro GENERATION_NUMBER_MAX = 0x3FFFFFFF to for commits whose
generation numbers are computed to be at least this value. We limit at
this value since it is the largest value that can be stored in the
commit-graph file using the 30 bits available to generation numbers. This
presents another case where a commit can have generation number equal to
that of a parent.
We use the macro GENERATION_NUMBER_V1_MAX = 0x3FFFFFFF for commits whose
topological levels (generation number v1) are computed to be at least
this value. We limit at this value since it is the largest value that
can be stored in the commit-graph file using the 30 bits available
to topological levels. This presents another case where a commit can
have generation number equal to that of a parent.
Design Details
--------------
@ -267,6 +289,35 @@ The merge strategy values (2 for the size multiple, 64,000 for the maximum
number of commits) could be extracted into config settings for full
flexibility.
## Handling Mixed Generation Number Chains
With the introduction of generation number v2 and generation data chunk, the
following scenario is possible:
1. "New" Git writes a commit-graph with the corrected commit dates.
2. "Old" Git writes a split commit-graph on top without corrected commit dates.
A naive approach of using the newest available generation number from
each layer would lead to violated expectations: the lower layer would
use corrected commit dates which are much larger than the topological
levels of the higher layer. For this reason, Git inspects the topmost
layer to see if the layer is missing corrected commit dates. In such a case
Git only uses topological level for generation numbers.
When writing a new layer in split commit-graph, we write corrected commit
dates if the topmost layer has corrected commit dates written. This
guarantees that if a layer has corrected commit dates, all lower layers
must have corrected commit dates as well.
When merging layers, we do not consider whether the merged layers had corrected
commit dates. Instead, the new layer will have corrected commit dates if the
layer below the new layer has corrected commit dates.
While writing or merging layers, if the new layer is the only layer, it will
have corrected commit dates when written by compatible versions of Git. Thus,
rewriting split commit-graph as a single file (`--split=replace`) creates a
single layer with corrected commit dates.
## Deleting graph-{hash} files
After a new tip file is written, some `graph-{hash}` files may no longer

View File

@ -38,11 +38,13 @@ void git_test_write_commit_graph_or_die(void)
#define GRAPH_CHUNKID_OIDFANOUT 0x4f494446 /* "OIDF" */
#define GRAPH_CHUNKID_OIDLOOKUP 0x4f49444c /* "OIDL" */
#define GRAPH_CHUNKID_DATA 0x43444154 /* "CDAT" */
#define GRAPH_CHUNKID_GENERATION_DATA 0x47444154 /* "GDAT" */
#define GRAPH_CHUNKID_GENERATION_DATA_OVERFLOW 0x47444f56 /* "GDOV" */
#define GRAPH_CHUNKID_EXTRAEDGES 0x45444745 /* "EDGE" */
#define GRAPH_CHUNKID_BLOOMINDEXES 0x42494458 /* "BIDX" */
#define GRAPH_CHUNKID_BLOOMDATA 0x42444154 /* "BDAT" */
#define GRAPH_CHUNKID_BASE 0x42415345 /* "BASE" */
#define MAX_NUM_CHUNKS 7
#define MAX_NUM_CHUNKS 9
#define GRAPH_DATA_WIDTH (the_hash_algo->rawsz + 16)
@ -61,9 +63,13 @@ void git_test_write_commit_graph_or_die(void)
#define GRAPH_MIN_SIZE (GRAPH_HEADER_SIZE + 4 * GRAPH_CHUNKLOOKUP_WIDTH \
+ GRAPH_FANOUT_SIZE + the_hash_algo->rawsz)
#define CORRECTED_COMMIT_DATE_OFFSET_OVERFLOW (1ULL << 31)
/* Remember to update object flag allocation in object.h */
#define REACHABLE (1u<<15)
define_commit_slab(topo_level_slab, uint32_t);
/* Keep track of the order in which commits are added to our list. */
define_commit_slab(commit_pos, int);
static struct commit_pos commit_pos = COMMIT_SLAB_INIT(1, commit_pos);
@ -99,7 +105,7 @@ uint32_t commit_graph_position(const struct commit *c)
return data ? data->graph_pos : COMMIT_NOT_FROM_GRAPH;
}
uint32_t commit_graph_generation(const struct commit *c)
timestamp_t commit_graph_generation(const struct commit *c)
{
struct commit_graph_data *data =
commit_graph_data_slab_peek(&commit_graph_data_slab, c);
@ -139,13 +145,17 @@ static struct commit_graph_data *commit_graph_data_at(const struct commit *c)
return data;
}
/*
* Should be used only while writing commit-graph as it compares
* generation value of commits by directly accessing commit-slab.
*/
static int commit_gen_cmp(const void *va, const void *vb)
{
const struct commit *a = *(const struct commit **)va;
const struct commit *b = *(const struct commit **)vb;
uint32_t generation_a = commit_graph_generation(a);
uint32_t generation_b = commit_graph_generation(b);
const timestamp_t generation_a = commit_graph_data_at(a)->generation;
const timestamp_t generation_b = commit_graph_data_at(b)->generation;
/* lower generation commits first */
if (generation_a < generation_b)
return -1;
@ -388,6 +398,20 @@ struct commit_graph *parse_commit_graph(struct repository *r,
graph->chunk_commit_data = data + chunk_offset;
break;
case GRAPH_CHUNKID_GENERATION_DATA:
if (graph->chunk_generation_data)
chunk_repeated = 1;
else
graph->chunk_generation_data = data + chunk_offset;
break;
case GRAPH_CHUNKID_GENERATION_DATA_OVERFLOW:
if (graph->chunk_generation_data_overflow)
chunk_repeated = 1;
else
graph->chunk_generation_data_overflow = data + chunk_offset;
break;
case GRAPH_CHUNKID_EXTRAEDGES:
if (graph->chunk_extra_edges)
chunk_repeated = 1;
@ -590,6 +614,21 @@ static struct commit_graph *load_commit_graph_chain(struct repository *r,
return graph_chain;
}
static void validate_mixed_generation_chain(struct commit_graph *g)
{
int read_generation_data;
if (!g)
return;
read_generation_data = !!g->chunk_generation_data;
while (g) {
g->read_generation_data = read_generation_data;
g = g->base_graph;
}
}
struct commit_graph *read_commit_graph_one(struct repository *r,
struct object_directory *odb)
{
@ -598,6 +637,8 @@ struct commit_graph *read_commit_graph_one(struct repository *r,
if (!g)
g = load_commit_graph_chain(r, odb);
validate_mixed_generation_chain(g);
return g;
}
@ -673,6 +714,20 @@ int generation_numbers_enabled(struct repository *r)
return !!first_generation;
}
int corrected_commit_dates_enabled(struct repository *r)
{
struct commit_graph *g;
if (!prepare_commit_graph(r))
return 0;
g = r->objects->commit_graph;
if (!g->num_commits)
return 0;
return g->read_generation_data;
}
struct bloom_filter_settings *get_bloom_filter_settings(struct repository *r)
{
struct commit_graph *g = r->objects->commit_graph;
@ -748,17 +803,41 @@ static void fill_commit_graph_info(struct commit *item, struct commit_graph *g,
{
const unsigned char *commit_data;
struct commit_graph_data *graph_data;
uint32_t lex_index;
uint32_t lex_index, offset_pos;
uint64_t date_high, date_low, offset;
while (pos < g->num_commits_in_base)
g = g->base_graph;
if (pos >= g->num_commits + g->num_commits_in_base)
die(_("invalid commit position. commit-graph is likely corrupt"));
lex_index = pos - g->num_commits_in_base;
commit_data = g->chunk_commit_data + GRAPH_DATA_WIDTH * lex_index;
graph_data = commit_graph_data_at(item);
graph_data->graph_pos = pos;
graph_data->generation = get_be32(commit_data + g->hash_len + 8) >> 2;
date_high = get_be32(commit_data + g->hash_len + 8) & 0x3;
date_low = get_be32(commit_data + g->hash_len + 12);
item->date = (timestamp_t)((date_high << 32) | date_low);
if (g->read_generation_data) {
offset = (timestamp_t)get_be32(g->chunk_generation_data + sizeof(uint32_t) * lex_index);
if (offset & CORRECTED_COMMIT_DATE_OFFSET_OVERFLOW) {
if (!g->chunk_generation_data_overflow)
die(_("commit-graph requires overflow generation data but has none"));
offset_pos = offset ^ CORRECTED_COMMIT_DATE_OFFSET_OVERFLOW;
graph_data->generation = get_be64(g->chunk_generation_data_overflow + 8 * offset_pos);
} else
graph_data->generation = item->date + offset;
} else
graph_data->generation = get_be32(commit_data + g->hash_len + 8) >> 2;
if (g->topo_levels)
*topo_level_slab_at(g->topo_levels, item) = get_be32(commit_data + g->hash_len + 8) >> 2;
}
static inline void set_commit_tree(struct commit *c, struct tree *t)
@ -772,38 +851,22 @@ static int fill_commit_in_graph(struct repository *r,
{
uint32_t edge_value;
uint32_t *parent_data_ptr;
uint64_t date_low, date_high;
struct commit_list **pptr;
struct commit_graph_data *graph_data;
const unsigned char *commit_data;
uint32_t lex_index;
while (pos < g->num_commits_in_base)
g = g->base_graph;
if (pos >= g->num_commits + g->num_commits_in_base)
die(_("invalid commit position. commit-graph is likely corrupt"));
fill_commit_graph_info(item, g, pos);
/*
* Store the "full" position, but then use the
* "local" position for the rest of the calculation.
*/
graph_data = commit_graph_data_at(item);
graph_data->graph_pos = pos;
lex_index = pos - g->num_commits_in_base;
commit_data = g->chunk_commit_data + (g->hash_len + 16) * lex_index;
item->object.parsed = 1;
set_commit_tree(item, NULL);
date_high = get_be32(commit_data + g->hash_len + 8) & 0x3;
date_low = get_be32(commit_data + g->hash_len + 12);
item->date = (timestamp_t)((date_high << 32) | date_low);
graph_data->generation = get_be32(commit_data + g->hash_len + 8) >> 2;
pptr = &item->parents;
edge_value = get_be32(commit_data + g->hash_len);
@ -943,6 +1006,7 @@ struct write_commit_graph_context {
struct oid_array oids;
struct packed_commit_list commits;
int num_extra_edges;
int num_generation_data_overflows;
unsigned long approx_nr_objects;
struct progress *progress;
int progress_done;
@ -961,8 +1025,10 @@ struct write_commit_graph_context {
report_progress:1,
split:1,
changed_paths:1,
order_by_pack:1;
order_by_pack:1,
write_generation_data:1;
struct topo_level_slab *topo_levels;
const struct commit_graph_opts *opts;
size_t total_bloom_filter_data_size;
const struct bloom_filter_settings *bloom_settings;
@ -1109,7 +1175,7 @@ static int write_graph_chunk_data(struct hashfile *f,
else
packedDate[0] = 0;
packedDate[0] |= htonl(commit_graph_data_at(*list)->generation << 2);
packedDate[0] |= htonl(*topo_level_slab_at(ctx->topo_levels, *list) << 2);
packedDate[1] = htonl((*list)->date);
hashwrite(f, packedDate, 8);
@ -1120,6 +1186,45 @@ static int write_graph_chunk_data(struct hashfile *f,
return 0;
}
static int write_graph_chunk_generation_data(struct hashfile *f,
struct write_commit_graph_context *ctx)
{
int i, num_generation_data_overflows = 0;
for (i = 0; i < ctx->commits.nr; i++) {
struct commit *c = ctx->commits.list[i];
timestamp_t offset = commit_graph_data_at(c)->generation - c->date;
display_progress(ctx->progress, ++ctx->progress_cnt);
if (offset > GENERATION_NUMBER_V2_OFFSET_MAX) {
offset = CORRECTED_COMMIT_DATE_OFFSET_OVERFLOW | num_generation_data_overflows;
num_generation_data_overflows++;
}
hashwrite_be32(f, offset);
}
return 0;
}
static int write_graph_chunk_generation_data_overflow(struct hashfile *f,
struct write_commit_graph_context *ctx)
{
int i;
for (i = 0; i < ctx->commits.nr; i++) {
struct commit *c = ctx->commits.list[i];
timestamp_t offset = commit_graph_data_at(c)->generation - c->date;
display_progress(ctx->progress, ++ctx->progress_cnt);
if (offset > GENERATION_NUMBER_V2_OFFSET_MAX) {
hashwrite_be32(f, offset >> 32);
hashwrite_be32(f, (uint32_t) offset);
}
}
return 0;
}
static int write_graph_chunk_extra_edges(struct hashfile *f,
struct write_commit_graph_context *ctx)
{
@ -1339,11 +1444,12 @@ static void compute_generation_numbers(struct write_commit_graph_context *ctx)
_("Computing commit graph generation numbers"),
ctx->commits.nr);
for (i = 0; i < ctx->commits.nr; i++) {
uint32_t generation = commit_graph_data_at(ctx->commits.list[i])->generation;
uint32_t level = *topo_level_slab_at(ctx->topo_levels, ctx->commits.list[i]);
timestamp_t corrected_commit_date = commit_graph_data_at(ctx->commits.list[i])->generation;
display_progress(ctx->progress, i + 1);
if (generation != GENERATION_NUMBER_INFINITY &&
generation != GENERATION_NUMBER_ZERO)
if (level != GENERATION_NUMBER_ZERO &&
corrected_commit_date != GENERATION_NUMBER_ZERO)
continue;
commit_list_insert(ctx->commits.list[i], &list);
@ -1351,29 +1457,40 @@ static void compute_generation_numbers(struct write_commit_graph_context *ctx)
struct commit *current = list->item;
struct commit_list *parent;
int all_parents_computed = 1;
uint32_t max_generation = 0;
uint32_t max_level = 0;
timestamp_t max_corrected_commit_date = 0;
for (parent = current->parents; parent; parent = parent->next) {
generation = commit_graph_data_at(parent->item)->generation;
level = *topo_level_slab_at(ctx->topo_levels, parent->item);
corrected_commit_date = commit_graph_data_at(parent->item)->generation;
if (generation == GENERATION_NUMBER_INFINITY ||
generation == GENERATION_NUMBER_ZERO) {
if (level == GENERATION_NUMBER_ZERO ||
corrected_commit_date == GENERATION_NUMBER_ZERO) {
all_parents_computed = 0;
commit_list_insert(parent->item, &list);
break;
} else if (generation > max_generation) {
max_generation = generation;
}
if (level > max_level)
max_level = level;
if (corrected_commit_date > max_corrected_commit_date)
max_corrected_commit_date = corrected_commit_date;
}
if (all_parents_computed) {
struct commit_graph_data *data = commit_graph_data_at(current);
data->generation = max_generation + 1;
pop_commit(&list);
if (data->generation > GENERATION_NUMBER_MAX)
data->generation = GENERATION_NUMBER_MAX;
if (max_level > GENERATION_NUMBER_V1_MAX - 1)
max_level = GENERATION_NUMBER_V1_MAX - 1;
*topo_level_slab_at(ctx->topo_levels, current) = max_level + 1;
if (current->date && current->date > max_corrected_commit_date)
max_corrected_commit_date = current->date - 1;
commit_graph_data_at(current)->generation = max_corrected_commit_date + 1;
if (commit_graph_data_at(current)->generation - current->date > GENERATION_NUMBER_V2_OFFSET_MAX)
ctx->num_generation_data_overflows++;
}
}
}
@ -1707,6 +1824,21 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)
chunks[2].id = GRAPH_CHUNKID_DATA;
chunks[2].size = (hashsz + 16) * ctx->commits.nr;
chunks[2].write_fn = write_graph_chunk_data;
if (git_env_bool(GIT_TEST_COMMIT_GRAPH_NO_GDAT, 0))
ctx->write_generation_data = 0;
if (ctx->write_generation_data) {
chunks[num_chunks].id = GRAPH_CHUNKID_GENERATION_DATA;
chunks[num_chunks].size = sizeof(uint32_t) * ctx->commits.nr;
chunks[num_chunks].write_fn = write_graph_chunk_generation_data;
num_chunks++;
}
if (ctx->num_generation_data_overflows) {
chunks[num_chunks].id = GRAPH_CHUNKID_GENERATION_DATA_OVERFLOW;
chunks[num_chunks].size = sizeof(timestamp_t) * ctx->num_generation_data_overflows;
chunks[num_chunks].write_fn = write_graph_chunk_generation_data_overflow;
num_chunks++;
}
if (ctx->num_extra_edges) {
chunks[num_chunks].id = GRAPH_CHUNKID_EXTRAEDGES;
chunks[num_chunks].size = 4 * ctx->num_extra_edges;
@ -1918,6 +2050,13 @@ static void split_graph_merge_strategy(struct write_commit_graph_context *ctx)
if (i < ctx->num_commit_graphs_after)
ctx->commit_graph_hash_after[i] = xstrdup(oid_to_hex(&g->oid));
/*
* If the topmost remaining layer has generation data chunk, the
* resultant layer also has generation data chunk.
*/
if (i == ctx->num_commit_graphs_after - 2)
ctx->write_generation_data = !!g->chunk_generation_data;
i--;
g = g->base_graph;
}
@ -2109,6 +2248,7 @@ int write_commit_graph(struct object_directory *odb,
int res = 0;
int replace = 0;
struct bloom_filter_settings bloom_settings = DEFAULT_BLOOM_FILTER_SETTINGS;
struct topo_level_slab topo_levels;
prepare_repo_settings(the_repository);
if (!the_repository->settings.core_commit_graph) {
@ -2126,6 +2266,8 @@ int write_commit_graph(struct object_directory *odb,
ctx->split = flags & COMMIT_GRAPH_WRITE_SPLIT ? 1 : 0;
ctx->opts = opts;
ctx->total_bloom_filter_data_size = 0;
ctx->write_generation_data = 1;
ctx->num_generation_data_overflows = 0;
bloom_settings.bits_per_entry = git_env_ulong("GIT_TEST_BLOOM_SETTINGS_BITS_PER_ENTRY",
bloom_settings.bits_per_entry);
@ -2135,6 +2277,18 @@ int write_commit_graph(struct object_directory *odb,
bloom_settings.max_changed_paths);
ctx->bloom_settings = &bloom_settings;
init_topo_level_slab(&topo_levels);
ctx->topo_levels = &topo_levels;
if (ctx->r->objects->commit_graph) {
struct commit_graph *g = ctx->r->objects->commit_graph;
while (g) {
g->topo_levels = &topo_levels;
g = g->base_graph;
}
}
if (flags & COMMIT_GRAPH_WRITE_BLOOM_FILTERS)
ctx->changed_paths = 1;
if (!(flags & COMMIT_GRAPH_NO_WRITE_BLOOM_FILTERS)) {
@ -2227,6 +2381,8 @@ int write_commit_graph(struct object_directory *odb,
} else
ctx->num_commit_graphs_after = 1;
validate_mixed_generation_chain(ctx->r->objects->commit_graph);
compute_generation_numbers(ctx);
if (ctx->changed_paths)
@ -2355,8 +2511,8 @@ int verify_commit_graph(struct repository *r, struct commit_graph *g, int flags)
for (i = 0; i < g->num_commits; i++) {
struct commit *graph_commit, *odb_commit;
struct commit_list *graph_parents, *odb_parents;
uint32_t max_generation = 0;
uint32_t generation;
timestamp_t max_generation = 0;
timestamp_t generation;
display_progress(progress, i + 1);
hashcpy(cur_oid.hash, g->chunk_oid_lookup + g->hash_len * i);
@ -2420,16 +2576,17 @@ int verify_commit_graph(struct repository *r, struct commit_graph *g, int flags)
continue;
/*
* If one of our parents has generation GENERATION_NUMBER_MAX, then
* our generation is also GENERATION_NUMBER_MAX. Decrement to avoid
* extra logic in the following condition.
* If we are using topological level and one of our parents has
* generation GENERATION_NUMBER_V1_MAX, then our generation is
* also GENERATION_NUMBER_V1_MAX. Decrement to avoid extra logic
* in the following condition.
*/
if (max_generation == GENERATION_NUMBER_MAX)
if (!g->read_generation_data && max_generation == GENERATION_NUMBER_V1_MAX)
max_generation--;
generation = commit_graph_generation(graph_commit);
if (generation != max_generation + 1)
graph_report(_("commit-graph generation for commit %s is %u != %u"),
if (generation < max_generation + 1)
graph_report(_("commit-graph generation for commit %s is %"PRItime" < %"PRItime),
oid_to_hex(&cur_oid),
generation,
max_generation + 1);

View File

@ -6,6 +6,7 @@
#include "oidset.h"
#define GIT_TEST_COMMIT_GRAPH "GIT_TEST_COMMIT_GRAPH"
#define GIT_TEST_COMMIT_GRAPH_NO_GDAT "GIT_TEST_COMMIT_GRAPH_NO_GDAT"
#define GIT_TEST_COMMIT_GRAPH_DIE_ON_PARSE "GIT_TEST_COMMIT_GRAPH_DIE_ON_PARSE"
#define GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS "GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS"
@ -63,16 +64,20 @@ struct commit_graph {
struct object_directory *odb;
uint32_t num_commits_in_base;
unsigned int read_generation_data;
struct commit_graph *base_graph;
const uint32_t *chunk_oid_fanout;
const unsigned char *chunk_oid_lookup;
const unsigned char *chunk_commit_data;
const unsigned char *chunk_generation_data;
const unsigned char *chunk_generation_data_overflow;
const unsigned char *chunk_extra_edges;
const unsigned char *chunk_base_graphs;
const unsigned char *chunk_bloom_indexes;
const unsigned char *chunk_bloom_data;
struct topo_level_slab *topo_levels;
struct bloom_filter_settings *bloom_filter_settings;
};
@ -90,6 +95,12 @@ struct commit_graph *parse_commit_graph(struct repository *r,
*/
int generation_numbers_enabled(struct repository *r);
/*
* Return 1 if and only if the repository has a commit-graph
* file and generation data chunk has been written for the file.
*/
int corrected_commit_dates_enabled(struct repository *r);
struct bloom_filter_settings *get_bloom_filter_settings(struct repository *r);
enum commit_graph_write_flags {
@ -144,12 +155,12 @@ void disable_commit_graph(struct repository *r);
struct commit_graph_data {
uint32_t graph_pos;
uint32_t generation;
timestamp_t generation;
};
/*
* Commits should be parsed before accessing generation, graph positions.
*/
uint32_t commit_graph_generation(const struct commit *);
timestamp_t commit_graph_generation(const struct commit *);
uint32_t commit_graph_position(const struct commit *);
#endif

View File

@ -32,14 +32,14 @@ static int queue_has_nonstale(struct prio_queue *queue)
static struct commit_list *paint_down_to_common(struct repository *r,
struct commit *one, int n,
struct commit **twos,
int min_generation)
timestamp_t min_generation)
{
struct prio_queue queue = { compare_commits_by_gen_then_commit_date };
struct commit_list *result = NULL;
int i;
uint32_t last_gen = GENERATION_NUMBER_INFINITY;
timestamp_t last_gen = GENERATION_NUMBER_INFINITY;
if (!min_generation)
if (!min_generation && !corrected_commit_dates_enabled(r))
queue.compare = compare_commits_by_commit_date;
one->object.flags |= PARENT1;
@ -58,10 +58,10 @@ static struct commit_list *paint_down_to_common(struct repository *r,
struct commit *commit = prio_queue_get(&queue);
struct commit_list *parents;
int flags;
uint32_t generation = commit_graph_generation(commit);
timestamp_t generation = commit_graph_generation(commit);
if (min_generation && generation > last_gen)
BUG("bad generation skip %8x > %8x at %s",
BUG("bad generation skip %"PRItime" > %"PRItime" at %s",
generation, last_gen,
oid_to_hex(&commit->object.oid));
last_gen = generation;
@ -177,12 +177,12 @@ static int remove_redundant(struct repository *r, struct commit **array, int cnt
repo_parse_commit(r, array[i]);
for (i = 0; i < cnt; i++) {
struct commit_list *common;
uint32_t min_generation = commit_graph_generation(array[i]);
timestamp_t min_generation = commit_graph_generation(array[i]);
if (redundant[i])
continue;
for (j = filled = 0; j < cnt; j++) {
uint32_t curr_generation;
timestamp_t curr_generation;
if (i == j || redundant[j])
continue;
filled_index[filled] = j;
@ -321,7 +321,7 @@ int repo_in_merge_bases_many(struct repository *r, struct commit *commit,
{
struct commit_list *bases;
int ret = 0, i;
uint32_t generation, max_generation = GENERATION_NUMBER_ZERO;
timestamp_t generation, max_generation = GENERATION_NUMBER_ZERO;
if (repo_parse_commit(r, commit))
return ret;
@ -470,7 +470,7 @@ static int in_commit_list(const struct commit_list *want, struct commit *c)
static enum contains_result contains_test(struct commit *candidate,
const struct commit_list *want,
struct contains_cache *cache,
uint32_t cutoff)
timestamp_t cutoff)
{
enum contains_result *cached = contains_cache_at(cache, candidate);
@ -506,11 +506,11 @@ static enum contains_result contains_tag_algo(struct commit *candidate,
{
struct contains_stack contains_stack = { 0, 0, NULL };
enum contains_result result;
uint32_t cutoff = GENERATION_NUMBER_INFINITY;
timestamp_t cutoff = GENERATION_NUMBER_INFINITY;
const struct commit_list *p;
for (p = want; p; p = p->next) {
uint32_t generation;
timestamp_t generation;
struct commit *c = p->item;
load_commit_graph_info(the_repository, c);
generation = commit_graph_generation(c);
@ -566,8 +566,8 @@ static int compare_commits_by_gen(const void *_a, const void *_b)
const struct commit *a = *(const struct commit * const *)_a;
const struct commit *b = *(const struct commit * const *)_b;
uint32_t generation_a = commit_graph_generation(a);
uint32_t generation_b = commit_graph_generation(b);
timestamp_t generation_a = commit_graph_generation(a);
timestamp_t generation_b = commit_graph_generation(b);
if (generation_a < generation_b)
return -1;
@ -580,7 +580,7 @@ int can_all_from_reach_with_flag(struct object_array *from,
unsigned int with_flag,
unsigned int assign_flag,
time_t min_commit_date,
uint32_t min_generation)
timestamp_t min_generation)
{
struct commit **list = NULL;
int i;
@ -681,13 +681,13 @@ int can_all_from_reach(struct commit_list *from, struct commit_list *to,
time_t min_commit_date = cutoff_by_min_date ? from->item->date : 0;
struct commit_list *from_iter = from, *to_iter = to;
int result;
uint32_t min_generation = GENERATION_NUMBER_INFINITY;
timestamp_t min_generation = GENERATION_NUMBER_INFINITY;
while (from_iter) {
add_object_array(&from_iter->item->object, NULL, &from_objs);
if (!parse_commit(from_iter->item)) {
uint32_t generation;
timestamp_t generation;
if (from_iter->item->date < min_commit_date)
min_commit_date = from_iter->item->date;
@ -701,7 +701,7 @@ int can_all_from_reach(struct commit_list *from, struct commit_list *to,
while (to_iter) {
if (!parse_commit(to_iter->item)) {
uint32_t generation;
timestamp_t generation;
if (to_iter->item->date < min_commit_date)
min_commit_date = to_iter->item->date;
@ -741,13 +741,13 @@ struct commit_list *get_reachable_subset(struct commit **from, int nr_from,
struct commit_list *found_commits = NULL;
struct commit **to_last = to + nr_to;
struct commit **from_last = from + nr_from;
uint32_t min_generation = GENERATION_NUMBER_INFINITY;
timestamp_t min_generation = GENERATION_NUMBER_INFINITY;
int num_to_find = 0;
struct prio_queue queue = { compare_commits_by_gen_then_commit_date };
for (item = to; item < to_last; item++) {
uint32_t generation;
timestamp_t generation;
struct commit *c = *item;
parse_commit(c);

View File

@ -87,7 +87,7 @@ int can_all_from_reach_with_flag(struct object_array *from,
unsigned int with_flag,
unsigned int assign_flag,
time_t min_commit_date,
uint32_t min_generation);
timestamp_t min_generation);
int can_all_from_reach(struct commit_list *from, struct commit_list *to,
int commit_date_cutoff);

View File

@ -753,8 +753,8 @@ int compare_commits_by_author_date(const void *a_, const void *b_,
int compare_commits_by_gen_then_commit_date(const void *a_, const void *b_, void *unused)
{
const struct commit *a = a_, *b = b_;
const uint32_t generation_a = commit_graph_generation(a),
generation_b = commit_graph_generation(b);
const timestamp_t generation_a = commit_graph_generation(a),
generation_b = commit_graph_generation(b);
/* newer commits first */
if (generation_a < generation_b)

View File

@ -11,9 +11,10 @@
#include "commit-slab.h"
#define COMMIT_NOT_FROM_GRAPH 0xFFFFFFFF
#define GENERATION_NUMBER_INFINITY 0xFFFFFFFF
#define GENERATION_NUMBER_MAX 0x3FFFFFFF
#define GENERATION_NUMBER_INFINITY ((1ULL << 63) - 1)
#define GENERATION_NUMBER_V1_MAX 0x3FFFFFFF
#define GENERATION_NUMBER_ZERO 0
#define GENERATION_NUMBER_V2_OFFSET_MAX ((1ULL << 31) - 1)
struct commit_list {
struct commit *item;

View File

@ -3270,7 +3270,7 @@ define_commit_slab(indegree_slab, int);
define_commit_slab(author_date_slab, timestamp_t);
struct topo_walk_info {
uint32_t min_generation;
timestamp_t min_generation;
struct prio_queue explore_queue;
struct prio_queue indegree_queue;
struct prio_queue topo_queue;
@ -3338,7 +3338,7 @@ static void explore_walk_step(struct rev_info *revs)
}
static void explore_to_depth(struct rev_info *revs,
uint32_t gen_cutoff)
timestamp_t gen_cutoff)
{
struct topo_walk_info *info = revs->topo_walk_info;
struct commit *c;
@ -3367,6 +3367,9 @@ static void indegree_walk_step(struct rev_info *revs)
struct commit *parent = p->item;
int *pi = indegree_slab_at(&info->indegree, parent);
if (repo_parse_commit_gently(revs->repo, parent, 1) < 0)
return;
if (*pi)
(*pi)++;
else
@ -3380,7 +3383,7 @@ static void indegree_walk_step(struct rev_info *revs)
}
static void compute_indegrees_to_depth(struct rev_info *revs,
uint32_t gen_cutoff)
timestamp_t gen_cutoff)
{
struct topo_walk_info *info = revs->topo_walk_info;
struct commit *c;
@ -3438,7 +3441,7 @@ static void init_topo_walk(struct rev_info *revs)
info->min_generation = GENERATION_NUMBER_INFINITY;
for (list = revs->commits; list; list = list->next) {
struct commit *c = list->item;
uint32_t generation;
timestamp_t generation;
if (repo_parse_commit_gently(revs->repo, c, 1))
continue;
@ -3506,7 +3509,7 @@ static void expand_topo_walk(struct rev_info *revs, struct commit *commit)
for (p = commit->parents; p; p = p->next) {
struct commit *parent = p->item;
int *pi;
uint32_t generation;
timestamp_t generation;
if (parent->object.flags & UNINTERESTING)
continue;

View File

@ -387,6 +387,9 @@ GIT_TEST_COMMIT_GRAPH=<boolean>, when true, forces the commit-graph to
be written after every 'git commit' command, and overrides the
'core.commitGraph' setting to true.
GIT_TEST_COMMIT_GRAPH_NO_GDAT=<boolean>, when true, forces the
commit-graph to be written without generation data chunk.
GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS=<boolean>, when true, forces
commit-graph write to compute and write changed path Bloom filters for
every 'git commit-graph write', as if the `--changed-paths` option was

View File

@ -33,6 +33,10 @@ int cmd__read_graph(int argc, const char **argv)
printf(" oid_lookup");
if (graph->chunk_commit_data)
printf(" commit_metadata");
if (graph->chunk_generation_data)
printf(" generation_data");
if (graph->chunk_generation_data_overflow)
printf(" generation_data_overflow");
if (graph->chunk_extra_edges)
printf(" extra_edges");
if (graph->chunk_bloom_indexes)

View File

@ -43,11 +43,11 @@ test_expect_success 'setup test - repo, commits, commit graph, log outputs' '
'
graph_read_expect () {
NUM_CHUNKS=5
NUM_CHUNKS=6
cat >expect <<- EOF
header: 43475048 1 $(test_oid oid_version) $NUM_CHUNKS 0
num_commits: $1
chunks: oid_fanout oid_lookup commit_metadata bloom_indexes bloom_data
chunks: oid_fanout oid_lookup commit_metadata generation_data bloom_indexes bloom_data
EOF
test-tool read-graph >actual &&
test_cmp expect actual

View File

@ -431,15 +431,33 @@ test_expect_success TAR_HUGE,LONG_IS_64BIT 'system tar can read our huge size' '
test_cmp expect actual
'
test_expect_success TIME_IS_64BIT 'set up repository with far-future commit' '
test_expect_success TIME_IS_64BIT 'set up repository with far-future (2^34 - 1) commit' '
rm -f .git/index &&
echo content >file &&
echo foo >file &&
git add file &&
GIT_COMMITTER_DATE="@68719476737 +0000" \
GIT_COMMITTER_DATE="@17179869183 +0000" \
git commit -m "tempori parendum"
'
test_expect_success TIME_IS_64BIT 'generate tar with future mtime' '
test_expect_success TIME_IS_64BIT 'generate tar with far-future mtime' '
git archive HEAD >future.tar
'
test_expect_success TAR_HUGE,TIME_IS_64BIT,TIME_T_IS_64BIT 'system tar can read our future mtime' '
echo 2514 >expect &&
tar_info future.tar | cut -d" " -f2 >actual &&
test_cmp expect actual
'
test_expect_success TIME_IS_64BIT 'set up repository with far-far-future (2^36 + 1) commit' '
rm -f .git/index &&
echo content >file &&
git add file &&
GIT_TEST_COMMIT_GRAPH=0 GIT_COMMITTER_DATE="@68719476737 +0000" \
git commit -m "tempori parendum"
'
test_expect_success TIME_IS_64BIT 'generate tar with far-far-future mtime' '
git archive HEAD >future.tar
'

View File

@ -76,7 +76,7 @@ graph_git_behavior 'no graph' full commits/3 commits/1
graph_read_expect() {
OPTIONAL=""
NUM_CHUNKS=3
if test ! -z $2
if test ! -z "$2"
then
OPTIONAL=" $2"
NUM_CHUNKS=$((3 + $(echo "$2" | wc -w)))
@ -103,14 +103,14 @@ test_expect_success 'exit with correct error on bad input to --stdin-commits' '
# valid commit and tree OID
git rev-parse HEAD HEAD^{tree} >in &&
git commit-graph write --stdin-commits <in &&
graph_read_expect 3
graph_read_expect 3 generation_data
'
test_expect_success 'write graph' '
cd "$TRASH_DIRECTORY/full" &&
git commit-graph write &&
test_path_is_file $objdir/info/commit-graph &&
graph_read_expect "3"
graph_read_expect "3" generation_data
'
test_expect_success POSIXPERM 'write graph has correct permissions' '
@ -219,7 +219,7 @@ test_expect_success 'write graph with merges' '
cd "$TRASH_DIRECTORY/full" &&
git commit-graph write &&
test_path_is_file $objdir/info/commit-graph &&
graph_read_expect "10" "extra_edges"
graph_read_expect "10" "generation_data extra_edges"
'
graph_git_behavior 'merge 1 vs 2' full merge/1 merge/2
@ -254,7 +254,7 @@ test_expect_success 'write graph with new commit' '
cd "$TRASH_DIRECTORY/full" &&
git commit-graph write &&
test_path_is_file $objdir/info/commit-graph &&
graph_read_expect "11" "extra_edges"
graph_read_expect "11" "generation_data extra_edges"
'
graph_git_behavior 'full graph, commit 8 vs merge 1' full commits/8 merge/1
@ -264,7 +264,7 @@ test_expect_success 'write graph with nothing new' '
cd "$TRASH_DIRECTORY/full" &&
git commit-graph write &&
test_path_is_file $objdir/info/commit-graph &&
graph_read_expect "11" "extra_edges"
graph_read_expect "11" "generation_data extra_edges"
'
graph_git_behavior 'cleared graph, commit 8 vs merge 1' full commits/8 merge/1
@ -274,7 +274,7 @@ test_expect_success 'build graph from latest pack with closure' '
cd "$TRASH_DIRECTORY/full" &&
cat new-idx | git commit-graph write --stdin-packs &&
test_path_is_file $objdir/info/commit-graph &&
graph_read_expect "9" "extra_edges"
graph_read_expect "9" "generation_data extra_edges"
'
graph_git_behavior 'graph from pack, commit 8 vs merge 1' full commits/8 merge/1
@ -287,7 +287,7 @@ test_expect_success 'build graph from commits with closure' '
git rev-parse merge/1 >>commits-in &&
cat commits-in | git commit-graph write --stdin-commits &&
test_path_is_file $objdir/info/commit-graph &&
graph_read_expect "6"
graph_read_expect "6" "generation_data"
'
graph_git_behavior 'graph from commits, commit 8 vs merge 1' full commits/8 merge/1
@ -297,7 +297,7 @@ test_expect_success 'build graph from commits with append' '
cd "$TRASH_DIRECTORY/full" &&
git rev-parse merge/3 | git commit-graph write --stdin-commits --append &&
test_path_is_file $objdir/info/commit-graph &&
graph_read_expect "10" "extra_edges"
graph_read_expect "10" "generation_data extra_edges"
'
graph_git_behavior 'append graph, commit 8 vs merge 1' full commits/8 merge/1
@ -307,7 +307,7 @@ test_expect_success 'build graph using --reachable' '
cd "$TRASH_DIRECTORY/full" &&
git commit-graph write --reachable &&
test_path_is_file $objdir/info/commit-graph &&
graph_read_expect "11" "extra_edges"
graph_read_expect "11" "generation_data extra_edges"
'
graph_git_behavior 'append graph, commit 8 vs merge 1' full commits/8 merge/1
@ -328,7 +328,7 @@ test_expect_success 'write graph in bare repo' '
cd "$TRASH_DIRECTORY/bare" &&
git commit-graph write &&
test_path_is_file $baredir/info/commit-graph &&
graph_read_expect "11" "extra_edges"
graph_read_expect "11" "generation_data extra_edges"
'
graph_git_behavior 'bare repo with graph, commit 8 vs merge 1' bare commits/8 merge/1
@ -454,8 +454,9 @@ test_expect_success 'warn on improper hash version' '
test_expect_success 'git commit-graph verify' '
cd "$TRASH_DIRECTORY/full" &&
git rev-parse commits/8 | git commit-graph write --stdin-commits &&
git commit-graph verify >output
git rev-parse commits/8 | GIT_TEST_COMMIT_GRAPH_NO_GDAT=1 git commit-graph write --stdin-commits &&
git commit-graph verify >output &&
graph_read_expect 9 extra_edges
'
NUM_COMMITS=9
@ -741,4 +742,56 @@ test_expect_success 'corrupt commit-graph write (missing tree)' '
)
'
# We test the overflow-related code with the following repo history:
#
# 4:F - 5:N - 6:U
# / \
# 1:U - 2:N - 3:U M:N
# \ /
# 7:N - 8:F - 9:N
#
# Here the commits denoted by U have committer date of zero seconds
# since Unix epoch, the commits denoted by N have committer date
# starting from 1112354055 seconds since Unix epoch (default committer
# date for the test suite), and the commits denoted by F have committer
# date of (2 ^ 31 - 2) seconds since Unix epoch.
#
# The largest offset observed is 2 ^ 31, just large enough to overflow.
#
test_expect_success 'set up and verify repo with generation data overflow chunk' '
objdir=".git/objects" &&
UNIX_EPOCH_ZERO="@0 +0000" &&
FUTURE_DATE="@2147483646 +0000" &&
test_oid_cache <<-EOF &&
oid_version sha1:1
oid_version sha256:2
EOF
cd "$TRASH_DIRECTORY" &&
mkdir repo &&
cd repo &&
git init &&
test_commit --date "$UNIX_EPOCH_ZERO" 1 &&
test_commit 2 &&
test_commit --date "$UNIX_EPOCH_ZERO" 3 &&
git commit-graph write --reachable &&
graph_read_expect 3 generation_data &&
test_commit --date "$FUTURE_DATE" 4 &&
test_commit 5 &&
test_commit --date "$UNIX_EPOCH_ZERO" 6 &&
git branch left &&
git reset --hard 3 &&
test_commit 7 &&
test_commit --date "$FUTURE_DATE" 8 &&
test_commit 9 &&
git branch right &&
git reset --hard 3 &&
test_merge M left right &&
git commit-graph write --reachable &&
graph_read_expect 10 "generation_data generation_data_overflow" &&
git commit-graph verify
'
graph_git_behavior 'generation data overflow chunk repo' repo left right
test_done

View File

@ -13,11 +13,11 @@ test_expect_success 'setup repo' '
infodir=".git/objects/info" &&
graphdir="$infodir/commit-graphs" &&
test_oid_cache <<-EOM
shallow sha1:1760
shallow sha256:2064
shallow sha1:2132
shallow sha256:2436
base sha1:1376
base sha256:1496
base sha1:1408
base sha256:1528
oid_version sha1:1
oid_version sha256:2
@ -31,9 +31,9 @@ graph_read_expect() {
NUM_BASE=$2
fi
cat >expect <<- EOF
header: 43475048 1 $(test_oid oid_version) 3 $NUM_BASE
header: 43475048 1 $(test_oid oid_version) 4 $NUM_BASE
num_commits: $1
chunks: oid_fanout oid_lookup commit_metadata
chunks: oid_fanout oid_lookup commit_metadata generation_data
EOF
test-tool read-graph >output &&
test_cmp expect output
@ -453,4 +453,185 @@ test_expect_success 'prevent regression for duplicate commits across layers' '
git -C dup commit-graph verify
'
NUM_FIRST_LAYER_COMMITS=64
NUM_SECOND_LAYER_COMMITS=16
NUM_THIRD_LAYER_COMMITS=7
NUM_FOURTH_LAYER_COMMITS=8
NUM_FIFTH_LAYER_COMMITS=16
SECOND_LAYER_SEQUENCE_START=$(($NUM_FIRST_LAYER_COMMITS + 1))
SECOND_LAYER_SEQUENCE_END=$(($SECOND_LAYER_SEQUENCE_START + $NUM_SECOND_LAYER_COMMITS - 1))
THIRD_LAYER_SEQUENCE_START=$(($SECOND_LAYER_SEQUENCE_END + 1))
THIRD_LAYER_SEQUENCE_END=$(($THIRD_LAYER_SEQUENCE_START + $NUM_THIRD_LAYER_COMMITS - 1))
FOURTH_LAYER_SEQUENCE_START=$(($THIRD_LAYER_SEQUENCE_END + 1))
FOURTH_LAYER_SEQUENCE_END=$(($FOURTH_LAYER_SEQUENCE_START + $NUM_FOURTH_LAYER_COMMITS - 1))
FIFTH_LAYER_SEQUENCE_START=$(($FOURTH_LAYER_SEQUENCE_END + 1))
FIFTH_LAYER_SEQUENCE_END=$(($FIFTH_LAYER_SEQUENCE_START + $NUM_FIFTH_LAYER_COMMITS - 1))
# Current split graph chain:
#
# 16 commits (No GDAT)
# ------------------------
# 64 commits (GDAT)
#
test_expect_success 'setup repo for mixed generation commit-graph-chain' '
graphdir=".git/objects/info/commit-graphs" &&
test_oid_cache <<-EOF &&
oid_version sha1:1
oid_version sha256:2
EOF
git init mixed &&
(
cd mixed &&
git config core.commitGraph true &&
git config gc.writeCommitGraph false &&
for i in $(test_seq $NUM_FIRST_LAYER_COMMITS)
do
test_commit $i &&
git branch commits/$i || return 1
done &&
git commit-graph write --reachable --split &&
graph_read_expect $NUM_FIRST_LAYER_COMMITS &&
test_line_count = 1 $graphdir/commit-graph-chain &&
for i in $(test_seq $SECOND_LAYER_SEQUENCE_START $SECOND_LAYER_SEQUENCE_END)
do
test_commit $i &&
git branch commits/$i || return 1
done &&
GIT_TEST_COMMIT_GRAPH_NO_GDAT=1 git commit-graph write --reachable --split=no-merge &&
test_line_count = 2 $graphdir/commit-graph-chain &&
test-tool read-graph >output &&
cat >expect <<-EOF &&
header: 43475048 1 $(test_oid oid_version) 4 1
num_commits: $NUM_SECOND_LAYER_COMMITS
chunks: oid_fanout oid_lookup commit_metadata
EOF
test_cmp expect output &&
git commit-graph verify &&
cat $graphdir/commit-graph-chain
)
'
# The new layer will be added without generation data chunk as it was not
# present on the layer underneath it.
#
# 7 commits (No GDAT)
# ------------------------
# 16 commits (No GDAT)
# ------------------------
# 64 commits (GDAT)
#
test_expect_success 'do not write generation data chunk if not present on existing tip' '
git clone mixed mixed-no-gdat &&
(
cd mixed-no-gdat &&
for i in $(test_seq $THIRD_LAYER_SEQUENCE_START $THIRD_LAYER_SEQUENCE_END)
do
test_commit $i &&
git branch commits/$i || return 1
done &&
git commit-graph write --reachable --split=no-merge &&
test_line_count = 3 $graphdir/commit-graph-chain &&
test-tool read-graph >output &&
cat >expect <<-EOF &&
header: 43475048 1 $(test_oid oid_version) 4 2
num_commits: $NUM_THIRD_LAYER_COMMITS
chunks: oid_fanout oid_lookup commit_metadata
EOF
test_cmp expect output &&
git commit-graph verify
)
'
# Number of commits in each layer of the split-commit graph before merge:
#
# 8 commits (No GDAT)
# ------------------------
# 7 commits (No GDAT)
# ------------------------
# 16 commits (No GDAT)
# ------------------------
# 64 commits (GDAT)
#
# The top two layers are merged and do not have generation data chunk as layer below them does
# not have generation data chunk.
#
# 15 commits (No GDAT)
# ------------------------
# 16 commits (No GDAT)
# ------------------------
# 64 commits (GDAT)
#
test_expect_success 'do not write generation data chunk if the topmost remaining layer does not have generation data chunk' '
git clone mixed-no-gdat mixed-merge-no-gdat &&
(
cd mixed-merge-no-gdat &&
for i in $(test_seq $FOURTH_LAYER_SEQUENCE_START $FOURTH_LAYER_SEQUENCE_END)
do
test_commit $i &&
git branch commits/$i || return 1
done &&
git commit-graph write --reachable --split --size-multiple 1 &&
test_line_count = 3 $graphdir/commit-graph-chain &&
test-tool read-graph >output &&
cat >expect <<-EOF &&
header: 43475048 1 $(test_oid oid_version) 4 2
num_commits: $(($NUM_THIRD_LAYER_COMMITS + $NUM_FOURTH_LAYER_COMMITS))
chunks: oid_fanout oid_lookup commit_metadata
EOF
test_cmp expect output &&
git commit-graph verify
)
'
# Number of commits in each layer of the split-commit graph before merge:
#
# 16 commits (No GDAT)
# ------------------------
# 15 commits (No GDAT)
# ------------------------
# 16 commits (No GDAT)
# ------------------------
# 64 commits (GDAT)
#
# The top three layers are merged and has generation data chunk as the topmost remaining layer
# has generation data chunk.
#
# 47 commits (GDAT)
# ------------------------
# 64 commits (GDAT)
#
test_expect_success 'write generation data chunk if topmost remaining layer has generation data chunk' '
git clone mixed-merge-no-gdat mixed-merge-gdat &&
(
cd mixed-merge-gdat &&
for i in $(test_seq $FIFTH_LAYER_SEQUENCE_START $FIFTH_LAYER_SEQUENCE_END)
do
test_commit $i &&
git branch commits/$i || return 1
done &&
git commit-graph write --reachable --split --size-multiple 1 &&
test_line_count = 2 $graphdir/commit-graph-chain &&
test-tool read-graph >output &&
cat >expect <<-EOF &&
header: 43475048 1 $(test_oid oid_version) 5 1
num_commits: $(($NUM_SECOND_LAYER_COMMITS + $NUM_THIRD_LAYER_COMMITS + $NUM_FOURTH_LAYER_COMMITS + $NUM_FIFTH_LAYER_COMMITS))
chunks: oid_fanout oid_lookup commit_metadata generation_data
EOF
test_cmp expect output
)
'
test_expect_success 'write generation data chunk when commit-graph chain is replaced' '
git clone mixed mixed-replace &&
(
cd mixed-replace &&
git commit-graph write --reachable --split=replace &&
test_path_is_file $graphdir/commit-graph-chain &&
test_line_count = 1 $graphdir/commit-graph-chain &&
verify_chain_files_exist $graphdir &&
graph_read_expect $(($NUM_FIRST_LAYER_COMMITS + $NUM_SECOND_LAYER_COMMITS)) &&
git commit-graph verify
)
'
test_done

View File

@ -18,6 +18,8 @@ GIT_COMMITTER_DATE="2006-12-12 23:28:00 +0100"
export GIT_COMMITTER_DATE
test_expect_success 'setup tests' '
GIT_TEST_COMMIT_GRAPH=0 &&
export GIT_TEST_COMMIT_GRAPH &&
echo 1 >a1 &&
git add a1 &&
GIT_AUTHOR_DATE="2006-12-12 23:00:00" git commit -m 1 a1 &&
@ -69,7 +71,7 @@ test_expect_success 'setup tests' '
'
test_expect_success 'combined merge conflicts' '
test_must_fail env GIT_TEST_COMMIT_GRAPH=0 git merge -m final G
test_must_fail git merge -m final G
'
test_expect_success 'result contains a conflict' '
@ -85,6 +87,7 @@ test_expect_success 'result contains a conflict' '
'
test_expect_success 'virtual trees were processed' '
# TODO: fragile test, relies on ambigious merge-base resolution
git ls-files --stage >out &&
cat >expect <<-EOF &&

View File

@ -55,10 +55,13 @@ test_expect_success 'setup' '
git show-ref -s commit-5-5 | git commit-graph write --stdin-commits &&
mv .git/objects/info/commit-graph commit-graph-half &&
chmod u+w commit-graph-half &&
GIT_TEST_COMMIT_GRAPH_NO_GDAT=1 git commit-graph write --reachable &&
mv .git/objects/info/commit-graph commit-graph-no-gdat &&
chmod u+w commit-graph-no-gdat &&
git config core.commitGraph true
'
run_three_modes () {
run_all_modes () {
test_when_finished rm -rf .git/objects/info/commit-graph &&
"$@" <input >actual &&
test_cmp expect actual &&
@ -67,11 +70,14 @@ run_three_modes () {
test_cmp expect actual &&
cp commit-graph-half .git/objects/info/commit-graph &&
"$@" <input >actual &&
test_cmp expect actual &&
cp commit-graph-no-gdat .git/objects/info/commit-graph &&
"$@" <input >actual &&
test_cmp expect actual
}
test_three_modes () {
run_three_modes test-tool reach "$@"
test_all_modes () {
run_all_modes test-tool reach "$@"
}
test_expect_success 'ref_newer:miss' '
@ -80,7 +86,7 @@ test_expect_success 'ref_newer:miss' '
B:commit-4-9
EOF
echo "ref_newer(A,B):0" >expect &&
test_three_modes ref_newer
test_all_modes ref_newer
'
test_expect_success 'ref_newer:hit' '
@ -89,7 +95,7 @@ test_expect_success 'ref_newer:hit' '
B:commit-2-3
EOF
echo "ref_newer(A,B):1" >expect &&
test_three_modes ref_newer
test_all_modes ref_newer
'
test_expect_success 'in_merge_bases:hit' '
@ -98,7 +104,7 @@ test_expect_success 'in_merge_bases:hit' '
B:commit-8-8
EOF
echo "in_merge_bases(A,B):1" >expect &&
test_three_modes in_merge_bases
test_all_modes in_merge_bases
'
test_expect_success 'in_merge_bases:miss' '
@ -107,7 +113,7 @@ test_expect_success 'in_merge_bases:miss' '
B:commit-5-9
EOF
echo "in_merge_bases(A,B):0" >expect &&
test_three_modes in_merge_bases
test_all_modes in_merge_bases
'
test_expect_success 'in_merge_bases_many:hit' '
@ -117,7 +123,7 @@ test_expect_success 'in_merge_bases_many:hit' '
X:commit-5-7
EOF
echo "in_merge_bases_many(A,X):1" >expect &&
test_three_modes in_merge_bases_many
test_all_modes in_merge_bases_many
'
test_expect_success 'in_merge_bases_many:miss' '
@ -127,7 +133,7 @@ test_expect_success 'in_merge_bases_many:miss' '
X:commit-8-6
EOF
echo "in_merge_bases_many(A,X):0" >expect &&
test_three_modes in_merge_bases_many
test_all_modes in_merge_bases_many
'
test_expect_success 'in_merge_bases_many:miss-heuristic' '
@ -137,7 +143,7 @@ test_expect_success 'in_merge_bases_many:miss-heuristic' '
X:commit-6-6
EOF
echo "in_merge_bases_many(A,X):0" >expect &&
test_three_modes in_merge_bases_many
test_all_modes in_merge_bases_many
'
test_expect_success 'is_descendant_of:hit' '
@ -148,7 +154,7 @@ test_expect_success 'is_descendant_of:hit' '
X:commit-1-1
EOF
echo "is_descendant_of(A,X):1" >expect &&
test_three_modes is_descendant_of
test_all_modes is_descendant_of
'
test_expect_success 'is_descendant_of:miss' '
@ -159,7 +165,7 @@ test_expect_success 'is_descendant_of:miss' '
X:commit-7-6
EOF
echo "is_descendant_of(A,X):0" >expect &&
test_three_modes is_descendant_of
test_all_modes is_descendant_of
'
test_expect_success 'get_merge_bases_many' '
@ -174,7 +180,7 @@ test_expect_success 'get_merge_bases_many' '
git rev-parse commit-5-6 \
commit-4-7 | sort
} >expect &&
test_three_modes get_merge_bases_many
test_all_modes get_merge_bases_many
'
test_expect_success 'reduce_heads' '
@ -196,7 +202,7 @@ test_expect_success 'reduce_heads' '
commit-2-8 \
commit-1-10 | sort
} >expect &&
test_three_modes reduce_heads
test_all_modes reduce_heads
'
test_expect_success 'can_all_from_reach:hit' '
@ -219,7 +225,7 @@ test_expect_success 'can_all_from_reach:hit' '
Y:commit-8-1
EOF
echo "can_all_from_reach(X,Y):1" >expect &&
test_three_modes can_all_from_reach
test_all_modes can_all_from_reach
'
test_expect_success 'can_all_from_reach:miss' '
@ -241,7 +247,7 @@ test_expect_success 'can_all_from_reach:miss' '
Y:commit-8-5
EOF
echo "can_all_from_reach(X,Y):0" >expect &&
test_three_modes can_all_from_reach
test_all_modes can_all_from_reach
'
test_expect_success 'can_all_from_reach_with_flag: tags case' '
@ -264,7 +270,7 @@ test_expect_success 'can_all_from_reach_with_flag: tags case' '
Y:commit-8-1
EOF
echo "can_all_from_reach_with_flag(X,_,_,0,0):1" >expect &&
test_three_modes can_all_from_reach_with_flag
test_all_modes can_all_from_reach_with_flag
'
test_expect_success 'commit_contains:hit' '
@ -280,8 +286,8 @@ test_expect_success 'commit_contains:hit' '
X:commit-9-3
EOF
echo "commit_contains(_,A,X,_):1" >expect &&
test_three_modes commit_contains &&
test_three_modes commit_contains --tag
test_all_modes commit_contains &&
test_all_modes commit_contains --tag
'
test_expect_success 'commit_contains:miss' '
@ -297,8 +303,8 @@ test_expect_success 'commit_contains:miss' '
X:commit-9-3
EOF
echo "commit_contains(_,A,X,_):0" >expect &&
test_three_modes commit_contains &&
test_three_modes commit_contains --tag
test_all_modes commit_contains &&
test_all_modes commit_contains --tag
'
test_expect_success 'rev-list: basic topo-order' '
@ -310,7 +316,7 @@ test_expect_success 'rev-list: basic topo-order' '
commit-6-2 commit-5-2 commit-4-2 commit-3-2 commit-2-2 commit-1-2 \
commit-6-1 commit-5-1 commit-4-1 commit-3-1 commit-2-1 commit-1-1 \
>expect &&
run_three_modes git rev-list --topo-order commit-6-6
run_all_modes git rev-list --topo-order commit-6-6
'
test_expect_success 'rev-list: first-parent topo-order' '
@ -322,7 +328,7 @@ test_expect_success 'rev-list: first-parent topo-order' '
commit-6-2 \
commit-6-1 commit-5-1 commit-4-1 commit-3-1 commit-2-1 commit-1-1 \
>expect &&
run_three_modes git rev-list --first-parent --topo-order commit-6-6
run_all_modes git rev-list --first-parent --topo-order commit-6-6
'
test_expect_success 'rev-list: range topo-order' '
@ -334,7 +340,7 @@ test_expect_success 'rev-list: range topo-order' '
commit-6-2 commit-5-2 commit-4-2 \
commit-6-1 commit-5-1 commit-4-1 \
>expect &&
run_three_modes git rev-list --topo-order commit-3-3..commit-6-6
run_all_modes git rev-list --topo-order commit-3-3..commit-6-6
'
test_expect_success 'rev-list: range topo-order' '
@ -346,7 +352,7 @@ test_expect_success 'rev-list: range topo-order' '
commit-6-2 commit-5-2 commit-4-2 \
commit-6-1 commit-5-1 commit-4-1 \
>expect &&
run_three_modes git rev-list --topo-order commit-3-8..commit-6-6
run_all_modes git rev-list --topo-order commit-3-8..commit-6-6
'
test_expect_success 'rev-list: first-parent range topo-order' '
@ -358,7 +364,7 @@ test_expect_success 'rev-list: first-parent range topo-order' '
commit-6-2 \
commit-6-1 commit-5-1 commit-4-1 \
>expect &&
run_three_modes git rev-list --first-parent --topo-order commit-3-8..commit-6-6
run_all_modes git rev-list --first-parent --topo-order commit-3-8..commit-6-6
'
test_expect_success 'rev-list: ancestry-path topo-order' '
@ -368,7 +374,7 @@ test_expect_success 'rev-list: ancestry-path topo-order' '
commit-6-4 commit-5-4 commit-4-4 commit-3-4 \
commit-6-3 commit-5-3 commit-4-3 \
>expect &&
run_three_modes git rev-list --topo-order --ancestry-path commit-3-3..commit-6-6
run_all_modes git rev-list --topo-order --ancestry-path commit-3-3..commit-6-6
'
test_expect_success 'rev-list: symmetric difference topo-order' '
@ -382,7 +388,7 @@ test_expect_success 'rev-list: symmetric difference topo-order' '
commit-3-8 commit-2-8 commit-1-8 \
commit-3-7 commit-2-7 commit-1-7 \
>expect &&
run_three_modes git rev-list --topo-order commit-3-8...commit-6-6
run_all_modes git rev-list --topo-order commit-3-8...commit-6-6
'
test_expect_success 'get_reachable_subset:all' '
@ -402,7 +408,7 @@ test_expect_success 'get_reachable_subset:all' '
commit-1-7 \
commit-5-6 | sort
) >expect &&
test_three_modes get_reachable_subset
test_all_modes get_reachable_subset
'
test_expect_success 'get_reachable_subset:some' '
@ -420,7 +426,7 @@ test_expect_success 'get_reachable_subset:some' '
git rev-parse commit-3-3 \
commit-1-7 | sort
) >expect &&
test_three_modes get_reachable_subset
test_all_modes get_reachable_subset
'
test_expect_success 'get_reachable_subset:none' '
@ -434,7 +440,7 @@ test_expect_success 'get_reachable_subset:none' '
Y:commit-2-8
EOF
echo "get_reachable_subset(X,Y)" >expect &&
test_three_modes get_reachable_subset
test_all_modes get_reachable_subset
'
test_done

View File

@ -218,6 +218,12 @@ test_commit () {
--signoff)
signoff="$1"
;;
--date)
notick=yes
GIT_COMMITTER_DATE="$2"
GIT_AUTHOR_DATE="$2"
shift
;;
-C)
indir="$2"
shift

View File

@ -493,7 +493,7 @@ static int got_oid(struct upload_pack_data *data,
static int ok_to_give_up(struct upload_pack_data *data)
{
uint32_t min_generation = GENERATION_NUMBER_ZERO;
timestamp_t min_generation = GENERATION_NUMBER_ZERO;
if (!data->have_obj.nr)
return 0;