1
0
Fork 0
mirror of https://github.com/git/git.git synced 2024-03-28 16:49:58 +01:00

Merge branch 'ds/sparse-index' into next

Both in-core and on-disk index has been updated to optionally omit
individual entries and replace them with the tree object that
corresponds to the directory that contains them when the "cone"
mode of sparse checkout is in use.

* ds/sparse-index: (21 commits)
  p2000: add sparse-index repos
  sparse-index: loose integration with cache_tree_verify()
  cache-tree: integrate with sparse directory entries
  sparse-checkout: disable sparse-index
  sparse-checkout: toggle sparse index from builtin
  sparse-index: add index.sparse config option
  sparse-index: check index conversion happens
  unpack-trees: allow sparse directories
  submodule: sparse-index should not collapse links
  sparse-index: convert from full to sparse
  sparse-index: add 'sdir' index extension
  sparse-checkout: hold pattern list in index
  unpack-trees: ensure full index
  test-tool: don't force full index
  test-read-cache: print cache entries with --table
  t1092: compare sparse-checkout to sparse-index
  sparse-index: implement ensure_full_index()
  sparse-index: add guard to ensure full index
  t1092: clean up script quoting
  t/perf: add performance test for sparse operations
  ...
This commit is contained in:
Junio C Hamano 2021-04-07 16:48:30 -07:00
commit f1290a7929
20 changed files with 988 additions and 40 deletions

View File

@ -14,6 +14,11 @@ index.recordOffsetTable::
Defaults to 'true' if index.threads has been explicitly enabled,
'false' otherwise.
index.sparse::
When enabled, write the index using sparse-directory entries. This
has no effect unless `core.sparseCheckout` and
`core.sparseCheckoutCone` are both enabled. Defaults to 'false'.
index.threads::
Specifies the number of threads to spawn when loading the index.
This is meant to reduce index load time on multiprocessor machines.

View File

@ -45,6 +45,20 @@ To avoid interfering with other worktrees, it first enables the
When `--cone` is provided, the `core.sparseCheckoutCone` setting is
also set, allowing for better performance with a limited set of
patterns (see 'CONE PATTERN SET' below).
+
Use the `--[no-]sparse-index` option to toggle the use of the sparse
index format. This reduces the size of the index to be more closely
aligned with your sparse-checkout definition. This can have significant
performance advantages for commands such as `git status` or `git add`.
This feature is still experimental. Some commands might be slower with
a sparse index until they are properly integrated with the feature.
+
**WARNING:** Using a sparse index requires modifying the index in a way
that is not completely understood by external tools. If you have trouble
with this compatibility, then run `git sparse-checkout init --no-sparse-index`
to rewrite your index to not be sparse. Older versions of Git will not
understand the sparse directory entries index extension and may fail to
interact with your repository until it is disabled.
'set'::
Write a set of patterns to the sparse-checkout file, as given as

View File

@ -44,6 +44,13 @@ Git index format
localization, no special casing of directory separator '/'). Entries
with the same name are sorted by their stage field.
An index entry typically represents a file. However, if sparse-checkout
is enabled in cone mode (`core.sparseCheckoutCone` is enabled) and the
`extensions.sparseIndex` extension is enabled, then the index may
contain entries for directories outside of the sparse-checkout definition.
These entries have mode `040000`, include the `SKIP_WORKTREE` bit, and
the path ends in a directory separator.
32-bit ctime seconds, the last time a file's metadata changed
this is stat(2) data
@ -385,3 +392,15 @@ The remaining data of each directory block is grouped by type:
in this block of entries.
- 32-bit count of cache entries in this block
== Sparse Directory Entries
When using sparse-checkout in cone mode, some entire directories within
the index can be summarized by pointing to a tree object instead of the
entire expanded list of paths within that tree. An index containing such
entries is a "sparse index". Index format versions 4 and less were not
implemented with such entries in mind. Thus, for these versions, an
index containing sparse directory entries will include this extension
with signature { 's', 'd', 'i', 'r' }. Like the split-index extension,
tools should avoid interacting with a sparse index unless they understand
this extension.

View File

@ -0,0 +1,175 @@
Git Sparse-Index Design Document
================================
The sparse-checkout feature allows users to focus a working directory on
a subset of the files at HEAD. The cone mode patterns, enabled by
`core.sparseCheckoutCone`, allow for very fast pattern matching to
discover which files at HEAD belong in the sparse-checkout cone.
Three important scale dimensions for a Git working directory are:
* `HEAD`: How many files are present at `HEAD`?
* Populated: How many files are within the sparse-checkout cone.
* Modified: How many files has the user modified in the working directory?
We will use big-O notation -- O(X) -- to denote how expensive certain
operations are in terms of these dimensions.
These dimensions are ordered by their magnitude: users (typically) modify
fewer files than are populated, and we can only populate files at `HEAD`.
Problems occur if there is an extreme imbalance in these dimensions. For
example, if `HEAD` contains millions of paths but the populated set has
only tens of thousands, then commands like `git status` and `git add` can
be dominated by operations that require O(`HEAD`) operations instead of
O(Populated). Primarily, the cost is in parsing and rewriting the index,
which is filled primarily with files at `HEAD` that are marked with the
`SKIP_WORKTREE` bit.
The sparse-index intends to take these commands that read and modify the
index from O(`HEAD`) to O(Populated). To do this, we need to modify the
index format in a significant way: add "sparse directory" entries.
With cone mode patterns, it is possible to detect when an entire
directory will have its contents outside of the sparse-checkout definition.
Instead of listing all of the files it contains as individual entries, a
sparse-index contains an entry with the directory name, referencing the
object ID of the tree at `HEAD` and marked with the `SKIP_WORKTREE` bit.
If we need to discover the details for paths within that directory, we
can parse trees to find that list.
At time of writing, sparse-directory entries violate expectations about the
index format and its in-memory data structure. There are many consumers in
the codebase that expect to iterate through all of the index entries and
see only files. In fact, these loops expect to see a reference to every
staged file. One way to handle this is to parse trees to replace a
sparse-directory entry with all of the files within that tree as the index
is loaded. However, parsing trees is slower than parsing the index format,
so that is a slower operation than if we left the index alone. The plan is
to make all of these integrations "sparse aware" so this expansion through
tree parsing is unnecessary and they use fewer resources than when using a
full index.
The implementation plan below follows four phases to slowly integrate with
the sparse-index. The intention is to incrementally update Git commands to
interact safely with the sparse-index without significant slowdowns. This
may not always be possible, but the hope is that the primary commands that
users need in their daily work are dramatically improved.
Phase I: Format and initial speedups
------------------------------------
During this phase, Git learns to enable the sparse-index and safely parse
one. Protections are put in place so that every consumer of the in-memory
data structure can operate with its current assumption of every file at
`HEAD`.
At first, every index parse will call a helper method,
`ensure_full_index()`, which scans the index for sparse-directory entries
(pointing to trees) and replaces them with the full list of paths (with
blob contents) by parsing tree objects. This will be slower in all cases.
The only noticeable change in behavior will be that the serialized index
file contains sparse-directory entries.
To start, we use a new required index extension, `sdir`, to allow
inserting sparse-directory entries into indexes with file format
versions 2, 3, and 4. This prevents Git versions that do not understand
the sparse-index from operating on one, while allowing tools that do not
understand the sparse-index to operate on repositories as long as they do
not interact with the index. A new format, index v5, will be introduced
that includes sparse-directory entries by default. It might also
introduce other features that have been considered for improving the
index, as well.
Next, consumers of the index will be guarded against operating on a
sparse-index by inserting calls to `ensure_full_index()` or
`expand_index_to_path()`. After these guards are in place, we can begin
leaving sparse-directory entries in the in-memory index structure.
Even after inserting these guards, we will keep expanding sparse-indexes
for most Git commands using the `command_requires_full_index` repository
setting. This setting will be on by default and disabled one builtin at a
time until we have sufficient confidence that all of the index operations
are properly guarded.
To complete this phase, the commands `git status` and `git add` will be
integrated with the sparse-index so that they operate with O(Populated)
performance. They will be carefully tested for operations within and
outside the sparse-checkout definition.
Phase II: Careful integrations
------------------------------
This phase focuses on ensuring that all index extensions and APIs work
well with a sparse-index. This requires significant increases to our test
coverage, especially for operations that interact with the working
directory outside of the sparse-checkout definition. Some of these
behaviors may not be the desirable ones, such as some tests already
marked for failure in `t1092-sparse-checkout-compatibility.sh`.
The index extensions that may require special integrations are:
* FS Monitor
* Untracked cache
While integrating with these features, we should look for patterns that
might lead to better APIs for interacting with the index. Coalescing
common usage patterns into an API call can reduce the number of places
where sparse-directories need to be handled carefully.
Phase III: Important command speedups
-------------------------------------
At this point, the patterns for testing and implementing sparse-directory
logic should be relatively stable. This phase focuses on updating some of
the most common builtins that use the index to operate as O(Populated).
Here is a potential list of commands that could be valuable to integrate
at this point:
* `git commit`
* `git checkout`
* `git merge`
* `git rebase`
Hopefully, commands such as `git merge` and `git rebase` can benefit
instead from merge algorithms that do not use the index as a data
structure, such as the merge-ORT strategy. As these topics mature, we
may enable the ORT strategy by default for repositories using the
sparse-index feature.
Along with `git status` and `git add`, these commands cover the majority
of users' interactions with the working directory. In addition, we can
integrate with these commands:
* `git grep`
* `git rm`
These have been proposed as some whose behavior could change when in a
repo with a sparse-checkout definition. It would be good to include this
behavior automatically when using a sparse-index. Some clarity is needed
to make the behavior switch clear to the user.
This phase is the first where parallel work might be possible without too
much conflicts between topics.
Phase IV: The long tail
-----------------------
This last phase is less a "phase" and more "the new normal" after all of
the previous work.
To start, the `command_requires_full_index` option could be removed in
favor of expanding only when hitting an API guard.
There are many Git commands that could use special attention to operate as
O(Populated), while some might be so rare that it is acceptable to leave
them with additional overhead when a sparse-index is present.
Here are some commands that might be useful to update:
* `git sparse-checkout set`
* `git am`
* `git clean`
* `git stash`

View File

@ -994,6 +994,7 @@ LIB_OBJS += setup.o
LIB_OBJS += shallow.o
LIB_OBJS += sideband.o
LIB_OBJS += sigchain.o
LIB_OBJS += sparse-index.o
LIB_OBJS += split-index.o
LIB_OBJS += stable-qsort.o
LIB_OBJS += strbuf.o

View File

@ -14,6 +14,7 @@
#include "unpack-trees.h"
#include "wt-status.h"
#include "quote.h"
#include "sparse-index.h"
static const char *empty_base = "";
@ -110,6 +111,8 @@ static int update_working_directory(struct pattern_list *pl)
if (is_index_unborn(r->index))
return UPDATE_SPARSITY_SUCCESS;
r->index->sparse_checkout_patterns = pl;
memset(&o, 0, sizeof(o));
o.verbose_update = isatty(2);
o.update = 1;
@ -138,6 +141,7 @@ static int update_working_directory(struct pattern_list *pl)
else
rollback_lock_file(&lock_file);
r->index->sparse_checkout_patterns = NULL;
return result;
}
@ -276,16 +280,20 @@ static int set_config(enum sparse_checkout_mode mode)
"core.sparseCheckoutCone",
mode == MODE_CONE_PATTERNS ? "true" : NULL);
if (mode == MODE_NO_PATTERNS)
set_sparse_index_config(the_repository, 0);
return 0;
}
static char const * const builtin_sparse_checkout_init_usage[] = {
N_("git sparse-checkout init [--cone]"),
N_("git sparse-checkout init [--cone] [--[no-]sparse-index]"),
NULL
};
static struct sparse_checkout_init_opts {
int cone_mode;
int sparse_index;
} init_opts;
static int sparse_checkout_init(int argc, const char **argv)
@ -300,11 +308,15 @@ static int sparse_checkout_init(int argc, const char **argv)
static struct option builtin_sparse_checkout_init_options[] = {
OPT_BOOL(0, "cone", &init_opts.cone_mode,
N_("initialize the sparse-checkout in cone mode")),
OPT_BOOL(0, "sparse-index", &init_opts.sparse_index,
N_("toggle the use of a sparse index")),
OPT_END(),
};
repo_read_index(the_repository);
init_opts.sparse_index = -1;
argc = parse_options(argc, argv, NULL,
builtin_sparse_checkout_init_options,
builtin_sparse_checkout_init_usage, 0);
@ -323,10 +335,20 @@ static int sparse_checkout_init(int argc, const char **argv)
sparse_filename = get_sparse_checkout_filename();
res = add_patterns_from_file_to_list(sparse_filename, "", 0, &pl, NULL, 0);
if (init_opts.sparse_index >= 0) {
if (set_sparse_index_config(the_repository, init_opts.sparse_index) < 0)
die(_("failed to modify sparse-index config"));
/* force an index rewrite */
repo_read_index(the_repository);
the_repository->index->updated_workdir = 1;
}
core_apply_sparse_checkout = 1;
/* If we already have a sparse-checkout file, use it. */
if (res >= 0) {
free(sparse_filename);
core_apply_sparse_checkout = 1;
return update_working_directory(NULL);
}
@ -348,6 +370,7 @@ static int sparse_checkout_init(int argc, const char **argv)
add_pattern(strbuf_detach(&pattern, NULL), empty_base, 0, &pl, 0);
strbuf_addstr(&pattern, "!/*/");
add_pattern(strbuf_detach(&pattern, NULL), empty_base, 0, &pl, 0);
pl.use_cone_patterns = init_opts.cone_mode;
return write_patterns_and_update(&pl);
}
@ -517,19 +540,18 @@ static int modify_pattern_list(int argc, const char **argv, enum modify_type m)
{
int result;
int changed_config = 0;
struct pattern_list pl;
memset(&pl, 0, sizeof(pl));
struct pattern_list *pl = xcalloc(1, sizeof(*pl));
switch (m) {
case ADD:
if (core_sparse_checkout_cone)
add_patterns_cone_mode(argc, argv, &pl);
add_patterns_cone_mode(argc, argv, pl);
else
add_patterns_literal(argc, argv, &pl);
add_patterns_literal(argc, argv, pl);
break;
case REPLACE:
add_patterns_from_input(&pl, argc, argv);
add_patterns_from_input(pl, argc, argv);
break;
}
@ -539,12 +561,13 @@ static int modify_pattern_list(int argc, const char **argv, enum modify_type m)
changed_config = 1;
}
result = write_patterns_and_update(&pl);
result = write_patterns_and_update(pl);
if (result && changed_config)
set_config(MODE_NO_PATTERNS);
clear_pattern_list(&pl);
clear_pattern_list(pl);
free(pl);
return result;
}
@ -614,6 +637,9 @@ static int sparse_checkout_disable(int argc, const char **argv)
strbuf_addstr(&match_all, "/*");
add_pattern(strbuf_detach(&match_all, NULL), empty_base, 0, &pl, 0);
prepare_repo_settings(the_repository);
the_repository->settings.sparse_index = 0;
if (update_working_directory(&pl))
die(_("error while refreshing working directory"));

View File

@ -6,6 +6,7 @@
#include "object-store.h"
#include "replace-object.h"
#include "promisor-remote.h"
#include "sparse-index.h"
#ifndef DEBUG_CACHE_TREE
#define DEBUG_CACHE_TREE 0
@ -255,6 +256,24 @@ static int update_one(struct cache_tree *it,
*skip_count = 0;
/*
* If the first entry of this region is a sparse directory
* entry corresponding exactly to 'base', then this cache_tree
* struct is a "leaf" in the data structure, pointing to the
* tree OID specified in the entry.
*/
if (entries > 0) {
const struct cache_entry *ce = cache[0];
if (S_ISSPARSEDIR(ce->ce_mode) &&
ce->ce_namelen == baselen &&
!strncmp(ce->name, base, baselen)) {
it->entry_count = 1;
oidcpy(&it->oid, &ce->oid);
return 1;
}
}
if (0 <= it->entry_count && has_object_file(&it->oid))
return it->entry_count;
@ -442,6 +461,8 @@ int cache_tree_update(struct index_state *istate, int flags)
if (i)
return i;
ensure_full_index(istate);
if (!istate->cache_tree)
istate->cache_tree = cache_tree();
@ -787,6 +808,19 @@ int cache_tree_matches_traversal(struct cache_tree *root,
return 0;
}
static void verify_one_sparse(struct repository *r,
struct index_state *istate,
struct cache_tree *it,
struct strbuf *path,
int pos)
{
struct cache_entry *ce = istate->cache[pos];
if (!S_ISSPARSEDIR(ce->ce_mode))
BUG("directory '%s' is present in index, but not sparse",
path->buf);
}
static void verify_one(struct repository *r,
struct index_state *istate,
struct cache_tree *it,
@ -809,6 +843,12 @@ static void verify_one(struct repository *r,
if (path->len) {
pos = index_name_pos(istate, path->buf, path->len);
if (pos >= 0) {
verify_one_sparse(r, istate, it, path, pos);
return;
}
pos = -pos - 1;
} else {
pos = 0;

18
cache.h
View File

@ -204,6 +204,8 @@ struct cache_entry {
#error "CE_EXTENDED_FLAGS out of range"
#endif
#define S_ISSPARSEDIR(m) ((m) == S_IFDIR)
/* Forward structure decls */
struct pathspec;
struct child_process;
@ -249,6 +251,8 @@ static inline unsigned int create_ce_mode(unsigned int mode)
{
if (S_ISLNK(mode))
return S_IFLNK;
if (S_ISSPARSEDIR(mode))
return S_IFDIR;
if (S_ISDIR(mode) || S_ISGITLINK(mode))
return S_IFGITLINK;
return S_IFREG | ce_permissions(mode);
@ -305,6 +309,7 @@ static inline unsigned int canon_mode(unsigned int mode)
struct split_index;
struct untracked_cache;
struct progress;
struct pattern_list;
struct index_state {
struct cache_entry **cache;
@ -319,7 +324,14 @@ struct index_state {
drop_cache_tree : 1,
updated_workdir : 1,
updated_skipworktree : 1,
fsmonitor_has_run_once : 1;
fsmonitor_has_run_once : 1,
/*
* sparse_index == 1 when sparse-directory
* entries exist. Requires sparse-checkout
* in cone mode.
*/
sparse_index : 1;
struct hashmap name_hash;
struct hashmap dir_hash;
struct object_id oid;
@ -329,6 +341,7 @@ struct index_state {
struct mem_pool *ce_mem_pool;
struct progress *progress;
struct repository *repo;
struct pattern_list *sparse_checkout_patterns;
};
/* Name hashing */
@ -722,6 +735,8 @@ int read_index_from(struct index_state *, const char *path,
const char *gitdir);
int is_index_unborn(struct index_state *);
void ensure_full_index(struct index_state *istate);
/* For use with `write_locked_index()`. */
#define COMMIT_LOCK (1 << 0)
#define SKIP_IF_UNCHANGED (1 << 1)
@ -1044,6 +1059,7 @@ struct repository_format {
int worktree_config;
int is_bare;
int hash_algo;
int sparse_index;
char *work_tree;
struct string_list unknown_extensions;
struct string_list v1_only_extensions;

View File

@ -25,6 +25,7 @@
#include "fsmonitor.h"
#include "thread-utils.h"
#include "progress.h"
#include "sparse-index.h"
/* Mask for the name length in ce_flags in the on-disk index */
@ -47,6 +48,7 @@
#define CACHE_EXT_FSMONITOR 0x46534D4E /* "FSMN" */
#define CACHE_EXT_ENDOFINDEXENTRIES 0x454F4945 /* "EOIE" */
#define CACHE_EXT_INDEXENTRYOFFSETTABLE 0x49454F54 /* "IEOT" */
#define CACHE_EXT_SPARSE_DIRECTORIES 0x73646972 /* "sdir" */
/* changes that can be kept in $GIT_DIR/index (basically all extensions) */
#define EXTMASK (RESOLVE_UNDO_CHANGED | CACHE_TREE_CHANGED | \
@ -101,6 +103,9 @@ static const char *alternate_index_output;
static void set_index_entry(struct index_state *istate, int nr, struct cache_entry *ce)
{
if (S_ISSPARSEDIR(ce->ce_mode))
istate->sparse_index = 1;
istate->cache[nr] = ce;
add_name_hash(istate, ce);
}
@ -999,8 +1004,14 @@ int verify_path(const char *path, unsigned mode)
c = *path++;
if ((c == '.' && !verify_dotfile(path, mode)) ||
is_dir_sep(c) || c == '\0')
is_dir_sep(c))
return 0;
/*
* allow terminating directory separators for
* sparse directory entries.
*/
if (c == '\0')
return S_ISDIR(mode);
} else if (c == '\\' && protect_ntfs) {
if (is_ntfs_dotgit(path))
return 0;
@ -1760,6 +1771,10 @@ static int read_index_extension(struct index_state *istate,
case CACHE_EXT_INDEXENTRYOFFSETTABLE:
/* already handled in do_read_index() */
break;
case CACHE_EXT_SPARSE_DIRECTORIES:
/* no content, only an indicator */
istate->sparse_index = 1;
break;
default:
if (*ext < 'A' || 'Z' < *ext)
return error(_("index uses %.4s extension, which we do not understand"),
@ -2273,6 +2288,12 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist)
trace2_data_intmax("index", the_repository, "read/cache_nr",
istate->cache_nr);
if (!istate->repo)
istate->repo = the_repository;
prepare_repo_settings(istate->repo);
if (istate->repo->settings.command_requires_full_index)
ensure_full_index(istate);
return istate->cache_nr;
unmap:
@ -3012,6 +3033,10 @@ static int do_write_index(struct index_state *istate, struct tempfile *tempfile,
if (err)
return -1;
}
if (istate->sparse_index) {
if (write_index_ext_header(&c, &eoie_c, newfd, CACHE_EXT_SPARSE_DIRECTORIES, 0) < 0)
return -1;
}
/*
* CACHE_EXT_ENDOFINDEXENTRIES must be written as the last entry before the SHA1
@ -3071,6 +3096,14 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
unsigned flags)
{
int ret;
int was_full = !istate->sparse_index;
ret = convert_to_sparse(istate);
if (ret) {
warning(_("failed to convert to a sparse-index"));
return ret;
}
/*
* TODO trace2: replace "the_repository" with the actual repo instance
@ -3082,6 +3115,9 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
trace2_region_leave_printf("index", "do_write_index", the_repository,
"%s", get_lock_file_path(lock));
if (was_full)
ensure_full_index(istate);
if (ret)
return ret;
if (flags & COMMIT_LOCK)
@ -3172,9 +3208,10 @@ static int write_shared_index(struct index_state *istate,
struct tempfile **temp)
{
struct split_index *si = istate->split_index;
int ret;
int ret, was_full = !istate->sparse_index;
move_cache_to_base_index(istate);
convert_to_sparse(istate);
trace2_region_enter_printf("index", "shared/do_write_index",
the_repository, "%s", get_tempfile_path(*temp));
@ -3182,6 +3219,9 @@ static int write_shared_index(struct index_state *istate,
trace2_region_leave_printf("index", "shared/do_write_index",
the_repository, "%s", get_tempfile_path(*temp));
if (was_full)
ensure_full_index(istate);
if (ret)
return ret;
ret = adjust_shared_perm(get_tempfile_path(*temp));

View File

@ -77,4 +77,19 @@ void prepare_repo_settings(struct repository *r)
UPDATE_DEFAULT_BOOL(r->settings.core_untracked_cache, UNTRACKED_CACHE_KEEP);
UPDATE_DEFAULT_BOOL(r->settings.fetch_negotiation_algorithm, FETCH_NEGOTIATION_DEFAULT);
/*
* This setting guards all index reads to require a full index
* over a sparse index. After suitable guards are placed in the
* codebase around uses of the index, this setting will be
* removed.
*/
r->settings.command_requires_full_index = 1;
/*
* Initialize this as off.
*/
r->settings.sparse_index = 0;
if (!repo_config_get_bool(r, "index.sparse", &value) && value)
r->settings.sparse_index = 1;
}

View File

@ -10,6 +10,7 @@
#include "object.h"
#include "lockfile.h"
#include "submodule-config.h"
#include "sparse-index.h"
/* The main repository */
static struct repository the_repo;
@ -261,6 +262,8 @@ void repo_clear(struct repository *repo)
int repo_read_index(struct repository *repo)
{
int res;
if (!repo->index)
CALLOC_ARRAY(repo->index, 1);
@ -270,7 +273,13 @@ int repo_read_index(struct repository *repo)
else if (repo->index->repo != repo)
BUG("repo's index should point back at itself");
return read_index_from(repo->index, repo->index_file, repo->gitdir);
res = read_index_from(repo->index, repo->index_file, repo->gitdir);
prepare_repo_settings(repo);
if (repo->settings.command_requires_full_index)
ensure_full_index(repo->index);
return res;
}
int repo_hold_locked_index(struct repository *repo,

View File

@ -41,6 +41,9 @@ struct repo_settings {
enum fetch_negotiation_setting fetch_negotiation_algorithm;
int core_multi_pack_index;
unsigned command_requires_full_index:1,
sparse_index:1;
};
struct repository {

285
sparse-index.c Normal file
View File

@ -0,0 +1,285 @@
#include "cache.h"
#include "repository.h"
#include "sparse-index.h"
#include "tree.h"
#include "pathspec.h"
#include "trace2.h"
#include "cache-tree.h"
#include "config.h"
#include "dir.h"
#include "fsmonitor.h"
static struct cache_entry *construct_sparse_dir_entry(
struct index_state *istate,
const char *sparse_dir,
struct cache_tree *tree)
{
struct cache_entry *de;
de = make_cache_entry(istate, S_IFDIR, &tree->oid, sparse_dir, 0, 0);
de->ce_flags |= CE_SKIP_WORKTREE;
return de;
}
/*
* Returns the number of entries "inserted" into the index.
*/
static int convert_to_sparse_rec(struct index_state *istate,
int num_converted,
int start, int end,
const char *ct_path, size_t ct_pathlen,
struct cache_tree *ct)
{
int i, can_convert = 1;
int start_converted = num_converted;
enum pattern_match_result match;
int dtype;
struct strbuf child_path = STRBUF_INIT;
struct pattern_list *pl = istate->sparse_checkout_patterns;
/*
* Is the current path outside of the sparse cone?
* Then check if the region can be replaced by a sparse
* directory entry (everything is sparse and merged).
*/
match = path_matches_pattern_list(ct_path, ct_pathlen,
NULL, &dtype, pl, istate);
if (match != NOT_MATCHED)
can_convert = 0;
for (i = start; can_convert && i < end; i++) {
struct cache_entry *ce = istate->cache[i];
if (ce_stage(ce) ||
S_ISGITLINK(ce->ce_mode) ||
!(ce->ce_flags & CE_SKIP_WORKTREE))
can_convert = 0;
}
if (can_convert) {
struct cache_entry *se;
se = construct_sparse_dir_entry(istate, ct_path, ct);
istate->cache[num_converted++] = se;
return 1;
}
for (i = start; i < end; ) {
int count, span, pos = -1;
const char *base, *slash;
struct cache_entry *ce = istate->cache[i];
/*
* Detect if this is a normal entry outside of any subtree
* entry.
*/
base = ce->name + ct_pathlen;
slash = strchr(base, '/');
if (slash)
pos = cache_tree_subtree_pos(ct, base, slash - base);
if (pos < 0) {
istate->cache[num_converted++] = ce;
i++;
continue;
}
strbuf_setlen(&child_path, 0);
strbuf_add(&child_path, ce->name, slash - ce->name + 1);
span = ct->down[pos]->cache_tree->entry_count;
count = convert_to_sparse_rec(istate,
num_converted, i, i + span,
child_path.buf, child_path.len,
ct->down[pos]->cache_tree);
num_converted += count;
i += span;
}
strbuf_release(&child_path);
return num_converted - start_converted;
}
static int set_index_sparse_config(struct repository *repo, int enable)
{
int res;
char *config_path = repo_git_path(repo, "config.worktree");
res = git_config_set_in_file_gently(config_path,
"index.sparse",
enable ? "true" : NULL);
free(config_path);
prepare_repo_settings(repo);
repo->settings.sparse_index = 1;
return res;
}
int set_sparse_index_config(struct repository *repo, int enable)
{
int res = set_index_sparse_config(repo, enable);
prepare_repo_settings(repo);
repo->settings.sparse_index = enable;
return res;
}
int convert_to_sparse(struct index_state *istate)
{
int test_env;
if (istate->split_index || istate->sparse_index ||
!core_apply_sparse_checkout || !core_sparse_checkout_cone)
return 0;
if (!istate->repo)
istate->repo = the_repository;
/*
* The GIT_TEST_SPARSE_INDEX environment variable triggers the
* index.sparse config variable to be on.
*/
test_env = git_env_bool("GIT_TEST_SPARSE_INDEX", -1);
if (test_env >= 0)
set_sparse_index_config(istate->repo, test_env);
/*
* Only convert to sparse if index.sparse is set.
*/
prepare_repo_settings(istate->repo);
if (!istate->repo->settings.sparse_index)
return 0;
if (!istate->sparse_checkout_patterns) {
istate->sparse_checkout_patterns = xcalloc(1, sizeof(struct pattern_list));
if (get_sparse_checkout_patterns(istate->sparse_checkout_patterns) < 0)
return 0;
}
if (!istate->sparse_checkout_patterns->use_cone_patterns) {
warning(_("attempting to use sparse-index without cone mode"));
return -1;
}
if (cache_tree_update(istate, 0)) {
warning(_("unable to update cache-tree, staying full"));
return -1;
}
remove_fsmonitor(istate);
trace2_region_enter("index", "convert_to_sparse", istate->repo);
istate->cache_nr = convert_to_sparse_rec(istate,
0, 0, istate->cache_nr,
"", 0, istate->cache_tree);
/* Clear and recompute the cache-tree */
cache_tree_free(&istate->cache_tree);
cache_tree_update(istate, 0);
istate->sparse_index = 1;
trace2_region_leave("index", "convert_to_sparse", istate->repo);
return 0;
}
static void set_index_entry(struct index_state *istate, int nr, struct cache_entry *ce)
{
ALLOC_GROW(istate->cache, nr + 1, istate->cache_alloc);
istate->cache[nr] = ce;
add_name_hash(istate, ce);
}
static int add_path_to_index(const struct object_id *oid,
struct strbuf *base, const char *path,
unsigned int mode, void *context)
{
struct index_state *istate = (struct index_state *)context;
struct cache_entry *ce;
size_t len = base->len;
if (S_ISDIR(mode))
return READ_TREE_RECURSIVE;
strbuf_addstr(base, path);
ce = make_cache_entry(istate, mode, oid, base->buf, 0, 0);
ce->ce_flags |= CE_SKIP_WORKTREE;
set_index_entry(istate, istate->cache_nr++, ce);
strbuf_setlen(base, len);
return 0;
}
void ensure_full_index(struct index_state *istate)
{
int i;
struct index_state *full;
struct strbuf base = STRBUF_INIT;
if (!istate || !istate->sparse_index)
return;
if (!istate->repo)
istate->repo = the_repository;
trace2_region_enter("index", "ensure_full_index", istate->repo);
/* initialize basics of new index */
full = xcalloc(1, sizeof(struct index_state));
memcpy(full, istate, sizeof(struct index_state));
/* then change the necessary things */
full->sparse_index = 0;
full->cache_alloc = (3 * istate->cache_alloc) / 2;
full->cache_nr = 0;
ALLOC_ARRAY(full->cache, full->cache_alloc);
for (i = 0; i < istate->cache_nr; i++) {
struct cache_entry *ce = istate->cache[i];
struct tree *tree;
struct pathspec ps;
if (!S_ISSPARSEDIR(ce->ce_mode)) {
set_index_entry(full, full->cache_nr++, ce);
continue;
}
if (!(ce->ce_flags & CE_SKIP_WORKTREE))
warning(_("index entry is a directory, but not sparse (%08x)"),
ce->ce_flags);
/* recursively walk into cd->name */
tree = lookup_tree(istate->repo, &ce->oid);
memset(&ps, 0, sizeof(ps));
ps.recursive = 1;
ps.has_wildcard = 1;
ps.max_depth = -1;
strbuf_setlen(&base, 0);
strbuf_add(&base, ce->name, strlen(ce->name));
read_tree_at(istate->repo, tree, &base, &ps,
add_path_to_index, full);
/* free directory entries. full entries are re-used */
discard_cache_entry(ce);
}
/* Copy back into original index. */
memcpy(&istate->name_hash, &full->name_hash, sizeof(full->name_hash));
istate->sparse_index = 0;
free(istate->cache);
istate->cache = full->cache;
istate->cache_nr = full->cache_nr;
istate->cache_alloc = full->cache_alloc;
strbuf_release(&base);
free(full);
/* Clear and recompute the cache-tree */
cache_tree_free(&istate->cache_tree);
cache_tree_update(istate, 0);
trace2_region_leave("index", "ensure_full_index", istate->repo);
}

11
sparse-index.h Normal file
View File

@ -0,0 +1,11 @@
#ifndef SPARSE_INDEX_H__
#define SPARSE_INDEX_H__
struct index_state;
void ensure_full_index(struct index_state *istate);
int convert_to_sparse(struct index_state *istate);
struct repository;
int set_sparse_index_config(struct repository *repo, int enable);
#endif

View File

@ -436,6 +436,9 @@ and "sha256".
GIT_TEST_WRITE_REV_INDEX=<boolean>, when true enables the
'pack.writeReverseIndex' setting.
GIT_TEST_SPARSE_INDEX=<boolean>, when true enables index writes to use the
sparse-index format by default.
Naming Tests
------------

View File

@ -1,36 +1,82 @@
#include "test-tool.h"
#include "cache.h"
#include "config.h"
#include "blob.h"
#include "commit.h"
#include "tree.h"
#include "sparse-index.h"
static void print_cache_entry(struct cache_entry *ce)
{
const char *type;
printf("%06o ", ce->ce_mode & 0177777);
if (S_ISSPARSEDIR(ce->ce_mode))
type = tree_type;
else if (S_ISGITLINK(ce->ce_mode))
type = commit_type;
else
type = blob_type;
printf("%s %s\t%s\n",
type,
oid_to_hex(&ce->oid),
ce->name);
}
static void print_cache(struct index_state *istate)
{
int i;
for (i = 0; i < istate->cache_nr; i++)
print_cache_entry(istate->cache[i]);
}
int cmd__read_cache(int argc, const char **argv)
{
struct repository *r = the_repository;
int i, cnt = 1;
const char *name = NULL;
int table = 0, expand = 0;
if (argc > 1 && skip_prefix(argv[1], "--print-and-refresh=", &name)) {
argc--;
argv++;
initialize_the_repository();
prepare_repo_settings(r);
r->settings.command_requires_full_index = 0;
for (++argv, --argc; *argv && starts_with(*argv, "--"); ++argv, --argc) {
if (skip_prefix(*argv, "--print-and-refresh=", &name))
continue;
if (!strcmp(*argv, "--table"))
table = 1;
else if (!strcmp(*argv, "--expand"))
expand = 1;
}
if (argc == 2)
cnt = strtol(argv[1], NULL, 0);
if (argc == 1)
cnt = strtol(argv[0], NULL, 0);
setup_git_directory();
git_config(git_default_config, NULL);
for (i = 0; i < cnt; i++) {
read_cache();
repo_read_index(r);
if (expand)
ensure_full_index(r->index);
if (name) {
int pos;
refresh_index(&the_index, REFRESH_QUIET,
refresh_index(r->index, REFRESH_QUIET,
NULL, NULL, NULL);
pos = index_name_pos(&the_index, name, strlen(name));
pos = index_name_pos(r->index, name, strlen(name));
if (pos < 0)
die("%s not in index", name);
printf("%s is%s up to date\n", name,
ce_uptodate(the_index.cache[pos]) ? "" : " not");
ce_uptodate(r->index->cache[pos]) ? "" : " not");
write_file(name, "%d\n", i);
}
discard_cache();
if (table)
print_cache(r->index);
discard_index(r->index);
}
return 0;
}

101
t/perf/p2000-sparse-operations.sh Executable file
View File

@ -0,0 +1,101 @@
#!/bin/sh
test_description="test performance of Git operations using the index"
. ./perf-lib.sh
test_perf_default_repo
SPARSE_CONE=f2/f4/f1
test_expect_success 'setup repo and indexes' '
git reset --hard HEAD &&
# Remove submodules from the example repo, because our
# duplication of the entire repo creates an unlikely data shape.
if git config --file .gitmodules --get-regexp "submodule.*.path" >modules
then
git rm $(awk "{print \$2}" modules) &&
git commit -m "remove submodules" || return 1
fi &&
echo bogus >a &&
cp a b &&
git add a b &&
git commit -m "level 0" &&
BLOB=$(git rev-parse HEAD:a) &&
OLD_COMMIT=$(git rev-parse HEAD) &&
OLD_TREE=$(git rev-parse HEAD^{tree}) &&
for i in $(test_seq 1 4)
do
cat >in <<-EOF &&
100755 blob $BLOB a
040000 tree $OLD_TREE f1
040000 tree $OLD_TREE f2
040000 tree $OLD_TREE f3
040000 tree $OLD_TREE f4
EOF
NEW_TREE=$(git mktree <in) &&
NEW_COMMIT=$(git commit-tree $NEW_TREE -p $OLD_COMMIT -m "level $i") &&
OLD_TREE=$NEW_TREE &&
OLD_COMMIT=$NEW_COMMIT || return 1
done &&
git sparse-checkout init --cone &&
git branch -f wide $OLD_COMMIT &&
git -c core.sparseCheckoutCone=true clone --branch=wide --sparse . full-index-v3 &&
(
cd full-index-v3 &&
git sparse-checkout init --cone &&
git sparse-checkout set $SPARSE_CONE &&
git config index.version 3 &&
git update-index --index-version=3
) &&
git -c core.sparseCheckoutCone=true clone --branch=wide --sparse . full-index-v4 &&
(
cd full-index-v4 &&
git sparse-checkout init --cone &&
git sparse-checkout set $SPARSE_CONE &&
git config index.version 4 &&
git update-index --index-version=4
) &&
git -c core.sparseCheckoutCone=true clone --branch=wide --sparse . sparse-index-v3 &&
(
cd sparse-index-v3 &&
git sparse-checkout init --cone --sparse-index &&
git sparse-checkout set $SPARSE_CONE &&
git config index.version 3 &&
git update-index --index-version=3
) &&
git -c core.sparseCheckoutCone=true clone --branch=wide --sparse . sparse-index-v4 &&
(
cd sparse-index-v4 &&
git sparse-checkout init --cone --sparse-index &&
git sparse-checkout set $SPARSE_CONE &&
git config index.version 4 &&
git update-index --index-version=4
)
'
test_perf_on_all () {
command="$@"
for repo in full-index-v3 full-index-v4 \
sparse-index-v3 sparse-index-v4
do
test_perf "$command ($repo)" "
(
cd $repo &&
echo >>$SPARSE_CONE/a &&
$command
)
"
done
}
test_perf_on_all git status
test_perf_on_all git add -A
test_perf_on_all git add .
test_perf_on_all git commit -a -m A
test_done

View File

@ -205,6 +205,19 @@ test_expect_success 'sparse-checkout disable' '
check_files repo a deep folder1 folder2
'
test_expect_success 'sparse-index enabled and disabled' '
git -C repo sparse-checkout init --cone --sparse-index &&
test_cmp_config -C repo true index.sparse &&
test-tool -C repo read-cache --table >cache &&
grep " tree " cache &&
git -C repo sparse-checkout disable &&
test-tool -C repo read-cache --table >cache &&
! grep " tree " cache &&
git -C repo config --list >config &&
! grep index.sparse config
'
test_expect_success 'cone mode: init and set' '
git -C repo sparse-checkout init --cone &&
git -C repo config --list >config &&

View File

@ -2,11 +2,15 @@
test_description='compare full workdir to sparse workdir'
GIT_TEST_SPLIT_INDEX=0
GIT_TEST_SPARSE_INDEX=
. ./test-lib.sh
test_expect_success 'setup' '
git init initial-repo &&
(
GIT_TEST_SPARSE_INDEX=0 &&
cd initial-repo &&
echo a >a &&
echo "after deep" >e &&
@ -87,39 +91,102 @@ init_repos () {
cp -r initial-repo sparse-checkout &&
git -C sparse-checkout reset --hard &&
git -C sparse-checkout sparse-checkout init --cone &&
cp -r initial-repo sparse-index &&
git -C sparse-index reset --hard &&
# initialize sparse-checkout definitions
git -C sparse-checkout sparse-checkout set deep
git -C sparse-checkout sparse-checkout init --cone &&
git -C sparse-checkout sparse-checkout set deep &&
git -C sparse-index sparse-checkout init --cone --sparse-index &&
test_cmp_config -C sparse-index true index.sparse &&
git -C sparse-index sparse-checkout set deep
}
run_on_sparse () {
(
cd sparse-checkout &&
$* >../sparse-checkout-out 2>../sparse-checkout-err
"$@" >../sparse-checkout-out 2>../sparse-checkout-err
) &&
(
cd sparse-index &&
"$@" >../sparse-index-out 2>../sparse-index-err
)
}
run_on_all () {
(
cd full-checkout &&
$* >../full-checkout-out 2>../full-checkout-err
"$@" >../full-checkout-out 2>../full-checkout-err
) &&
run_on_sparse $*
run_on_sparse "$@"
}
test_all_match () {
run_on_all $* &&
run_on_all "$@" &&
test_cmp full-checkout-out sparse-checkout-out &&
test_cmp full-checkout-err sparse-checkout-err
test_cmp full-checkout-out sparse-index-out &&
test_cmp full-checkout-err sparse-checkout-err &&
test_cmp full-checkout-err sparse-index-err
}
test_sparse_match () {
run_on_sparse "$@" &&
test_cmp sparse-checkout-out sparse-index-out &&
test_cmp sparse-checkout-err sparse-index-err
}
test_expect_success 'sparse-index contents' '
init_repos &&
test-tool -C sparse-index read-cache --table >cache &&
for dir in folder1 folder2 x
do
TREE=$(git -C sparse-index rev-parse HEAD:$dir) &&
grep "040000 tree $TREE $dir/" cache \
|| return 1
done &&
git -C sparse-index sparse-checkout set folder1 &&
test-tool -C sparse-index read-cache --table >cache &&
for dir in deep folder2 x
do
TREE=$(git -C sparse-index rev-parse HEAD:$dir) &&
grep "040000 tree $TREE $dir/" cache \
|| return 1
done &&
git -C sparse-index sparse-checkout set deep/deeper1 &&
test-tool -C sparse-index read-cache --table >cache &&
for dir in deep/deeper2 folder1 folder2 x
do
TREE=$(git -C sparse-index rev-parse HEAD:$dir) &&
grep "040000 tree $TREE $dir/" cache \
|| return 1
done &&
# Disabling the sparse-index removes tree entries with full ones
git -C sparse-index sparse-checkout init --no-sparse-index &&
test-tool -C sparse-index read-cache --table >cache &&
! grep "040000 tree" cache &&
test_sparse_match test-tool read-cache --table
'
test_expect_success 'expanded in-memory index matches full index' '
init_repos &&
test_sparse_match test-tool read-cache --expand --table
'
test_expect_success 'status with options' '
init_repos &&
test_sparse_match ls &&
test_all_match git status --porcelain=v2 &&
test_all_match git status --porcelain=v2 -z -u &&
test_all_match git status --porcelain=v2 -uno &&
run_on_all "touch README.md" &&
run_on_all touch README.md &&
test_all_match git status --porcelain=v2 &&
test_all_match git status --porcelain=v2 -z -u &&
test_all_match git status --porcelain=v2 -uno &&
@ -135,7 +202,7 @@ test_expect_success 'add, commit, checkout' '
write_script edit-contents <<-\EOF &&
echo text >>$1
EOF
run_on_all "../edit-contents README.md" &&
run_on_all ../edit-contents README.md &&
test_all_match git add README.md &&
test_all_match git status --porcelain=v2 &&
@ -144,7 +211,7 @@ test_expect_success 'add, commit, checkout' '
test_all_match git checkout HEAD~1 &&
test_all_match git checkout - &&
run_on_all "../edit-contents README.md" &&
run_on_all ../edit-contents README.md &&
test_all_match git add -A &&
test_all_match git status --porcelain=v2 &&
@ -153,7 +220,7 @@ test_expect_success 'add, commit, checkout' '
test_all_match git checkout HEAD~1 &&
test_all_match git checkout - &&
run_on_all "../edit-contents deep/newfile" &&
run_on_all ../edit-contents deep/newfile &&
test_all_match git status --porcelain=v2 -uno &&
test_all_match git status --porcelain=v2 &&
@ -186,7 +253,7 @@ test_expect_success 'diff --staged' '
write_script edit-contents <<-\EOF &&
echo text >>README.md
EOF
run_on_all "../edit-contents" &&
run_on_all ../edit-contents &&
test_all_match git diff &&
test_all_match git diff --staged &&
@ -252,6 +319,17 @@ test_expect_failure 'checkout and reset (mixed)' '
test_all_match git reset update-folder2
'
# Ensure that sparse-index behaves identically to
# sparse-checkout with a full index.
test_expect_success 'checkout and reset (mixed) [sparse]' '
init_repos &&
test_sparse_match git checkout -b reset-test update-deep &&
test_sparse_match git reset deepest &&
test_sparse_match git reset update-folder1 &&
test_sparse_match git reset update-folder2
'
test_expect_success 'merge' '
init_repos &&
@ -280,7 +358,7 @@ test_expect_success 'clean' '
echo bogus >>.gitignore &&
run_on_all cp ../.gitignore . &&
test_all_match git add .gitignore &&
test_all_match git commit -m ignore-bogus-files &&
test_all_match git commit -m "ignore bogus files" &&
run_on_sparse mkdir folder1 &&
run_on_all touch folder1/bogus &&
@ -288,14 +366,51 @@ test_expect_success 'clean' '
test_all_match git status --porcelain=v2 &&
test_all_match git clean -f &&
test_all_match git status --porcelain=v2 &&
test_sparse_match ls &&
test_sparse_match ls folder1 &&
test_all_match git clean -xf &&
test_all_match git status --porcelain=v2 &&
test_sparse_match ls &&
test_sparse_match ls folder1 &&
test_all_match git clean -xdf &&
test_all_match git status --porcelain=v2 &&
test_sparse_match ls &&
test_sparse_match ls folder1 &&
test_path_is_dir sparse-checkout/folder1
test_sparse_match test_path_is_dir folder1
'
test_expect_success 'submodule handling' '
init_repos &&
test_all_match mkdir modules &&
test_all_match touch modules/a &&
test_all_match git add modules &&
test_all_match git commit -m "add modules directory" &&
run_on_all git submodule add "$(pwd)/initial-repo" modules/sub &&
test_all_match git commit -m "add submodule" &&
# having a submodule prevents "modules" from collapse
test-tool -C sparse-index read-cache --table >cache &&
grep "100644 blob .* modules/a" cache &&
grep "160000 commit $(git -C initial-repo rev-parse HEAD) modules/sub" cache
'
test_expect_success 'sparse-index is expanded and converted back' '
init_repos &&
GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \
git -C sparse-index -c core.fsmonitor="" reset --hard &&
test_region index convert_to_sparse trace2.txt &&
test_region index ensure_full_index trace2.txt &&
rm trace2.txt &&
GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \
git -C sparse-index -c core.fsmonitor="" status -uno &&
test_region index ensure_full_index trace2.txt
'
test_done

View File

@ -750,9 +750,13 @@ static int index_pos_by_traverse_info(struct name_entry *names,
strbuf_make_traverse_path(&name, info, names->path, names->pathlen);
strbuf_addch(&name, '/');
pos = index_name_pos(o->src_index, name.buf, name.len);
if (pos >= 0)
BUG("This is a directory and should not exist in index");
pos = -pos - 1;
if (pos >= 0) {
if (!o->src_index->sparse_index ||
!(o->src_index->cache[pos]->ce_flags & CE_SKIP_WORKTREE))
BUG("This is a directory and should not exist in index");
} else {
pos = -pos - 1;
}
if (pos >= o->src_index->cache_nr ||
!starts_with(o->src_index->cache[pos]->name, name.buf) ||
(pos > 0 && starts_with(o->src_index->cache[pos-1]->name, name.buf)))
@ -1571,6 +1575,7 @@ static int verify_absent(const struct cache_entry *,
*/
int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options *o)
{
struct repository *repo = the_repository;
int i, ret;
static struct cache_entry *dfc;
struct pattern_list pl;
@ -1582,6 +1587,12 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options
trace_performance_enter();
trace2_region_enter("unpack_trees", "unpack_trees", the_repository);
prepare_repo_settings(repo);
if (repo->settings.command_requires_full_index) {
ensure_full_index(o->src_index);
ensure_full_index(o->dst_index);
}
if (!core_apply_sparse_checkout || !o->update)
o->skip_sparse_checkout = 1;
if (!o->skip_sparse_checkout && !o->pl) {