mirror/ git
1
0
Fork 0
mirror of https://github.com/git/git.git synced 2024-05-30 13:46:08 +02:00
Code

Merge branch 'en/sparse-cone-becomes-default' 

Deprecate non-cone mode of the sparse-checkout feature.

* en/sparse-cone-becomes-default:
  Documentation: some sparsity wording clarifications
  git-sparse-checkout.txt: mark non-cone mode as deprecated
  git-sparse-checkout.txt: flesh out pattern set sections a bit
  git-sparse-checkout.txt: add a new EXAMPLES section
  git-sparse-checkout.txt: shuffle some sections and mark as internal
  git-sparse-checkout.txt: update docs for deprecation of 'init'
  git-sparse-checkout.txt: wording updates for the cone mode default
  sparse-checkout: make --cone the default
  tests: stop assuming --no-cone is the default mode for sparse-checkout
This commit is contained in:
Junio C Hamano 2022-06-03 14:30:33 -07:00
parent 2668e3608e 5d4b293340
commit 377d347eb3
10 changed files with 267 additions and 100 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
Documentation/config/core.txt
View File

@ -698,8 +698,10 @@ core.sparseCheckout::
core.sparseCheckoutCone::
Enables the "cone mode" of the sparse checkout feature. When the
sparse-checkout file contains a limited set of patterns, then this
mode provides significant performance advantages. See
sparse-checkout file contains a limited set of patterns, this
mode provides significant performance advantages. The "non
cone mode" can be requested to allow specifying a more flexible
patterns by setting this variable to 'false'. See
linkgit:git-sparse-checkout[1] for more information.
core.abbrev::

11
Documentation/git-read-tree.txt
View File

@ -375,10 +375,13 @@ have finished your work-in-progress), attempt the merge again.
SPARSE CHECKOUT
---------------
Note: The `update-index` and `read-tree` primitives for supporting the
skip-worktree bit predated the introduction of
linkgit:git-sparse-checkout[1]. Users are encouraged to use
`sparse-checkout` in preference to these low-level primitives.
Note: The skip-worktree capabilities in linkgit:git-update-index[1]
and `read-tree` predated the introduction of
linkgit:git-sparse-checkout[1]. Users are encouraged to use the
`sparse-checkout` command in preference to these plumbing commands for
sparse-checkout/skip-worktree related needs. However, the information
below might be useful to users trying to understand the pattern style
used in non-cone mode of the `sparse-checkout` command.
"Sparse checkout" allows populating the working directory sparsely.
It uses the skip-worktree bit (see linkgit:git-update-index[1]) to

315
Documentation/git-sparse-checkout.txt
View File

@ -15,15 +15,15 @@ SYNOPSIS
DESCRIPTION
-----------
This command is used to create sparse checkouts, which means that it
changes the working tree from having all tracked files present, to only
have a subset of them. It can also switch which subset of files are
present, or undo and go back to having all tracked files present in the
working copy.
This command is used to create sparse checkouts, which change the
working tree from having all tracked files present to only having a
subset of those files. It can also switch which subset of files are
present, or undo and go back to having all tracked files present in
the working copy.
The subset of files is chosen by providing a list of directories in
cone mode (which is recommended), or by providing a list of patterns
in non-cone mode.
cone mode (the default), or by providing a list of patterns in
non-cone mode.
When in a sparse-checkout, other Git commands behave a bit differently.
For example, switching branches will not update paths outside the
@ -44,9 +44,9 @@ COMMANDS
Enable the necessary sparse-checkout config settings
(`core.sparseCheckout`, `core.sparseCheckoutCone`, and
`index.sparse`) if they are not already set to the desired values,
and write a set of patterns to the sparse-checkout file from the
list of arguments following the 'set' subcommand. Update the
working directory to match the new patterns.
populate the sparse-checkout file from the list of arguments
following the 'set' subcommand, and update the working directory to
match.
+
To ensure that adjusting the sparse-checkout settings within a worktree
does not alter the sparse-checkout settings in other worktrees, the 'set'
@ -60,22 +60,20 @@ When the `--stdin` option is provided, the directories or patterns are
read from standard in as a newline-delimited list instead of from the
arguments.
+
When `--cone` is passed or `core.sparseCheckoutCone` is enabled, the
input list is considered a list of directories. This allows for
better performance with a limited set of patterns (see 'CONE PATTERN
SET' below). The input format matches the output of `git ls-tree
--name-only`. This includes interpreting pathnames that begin with a
double quote (") as C-style quoted strings. Note that the set command
will write patterns to the sparse-checkout file to include all files
contained in those directories (recursively) as well as files that are
siblings of ancestor directories. This may become the default in the
future; --no-cone can be passed to request non-cone mode.
By default, the input list is considered a list of directories, matching
the output of `git ls-tree -d --name-only`. This includes interpreting
pathnames that begin with a double quote (") as C-style quoted strings.
Note that all files under the specified directories (at any depth) will
be included in the sparse checkout, as well as files that are siblings
of either the given directory or any of its ancestors (see 'CONE PATTERN
SET' below for more details). In the past, this was not the default,
and `--cone` needed to be specified or `core.sparseCheckoutCone` needed
to be enabled.
+
When `--no-cone` is passed or `core.sparseCheckoutCone` is not enabled,
the input list is considered a list of patterns. This mode is harder
to use and less performant, and is thus not recommended. See the
"Sparse Checkout" section of linkgit:git-read-tree[1] and the "Pattern
Set" sections below for more details.
When `--no-cone` is passed, the input list is considered a list of
patterns. This mode has a number of drawbacks, including not working
with some options like `--sparse-index`. As explained in the
"Non-cone Problems" section below, we do not recommend using it.
+
Use the `--[no-]sparse-index` option to use a sparse index (the
default is to not use it). A sparse index reduces the size of the
@ -137,8 +135,45 @@ paths to pass to a subsequent 'set' or 'add' command. However,
the disable command, so the easy restore of calling a plain `init`
decreased in utility.
SPARSE CHECKOUT
---------------
EXAMPLES
--------
`git sparse-checkout set MY/DIR1 SUB/DIR2`::
Change to a sparse checkout with all files (at any depth) under
MY/DIR1/ and SUB/DIR2/ present in the working copy (plus all
files immediately under MY/ and SUB/ and the toplevel
directory). If already in a sparse checkout, change which files
are present in the working copy to this new selection. Note
that this command will also delete all ignored files in any
directory that no longer has either tracked or
non-ignored-untracked files present.
`git sparse-checkout disable`::
Repopulate the working directory with all files, disabling sparse
checkouts.
`git sparse-checkout add SOME/DIR/ECTORY`::
Add all files under SOME/DIR/ECTORY/ (at any depth) to the
sparse checkout, as well as all files immediately under
SOME/DIR/ and immediately under SOME/. Must already be in a
sparse checkout before using this command.
`git sparse-checkout reapply`::
It is possible for commands to update the working tree in a
way that does not respect the selected sparsity directories.
This can come from tools external to Git writing files, or
even affect Git commands because of either special cases (such
as hitting conflicts when merging/rebasing), or because some
commands didn't fully support sparse checkouts (e.g. the old
`recursive` merge backend had only limited support). This
command reapplies the existing sparse directory specifications
to make the working directory match.
INTERNALS -- SPARSE CHECKOUT
----------------------------
"Sparse checkout" allows populating the working directory sparsely. It
uses the skip-worktree bit (see linkgit:git-update-index[1]) to tell Git
@ -155,31 +190,133 @@ directory, it updates the skip-worktree bits in the index based
on this file. The files matching the patterns in the file will
appear in the working directory, and the rest will not.
To enable the sparse-checkout feature, run `git sparse-checkout set` to
set the patterns you want to use.
INTERNALS -- NON-CONE PROBLEMS
------------------------------
To repopulate the working directory with all files, use the
`git sparse-checkout disable` command.
The `$GIT_DIR/info/sparse-checkout` file populated by the `set` and
`add` subcommands is defined to be a bunch of patterns (one per line)
using the same syntax as `.gitignore` files. In cone mode, these
patterns are restricted to matching directories (and users only ever
need supply or see directory names), while in non-cone mode any
gitignore-style pattern is permitted. Using the full gitignore-style
patterns in non-cone mode has a number of shortcomings:
* Fundamentally, it makes various worktree-updating processes (pull,
merge, rebase, switch, reset, checkout, etc.) require O(N*M) pattern
matches, where N is the number of patterns and M is the number of
paths in the index. This scales poorly.
* Avoiding the scaling issue has to be done via limiting the number
of patterns via specifying leading directory name or glob.
* Passing globs on the command line is error-prone as users may
forget to quote the glob, causing the shell to expand it into all
matching files and pass them all individually along to
sparse-checkout set/add. While this could also be a problem with
e.g. "git grep -- *.c", mistakes with grep/log/status appear in
the immediate output. With sparse-checkout, the mistake gets
recorded at the time the sparse-checkout command is run and might
not be problematic until the user later switches branches or rebases
or merges, thus putting a delay between the user's error and when
they have a chance to catch/notice it.
* Related to the previous item, sparse-checkout has an 'add'
subcommand but no 'remove' subcommand. Even if a 'remove'
subcommand were added, undoing an accidental unquoted glob runs
the risk of "removing too much", as it may remove entries that had
been included before the accidental add.
* Non-cone mode uses gitignore-style patterns to select what to
*include* (with the exception of negated patterns), while
.gitignore files use gitignore-style patterns to select what to
*exclude* (with the exception of negated patterns). The
documentation on gitignore-style patterns usually does not talk in
terms of matching or non-matching, but on what the user wants to
"exclude". This can cause confusion for users trying to learn how
to specify sparse-checkout patterns to get their desired behavior.
* Every other git subcommand that wants to provide "special path
pattern matching" of some sort uses pathspecs, but non-cone mode
for sparse-checkout uses gitignore patterns, which feels
inconsistent.
* It has edge cases where the "right" behavior is unclear. Two examples:
First, two users are in a subdirectory, and the first runs
git sparse-checkout set '/toplevel-dir/*.c'
while the second runs
git sparse-checkout set relative-dir
Should those arguments be transliterated into
current/subdirectory/toplevel-dir/*.c
and
current/subdirectory/relative-dir
before inserting into the sparse-checkout file? The user who typed
the first command is probably aware that arguments to set/add are
supposed to be patterns in non-cone mode, and probably would not be
happy with such a transliteration. However, many gitignore-style
patterns are just paths, which might be what the user who typed the
second command was thinking, and they'd be upset if their argument
wasn't transliterated.
Second, what should bash-completion complete on for set/add commands
for non-cone users? If it suggests paths, is it exacerbating the
problem above? Also, if it suggests paths, what if the user has a
file or directory that begins with either a '!' or '#' or has a '*',
'\', '?', '[', or ']' in its name? And if it suggests paths, will
it complete "/pro" to "/proc" (in the root filesytem) rather than to
"/progress.txt" in the current directory? (Note that users are
likely to want to start paths with a leading '/' in non-cone mode,
for the same reason that .gitignore files often have one.)
Completing on files or directories might give nasty surprises in
all these cases.
* The excessive flexibility made other extensions essentially
impractical. `--sparse-index` is likely impossible in non-cone
mode; even if it is somehow feasible, it would have been far more
work to implement and may have been too slow in practice. Some
ideas for adding coupling between partial clones and sparse
checkouts are only practical with a more restricted set of paths
as well.
For all these reasons, non-cone mode is deprecated. Please switch to
using cone mode.
FULL PATTERN SET
----------------
INTERNALS -- CONE MODE HANDLING
-------------------------------
By default, the sparse-checkout file uses the same syntax as `.gitignore`
files.
The "cone mode", which is the default, lets you specify only what
directories to include. For any directory specified, all paths below
that directory will be included, and any paths immediately under
leading directories (including the toplevel directory) will also be
included. Thus, if you specified the directory
Documentation/technical/
then your sparse checkout would contain:
While `$GIT_DIR/info/sparse-checkout` is usually used to specify what
files are included, you can also specify what files are _not_ included,
using negative patterns. For example, to remove the file `unwanted`:
* all files in the toplevel-directory
* all files immediately under Documentation/
* all files at any depth under Documentation/technical/
----------------
/*
!unwanted
----------------
Also, in cone mode, even if no directories are specified, then the
files in the toplevel directory will be included.
When changing the sparse-checkout patterns in cone mode, Git will inspect each
tracked directory that is not within the sparse-checkout cone to see if it
contains any untracked files. If all of those files are ignored due to the
`.gitignore` patterns, then the directory will be deleted. If any of the
untracked files within that directory is not ignored, then no deletions will
occur within that directory and a warning message will appear. If these files
are important, then reset your sparse-checkout definition so they are included,
use `git add` and `git commit` to store them, then remove any remaining files
manually to ensure Git can behave optimally.
See also the "Internals -- Cone Pattern Set" section to learn how the
directories are transformed under the hood into a subset of the
Full Pattern Set of sparse-checkout.
CONE PATTERN SET
----------------
INTERNALS -- FULL PATTERN SET
-----------------------------
The full pattern set allows for arbitrary pattern matches and complicated
inclusion/exclusion rules. These can result in O(N*M) pattern matches when
@ -187,32 +324,62 @@ updating the index, where N is the number of patterns and M is the number
of paths in the index. To combat this performance issue, a more restricted
pattern set is allowed when `core.sparseCheckoutCone` is enabled.
The accepted patterns in the cone pattern set are:
The sparse-checkout file uses the same syntax as `.gitignore` files;
see linkgit:gitignore[5] for details. Here, though, the patterns are
usually being used to select which files to include rather than which
files to exclude. (However, it can get a bit confusing since
gitignore-style patterns have negations defined by patterns which
begin with a '!', so you can also select files to _not_ include.)
For example, to select everything, and then to remove the file
`unwanted` (so that every file will appear in your working tree except
the file named `unwanted`):
git sparse-checkout set --no-cone '/*' '!unwanted'
These patterns are just placed into the
`$GIT_DIR/info/sparse-checkout` as-is, so the contents of that file
at this point would be
----------------
/*
!unwanted
----------------
See also the "Sparse Checkout" section of linkgit:git-read-tree[1] to
learn more about the gitignore-style patterns used in sparse
checkouts.
INTERNALS -- CONE PATTERN SET
-----------------------------
In cone mode, only directories are accepted, but they are translated into
the same gitignore-style patterns used in the full pattern set. We refer
to the particular patterns used in those mode as being of one of two types:
1. *Recursive:* All paths inside a directory are included.
2. *Parent:* All files immediately inside a directory are included.
In addition to the above two patterns, we also expect that all files in the
root directory are included. If a recursive pattern is added, then all
leading directories are added as parent patterns.
By default, when running `git sparse-checkout init`, the root directory is
added as a parent pattern. At this point, the sparse-checkout file contains
the following patterns:
Since cone mode always includes files at the toplevel, when running
`git sparse-checkout set` with no directories specified, the toplevel
directory is added as a parent pattern. At this point, the
sparse-checkout file contains the following patterns:
----------------
/*
!/*/
----------------
This says "include everything in root, but nothing two levels below root."
This says "include everything immediately under the toplevel
directory, but nothing at any level below that."
When in cone mode, the `git sparse-checkout set` subcommand takes a list of
directories instead of a list of sparse-checkout patterns. In this mode,
the command `git sparse-checkout set A/B/C` sets the directory `A/B/C` as
a recursive pattern, the directories `A` and `A/B` are added as parent
patterns. The resulting sparse-checkout file is now
When in cone mode, the `git sparse-checkout set` subcommand takes a
list of directories. The command `git sparse-checkout set A/B/C` sets
the directory `A/B/C` as a recursive pattern, the directories `A` and
`A/B` are added as parent patterns. The resulting sparse-checkout file
is now
----------------
/*
@ -227,14 +394,18 @@ patterns. The resulting sparse-checkout file is now
Here, order matters, so the negative patterns are overridden by the positive
patterns that appear lower in the file.
If `core.sparseCheckoutCone=true`, then Git will parse the sparse-checkout file
expecting patterns of these types. Git will warn if the patterns do not match.
If the patterns do match the expected format, then Git will use faster hash-
based algorithms to compute inclusion in the sparse-checkout.
Unless `core.sparseCheckoutCone` is explicitly set to `false`, Git will
parse the sparse-checkout file expecting patterns of these types. Git will
warn if the patterns do not match. If the patterns do match the expected
format, then Git will use faster hash-based algorithms to compute inclusion
in the sparse-checkout. If they do not match, git will behave as though
`core.sparseCheckoutCone` was false, regardless of its setting.
In the cone mode case, the `git sparse-checkout list` subcommand will list the
directories that define the recursive patterns. For the example sparse-checkout
file above, the output is as follows:
In the cone mode case, despite the fact that full patterns are written
to the $GIT_DIR/info/sparse-checkout file, the `git sparse-checkout
list` subcommand will list the directories that define the recursive
patterns. For the example sparse-checkout file above, the output is as
follows:
--------------------------
$ git sparse-checkout list
@ -246,19 +417,9 @@ case-insensitive check. This corrects for case mismatched filenames in the
'git sparse-checkout set' command to reflect the expected cone in the working
directory.
When changing the sparse-checkout patterns in cone mode, Git will inspect each
tracked directory that is not within the sparse-checkout cone to see if it
contains any untracked files. If all of those files are ignored due to the
`.gitignore` patterns, then the directory will be deleted. If any of the
untracked files within that directory is not ignored, then no deletions will
occur within that directory and a warning message will appear. If these files
are important, then reset your sparse-checkout definition so they are included,
use `git add` and `git commit` to store them, then remove any remaining files
manually to ensure Git can behave optimally.
SUBMODULES
----------
INTERNALS -- SUBMODULES
-----------------------
If your repository contains one or more submodules, then submodules
are populated based on interactions with the `git submodule` command.

2
builtin/sparse-checkout.c
View File

@ -395,7 +395,7 @@ static int update_modes(int *cone_mode, int *sparse_index)
/* Set cone/non-cone mode appropriately */
core_apply_sparse_checkout = 1;
if (*cone_mode == 1) {
if (*cone_mode == 1 || *cone_mode == -1) {
mode = MODE_CONE_PATTERNS;
core_sparse_checkout_cone = 1;
} else {

15
t/t1091-sparse-checkout-builtin.sh
View File

@ -72,7 +72,7 @@ test_expect_success 'git sparse-checkout list (populated)' '
'
test_expect_success 'git sparse-checkout init' '
git -C repo sparse-checkout init &&
git -C repo sparse-checkout init --no-cone &&
cat >expect <<-\EOF &&
/*
!/*/
@ -111,6 +111,7 @@ test_expect_success 'init with existing sparse-checkout' '
test_expect_success 'clone --sparse' '
git clone --sparse "file://$(pwd)/repo" clone &&
git -C clone sparse-checkout reapply --no-cone &&
git -C clone sparse-checkout list >actual &&
cat >expect <<-\EOF &&
/*
@ -124,7 +125,7 @@ test_expect_success 'switching to cone mode with non-cone mode patterns' '
git init bad-patterns &&
(
cd bad-patterns &&
git sparse-checkout init &&
git sparse-checkout init --no-cone &&
git sparse-checkout add dir &&
git config --worktree core.sparseCheckoutCone true &&
test_must_fail git sparse-checkout add dir 2>err &&
@ -402,7 +403,7 @@ test_expect_success 'revert to old sparse-checkout on empty update' '
git sparse-checkout set nothing 2>err &&
test_i18ngrep ! "Sparse checkout leaves no entry on working directory" err &&
test_i18ngrep ! ".git/index.lock" err &&
git sparse-checkout set file
git sparse-checkout set --no-cone file
)
'
@ -424,7 +425,7 @@ test_expect_success 'sparse-checkout (init|set|disable) warns with dirty status'
git clone repo dirty &&
echo dirty >dirty/folder1/a &&
git -C dirty sparse-checkout init 2>err &&
git -C dirty sparse-checkout init --no-cone 2>err &&
test_i18ngrep "warning.*The following paths are not up to date" err &&
git -C dirty sparse-checkout set /folder2/* /deep/deeper1/* 2>err &&
@ -435,7 +436,7 @@ test_expect_success 'sparse-checkout (init|set|disable) warns with dirty status'
test_must_be_empty err &&
git -C dirty reset --hard &&
git -C dirty sparse-checkout init &&
git -C dirty sparse-checkout init --no-cone &&
git -C dirty sparse-checkout set /folder2/* /deep/deeper1/* &&
test_path_is_missing dirty/folder1/a &&
git -C dirty sparse-checkout disable &&
@ -451,7 +452,7 @@ test_expect_success 'sparse-checkout (init|set|disable) warns with unmerged stat
EOF
git -C unmerged update-index --index-info <input &&
git -C unmerged sparse-checkout init 2>err &&
git -C unmerged sparse-checkout init --no-cone 2>err &&
test_i18ngrep "warning.*The following paths are unmerged" err &&
git -C unmerged sparse-checkout set /folder2/* /deep/deeper1/* 2>err &&
@ -462,7 +463,7 @@ test_expect_success 'sparse-checkout (init|set|disable) warns with unmerged stat
test_i18ngrep "warning.*The following paths are unmerged" err &&
git -C unmerged reset --hard &&
git -C unmerged sparse-checkout init &&
git -C unmerged sparse-checkout init --no-cone &&
git -C unmerged sparse-checkout set /folder2/* /deep/deeper1/* &&
git -C unmerged sparse-checkout disable
'

6
t/t3602-rm-sparse-checkout.sh
View File

@ -30,7 +30,7 @@ test_expect_success 'setup' "
for opt in "" -f --dry-run
do
test_expect_success "rm${opt:+ $opt} does not remove sparse entries" '
git sparse-checkout set a &&
git sparse-checkout set --no-cone a &&
test_must_fail git rm $opt b 2>stderr &&
test_cmp b_error_and_hint stderr &&
git ls-files --error-unmatch b
@ -118,7 +118,7 @@ test_expect_success 'can remove files from non-sparse dir' '
test_commit w/f &&
test_commit x/y/f &&
git sparse-checkout set w !/x y/ &&
git sparse-checkout set --no-cone w !/x y/ &&
git rm w/f.t x/y/f.t 2>stderr &&
test_must_be_empty stderr
'
@ -128,7 +128,7 @@ test_expect_success 'refuse to remove non-skip-worktree file from sparse dir' '
git sparse-checkout disable &&
mkdir -p x/y/z &&
test_commit x/y/z/f &&
git sparse-checkout set !/x y/ !x/y/z &&
git sparse-checkout set --no-cone !/x y/ !x/y/z &&
git update-index --no-skip-worktree x/y/z/f.t &&
test_must_fail git rm x/y/z/f.t 2>stderr &&

4
t/t3705-add-sparse-checkout.sh
View File

@ -166,7 +166,7 @@ test_expect_success 'do not warn when pathspec matches dense entries' '
test_expect_success 'git add fails outside of sparse-checkout definition' '
test_when_finished git sparse-checkout disable &&
test_commit a &&
git sparse-checkout init &&
git sparse-checkout init --no-cone &&
git sparse-checkout set a &&
echo >>sparse_entry &&
@ -208,7 +208,7 @@ test_expect_success 'add obeys advice.updateSparsePath' '
'
test_expect_success 'add allows sparse entries with --sparse' '
git sparse-checkout set a &&
git sparse-checkout set --no-cone a &&
echo modified >sparse_entry &&
test_must_fail git add sparse_entry &&
test_sparse_entry_unchanged &&

4
t/t6428-merge-conflicts-sparse.sh
View File

@ -87,7 +87,7 @@ test_expect_success 'conflicting entries written to worktree even if sparse' '
test_path_is_file numerals &&
git sparse-checkout init &&
git sparse-checkout set README &&
git sparse-checkout set --no-cone README &&
test_path_is_file README &&
test_path_is_missing numerals &&
@ -123,7 +123,7 @@ test_expect_success 'present-despite-SKIP_WORKTREE handled reasonably' '
test_path_is_file numerals &&
git sparse-checkout init &&
git sparse-checkout set README &&
git sparse-checkout set --no-cone README &&
test_path_is_file README &&
test_path_is_missing numerals &&

2
t/t7002-mv-sparse-checkout.sh
View File

@ -27,7 +27,7 @@ test_expect_success 'setup' "
test_expect_success 'mv refuses to move sparse-to-sparse' '
test_when_finished rm -f e &&
git reset --hard &&
git sparse-checkout set a &&
git sparse-checkout set --no-cone a &&
touch b &&
test_must_fail git mv b e 2>stderr &&
cat sparse_error_header >expect &&

2
t/t7012-skip-worktree-writing.sh
View File

@ -151,7 +151,7 @@ test_expect_success 'stash restore in sparse checkout' '
git stash push &&
git sparse-checkout set subdir &&
git sparse-checkout set --no-cone subdir &&
# Ensure after sparse-checkout we only have expected files
cat >expect <<-EOF &&