mirror/ git
1
0
Fork 0
mirror of https://github.com/git/git.git synced 2024-05-25 18:16:11 +02:00
Code
git/t/t1091-sparse-checkout-built...
Derrick Stolee 55dfcf9591 sparse-checkout: clear tracked sparse dirs 
When changing the scope of a sparse-checkout using cone mode, we might
have some tracked directories go out of scope. The current logic removes
the tracked files from within those directories, but leaves the ignored
files within those directories. This is a bit unexpected to users who
have given input to Git saying they don't need those directories
anymore.

This is something that is new to the cone mode pattern type: the user
has explicitly said "I want these directories and _not_ those
directories." The typical sparse-checkout patterns more generally apply
to "I want files with with these patterns" so it is natural to leave
ignored files as they are. This focus on directories in cone mode
provides us an opportunity to change the behavior.

Leaving these ignored files in the sparse directories makes it
impossible to gain performance benefits in the sparse index. When we
track into these directories, we need to know if the files are ignored
or not, which might depend on the _tracked_ .gitignore file(s) within
the sparse directory. This depends on the indexed version of the file,
so the sparse directory must be expanded.

We must take special care to look for untracked, non-ignored files in
these directories before deleting them. We do not want to delete any
meaningful work that the users were doing in those directories and
perhaps forgot to add and commit before switching sparse-checkout
definitions. Since those untracked files might be code files that
generated ignored build output, also do not delete any ignored files
from these directories in that case. The users can recover their state
by resetting their sparse-checkout definition to include that directory
and continue. Alternatively, they can see the warning that is presented
and delete the directory themselves to regain the performance they
expect.

By deleting the sparse directories when changing scope (or running 'git
sparse-checkout reapply') we regain these performance benefits as if the
repository was in a clean state.

Since these ignored files are frequently build output or helper files
from IDEs, the users should not need the files now that the tracked
files are removed. If the tracked files reappear, then they will have
newer timestamps than the build artifacts, so the artifacts will need to
be regenerated anyway.

Use the sparse-index as a data structure in order to find the sparse
directories that can be safely deleted. Re-expand the index to a full
one if it was full before.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-07 22:41:10 -07:00

705 lines
19 KiB
Bash
Executable File
Raw Blame History

#!/bin/sh
test_description='sparse checkout builtin tests'
GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
. ./test-lib.sh
list_files() {
# Do not replace this with 'ls "$1"', as "ls" with BSD-lineage
# enables "-A" by default for root and ends up including ".git" and
# such in its output. (Note, though, that running the test suite as
# root is generally not recommended.)
(cd "$1" && printf '%s\n' *)
}
check_files() {
list_files "$1" >actual &&
shift &&
printf "%s\n" $@ >expect &&
test_cmp expect actual
}
test_expect_success 'setup' '
git init repo &&
(
cd repo &&
echo "initial" >a &&
mkdir folder1 folder2 deep &&
mkdir deep/deeper1 deep/deeper2 &&
mkdir deep/deeper1/deepest &&
cp a folder1 &&
cp a folder2 &&
cp a deep &&
cp a deep/deeper1 &&
cp a deep/deeper2 &&
cp a deep/deeper1/deepest &&
git add . &&
git commit -m "initial commit"
)
'
test_expect_success 'git sparse-checkout list (empty)' '
git -C repo sparse-checkout list >list 2>err &&
test_must_be_empty list &&
test_i18ngrep "this worktree is not sparse (sparse-checkout file may not exist)" err
'
test_expect_success 'git sparse-checkout list (populated)' '
test_when_finished rm -f repo/.git/info/sparse-checkout &&
cat >repo/.git/info/sparse-checkout <<-\EOF &&
/folder1/*
/deep/
**/a
!*bin*
EOF
cp repo/.git/info/sparse-checkout expect &&
git -C repo sparse-checkout list >list &&
test_cmp expect list
'
test_expect_success 'git sparse-checkout init' '
git -C repo sparse-checkout init &&
cat >expect <<-\EOF &&
/*
!/*/
EOF
test_cmp expect repo/.git/info/sparse-checkout &&
test_cmp_config -C repo true core.sparsecheckout &&
check_files repo a
'
test_expect_success 'git sparse-checkout list after init' '
git -C repo sparse-checkout list >actual &&
cat >expect <<-\EOF &&
/*
!/*/
EOF
test_cmp expect actual
'
test_expect_success 'init with existing sparse-checkout' '
echo "*folder*" >> repo/.git/info/sparse-checkout &&
git -C repo sparse-checkout init &&
cat >expect <<-\EOF &&
/*
!/*/
*folder*
EOF
test_cmp expect repo/.git/info/sparse-checkout &&
check_files repo a folder1 folder2
'
test_expect_success 'clone --sparse' '
git clone --sparse "file://$(pwd)/repo" clone &&
git -C clone sparse-checkout list >actual &&
cat >expect <<-\EOF &&
/*
!/*/
EOF
test_cmp expect actual &&
check_files clone a
'
test_expect_success 'interaction with clone --no-checkout (unborn index)' '
git clone --no-checkout "file://$(pwd)/repo" clone_no_checkout &&
git -C clone_no_checkout sparse-checkout init --cone &&
git -C clone_no_checkout sparse-checkout set folder1 &&
git -C clone_no_checkout sparse-checkout list >actual &&
cat >expect <<-\EOF &&
folder1
EOF
test_cmp expect actual &&
# nothing checked out, expect "No such file or directory"
! ls clone_no_checkout/* >actual &&
test_must_be_empty actual &&
test_path_is_missing clone_no_checkout/.git/index &&
# No branch is checked out until we manually switch to one
git -C clone_no_checkout switch main &&
test_path_is_file clone_no_checkout/.git/index &&
check_files clone_no_checkout a folder1
'
test_expect_success 'set enables config' '
git init empty-config &&
(
cd empty-config &&
test_commit test file &&
test_path_is_missing .git/config.worktree &&
git sparse-checkout set nothing &&
test_path_is_file .git/config.worktree &&
test_cmp_config true core.sparseCheckout
)
'
test_expect_success 'set sparse-checkout using builtin' '
git -C repo sparse-checkout set "/*" "!/*/" "*folder*" &&
cat >expect <<-\EOF &&
/*
!/*/
*folder*
EOF
git -C repo sparse-checkout list >actual &&
test_cmp expect actual &&
test_cmp expect repo/.git/info/sparse-checkout &&
check_files repo a folder1 folder2
'
test_expect_success 'set sparse-checkout using --stdin' '
cat >expect <<-\EOF &&
/*
!/*/
/folder1/
/folder2/
EOF
git -C repo sparse-checkout set --stdin <expect &&
git -C repo sparse-checkout list >actual &&
test_cmp expect actual &&
test_cmp expect repo/.git/info/sparse-checkout &&
check_files repo "a folder1 folder2"
'
test_expect_success 'add to sparse-checkout' '
cat repo/.git/info/sparse-checkout >expect &&
cat >add <<-\EOF &&
pattern1
/folder1/
pattern2
EOF
cat add >>expect &&
git -C repo sparse-checkout add --stdin <add &&
git -C repo sparse-checkout list >actual &&
test_cmp expect actual &&
test_cmp expect repo/.git/info/sparse-checkout &&
check_files repo "a folder1 folder2"
'
test_expect_success 'cone mode: match patterns' '
git -C repo config --worktree core.sparseCheckoutCone true &&
rm -rf repo/a repo/folder1 repo/folder2 &&
git -C repo read-tree -mu HEAD 2>err &&
test_i18ngrep ! "disabling cone patterns" err &&
git -C repo reset --hard &&
check_files repo a folder1 folder2
'
test_expect_success 'cone mode: warn on bad pattern' '
test_when_finished mv sparse-checkout repo/.git/info/ &&
cp repo/.git/info/sparse-checkout . &&
echo "!/deep/deeper/*" >>repo/.git/info/sparse-checkout &&
git -C repo read-tree -mu HEAD 2>err &&
test_i18ngrep "unrecognized negative pattern" err
'
test_expect_success 'sparse-checkout disable' '
test_when_finished rm -rf repo/.git/info/sparse-checkout &&
git -C repo sparse-checkout disable &&
test_path_is_file repo/.git/info/sparse-checkout &&
git -C repo config --list >config &&
test_must_fail git config core.sparseCheckout &&
check_files repo a deep folder1 folder2
'
test_expect_success 'sparse-index enabled and disabled' '
git -C repo sparse-checkout init --cone --sparse-index &&
test_cmp_config -C repo true index.sparse &&
test-tool -C repo read-cache --table >cache &&
grep " tree " cache &&
git -C repo sparse-checkout disable &&
test-tool -C repo read-cache --table >cache &&
! grep " tree " cache &&
git -C repo config --list >config &&
! grep index.sparse config
'
test_expect_success 'cone mode: init and set' '
git -C repo sparse-checkout init --cone &&
git -C repo config --list >config &&
test_i18ngrep "core.sparsecheckoutcone=true" config &&
list_files repo >dir &&
echo a >expect &&
test_cmp expect dir &&
git -C repo sparse-checkout set deep/deeper1/deepest/ 2>err &&
test_must_be_empty err &&
check_files repo a deep &&
check_files repo/deep a deeper1 &&
check_files repo/deep/deeper1 a deepest &&
cat >expect <<-\EOF &&
/*
!/*/
/deep/
!/deep/*/
/deep/deeper1/
!/deep/deeper1/*/
/deep/deeper1/deepest/
EOF
test_cmp expect repo/.git/info/sparse-checkout &&
git -C repo sparse-checkout set --stdin 2>err <<-\EOF &&
folder1
folder2
EOF
test_must_be_empty err &&
check_files repo a folder1 folder2
'
test_expect_success 'cone mode: list' '
cat >expect <<-\EOF &&
folder1
folder2
EOF
git -C repo sparse-checkout set --stdin <expect &&
git -C repo sparse-checkout list >actual 2>err &&
test_must_be_empty err &&
test_cmp expect actual
'
test_expect_success 'cone mode: set with nested folders' '
git -C repo sparse-checkout set deep deep/deeper1/deepest 2>err &&
test_line_count = 0 err &&
cat >expect <<-\EOF &&
/*
!/*/
/deep/
EOF
test_cmp repo/.git/info/sparse-checkout expect
'
test_expect_success 'cone mode: add independent path' '
git -C repo sparse-checkout set deep/deeper1 &&
git -C repo sparse-checkout add folder1 &&
cat >expect <<-\EOF &&
/*
!/*/
/deep/
!/deep/*/
/deep/deeper1/
/folder1/
EOF
test_cmp expect repo/.git/info/sparse-checkout &&
check_files repo a deep folder1
'
test_expect_success 'cone mode: add sibling path' '
git -C repo sparse-checkout set deep/deeper1 &&
git -C repo sparse-checkout add deep/deeper2 &&
cat >expect <<-\EOF &&
/*
!/*/
/deep/
!/deep/*/
/deep/deeper1/
/deep/deeper2/
EOF
test_cmp expect repo/.git/info/sparse-checkout &&
check_files repo a deep
'
test_expect_success 'cone mode: add parent path' '
git -C repo sparse-checkout set deep/deeper1 folder1 &&
git -C repo sparse-checkout add deep &&
cat >expect <<-\EOF &&
/*
!/*/
/deep/
/folder1/
EOF
test_cmp expect repo/.git/info/sparse-checkout &&
check_files repo a deep folder1
'
test_expect_success 'not-up-to-date does not block rest of sparsification' '
test_when_finished git -C repo sparse-checkout disable &&
test_when_finished git -C repo reset --hard &&
git -C repo sparse-checkout set deep &&
echo update >repo/deep/deeper2/a &&
cp repo/.git/info/sparse-checkout expect &&
test_write_lines "!/deep/*/" "/deep/deeper1/" >>expect &&
git -C repo sparse-checkout set deep/deeper1 2>err &&
test_i18ngrep "The following paths are not up to date" err &&
test_cmp expect repo/.git/info/sparse-checkout &&
check_files repo/deep a deeper1 deeper2 &&
check_files repo/deep/deeper1 a deepest &&
check_files repo/deep/deeper1/deepest a &&
check_files repo/deep/deeper2 a
'
test_expect_success 'revert to old sparse-checkout on empty update' '
git init empty-test &&
(
echo >file &&
git add file &&
git commit -m "test" &&
git sparse-checkout set nothing 2>err &&
test_i18ngrep ! "Sparse checkout leaves no entry on working directory" err &&
test_i18ngrep ! ".git/index.lock" err &&
git sparse-checkout set file
)
'
test_expect_success 'fail when lock is taken' '
test_when_finished rm -rf repo/.git/info/sparse-checkout.lock &&
touch repo/.git/info/sparse-checkout.lock &&
test_must_fail git -C repo sparse-checkout set deep 2>err &&
test_i18ngrep "Unable to create .*\.lock" err
'
test_expect_success '.gitignore should not warn about cone mode' '
git -C repo config --worktree core.sparseCheckoutCone true &&
echo "**/bin/*" >repo/.gitignore &&
git -C repo reset --hard 2>err &&
test_i18ngrep ! "disabling cone patterns" err
'
test_expect_success 'sparse-checkout (init|set|disable) warns with dirty status' '
git clone repo dirty &&
echo dirty >dirty/folder1/a &&
git -C dirty sparse-checkout init 2>err &&
test_i18ngrep "warning.*The following paths are not up to date" err &&
git -C dirty sparse-checkout set /folder2/* /deep/deeper1/* 2>err &&
test_i18ngrep "warning.*The following paths are not up to date" err &&
test_path_is_file dirty/folder1/a &&
git -C dirty sparse-checkout disable 2>err &&
test_must_be_empty err &&
git -C dirty reset --hard &&
git -C dirty sparse-checkout init &&
git -C dirty sparse-checkout set /folder2/* /deep/deeper1/* &&
test_path_is_missing dirty/folder1/a &&
git -C dirty sparse-checkout disable &&
test_path_is_file dirty/folder1/a
'
test_expect_success 'sparse-checkout (init|set|disable) warns with unmerged status' '
git clone repo unmerged &&
cat >input <<-EOF &&
0 $ZERO_OID folder1/a
100644 $(git -C unmerged rev-parse HEAD:folder1/a) 1 folder1/a
EOF
git -C unmerged update-index --index-info <input &&
git -C unmerged sparse-checkout init 2>err &&
test_i18ngrep "warning.*The following paths are unmerged" err &&
git -C unmerged sparse-checkout set /folder2/* /deep/deeper1/* 2>err &&
test_i18ngrep "warning.*The following paths are unmerged" err &&
test_path_is_file dirty/folder1/a &&
git -C unmerged sparse-checkout disable 2>err &&
test_i18ngrep "warning.*The following paths are unmerged" err &&
git -C unmerged reset --hard &&
git -C unmerged sparse-checkout init &&
git -C unmerged sparse-checkout set /folder2/* /deep/deeper1/* &&
git -C unmerged sparse-checkout disable
'
test_expect_success 'sparse-checkout reapply' '
git clone repo tweak &&
echo dirty >tweak/deep/deeper2/a &&
cat >input <<-EOF &&
0 $ZERO_OID folder1/a
100644 $(git -C tweak rev-parse HEAD:folder1/a) 1 folder1/a
EOF
git -C tweak update-index --index-info <input &&
git -C tweak sparse-checkout init --cone 2>err &&
test_i18ngrep "warning.*The following paths are not up to date" err &&
test_i18ngrep "warning.*The following paths are unmerged" err &&
git -C tweak sparse-checkout set folder2 deep/deeper1 2>err &&
test_i18ngrep "warning.*The following paths are not up to date" err &&
test_i18ngrep "warning.*The following paths are unmerged" err &&
git -C tweak sparse-checkout reapply 2>err &&
test_i18ngrep "warning.*The following paths are not up to date" err &&
test_path_is_file tweak/deep/deeper2/a &&
test_i18ngrep "warning.*The following paths are unmerged" err &&
test_path_is_file tweak/folder1/a &&
git -C tweak checkout HEAD deep/deeper2/a &&
git -C tweak sparse-checkout reapply 2>err &&
test_i18ngrep ! "warning.*The following paths are not up to date" err &&
test_path_is_missing tweak/deep/deeper2/a &&
test_i18ngrep "warning.*The following paths are unmerged" err &&
test_path_is_file tweak/folder1/a &&
git -C tweak add folder1/a &&
git -C tweak sparse-checkout reapply 2>err &&
test_must_be_empty err &&
test_path_is_missing tweak/deep/deeper2/a &&
test_path_is_missing tweak/folder1/a &&
git -C tweak sparse-checkout disable
'
test_expect_success 'cone mode: set with core.ignoreCase=true' '
rm repo/.git/info/sparse-checkout &&
git -C repo sparse-checkout init --cone &&
git -C repo -c core.ignoreCase=true sparse-checkout set folder1 &&
cat >expect <<-\EOF &&
/*
!/*/
/folder1/
EOF
test_cmp expect repo/.git/info/sparse-checkout &&
check_files repo a folder1
'
test_expect_success 'interaction with submodules' '
git clone repo super &&
(
cd super &&
mkdir modules &&
git submodule add ../repo modules/child &&
git add . &&
git commit -m "add submodule" &&
git sparse-checkout init --cone &&
git sparse-checkout set folder1
) &&
check_files super a folder1 modules &&
check_files super/modules/child a deep folder1 folder2
'
test_expect_success 'different sparse-checkouts with worktrees' '
git -C repo worktree add --detach ../worktree &&
check_files worktree "a deep folder1 folder2" &&
git -C worktree sparse-checkout init --cone &&
git -C repo sparse-checkout set folder1 &&
git -C worktree sparse-checkout set deep/deeper1 &&
check_files repo a folder1 &&
check_files worktree a deep
'
test_expect_success 'set using filename keeps file on-disk' '
git -C repo sparse-checkout set a deep &&
cat >expect <<-\EOF &&
/*
!/*/
/a/
/deep/
EOF
test_cmp expect repo/.git/info/sparse-checkout &&
check_files repo a deep
'
check_read_tree_errors () {
REPO=$1
FILES=$2
ERRORS=$3
git -C $REPO -c core.sparseCheckoutCone=false read-tree -mu HEAD 2>err &&
test_must_be_empty err &&
check_files $REPO "$FILES" &&
git -C $REPO read-tree -mu HEAD 2>err &&
if test -z "$ERRORS"
then
test_must_be_empty err
else
test_i18ngrep "$ERRORS" err
fi &&
check_files $REPO $FILES
}
test_expect_success 'pattern-checks: /A/**' '
cat >repo/.git/info/sparse-checkout <<-\EOF &&
/*
!/*/
/folder1/**
EOF
check_read_tree_errors repo "a folder1" "disabling cone pattern matching"
'
test_expect_success 'pattern-checks: /A/**/B/' '
cat >repo/.git/info/sparse-checkout <<-\EOF &&
/*
!/*/
/deep/**/deepest
EOF
check_read_tree_errors repo "a deep" "disabling cone pattern matching" &&
check_files repo/deep "deeper1" &&
check_files repo/deep/deeper1 "deepest"
'
test_expect_success 'pattern-checks: too short' '
cat >repo/.git/info/sparse-checkout <<-\EOF &&
/*
!/*/
/
EOF
check_read_tree_errors repo "a" "disabling cone pattern matching"
'
test_expect_success 'pattern-checks: not too short' '
cat >repo/.git/info/sparse-checkout <<-\EOF &&
/*
!/*/
/b/
EOF
git -C repo read-tree -mu HEAD 2>err &&
test_must_be_empty err &&
check_files repo a
'
test_expect_success 'pattern-checks: trailing "*"' '
cat >repo/.git/info/sparse-checkout <<-\EOF &&
/*
!/*/
/a*
EOF
check_read_tree_errors repo "a" "disabling cone pattern matching"
'
test_expect_success 'pattern-checks: starting "*"' '
cat >repo/.git/info/sparse-checkout <<-\EOF &&
/*
!/*/
*eep/
EOF
check_read_tree_errors repo "a deep" "disabling cone pattern matching"
'
test_expect_success 'pattern-checks: contained glob characters' '
for c in "[a]" "\\" "?" "*"
do
cat >repo/.git/info/sparse-checkout <<-EOF &&
/*
!/*/
something$c-else/
EOF
check_read_tree_errors repo "a" "disabling cone pattern matching"
done
'
test_expect_success BSLASHPSPEC 'pattern-checks: escaped characters' '
git clone repo escaped &&
TREEOID=$(git -C escaped rev-parse HEAD:folder1) &&
NEWTREE=$(git -C escaped mktree <<-EOF
$(git -C escaped ls-tree HEAD)
040000 tree $TREEOID zbad\\dir
040000 tree $TREEOID zdoes*exist
040000 tree $TREEOID zglob[!a]?
EOF
) &&
COMMIT=$(git -C escaped commit-tree $NEWTREE -p HEAD) &&
git -C escaped reset --hard $COMMIT &&
check_files escaped "a deep folder1 folder2 zbad\\dir zdoes*exist" zglob[!a]? &&
git -C escaped sparse-checkout init --cone &&
git -C escaped sparse-checkout set zbad\\dir/bogus "zdoes*not*exist" "zdoes*exist" "zglob[!a]?" &&
cat >expect <<-\EOF &&
/*
!/*/
/zbad\\dir/
!/zbad\\dir/*/
/zbad\\dir/bogus/
/zdoes\*exist/
/zdoes\*not\*exist/
/zglob\[!a]\?/
EOF
test_cmp expect escaped/.git/info/sparse-checkout &&
check_read_tree_errors escaped "a zbad\\dir zdoes*exist zglob[!a]?" &&
git -C escaped ls-tree -d --name-only HEAD >list-expect &&
git -C escaped sparse-checkout set --stdin <list-expect &&
cat >expect <<-\EOF &&
/*
!/*/
/deep/
/folder1/
/folder2/
/zbad\\dir/
/zdoes\*exist/
/zglob\[!a]\?/
EOF
test_cmp expect escaped/.git/info/sparse-checkout &&
check_files escaped "a deep folder1 folder2 zbad\\dir zdoes*exist" zglob[!a]? &&
git -C escaped sparse-checkout list >list-actual &&
test_cmp list-expect list-actual
'
test_expect_success MINGW 'cone mode replaces backslashes with slashes' '
git -C repo sparse-checkout set deep\\deeper1 &&
cat >expect <<-\EOF &&
/*
!/*/
/deep/
!/deep/*/
/deep/deeper1/
EOF
test_cmp expect repo/.git/info/sparse-checkout &&
check_files repo a deep &&
check_files repo/deep a deeper1
'
test_expect_success 'cone mode clears ignored subdirectories' '
rm repo/.git/info/sparse-checkout &&
git -C repo sparse-checkout init --cone &&
git -C repo sparse-checkout set deep/deeper1 &&
cat >repo/.gitignore <<-\EOF &&
obj/
*.o
EOF
git -C repo add .gitignore &&
git -C repo commit -m ".gitignore" &&
mkdir -p repo/obj repo/folder1/obj repo/deep/deeper2/obj &&
for file in folder1/obj/a obj/a folder1/file.o folder1.o \
deep/deeper2/obj/a deep/deeper2/file.o file.o
do
echo ignored >repo/$file || return 1
done &&
git -C repo status --porcelain=v2 >out &&
test_must_be_empty out &&
git -C repo sparse-checkout reapply &&
test_path_is_missing repo/folder1 &&
test_path_is_missing repo/deep/deeper2 &&
test_path_is_dir repo/obj &&
test_path_is_file repo/file.o &&
git -C repo status --porcelain=v2 >out &&
test_must_be_empty out &&
git -C repo sparse-checkout set deep/deeper2 &&
test_path_is_missing repo/deep/deeper1 &&
test_path_is_dir repo/deep/deeper2 &&
test_path_is_dir repo/obj &&
test_path_is_file repo/file.o &&
>repo/deep/deeper2/ignored.o &&
>repo/deep/deeper2/untracked &&
# When an untracked file is in the way, all untracked files
# (even ignored files) are preserved.
git -C repo sparse-checkout set folder1 2>err &&
grep "contains untracked files" err &&
test_path_is_file repo/deep/deeper2/ignored.o &&
test_path_is_file repo/deep/deeper2/untracked &&
# The rest of the cone matches expectation
test_path_is_missing repo/deep/deeper1 &&
test_path_is_dir repo/obj &&
test_path_is_file repo/file.o &&
git -C repo status --porcelain=v2 >out &&
echo "? deep/deeper2/untracked" >expect &&
test_cmp expect out
'
test_done
View Git Blame Copy Permalink