When there are multiple subtrees present in a repository and they are
all using 'git subtree split', the 'split' command can take a
significant (and constantly growing) amount of time to run even when
using the '--rejoin' flag. This is due to the fact that when processing
commits to determine the last known split to start from when looking
for changes, if there has been a split/merge done from another subtree
there will be 2 split commits, one mainline and one subtree, for the
second subtree that are part of the processing. The non-mainline
subtree split commit will cause the processing to always need to search
the entire history of the given subtree as part of its processing even
though those commits are totally irrelevant to the current subtree
split being run.
To see this in practice you can use the open source GitHub repo
'apollo-ios-dev' and do the following in order:
-Make a changes to a file in 'apollo-ios' and 'apollo-ios-codegen'
directories
-Create a commit containing these changes
-Do a split on apollo-ios-codegen
- Do a fetch on the subtree repo
- git fetch git@github.com:apollographql/apollo-ios-codegen.git
- git subtree split --prefix=apollo-ios-codegen --squash --rejoin
- Depending on the current state of the 'apollo-ios-dev' repo
you may see the issue at this point if the last split was on
apollo-ios
-Do a split on apollo-ios
- Do a fetch on the subtree repo
- git fetch git@github.com:apollographql/apollo-ios.git
- git subtree split --prefix=apollo-ios --squash --rejoin
-Make changes to a file in apollo-ios-codegen
-Create a commit containing the change(s)
-Do a split on apollo-ios-codegen
- git subtree split --prefix=apollo-ios-codegen --squash --rejoin
-To see that the patch fixes the issue you can use the custom subtree
script in the repo so following the same steps as above, except
instead of using 'git subtree ...' for the commands use
'git-subtree.sh ...' for the commands
You will see that the final split is looking for the last split
on apollo-ios-codegen to use as it's starting point to process
commits. Since there is a split commit from apollo-ios in between the
2 splits run on apollo-ios-codegen, the processing ends up traversing
the entire history of apollo-ios which increases the time it takes to
do a split based on how long of a history apollo-ios has, while none
of these commits are relevant to the split being done on
apollo-ios-codegen.
So this commit makes a change to the processing of commits for the
split command in order to ignore non-mainline commits from other
subtrees such as apollo-ios in the above breakdown by adding a new
function 'should_ignore_subtree_commit' which is called during
'process_split_commit'. This allows the split/rejoin processing to
still function as expected but removes all of the unnecessary
processing that takes place currently which greatly inflates the
processing time. In the above example, previously the final split
would take ~10-12 minutes, while after this fix it takes seconds.
Added a test to validate that the proposed fix
solves the issue.
The test accomplishes this by checking the output
of the split command to ensure the output from
the progress of 'process_split_commit' function
that represents the 'extracount' of commits
processed remains at 0, meaning none of the commits
from the second subtree were processed.
This was tested against the original functionality
to show the test failed, and then with this fix
to show the test passes.
This illustrated that when using multiple subtrees,
A and B, when doing a split on subtree B, the
processing does not traverse the entire history
of subtree A which is unnecessary and would cause
the 'extracount' of processed commits to climb
based on the number of commits in the history of
subtree A.
Signed-off-by: Zach FettersMoore <zach.fetters@apollographql.com>
Reviewed-by: Christian Couder <christian.couder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a "[no-]" prefix to options without the flag PARSE_OPT_NONEG to
document the fact that you can negate them.
This looks a bit strange for options that already start with "no-", e.g.
for the option --no-name of git show-branch:
--[no-]no-name suppress naming strings
You can actually use --no-no-name as an alias of --name, so the short
help is not wrong. If we strip off any of the "no-"s, we lose either
the ability to see if the remaining one belongs to the documented
variant or to see if it can be negated.
Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The previous commit fixed a failure in 'git subtree merge --squash' when
the previous squash-merge merged an annotated tag of the subtree
repository which is missing locally.
The same failure happens in 'git subtree split', either directly or when
called by 'git subtree push', under the same circumstances: 'cmd_split'
invokes 'find_existing_splits', which loops through previous commits and
invokes 'git rev-parse' (via 'process_subtree_split_trailer') on the
value of any 'git subtree-split' trailer it finds. This fails if this
value is the hash of an annotated tag which is missing locally.
Add a new optional argument 'repository' to 'cmd_split' and
'find_existing_splits', and invoke 'cmd_split' with that argument from
'cmd_push'. This allows 'process_subtree_split_trailer' to try to fetch
the missing tag from the 'repository' if it's not available locally,
mirroring the new behaviour of 'git subtree pull' and 'git subtree
merge'.
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When 'git subtree merge --squash $ref' is invoked, either directly or
through 'git subtree pull --squash $repo $ref', the code looks for the
latest squash merge of the subtree in order to create the new merge
commit as a child of the previous squash merge.
This search is done in function 'process_subtree_split_trailer', invoked
by 'find_latest_squash', which looks for the most recent commit with a
'git-subtree-split' trailer; that trailer's value is the object name in
the subtree repository of the ref that was last squash-merged. The
function verifies that this object is present locally with 'git
rev-parse', and aborts if it's not.
The hash referenced by the 'git-subtree-split' trailer is guaranteed to
correspond to a commit since it is the result of running 'git rev-parse
-q --verify "$1^{commit}"' on the first argument of 'cmd_merge' (this
corresponds to 'rev' in 'cmd_merge' which is passed through to
'new_squash_commit' and 'squash_msg').
But this is only the case since e4f8baa88a (subtree: parse revs in
individual cmd_ functions, 2021-04-27), which went into Git 2.32. Before
that commit, 'cmd_merge' verified the revision it was given using 'git
rev-parse --revs-only "$@"'. Such an invocation, when fed the name of an
annotated tag, would return the hash of the tag, not of the commit
referenced by the tag.
This leads to a failure in 'find_latest_squash' when squash-merging if
the most recent squash-merge merged an annotated tag of the subtree
repository, using a pre-2.32 version of 'git subtree', unless that
previous annotated tag is present locally (which is not usually the
case).
We can fix this by fetching the object directly by its hash in
'process_subtree_split_trailer' when 'git rev-parse' fails, but in order
to do so we need to know the name or URL of the subtree repository.
This is not possible in general for 'git subtree merge', but is easy
when it is invoked through 'git subtree pull' since in that case the
subtree repository is passed by the user at the command line.
Allow the 'git subtree pull' scenario to work out-of-the-box by adding
an optional 'repository' argument to functions 'cmd_merge',
'find_latest_squash' and 'process_subtree_split_trailer', and invoke
'cmd_merge' with that 'repository' argument in 'cmd_pull'.
If 'repository' is absent in 'process_subtree_split_trailer', instruct
the user to try fetching the missing object directly.
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Just as was done in 0008d12284 (submodule: prefix die messages with
'fatal', 2021-07-10) for 'git-submodule.sh', make the 'die' messages
outputed by 'git-subtree.sh' more in line with the rest of the code base
by prefixing them with "fatal: ", and do not capitalize their first
letter.
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The top-level &&-chain checker built into t/test-lib.sh causes tests to
magically exit with code 117 if the &&-chain is broken. However, it has
the shortcoming that the magic does not work within `{...}` groups,
`(...)` subshells, `$(...)` substitutions, or within bodies of compound
statements, such as `if`, `for`, `while`, `case`, etc. `chainlint.sed`
partly fills in the gap by catching broken &&-chains in `(...)`
subshells, but bugs can still lurk behind broken &&-chains in the other
cases.
Fix broken &&-chains in `$(...)` command substitutions in order to
reduce the number of possible lurking bugs.
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Don't silently ignore a flag that's invalid for a given subcommand. The
user expected it to do something; we should tell the user that they are
mistaken, instead of surprising the user.
It could be argued that this change might break existing users. I'd
argue that those existing users are already broken, and they just don't
know it. Let them know that they're broken.
Signed-off-by: Luke Shumaker <lukeshu@datawire.io>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
'git subtree split' lets you specify a rev other than HEAD. 'git push'
lets you specify a mapping between a local thing and a remot ref. So
smash those together, and have 'git subtree push' let you specify which
local thing to run split on and push the result of that split to the
remote ref.
Signed-off-by: Luke Shumaker <lukeshu@datawire.io>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
'push' does a 'split' internally, but it doesn't pass flags through to the
'split'. This is silly, if you need to pass flags to 'split', then it
means that you can't use 'push'!
So, have 'push' accept 'split' flags, and pass them through to 'split'.
Add tests for this by copying split's tests with minimal modification.
Signed-off-by: Luke Shumaker <lukeshu@datawire.io>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Besides being a genuinely useful thing to do, this also just makes sense
and harmonizes which flags may be used when. `git subtree split
--rejoin` amounts to "automatically go ahead and do a `git subtree
merge` after doing the main `git subtree split`", so it's weird and
arbitrary that you can't pass `--squash` to `git subtree split --rejoin`
like you can `git subtree merge`. It's weird that `git subtree split
--rejoin` inherits `git subtree merge`'s `--message` but not `--squash`.
Reconcile the situation by just having `split --rejoin` actually just
call `merge` internally (or call `add` instead, as appropriate), so it
can get access to the full `merge` behavior, including `--squash`.
Signed-off-by: Luke Shumaker <lukeshu@datawire.io>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The 'pull' and 'push' subcommands deserve their own sections in the tests.
Add some basic tests for them.
Signed-off-by: Luke Shumaker <lukeshu@datawire.io>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It's a dumb test, but it's surprisingly easy to break.
Signed-off-by: Luke Shumaker <lukeshu@datawire.io>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t7900-subtree.sh defines a helper function named last_commit_message.
However, it only returns the subject line of the commit message, not the
entire commit message. So rename it, to make the name less confusing.
Signed-off-by: Luke Shumaker <lukeshu@datawire.io>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As far as I can tell, this test isn't actually testing anything, because
someone forgot to tack on `--name-only` to `git log`. This seems to
have been the case since the test was first written, back in fa16ab36ad
("test.sh: make sure no commit changes more than one file at a time.",
2009-04-26), unless `git log` used to do that by default and didn't need
the flag back then?
Convincing myself that it's not actually testing anything was tricky,
the code is a little hard to reason about. It can be made a lot simpler
if instead of trying to parse all of the info from a single `git log`,
we're OK calling `git log` from inside of a loop. And it's my opinion
that tests are not the place for clever optimized code.
So, fix and simplify the test, so that it's actually testing something
and is simpler to reason about.
Signed-off-by: Luke Shumaker <lukeshu@datawire.io>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t7900-subtree.sh defines its own `check_equal A B` function, instead of
just using `test A = B` like all of the other tests. Don't be special,
get rid of `check_equal` in favor of `test`.
Signed-off-by: Luke Shumaker <lukeshu@datawire.io>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It's unclear what the purpose of t7900-subtree.sh's
`subtree_test_create_repo` helper function is. It wraps test-lib.sh's,
`test_create_repo` but follows that up by setting log.date=relative. Why
does it set log.date=relative?
My first guess was that at one point the tests required that, but no
longer do, and that the function is now vestigial. I even wrote a patch
to get rid of it and was moments away from `git send-email`ing it.
However, by chance when looking for something else in the history, I
discovered the true reason, from e7aac44ed2 (contrib/subtree: ignore
log.date configuration, 2015-07-21). It's testing that setting
log.date=relative doesn't break `git subtree`, as at one point in the past
that did break `git subtree`.
So, add a comment about this, to avoid future such confusion.
And while at it, go ahead and (1) touch up the function to avoid a
pointless subshell and (2) update the one test that didn't use it.
Signed-off-by: Luke Shumaker <lukeshu@datawire.io>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The formatting in t7900-subtree.sh isn't even consistent throughout the
file. Fix that; make it consistent throughout the file.
Signed-off-by: Luke Shumaker <lukeshu@datawire.io>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use test-lib.sh's `test_count`, instead instead of having
t7900-subtree.sh do its own book-keeping with `subtree_test_count` that
has to be explicitly incremented by calling `next_test`.
Signed-off-by: Luke Shumaker <lukeshu@datawire.io>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Most of the tests had been converted to support
`GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main`, but `contrib/subtree/t/`
hadn't.
Convert it. Most of the mentions of 'master' can just be replaced with
'HEAD'.
Signed-off-by: Luke Shumaker <lukeshu@datawire.io>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This reverts commit 508fd8e8ba.
In 6e6029a8 (fmt-merge-msg: allow merge destination to be omitted again)
we get back the behavior where merges against 'master', by default, do
not include "into 'master'" at the end of the merge message. This test
fix is no longer needed.
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We're starting to stop treating `master' specially in fmt-merge-msg.
Adjust the test to reflect that change.
Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This mixture of quoting, pipes, and here-docs to produce expected
results in shell variables is difficult to follow. Simplify by using
simpler constructs that write output to files instead.
Noticed because without this patch, t/chainlint is not able to
understand the script in order to validate that its subshells use an
unbroken &&-chain, causing "make -C contrib/subtree test" to fail with
error: bug in the test script: broken &&-chain or run-away HERE-DOC:
in t7900.21.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Follow the Oxford style, which says to use "up-to-date" before the noun,
but "up to date" after it. Don't change plumbing (specifically
send-pack.c, but transport.c (git push) also has the same string).
This was produced by grepping for "up-to-date" and "up to date". It
turned out we only had to edit in one direction, removing the hyphens.
Fix a typo in Documentation/git-diff-index.txt while we're there.
Reported-by: Jeffrey Manian <jeffrey.manian@gmail.com>
Reported-by: STEVEN WHITE <stevencharleswhitevoices@gmail.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git merge" in v2.9 prevents merging unrelated histories.
"git subtree split --rejoin" creates unrelated histories when
creating a split repo from a raw sub-directory that did not
originate from an invocation of "git subtree add".
Restore the original behavior by passing --allow-unrelated-histories
when merging subtrees. This ensures that the synthetic history
created by "git subtree split" can be merged.
Add a test to ensure that this feature works as advertised.
Reported-by: Brett Cundal <brett.cundal@iugome.com>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Allow whitespace in arguments to subtree_test_create_repo.
Add missing && chains.
Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
'git subtree split' can incorrectly skip a merge even when both parents
act on the subtree, provided the merge results in a tree identical to
one of the parents. Fix by copying the merge if at least one parent is
non-identical, and the non-identical parent is not an ancestor of the
identical parent.
Also, add a test case which checks that a descendant remains a
descendent on the subtree in this case.
Signed-off-by: Dave Ware <davidw@realtimegenomics.com>
Reviewed-by: David A. Greene <greened@obbligato.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
'git subtree merge' will fail if the argument of '--prefix' has a slash
appended.
Signed-off-by: Techlive Zheng <techlivezheng@gmail.com>
Signed-off-by: David A. Greene <greened@obbligato.org>
Signed-off-by: Jeff King <peff@peff.net>
Each test runs a full repository creation and any subtree actions
needed to perform the test. Each test starts with a clean slate,
making debugging and post-mortem analysis much easier.
Signed-off-by: Techlive Zheng <techlivezheng@gmail.com>
Signed-off-by: David A. Greene <greened@obbligato.org>
Signed-off-by: Jeff King <peff@peff.net>
Add tests to check various options to split. Check combinations of
--prefix, --message, --annotate, --branch and --rejoin.
Signed-off-by: Techlive Zheng <techlivezheng@gmail.com>
Signed-off-by: David A. Greene <greened@obbligato.org>
Signed-off-by: Jeff King <peff@peff.net>
Add some tests for various merge operations. Test combinations of merge
with --message, --prefix and --squash.
Signed-off-by: Techlive Zheng <techlivezheng@gmail.com>
Signed-off-by: David A. Greene <greened@obbligato.org>
Signed-off-by: Jeff King <peff@peff.net>
Add some tests to check various options to subtree add. These test
various combinations of --message, --prefix and --squash.
Signed-off-by: Techlive Zheng <techlivezheng@gmail.com>
Signed-off-by: David A. Greene <greened@obbligato.org>
Signed-off-by: Jeff King <peff@peff.net>
Test that a merge from a non-existant subtree fails.
Signed-off-by: Techlive Zheng <techlivezheng@gmail.com>
Signed-off-by: David A. Greene <greened@obbligato.org>
Signed-off-by: Jeff King <peff@peff.net>
Mostly prepare for the later tests refactoring. This moves some
common code to helper functions and generally cleans things up to be
more presentable.
Signed-off-by: Techlive Zheng <techlivezheng@gmail.com>
Signed-off-by: David A. Greene <greened@obbligato.org>
Signed-off-by: Jeff King <peff@peff.net>
Remote repository may have spaces in its path, so take it into account.
Also, as far as there are no tests for the `push` command, add them.
Signed-off-by: Alexey Shumkin <Alex.Crezoff@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In common case there can be spaces in a subdirectory name. Change tests
accorgingly to this statement.
Also, as far as a call to the `rejoin_msg` function (in `cmd_split`)
does not take into account such a case this patch fixes commit message
when `--rejoin` option is set .
Besides, as `fixnl` and `multiline` functions did not take into account
the "new" tested "space in a subdirectory name" case they become unused
and redundant, so they are removed.
Signed-off-by: Alexey Shumkin <Alex.Crezoff@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git subtree" (in contrib/) depended on "git log" output to be
stable, which was a no-no. Apply a workaround to force a
particular date format.
* da/subtree-date-confusion:
contrib/subtree: ignore log.date configuration
git-subtree's log format string uses "%ad" and "%cd", which
respect the user's configured log.date value.
This is problematic for git-subtree because it needs to use real
dates so that copied commits come through unchanged.
Add a test and tweak the format strings to use %aD and %cD
so that the default date format is used instead.
Reported-by: Bryan Jacobs <b@q3q.us>
Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There's no need to switch branches to parse another branch's ancestry.
Signed-off-by: Charles Bailey <cbailey32@bloomberg.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This fixes two instances where a &&-chain was broken in the subtree
tests and fixes a test error that was revealed because of this.
Many tests in t7900-subtree.sh make a commit and then use 'undo' to
reset the state for the next test. In the 'check hash of split' test,
an 'undo' was being invoked after a 'subtree split' even though the
particular invocation of 'subtree split' did not actually make a commit.
The subsequent check_equal was failing, but this failure was masked by
that broken &&-chain.
Removing this undo causes the failing check_equal to succeed but breaks
the a check_equal later on in the same test.
It turns out that an earlier test ('check if --message for merge works
with squash too') makes a commit but doesn't 'undo' to the state
expected by the remaining tests. None of the intervening tests cared
enough about the state of the test repo to fail and the spurious 'undo'
in 'check hash of split' restored the expected state for any remaining
test that might care.
Adding the missing 'undo' to 'check if --message for merge works
with squash too' and removing the spurious one from 'check hash of
split' fixes all tests once the &&-chains are completed.
Signed-off-by: Charles Bailey <cbailey32@bloomberg.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Although subtrees tests uses more spaces for indentation than tabs,
there are still quite a lot of lines indented with tabs. As tabs conform
with Git coding guidelines resolve the inconsistency in favour of tabs.
Signed-off-by: Charles Bailey <cbailey32@bloomberg.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Found by check-non-portable-shell.pl
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The Git CodingGuidelines prefer the $(...) construct for command
substitution instead of using the backquotes `...`.
The backquoted form is the traditional method for command
substitution, and is supported by POSIX. However, all but the
simplest uses become complicated quickly. In particular, embedded
command substitutions and/or the use of double quotes require
careful escaping with the backslash character.
The patch was generated by:
for _f in $(find . -name "*.sh")
do
sed -i 's@`\(.*\)`@$(\1)@g' ${_f}
done
and then carefully proof-read.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Reviewed-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
`echo -n` is non-portable. The POSIX specification says:
Conforming applications that wish to do prompting without <newline>
characters or that could possibly be expecting to echo a -n, should
use the printf utility derived from the Ninth Edition system.
Since all of the affected shell scripts use a POSIX shell shebang,
replace `echo -n` invocations with printf.
Signed-off-by: Lukas Fleischer <git@cryptocrack.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Noticed with Lucas De Marchi's codespell tool.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Acked-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use %B to format the commit message and body to avoid an extra newline
if a commit only has a subject line.
Signed-off-by: Techlive Zheng <techlivezheng@gmail.com>
Signed-off-by: David A. Greene <greened@obbligato.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Beat Bolli
Zach FettersMoore
René Scharfe
Philippe Blain
Eric Sunshine
Luke Shumaker
Emily Shaffer
Đoàn Trần Công Danh
Jonathan Nieder
Martin Ågren
Ville Skyttä
David Aguilar
Dave Ware
Techlive Zheng
Alexey Shumkin
Junio C Hamano
Charles Bailey
Elia Pinto
Lukas Fleischer
Stefano Lattarini