From ac45f68866fbdbe14384d1b8964824ffac9d1e6b Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Fri, 22 Mar 2024 03:23:36 +0100 Subject: [PATCH 1/3] t7800: improve test descriptions with empty arguments Some of the tests in t7800 are executed repeatedly in a loop with different arguments. To distinguish these tests, the value of that variable is rendered into the test title. But given that one of the values is the empty string, it results in a somewhat awkward test name: difftool ignores exit code Improve this by printing "without options" in case the value is empty. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- t/t7800-difftool.sh | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh index 96ae5d5880..80bf108f54 100755 --- a/t/t7800-difftool.sh +++ b/t/t7800-difftool.sh @@ -93,40 +93,40 @@ test_expect_success 'difftool forwards arguments to diff' ' for opt in '' '--dir-diff' do - test_expect_success "difftool ${opt} ignores exit code" " + test_expect_success "difftool ${opt:-without options} ignores exit code" " test_config difftool.error.cmd false && git difftool ${opt} -y -t error branch " - test_expect_success "difftool ${opt} forwards exit code with --trust-exit-code" " + test_expect_success "difftool ${opt:-without options} forwards exit code with --trust-exit-code" " test_config difftool.error.cmd false && test_must_fail git difftool ${opt} -y --trust-exit-code -t error branch " - test_expect_success "difftool ${opt} forwards exit code with --trust-exit-code for built-ins" " + test_expect_success "difftool ${opt:-without options} forwards exit code with --trust-exit-code for built-ins" " test_config difftool.vimdiff.path false && test_must_fail git difftool ${opt} -y --trust-exit-code -t vimdiff branch " - test_expect_success "difftool ${opt} honors difftool.trustExitCode = true" " + test_expect_success "difftool ${opt:-without options} honors difftool.trustExitCode = true" " test_config difftool.error.cmd false && test_config difftool.trustExitCode true && test_must_fail git difftool ${opt} -y -t error branch " - test_expect_success "difftool ${opt} honors difftool.trustExitCode = false" " + test_expect_success "difftool ${opt:-without options} honors difftool.trustExitCode = false" " test_config difftool.error.cmd false && test_config difftool.trustExitCode false && git difftool ${opt} -y -t error branch " - test_expect_success "difftool ${opt} ignores exit code with --no-trust-exit-code" " + test_expect_success "difftool ${opt:-without options} ignores exit code with --no-trust-exit-code" " test_config difftool.error.cmd false && test_config difftool.trustExitCode true && git difftool ${opt} -y --no-trust-exit-code -t error branch " - test_expect_success "difftool ${opt} stops on error with --trust-exit-code" " + test_expect_success "difftool ${opt:-without options} stops on error with --trust-exit-code" " test_when_finished 'rm -f for-diff .git/fail-right-file' && test_when_finished 'git reset -- for-diff' && write_script .git/fail-right-file <<-\EOF && @@ -140,7 +140,7 @@ do test_line_count = 1 actual " - test_expect_success "difftool ${opt} honors exit status if command not found" " + test_expect_success "difftool ${opt:-without options} honors exit status if command not found" " test_config difftool.nonexistent.cmd i-dont-exist && test_config difftool.trustExitCode false && if test "${opt}" = '--dir-diff' From c559677c1fc45494ae45adafb35995992740e836 Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Fri, 22 Mar 2024 03:23:41 +0100 Subject: [PATCH 2/3] t7800: use single quotes for test bodies In eb84c8b6ce (git-difftool--helper: honor `--trust-exit-code` with `--dir-diff`, 2024-02-20) we have started to loop around some of the tests in t7800 so that they are reexecuted with slightly different arguments. As part of that refactoring the quoting of test bodies was changed from single quotes (') to double quotes (") so that the value of the loop variable is accessible to the body. As the test body is later on passed to eval this change was not required though. Let's revert it back to use single quotes as usual in our tests. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- t/t7800-difftool.sh | 40 ++++++++++++++++++++-------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh index 80bf108f54..cc917b257e 100755 --- a/t/t7800-difftool.sh +++ b/t/t7800-difftool.sh @@ -93,42 +93,42 @@ test_expect_success 'difftool forwards arguments to diff' ' for opt in '' '--dir-diff' do - test_expect_success "difftool ${opt:-without options} ignores exit code" " + test_expect_success "difftool ${opt:-without options} ignores exit code" ' test_config difftool.error.cmd false && git difftool ${opt} -y -t error branch - " + ' - test_expect_success "difftool ${opt:-without options} forwards exit code with --trust-exit-code" " + test_expect_success "difftool ${opt:-without options} forwards exit code with --trust-exit-code" ' test_config difftool.error.cmd false && test_must_fail git difftool ${opt} -y --trust-exit-code -t error branch - " + ' - test_expect_success "difftool ${opt:-without options} forwards exit code with --trust-exit-code for built-ins" " + test_expect_success "difftool ${opt:-without options} forwards exit code with --trust-exit-code for built-ins" ' test_config difftool.vimdiff.path false && test_must_fail git difftool ${opt} -y --trust-exit-code -t vimdiff branch - " + ' - test_expect_success "difftool ${opt:-without options} honors difftool.trustExitCode = true" " + test_expect_success "difftool ${opt:-without options} honors difftool.trustExitCode = true" ' test_config difftool.error.cmd false && test_config difftool.trustExitCode true && test_must_fail git difftool ${opt} -y -t error branch - " + ' - test_expect_success "difftool ${opt:-without options} honors difftool.trustExitCode = false" " + test_expect_success "difftool ${opt:-without options} honors difftool.trustExitCode = false" ' test_config difftool.error.cmd false && test_config difftool.trustExitCode false && git difftool ${opt} -y -t error branch - " + ' - test_expect_success "difftool ${opt:-without options} ignores exit code with --no-trust-exit-code" " + test_expect_success "difftool ${opt:-without options} ignores exit code with --no-trust-exit-code" ' test_config difftool.error.cmd false && test_config difftool.trustExitCode true && git difftool ${opt} -y --no-trust-exit-code -t error branch - " + ' - test_expect_success "difftool ${opt:-without options} stops on error with --trust-exit-code" " - test_when_finished 'rm -f for-diff .git/fail-right-file' && - test_when_finished 'git reset -- for-diff' && + test_expect_success "difftool ${opt:-without options} stops on error with --trust-exit-code" ' + test_when_finished "rm -f for-diff .git/fail-right-file" && + test_when_finished "git reset -- for-diff" && write_script .git/fail-right-file <<-\EOF && echo failed exit 1 @@ -138,19 +138,19 @@ do test_must_fail git difftool ${opt} -y --trust-exit-code \ --extcmd .git/fail-right-file branch >actual && test_line_count = 1 actual - " + ' - test_expect_success "difftool ${opt:-without options} honors exit status if command not found" " + test_expect_success "difftool ${opt:-without options} honors exit status if command not found" ' test_config difftool.nonexistent.cmd i-dont-exist && test_config difftool.trustExitCode false && - if test "${opt}" = '--dir-diff' + if test "${opt}" = --dir-diff then expected_code=127 else expected_code=128 fi && - test_expect_code \${expected_code} git difftool ${opt} -y -t nonexistent branch - " + test_expect_code ${expected_code} git difftool ${opt} -y -t nonexistent branch + ' done test_expect_success 'difftool honors --gui' ' From 7c4449eb31b60e9a39fa22ef8d18277f45610db2 Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Fri, 22 Mar 2024 03:23:46 +0100 Subject: [PATCH 3/3] t/README: document how to loop around test cases In some cases it makes sense to loop around test cases so that we can execute the same test with slightly different arguments. There are some gotchas around quoting here though that are easy to miss and that may lead to easy-to-miss errors and portability issues. Document the proper way to do this in "t/README". Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- t/README | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/t/README b/t/README index 36463d0742..4422ab9b98 100644 --- a/t/README +++ b/t/README @@ -721,6 +721,26 @@ The "do's:" Note that we still &&-chain the loop to propagate failures from earlier commands. + - Repeat tests with slightly different arguments in a loop. + + In some cases it may make sense to re-run the same set of tests with + different options or commands to ensure that the command behaves + despite the different parameters. This can be achieved by looping + around a specific parameter: + + for arg in '' "--foo" + do + test_expect_success "test command ${arg:-without arguments}" ' + command $arg + ' + done + + Note that while the test title uses double quotes ("), the test body + should continue to use single quotes (') to avoid breakage in case the + values contain e.g. quoting characters. The loop variable will be + accessible regardless of the single quotes as the test body is passed + to `eval`. + And here are the "don'ts:"