Allows N instances of tests run in parallel, each running 1/N parts
of the test suite under Valgrind, to speed things up.
* tr/test-v-and-v-subtest-only:
perf-lib: fix start/stop of perf tests
test-lib: support running tests under valgrind in parallel
test-lib: allow prefixing a custom string before "ok N" etc.
test-lib: valgrind for only tests matching a pattern
test-lib: verbose mode for only tests matching a pattern
test-lib: self-test that --verbose works
test-lib: rearrange start/end of test_expect_* and test_skip
test-lib: refactor $GIT_SKIP_TESTS matching
test-lib: enable MALLOC_* for the actual tests
With the new --valgrind-only=<pattern> option, one can enable
--valgrind at a per-test granularity, exactly analogous to
--verbose-only from the previous commit.
The options are wired such that --valgrind implies --verbose (as
before), but --valgrind-only=<pattern> implies
--verbose-only=<pattern> unless --verbose is also in effect.
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With the new --verbose-only=<pattern> option, one can enable --verbose
at a per-test granularity. The pattern is matched against the test
number, e.g.
./t0000-basic.sh --verbose-only='2[0-2]'
to see only the full output of test 20-22, while showing the rest in the
one-liner format.
As suggested by Jeff King, this takes care to wrap the entire
test_expect_* block, but nothing else, in the verbose toggling. We
can use the test_start/end functions from the previous commit for the
purpose.
This is arguably not *too* useful on its own, but makes the next patch
easier to follow.
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Many tests that check the behaviour of symbolic links stored in the
index or the tree objects do not have to be skipped on a filesystem
that lack symbolic link support.
* js/test-ln-s-add:
t4011: remove SYMLINKS prerequisite
t6035: use test_ln_s_add to remove SYMLINKS prerequisite
t3509, t4023, t4114: use test_ln_s_add to remove SYMLINKS prerequisite
t3100: use test_ln_s_add to remove SYMLINKS prerequisite
t3030: use test_ln_s_add to remove SYMLINKS prerequisite
t0000: use test_ln_s_add to remove SYMLINKS prerequisite
tests: use test_ln_s_add to remove SYMLINKS prerequisite (trivial cases)
tests: introduce test_ln_s_add
t3010: modernize style
test-chmtime: Fix exit code on Windows
Add a new function that creates a symbolic link and adds it to the index
to be used in cases where a symbolic link is not required on the file
system. We will use it to remove many SYMLINKS prerequisites from test
cases.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When a test wants to make sure there is no <string> in an output
file, we should just say "! grep string output".
"test_must_fail" is there only to test Git command and catch unusual
deaths we know about (e.g. segv) as an error, not as an expected
failure. "test_must_fail grep string output" is unnecessary, as
we are not making sure the system binaries do not dump core or
anything like that.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
Typo fix: replacing it's -> its
t: make PIPE a standard test prerequisite
archive: clarify explanation of --worktree-attributes
t/README: --immediate skips cleanup commands for failed tests
The 'PIPE' test prerequisite was already defined identically by t9010
and t9300, therefore it makes sense to make it a predefined
prerequisite.
Signed-off-by: Adam Spiers <git@adamspiers.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With --valgrind=memcheck-fast, the tests run under memcheck but
without the autodetected --track-origins. If you just run valgrind to
see *if* there is any memory issue with your program, the extra
information is not needed, and it comes at a roughly 30% hit in
runtime.
While it is possible to achieve the same through GIT_VALGRIND_OPTIONS,
this should be more discoverable and hopefully encourage more users to
run their tests with valgrind.
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Acked-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Running tests under helgrind and DRD recently proved useful in
tracking down thread interaction issues. This can unfortunately not
be done through GIT_VALGRIND_OPTIONS because any tool other than
memcheck would complain about unknown options.
Let --valgrind take an optional parameter that describes the valgrind
tool to invoke. The default mode is to run memcheck as before.
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Acked-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This was missed in 3da9365 (Tests: let --valgrind imply --verbose and
--tee, 2009-02-04).
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Acked-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
UTF8 behaviour of the filesystem (conversion from nfd to nfc) plays a
role in several tests and is tested in several tests. Therefore, move
the test from t0050 into the test lib and use the prerequisite in t0050.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Case insensitivity plays a role in several tests and is tested in several
tests. Therefore, move the test from t003 into the test lib and use the
prerequisite in t0003.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a few more advices that we often have to give to new test
writers.
Also update an example where a double quote pair is used to enclose
a test body to use a single quote pair, which is more readable and
more importantly gives saner semantics for variable substitution.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
I'm no longer running the Git smoke testing service at
smoke.git.nix.is due to Smolder being a fragile piece of software not
having time to follow through on making it easy for third parties to
run and submit their own smoke tests.
So remove the support in Git for sending smoke tests to
smoke.git.nix.is, it's still easy to modify the test suite to submit
smokes somewhere else.
This reverts the following commits:
Revert "t/README: Add SMOKE_{COMMENT,TAGS}= to smoke_report target" -- e38efac87d
Revert "t/README: Document the Smoke testing" -- d15e9ebc5c
Revert "t/Makefile: Create test-results dir for smoke target" -- 617344d77b
Revert "tests: Infrastructure for Git smoke testing" -- b6b84d1b74
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since 781f76b15 (test-lib: redirect stdin of tests) you can't simply put a
"bash &&" into a test for debugging purposes anymore. Instead you'll have
to use "bash <&6 >&3 2>&4".
As that invocation is not that easy to remember add the test_pause
convenience function. It invokes "$SHELL_PATH" to provide a sane shell
for the user.
This function also checks if the -v flag is given and will error out if
that is not the case instead of letting the test hang until ^D is pressed.
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* mk/grep-pcre:
git-grep: Fix problems with recently added tests
git-grep: Update tests (mainly for -P)
Makefile: Pass USE_LIBPCRE down in GIT-BUILD-OPTIONS
git-grep: update tests now regexp type is "last one wins"
git-grep: do not die upon -F/-P when grep.extendedRegexp is set.
git-grep: Bail out when -P is used with -F or -E
grep: Add basic tests
configure: Check for libpcre
git-grep: Learn PCRE
grep: Extract compile_regexp_failed() from compile_regexp()
grep: Fix a typo in a comment
grep: Put calls to fixmatch() and regmatch() into patmatch()
contrib/completion: --line-number to git grep
Documentation: Add --line-number to git-grep synopsis
This modest patch adds simple tests for git grep -P/--perl-regexp and
its interoperation with -i and -w.
Tests are only enabled when prerequisite LIBPCRE is defined (it's
automatically set based on USE_LIBPCRE in test-lib.sh).
Signed-off-by: Michał Kiedrowicz <michal.kiedrowicz@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Document all test function arguments in the same way.
While at it, tweak the description of test_path_is_* (thanks to Junio),
and correct some grammatical errors.
Signed-off-by: Mathias Lafeldt <misfire@debugon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
contrib/thunderbird-patch-inline: do not require bash to run the script
t8001: check the exit status of the command being tested
strbuf.h: remove a tad stale docs-in-comment and reference api-doc instead
Typos: t/README
Documentation/config.txt: make truth value of numbers more explicit
git-pack-objects.txt: fix grammatical errors
parse-remote: replace unnecessary sed invocation
* maint:
Update draft release notes to 1.7.4.2
Work around broken ln on solaris as used in t8006
t/README: Add a note about running commands under valgrind
The test suite runs valgrind with certain options activated. Add a
note saying how to run commands under the same conditions as the test
suite does.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This adds just a "do it this way" instruction without a lot of explanation,
because the details are too complex to be explained at this point.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some tests check their output with code like the following:
test "$(git ls-files -u B | wc -l)" -eq 3 || {
echo "BAD: should have left stages for B"
return 1
}
The verbose failure condition is used because test, unlike
diff, does not print any useful information about the
nature of the failure when it fails.
Introduce a test_line_count function to help. If used like
git ls-files -u B >output &&
test_line_count -eq 3 output
it will produce output like
test_line_count: line count for output !-eq 3
100644 b023018cabc396e7692c70bbf5784a93d3f738ab 2 hi.c
100644 45b983be36b73c0788dc9cbcb76cbb80fc7bb057 3 hi.c
on failure.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
You can run "make DEFAULT_TEST_TARGET=prove test" to run the test under
"prove" (or $(PROVE) if set). The output is a bit easier to read when
running many tests in parallel.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Liked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Liked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change test_expect_code to be a normal test command instead of a
top-level command.
As a top-level command it would fail in cases like:
test_expect_code 1 'phoney' '
foo && bar && (exit 1)
'
Here the test might incorrectly succeed if "foo" or "bar" happened to
fail with exit status 1. Instead we now do:
test_expect_success 'phoney' '
foo && bar && test_expect_code 1 "(exit 1)"
'
Which will only succeed if "foo" and "bar" return status 0, and "(exit
1)" returns status 1. Note that test_expect_code has been made slightly
noisier, as it reports the exit code it receives even upon success.
Some test code in t0000-basic.sh relied on the old semantics of
test_expect_code to test the test_when_finished command. I've
converted that code to use an external test similar to the TODO test I
added in v1.7.3-rc0~2^2~3.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Acked-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* ab/test-2: (51 commits)
tests: factor HOME=$(pwd) in test-lib.sh
test-lib: use subshell instead of cd $new && .. && cd $old
tests: simplify "missing PREREQ" message
t/t0000-basic.sh: Run the passing TODO test inside its own test-lib
test-lib: Allow overriding of TEST_DIRECTORY
test-lib: Use "$GIT_BUILD_DIR" instead of "$TEST_DIRECTORY"/../
test-lib: Use $TEST_DIRECTORY or $GIT_BUILD_DIR instead of $(pwd) and ../
test: Introduce $GIT_BUILD_DIR
cvs tests: do not touch test CVS repositories shipped with source
t/t9602-cvsimport-branches-tags.sh: Add a PERL prerequisite
t/t9601-cvsimport-vendor-branch.sh: Add a PERL prerequisite
t/t7105-reset-patch.sh: Add a PERL prerequisite
t/t9001-send-email.sh: convert setup code to tests
t/t9001-send-email.sh: change from skip_all=* to prereq skip
t/t9001-send-email.sh: Remove needless PROG=* assignment
t/t9600-cvsimport.sh: change from skip_all=* to prereq skip
lib-patch-mode tests: change from skip_all=* to prereq skip
t/t3701-add-interactive.sh: change from skip_all=* to prereq skip
tests: Move FILEMODE prerequisite to lib-prereq-FILEMODE.sh
t/Makefile: Create test-results dir for smoke target
...
Conflicts:
t/t6035-merge-dir-to-symlink.sh
The example I initially added to "Skipping tests" wasn't very
good. We'd rather skip tests using the three-arg prereq form to the
test_* functions, not bail out with a skip message.
Change the documentation to reflect that, but retain the bailout
example under a disclaimer which explains that it's probably not a
good idea to use it.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Having no coverage at all is almost always a bad sign, but trying to
attain 100% coverage everywhere is usually a waste of time. Add a
paragraph to explain this to future test writers.
Inspired-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Document how test writers can generate coverage reports, to ensure
that their tests are really testing the code they think they're
testing.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The smoke server supports a free form text field with comments about a
report, and a comma delimited list of tags. Change the smoke_report
target to expose this functionality. Now smokers can send more data
that explains and categorizes the reports they're submitting.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git now has a smoke testing service at http://smoke.git.nix.is that
anyone can send reports to. Change the t/README file to mention this.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some tests depend on not being able to write to files after chmod
-w. This doesn't work when running the tests as root.
Change test-lib.sh to test if this works, and if so it sets a new
SANITY test prerequisite. The tests that use this previously failed
when run under root.
There was already a test for this in t3600-rm.sh, added by Junio C
Hamano in 2283645 in 2006. That check now uses the new SANITY
prerequisite.
Some of this was resurrected from the "Tests in Cygwin" thread in May
2009:
http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118385
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The README for the test library suggested that you grep the
test-lib.sh for test_set_prereq to see what the preset prerequisites
were.
Remove that bit, and write a section explaining all the preset
prerequisites. Most of the text was lifted from from Junio C Hamano
and Johannes Sixt, See the "Tests in Cygwin" thread in May 2009 for
the originals:
http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118385http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118434
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the test_have_prereq function in test-lib.sh to support a
comma-separated list of prerequisites. This is useful for tests that
need e.g. both POSIXPERM and SANITY.
The implementation was stolen from Junio C Hamano and Johannes Sixt,
the tests and documentation were not. See the "Tests in Cygwin" thread
in May 2009 for the originals:
http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118385http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118434
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The helper functions are implemented, documented, and used in a few
places to validate them, but not everywhere to avoid useless code churn.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The correct advice should have been taken from c289c31 (t/t7006: ignore
return status of shell's unset builtin, 2010-06-02). A real-life issue
we experienced was with "unset", not with "export" (exporting an
unset variable may have similar portability issues, though).
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some have found the wording of the description to be somewhat ambiguous
with respect to when it is desirable to use test_must_fail instead of
"! <git-command>". Tweak the wording somewhat to hopefully clarify that
it is _because_ test_must_fail can detect segmentation fault that it is
desirable to use it instead of "! <git-command>".
Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There is no documentation in t/README for test_must_fail,
test_might_fail, test_cmp, or test_when_finished.
Reported-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a "Do's, don'ts & things to keep in mind" subsection to the
"Writing Tests" documentation. Much of this is based on Junio C
Hamano's "Test your stuff" section in
<7vhbkj2kcr.fsf@alter.siamese.dyndns.org>.
I turned it into a list of do's and don'ts to make it easier to skim
it, and integrated my note that a TAP harness will get confused if you
print "ok" or "not ok" at the beginning of a line.
Thad had to be fixed in 335f87871fe5aa6b3fd55b2b4e80f16fe9681483 when
TAP support was introduced.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
