1
0
Fork 0
mirror of https://github.com/git/git.git synced 2024-05-06 18:06:13 +02:00
git/ci/print-test-failures.sh

95 lines
2.6 KiB
Bash
Raw Normal View History

#!/bin/sh
#
# Print output of failing tests
#
. ${0%/*}/lib.sh
# Tracing executed commands would produce too much noise in the loop below.
set +x
cd "${TEST_OUTPUT_DIRECTORY:-t/}"
travis-ci: include the trash directories of failed tests in the trace log The trash directory of a failed test might contain invaluable information about the cause of the failure, but we have no access to the trash directories of Travis CI build jobs. The only feedback we get from there is the build job's trace log, so... Modify 'ci/print-test-failures.sh' to create a tar.gz archive of the trash directory of each failed test, encode that archive with base64, and print the resulting block of ASCII text, so it gets embedded in the trace log. Furthermore, run tests with '--immediate' to faithfully preserve the failed state. Extracting the trash directories from the trace log turned out to be a bit of a hassle, partly because of the size of these logs (usually resulting in several hundreds or even thousands of lines of base64-encoded text), and partly because these logs have CRLF, CRCRLF and occasionally even CRCRCRLF line endings, which cause 'base64 -d' from coreutils to complain about "invalid input". For convenience add a small script 'ci/util/extract-trash-dirs.sh', which will extract and unpack all base64-encoded trash directories embedded in the log fed to its standard input, and include an example command to be copy-pasted into a terminal to do it all at the end of the failure report. A few of our tests create sizeable trash directories, so limit the size of each included base64-encoded block, let's say, to 1MB. And just in case something fundamental gets broken and a lot of tests fail at once, don't include trash directories when the combined size of the included base64-encoded blocks would exceed 1MB. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-01 00:56:12 +02:00
if ! ls test-results/*.exit >/dev/null 2>/dev/null
then
echo "Build job failed before the tests could have been run"
exit
fi
travis-ci: include the trash directories of failed tests in the trace log The trash directory of a failed test might contain invaluable information about the cause of the failure, but we have no access to the trash directories of Travis CI build jobs. The only feedback we get from there is the build job's trace log, so... Modify 'ci/print-test-failures.sh' to create a tar.gz archive of the trash directory of each failed test, encode that archive with base64, and print the resulting block of ASCII text, so it gets embedded in the trace log. Furthermore, run tests with '--immediate' to faithfully preserve the failed state. Extracting the trash directories from the trace log turned out to be a bit of a hassle, partly because of the size of these logs (usually resulting in several hundreds or even thousands of lines of base64-encoded text), and partly because these logs have CRLF, CRCRLF and occasionally even CRCRCRLF line endings, which cause 'base64 -d' from coreutils to complain about "invalid input". For convenience add a small script 'ci/util/extract-trash-dirs.sh', which will extract and unpack all base64-encoded trash directories embedded in the log fed to its standard input, and include an example command to be copy-pasted into a terminal to do it all at the end of the failure report. A few of our tests create sizeable trash directories, so limit the size of each included base64-encoded block, let's say, to 1MB. And just in case something fundamental gets broken and a lot of tests fail at once, don't include trash directories when the combined size of the included base64-encoded blocks would exceed 1MB. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-01 00:56:12 +02:00
case "$jobname" in
osx-clang|osx-gcc)
# base64 in OSX doesn't wrap its output at 76 columns by
# default, but prints a single, very long line.
base64_opts="-b 76"
;;
esac
combined_trash_size=0
for TEST_EXIT in test-results/*.exit
do
if [ "$(cat "$TEST_EXIT")" != "0" ]
then
TEST_OUT="${TEST_EXIT%exit}out"
echo "------------------------------------------------------------------------"
echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
echo "------------------------------------------------------------------------"
cat "${TEST_OUT}"
travis-ci: include the trash directories of failed tests in the trace log The trash directory of a failed test might contain invaluable information about the cause of the failure, but we have no access to the trash directories of Travis CI build jobs. The only feedback we get from there is the build job's trace log, so... Modify 'ci/print-test-failures.sh' to create a tar.gz archive of the trash directory of each failed test, encode that archive with base64, and print the resulting block of ASCII text, so it gets embedded in the trace log. Furthermore, run tests with '--immediate' to faithfully preserve the failed state. Extracting the trash directories from the trace log turned out to be a bit of a hassle, partly because of the size of these logs (usually resulting in several hundreds or even thousands of lines of base64-encoded text), and partly because these logs have CRLF, CRCRLF and occasionally even CRCRCRLF line endings, which cause 'base64 -d' from coreutils to complain about "invalid input". For convenience add a small script 'ci/util/extract-trash-dirs.sh', which will extract and unpack all base64-encoded trash directories embedded in the log fed to its standard input, and include an example command to be copy-pasted into a terminal to do it all at the end of the failure report. A few of our tests create sizeable trash directories, so limit the size of each included base64-encoded block, let's say, to 1MB. And just in case something fundamental gets broken and a lot of tests fail at once, don't include trash directories when the combined size of the included base64-encoded blocks would exceed 1MB. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-01 00:56:12 +02:00
test_name="${TEST_EXIT%.exit}"
test_name="${test_name##*/}"
trash_dir="trash directory.$test_name"
case "$CI_TYPE" in
azure-pipelines)
mkdir -p failed-test-artifacts
mv "$trash_dir" failed-test-artifacts
continue
;;
github-actions)
mkdir -p failed-test-artifacts
echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
continue
;;
ci: add support for GitLab CI We already support Azure Pipelines and GitHub Workflows in the Git project, but until now we do not have support for GitLab CI. While it is arguably not in the interest of the Git project to maintain a ton of different CI platforms, GitLab has recently ramped up its efforts and tries to contribute to the Git project more regularly. Part of a problem we hit at GitLab rather frequently is that our own, custom CI setup we have is so different to the setup that the Git project has. More esoteric jobs like "linux-TEST-vars" that also set a couple of environment variables do not exist in GitLab's custom CI setup, and maintaining them to keep up with what Git does feels like wasted time. The result is that we regularly send patch series upstream that fail to compile or pass tests in GitHub Workflows. We would thus like to integrate the GitLab CI configuration into the Git project to help us send better patch series upstream and thus reduce overhead for the maintainer. Results of these pipeline runs will be made available (at least) in GitLab's mirror of the Git project at [1]. This commit introduces the integration into our regular CI scripts so that most of the setup continues to be shared across all of the CI solutions. Note that as the builds on GitLab CI run as unprivileged user, we need to pull in both sudo and shadow packages to our Alpine based job to set this up. [1]: https://gitlab.com/gitlab-org/git Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-09 09:05:54 +01:00
gitlab-ci)
mkdir -p failed-test-artifacts
cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
continue
;;
*)
echo "Unhandled CI type: $CI_TYPE" >&2
exit 1
;;
esac
travis-ci: include the trash directories of failed tests in the trace log The trash directory of a failed test might contain invaluable information about the cause of the failure, but we have no access to the trash directories of Travis CI build jobs. The only feedback we get from there is the build job's trace log, so... Modify 'ci/print-test-failures.sh' to create a tar.gz archive of the trash directory of each failed test, encode that archive with base64, and print the resulting block of ASCII text, so it gets embedded in the trace log. Furthermore, run tests with '--immediate' to faithfully preserve the failed state. Extracting the trash directories from the trace log turned out to be a bit of a hassle, partly because of the size of these logs (usually resulting in several hundreds or even thousands of lines of base64-encoded text), and partly because these logs have CRLF, CRCRLF and occasionally even CRCRCRLF line endings, which cause 'base64 -d' from coreutils to complain about "invalid input". For convenience add a small script 'ci/util/extract-trash-dirs.sh', which will extract and unpack all base64-encoded trash directories embedded in the log fed to its standard input, and include an example command to be copy-pasted into a terminal to do it all at the end of the failure report. A few of our tests create sizeable trash directories, so limit the size of each included base64-encoded block, let's say, to 1MB. And just in case something fundamental gets broken and a lot of tests fail at once, don't include trash directories when the combined size of the included base64-encoded blocks would exceed 1MB. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-01 00:56:12 +02:00
trash_tgz_b64="trash.$test_name.base64"
if [ -d "$trash_dir" ]
then
tar czp "$trash_dir" |base64 $base64_opts >"$trash_tgz_b64"
trash_size=$(wc -c <"$trash_tgz_b64")
if [ $trash_size -gt 1048576 ]
then
# larger than 1MB
echo "$(tput setaf 1)Didn't include the trash directory of '$test_name' in the trace log, it's too big$(tput sgr0)"
continue
fi
new_combined_trash_size=$(($combined_trash_size + $trash_size))
if [ $new_combined_trash_size -gt 1048576 ]
then
echo "$(tput setaf 1)Didn't include the trash directory of '$test_name' in the trace log, there is plenty of trash in there already.$(tput sgr0)"
continue
fi
combined_trash_size=$new_combined_trash_size
# DO NOT modify these two 'echo'-ed strings below
# without updating 'ci/util/extract-trash-dirs.sh'
# as well.
echo "$(tput setaf 1)Start of trash directory of '$test_name':$(tput sgr0)"
cat "$trash_tgz_b64"
echo "$(tput setaf 1)End of trash directory of '$test_name'$(tput sgr0)"
fi
fi
done