mirror of
https://github.com/git/git.git
synced 2024-05-19 14:06:37 +02:00
e88842ee1c
Replace the "grep" we run to exclude certain programs from the
generated output with a pure-shell loop that strips out the comments,
and sees if the "cmd" we're reading is on a list of excluded
programs. This uses a trick similar to test_have_prereq() in
test-lib-functions.sh.
On my *nix system this makes things quite a bit slower compared to
HEAD~:
o
'sh generate-cmdlist.sh.old command-list.txt' ran
1.56 ± 0.11 times faster than 'sh generate-cmdlist.sh command-list.txt'
18.00 ± 0.19 times faster than 'sh generate-cmdlist.sh.master command-list.txt'
But when I tried running generate-cmdlist.sh 100 times in CI I found
that it helped across the board even on OSX & Linux. I tried testing
it in CI with this ad-hoc few-liner:
for i in $(seq -w 0 11 | sort -nr)
do
git show HEAD~$i:generate-cmdlist.sh >generate-cmdlist-HEAD$i.sh &&
git add generate-cmdlist* &&
cp t/t0000-generate-cmdlist.sh t/t00$i-generate-cmdlist.sh || : &&
perl -pi -e "s/HEAD0/HEAD$i/g" t/t00$i-generate-cmdlist.sh &&
git add t/t00*.sh
done && git commit -m"generated it"
Here HEAD~02 and the t0002* file refers to this change, and HEAD~03
and t0003* file to the preceding commit, the relevant results were:
linux-gcc:
[12:05:33] t0002-generate-cmdlist.sh .. ok 14 ms ( 0.00 usr 0.00 sys + 3.64 cusr 3.09 csys = 6.73 CPU)
[12:05:30] t0003-generate-cmdlist.sh .. ok 32 ms ( 0.00 usr 0.00 sys + 2.66 cusr 1.81 csys = 4.47 CPU)
osx-gcc:
[11:58:04] t0002-generate-cmdlist.sh .. ok 80081 ms ( 0.02 usr 0.02 sys + 17.80 cusr 10.07 csys = 27.91 CPU)
[11:58:16] t0003-generate-cmdlist.sh .. ok 92127 ms ( 0.02 usr 0.01 sys + 22.54 cusr 14.27 csys = 36.84 CPU)
vs-test:
[12:03:14] t0002-generate-cmdlist.sh .. ok 30 s ( 0.02 usr 0.00 sys + 13.14 cusr 26.19 csys = 39.35 CPU)
[12:03:20] t0003-generate-cmdlist.sh .. ok 32 s ( 0.00 usr 0.02 sys + 13.25 cusr 26.10 csys = 39.37 CPU)
I.e. even on *nix running 100 of these in a loop was up to ~2x faster
in absolute runtime, I suspect it's due factors that are exacerbated
in the CI, e.g. much slower process startup due to some platform
limits, or a slower FS.
The "cut -d" change here is because we're not emitting the
40-character aligned output anymore, i.e. we'll get the output from
command_list() now, not an as-is line from command-list.txt.
This also makes the parsing more reliable, as we could tweak the
whitespace alignment without breaking this parser. Let's reword a
now-inaccurate comment in "command-list.txt" describing that previous
alignment limitation. We'll still need the "### command-list [...]"
line due to the "Documentation/cmd-list.perl" logic added in
11c6659d85 (command-list: prepare machinery for upcoming "common
groups" section, 2015-05-21).
There was a proposed change subsequent to this one[3] which continued
moving more logic into the "command_list() function, i.e. replaced the
"cut | tr | grep" chain in "category_list()" with an argument to
"command_list()".
That change might have had a bit of an effect, but not as much as the
preceding commit, so I decided to drop it. The relevant performance
numbers from it were:
linux-gcc:
[12:05:33] t0001-generate-cmdlist.sh .. ok 13 ms ( 0.00 usr 0.00 sys + 3.33 cusr 2.78 csys = 6.11 CPU)
[12:05:33] t0002-generate-cmdlist.sh .. ok 14 ms ( 0.00 usr 0.00 sys + 3.64 cusr 3.09 csys = 6.73 CPU)
osx-gcc:
[11:58:03] t0001-generate-cmdlist.sh .. ok 78416 ms ( 0.02 usr 0.01 sys + 11.78 cusr 6.22 csys = 18.03 CPU)
[11:58:04] t0002-generate-cmdlist.sh .. ok 80081 ms ( 0.02 usr 0.02 sys + 17.80 cusr 10.07 csys = 27.91 CPU)
vs-test:
[12:03:20] t0001-generate-cmdlist.sh .. ok 34 s ( 0.00 usr 0.03 sys + 12.42 cusr 19.55 csys = 32.00 CPU)
[12:03:14] t0002-generate-cmdlist.sh .. ok 30 s ( 0.02 usr 0.00 sys + 13.14 cusr 26.19 csys = 39.35 CPU)
As above HEAD~2 and t0002* are testing the code in this commit (and
the line is the same), but HEAD~1 and t0001* are testing that dropped
change in [3].
1. https://lore.kernel.org/git/cover-v2-00.10-00000000000-20211022T193027Z-avarab@gmail.com/
2. https://lore.kernel.org/git/patch-v2-08.10-83318d6c0da-20211022T193027Z-avarab@gmail.com/
3. https://lore.kernel.org/git/patch-v2-10.10-e10a43756d1-20211022T193027Z-avarab@gmail.com/
Signed-off-by: Junio C Hamano <gitster@pobox.com>
108 lines
1.6 KiB
Bash
Executable File
108 lines
1.6 KiB
Bash
Executable File
#!/bin/sh
|
|
|
|
die () {
|
|
echo "$@" >&2
|
|
exit 1
|
|
}
|
|
|
|
command_list () {
|
|
while read cmd rest
|
|
do
|
|
case "$cmd" in
|
|
"#"* | '')
|
|
# Ignore comments and allow empty lines
|
|
continue
|
|
;;
|
|
*)
|
|
case "$exclude_programs" in
|
|
*":$cmd:"*)
|
|
;;
|
|
*)
|
|
echo "$cmd $rest"
|
|
;;
|
|
esac
|
|
esac
|
|
done <"$1"
|
|
}
|
|
|
|
category_list () {
|
|
command_list "$1" |
|
|
cut -d' ' -f2- |
|
|
tr ' ' '\012' |
|
|
grep -v '^$' |
|
|
LC_ALL=C sort -u
|
|
}
|
|
|
|
define_categories () {
|
|
echo
|
|
echo "/* Command categories */"
|
|
bit=0
|
|
category_list "$1" |
|
|
while read cat
|
|
do
|
|
echo "#define CAT_$cat (1UL << $bit)"
|
|
bit=$(($bit+1))
|
|
done
|
|
test "$bit" -gt 32 && die "Urgh.. too many categories?"
|
|
}
|
|
|
|
define_category_names () {
|
|
echo
|
|
echo "/* Category names */"
|
|
echo "static const char *category_names[] = {"
|
|
bit=0
|
|
category_list "$1" |
|
|
while read cat
|
|
do
|
|
echo " \"$cat\", /* (1UL << $bit) */"
|
|
bit=$(($bit+1))
|
|
done
|
|
echo " NULL"
|
|
echo "};"
|
|
}
|
|
|
|
print_command_list () {
|
|
echo "static struct cmdname_help command_list[] = {"
|
|
|
|
command_list "$1" |
|
|
while read cmd rest
|
|
do
|
|
synopsis=
|
|
while read line
|
|
do
|
|
case "$line" in
|
|
"$cmd - "*)
|
|
synopsis=${line#$cmd - }
|
|
break
|
|
;;
|
|
esac
|
|
done <"Documentation/$cmd.txt"
|
|
|
|
printf '\t{ "%s", N_("%s"), 0' "$cmd" "$synopsis"
|
|
printf " | CAT_%s" $rest
|
|
echo " },"
|
|
done
|
|
echo "};"
|
|
}
|
|
|
|
exclude_programs=:
|
|
while test "--exclude-program" = "$1"
|
|
do
|
|
shift
|
|
exclude_programs="$exclude_programs$1:"
|
|
shift
|
|
done
|
|
|
|
echo "/* Automatically generated by generate-cmdlist.sh */
|
|
struct cmdname_help {
|
|
const char *name;
|
|
const char *help;
|
|
uint32_t category;
|
|
};
|
|
"
|
|
define_categories "$1"
|
|
echo
|
|
define_category_names "$1"
|
|
echo
|
|
print_command_list "$1"
|