1
0
Fork 0
mirror of https://github.com/git/git.git synced 2024-05-18 13:36:26 +02:00
git/t/helper/test-path-utils.c

506 lines
13 KiB
C
Raw Normal View History

#include "test-tool.h"
#include "cache.h"
#include "abspath.h"
#include "environment.h"
#include "setup.h"
#include "string-list.h"
#include "trace.h"
#include "utf8.h"
longest_ancestor_length(): require prefix list entries to be normalized Move the responsibility for normalizing prefixes from longest_ancestor_length() to its callers. Use slightly different normalizations at the two callers: In setup_git_directory_gently_1(), use the old normalization, which ignores paths that are not usable. In the next commit we will change this caller to also resolve symlinks in the paths from GIT_CEILING_DIRECTORIES as part of the normalization. In "test-path-utils longest_ancestor_length", use the old normalization, but die() if any paths are unusable. Also change t0060 to only pass normalized paths to the test program (no empty entries or non-absolute paths, strip trailing slashes from the paths, and remove tests that thereby become redundant). The point of this change is to reduce the scope of the ancestor_length tests in t0060 from testing normalization+longest_prefix to testing only mostly longest_prefix. This is necessary because when setup_git_directory_gently_1() starts resolving symlinks as part of its normalization, it will not be reasonable to do the same in the test suite, because that would make the test results depend on the contents of the root directory of the filesystem on which the test is run. HOWEVER: under Windows, bash mangles arguments that look like absolute POSIX paths into DOS paths. So we have to retain the level of normalization done by normalize_path_copy() to convert the bash-mangled DOS paths (which contain backslashes) into paths that use forward slashes. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jeff King <peff@peff.net>
2012-10-28 17:16:25 +01:00
/*
* A "string_list_each_func_t" function that normalizes an entry from
* GIT_CEILING_DIRECTORIES. If the path is unusable for some reason,
* die with an explanation.
*/
static int normalize_ceiling_entry(struct string_list_item *item,
void *data UNUSED)
longest_ancestor_length(): require prefix list entries to be normalized Move the responsibility for normalizing prefixes from longest_ancestor_length() to its callers. Use slightly different normalizations at the two callers: In setup_git_directory_gently_1(), use the old normalization, which ignores paths that are not usable. In the next commit we will change this caller to also resolve symlinks in the paths from GIT_CEILING_DIRECTORIES as part of the normalization. In "test-path-utils longest_ancestor_length", use the old normalization, but die() if any paths are unusable. Also change t0060 to only pass normalized paths to the test program (no empty entries or non-absolute paths, strip trailing slashes from the paths, and remove tests that thereby become redundant). The point of this change is to reduce the scope of the ancestor_length tests in t0060 from testing normalization+longest_prefix to testing only mostly longest_prefix. This is necessary because when setup_git_directory_gently_1() starts resolving symlinks as part of its normalization, it will not be reasonable to do the same in the test suite, because that would make the test results depend on the contents of the root directory of the filesystem on which the test is run. HOWEVER: under Windows, bash mangles arguments that look like absolute POSIX paths into DOS paths. So we have to retain the level of normalization done by normalize_path_copy() to convert the bash-mangled DOS paths (which contain backslashes) into paths that use forward slashes. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jeff King <peff@peff.net>
2012-10-28 17:16:25 +01:00
{
char *ceil = item->string;
longest_ancestor_length(): require prefix list entries to be normalized Move the responsibility for normalizing prefixes from longest_ancestor_length() to its callers. Use slightly different normalizations at the two callers: In setup_git_directory_gently_1(), use the old normalization, which ignores paths that are not usable. In the next commit we will change this caller to also resolve symlinks in the paths from GIT_CEILING_DIRECTORIES as part of the normalization. In "test-path-utils longest_ancestor_length", use the old normalization, but die() if any paths are unusable. Also change t0060 to only pass normalized paths to the test program (no empty entries or non-absolute paths, strip trailing slashes from the paths, and remove tests that thereby become redundant). The point of this change is to reduce the scope of the ancestor_length tests in t0060 from testing normalization+longest_prefix to testing only mostly longest_prefix. This is necessary because when setup_git_directory_gently_1() starts resolving symlinks as part of its normalization, it will not be reasonable to do the same in the test suite, because that would make the test results depend on the contents of the root directory of the filesystem on which the test is run. HOWEVER: under Windows, bash mangles arguments that look like absolute POSIX paths into DOS paths. So we have to retain the level of normalization done by normalize_path_copy() to convert the bash-mangled DOS paths (which contain backslashes) into paths that use forward slashes. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jeff King <peff@peff.net>
2012-10-28 17:16:25 +01:00
if (!*ceil)
longest_ancestor_length(): require prefix list entries to be normalized Move the responsibility for normalizing prefixes from longest_ancestor_length() to its callers. Use slightly different normalizations at the two callers: In setup_git_directory_gently_1(), use the old normalization, which ignores paths that are not usable. In the next commit we will change this caller to also resolve symlinks in the paths from GIT_CEILING_DIRECTORIES as part of the normalization. In "test-path-utils longest_ancestor_length", use the old normalization, but die() if any paths are unusable. Also change t0060 to only pass normalized paths to the test program (no empty entries or non-absolute paths, strip trailing slashes from the paths, and remove tests that thereby become redundant). The point of this change is to reduce the scope of the ancestor_length tests in t0060 from testing normalization+longest_prefix to testing only mostly longest_prefix. This is necessary because when setup_git_directory_gently_1() starts resolving symlinks as part of its normalization, it will not be reasonable to do the same in the test suite, because that would make the test results depend on the contents of the root directory of the filesystem on which the test is run. HOWEVER: under Windows, bash mangles arguments that look like absolute POSIX paths into DOS paths. So we have to retain the level of normalization done by normalize_path_copy() to convert the bash-mangled DOS paths (which contain backslashes) into paths that use forward slashes. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jeff King <peff@peff.net>
2012-10-28 17:16:25 +01:00
die("Empty path is not supported");
if (!is_absolute_path(ceil))
die("Path \"%s\" is not absolute", ceil);
if (normalize_path_copy(ceil, ceil) < 0)
longest_ancestor_length(): require prefix list entries to be normalized Move the responsibility for normalizing prefixes from longest_ancestor_length() to its callers. Use slightly different normalizations at the two callers: In setup_git_directory_gently_1(), use the old normalization, which ignores paths that are not usable. In the next commit we will change this caller to also resolve symlinks in the paths from GIT_CEILING_DIRECTORIES as part of the normalization. In "test-path-utils longest_ancestor_length", use the old normalization, but die() if any paths are unusable. Also change t0060 to only pass normalized paths to the test program (no empty entries or non-absolute paths, strip trailing slashes from the paths, and remove tests that thereby become redundant). The point of this change is to reduce the scope of the ancestor_length tests in t0060 from testing normalization+longest_prefix to testing only mostly longest_prefix. This is necessary because when setup_git_directory_gently_1() starts resolving symlinks as part of its normalization, it will not be reasonable to do the same in the test suite, because that would make the test results depend on the contents of the root directory of the filesystem on which the test is run. HOWEVER: under Windows, bash mangles arguments that look like absolute POSIX paths into DOS paths. So we have to retain the level of normalization done by normalize_path_copy() to convert the bash-mangled DOS paths (which contain backslashes) into paths that use forward slashes. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jeff King <peff@peff.net>
2012-10-28 17:16:25 +01:00
die("Path \"%s\" could not be normalized", ceil);
return 1;
}
static void normalize_argv_string(const char **var, const char *input)
{
if (!strcmp(input, "<null>"))
*var = NULL;
else if (!strcmp(input, "<empty>"))
*var = "";
else
*var = input;
if (*var && (**var == '<' || **var == '('))
die("Bad value: %s\n", input);
}
struct test_data {
const char *from; /* input: transform from this ... */
const char *to; /* output: ... to this. */
const char *alternative; /* output: ... or this. */
};
/*
* Compatibility wrappers for OpenBSD, whose basename(3) and dirname(3)
* have const parameters.
*/
static char *posix_basename(char *path)
{
return basename(path);
}
static char *posix_dirname(char *path)
{
return dirname(path);
}
static int test_function(struct test_data *data, char *(*func)(char *input),
const char *funcname)
{
int failed = 0, i;
char buffer[1024];
char *to;
for (i = 0; data[i].to; i++) {
if (!data[i].from)
to = func(NULL);
else {
xsnprintf(buffer, sizeof(buffer), "%s", data[i].from);
to = func(buffer);
}
if (!strcmp(to, data[i].to))
continue;
if (!data[i].alternative)
error("FAIL: %s(%s) => '%s' != '%s'\n",
funcname, data[i].from, to, data[i].to);
else if (!strcmp(to, data[i].alternative))
continue;
else
error("FAIL: %s(%s) => '%s' != '%s', '%s'\n",
funcname, data[i].from, to, data[i].to,
data[i].alternative);
failed = 1;
}
return failed;
}
static struct test_data basename_data[] = {
/* --- POSIX type paths --- */
{ NULL, "." },
{ "", "." },
{ ".", "." },
{ "..", ".." },
{ "/", "/" },
{ "//", "/", "//" },
{ "///", "/", "//" },
{ "////", "/", "//" },
{ "usr", "usr" },
{ "/usr", "usr" },
{ "/usr/", "usr" },
{ "/usr//", "usr" },
{ "/usr/lib", "lib" },
{ "usr/lib", "lib" },
{ "usr/lib///", "lib" },
#if defined(__MINGW32__) || defined(_MSC_VER)
/* --- win32 type paths --- */
{ "\\usr", "usr" },
{ "\\usr\\", "usr" },
{ "\\usr\\\\", "usr" },
{ "\\usr\\lib", "lib" },
{ "usr\\lib", "lib" },
{ "usr\\lib\\\\\\", "lib" },
{ "C:/usr", "usr" },
{ "C:/usr", "usr" },
{ "C:/usr/", "usr" },
{ "C:/usr//", "usr" },
{ "C:/usr/lib", "lib" },
{ "C:usr/lib", "lib" },
{ "C:usr/lib///", "lib" },
{ "C:", "." },
{ "C:a", "a" },
{ "C:/", "/" },
{ "C:///", "/" },
{ "\\", "\\", "/" },
{ "\\\\", "\\", "/" },
{ "\\\\\\", "\\", "/" },
#endif
{ NULL, NULL }
};
static struct test_data dirname_data[] = {
/* --- POSIX type paths --- */
{ NULL, "." },
{ "", "." },
{ ".", "." },
{ "..", "." },
{ "/", "/" },
{ "//", "/", "//" },
{ "///", "/", "//" },
{ "////", "/", "//" },
{ "usr", "." },
{ "/usr", "/" },
{ "/usr/", "/" },
{ "/usr//", "/" },
{ "/usr/lib", "/usr" },
{ "usr/lib", "usr" },
{ "usr/lib///", "usr" },
#if defined(__MINGW32__) || defined(_MSC_VER)
/* --- win32 type paths --- */
{ "\\", "\\" },
{ "\\\\", "\\\\" },
{ "\\usr", "\\" },
{ "\\usr\\", "\\" },
{ "\\usr\\\\", "\\" },
{ "\\usr\\lib", "\\usr" },
{ "usr\\lib", "usr" },
{ "usr\\lib\\\\\\", "usr" },
{ "C:a", "C:." },
{ "C:/", "C:/" },
{ "C:///", "C:/" },
{ "C:/usr", "C:/" },
{ "C:/usr/", "C:/" },
{ "C:/usr//", "C:/" },
{ "C:/usr/lib", "C:/usr" },
{ "C:usr/lib", "C:usr" },
{ "C:usr/lib///", "C:usr" },
{ "\\\\\\", "\\" },
{ "\\\\\\\\", "\\" },
{ "C:", "C:.", "." },
#endif
{ NULL, NULL }
};
static int check_dotfile(const char *x, const char **argv,
int (*is_hfs)(const char *),
int (*is_ntfs)(const char *))
{
int res = 0, expect = 1;
for (; *argv; argv++) {
if (!strcmp("--not", *argv))
expect = !expect;
else if (expect != (is_hfs(*argv) || is_ntfs(*argv)))
res = error("'%s' is %s.git%s", *argv,
expect ? "not " : "", x);
else
fprintf(stderr, "ok: '%s' is %s.git%s\n",
*argv, expect ? "" : "not ", x);
}
return !!res;
}
ci: parallelize testing on Windows The fact that Git's test suite is implemented in Unix shell script that is as portable as we can muster, combined with the fact that Unix shell scripting is foreign to Windows (and therefore has to be emulated), results in pretty abysmal speed of the test suite on that platform, for pretty much no other reason than that language choice. For comparison: while the Linux build & test is typically done within about 8 minutes, the Windows build & test typically lasts about 80 minutes in Azure Pipelines. To help with that, let's use the Azure Pipeline feature where you can parallelize jobs, make jobs depend on each other, and pass artifacts between them. The tests are distributed using the following heuristic: listing all test scripts ordered by size in descending order (as a cheap way to estimate the overall run time), every Nth script is run (where N is the total number of parallel jobs), starting at the index corresponding to the parallel job. This slicing is performed by a new function that is added to the `test-tool`. To optimize the overall runtime of the entire Pipeline, we need to move the Windows jobs to the beginning (otherwise there would be a very decent chance for the Pipeline to be run only the Windows build, while all the parallel Windows test jobs wait for this single one). We use Azure Pipelines Artifacts for both the minimal Git for Windows SDK as well as the built executables, as deduplication and caching close to the agents makes that really fast. For comparison: while downloading and unpacking the minimal Git for Windows SDK via PowerShell takes only one minute (down from anywhere between 2.5 to 7 when using a shallow clone), uploading it as Pipeline Artifact takes less than 30s and downloading and unpacking less than 20s (sometimes even as little as only twelve seconds). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 15:19:38 +01:00
static int cmp_by_st_size(const void *a, const void *b)
{
intptr_t x = (intptr_t)((struct string_list_item *)a)->util;
intptr_t y = (intptr_t)((struct string_list_item *)b)->util;
return x > y ? -1 : (x < y ? +1 : 0);
}
/*
* A very simple, reproducible pseudo-random generator. Copied from
* `test-genrandom.c`.
*/
static uint64_t my_random_value = 1234;
static uint64_t my_random(void)
{
my_random_value = my_random_value * 1103515245 + 12345;
return my_random_value;
}
/*
* A fast approximation of the square root, without requiring math.h.
*
* It uses Newton's method to approximate the solution of 0 = x^2 - value.
*/
static double my_sqrt(double value)
{
const double epsilon = 1e-6;
double x = value;
if (value == 0)
return 0;
for (;;) {
double delta = (value / x - x) / 2;
if (delta < epsilon && delta > -epsilon)
return x + delta;
x += delta;
}
}
static int protect_ntfs_hfs_benchmark(int argc, const char **argv)
{
size_t i, j, nr, min_len = 3, max_len = 20;
char **names;
int repetitions = 15, file_mode = 0100644;
uint64_t begin, end;
double m[3][2], v[3][2];
uint64_t cumul;
double cumul2;
if (argc > 1 && !strcmp(argv[1], "--with-symlink-mode")) {
file_mode = 0120000;
argc--;
argv++;
}
nr = argc > 1 ? strtoul(argv[1], NULL, 0) : 1000000;
ALLOC_ARRAY(names, nr);
if (argc > 2) {
min_len = strtoul(argv[2], NULL, 0);
if (argc > 3)
max_len = strtoul(argv[3], NULL, 0);
if (min_len > max_len)
die("min_len > max_len");
}
for (i = 0; i < nr; i++) {
size_t len = min_len + (my_random() % (max_len + 1 - min_len));
names[i] = xmallocz(len);
while (len > 0)
names[i][--len] = (char)(' ' + (my_random() % ('\x7f' - ' ')));
}
for (protect_ntfs = 0; protect_ntfs < 2; protect_ntfs++)
for (protect_hfs = 0; protect_hfs < 2; protect_hfs++) {
cumul = 0;
cumul2 = 0;
for (i = 0; i < repetitions; i++) {
begin = getnanotime();
for (j = 0; j < nr; j++)
verify_path(names[j], file_mode);
end = getnanotime();
printf("protect_ntfs = %d, protect_hfs = %d: %lfms\n", protect_ntfs, protect_hfs, (end-begin) / (double)1e6);
cumul += end - begin;
cumul2 += (end - begin) * (end - begin);
}
m[protect_ntfs][protect_hfs] = cumul / (double)repetitions;
v[protect_ntfs][protect_hfs] = my_sqrt(cumul2 / (double)repetitions - m[protect_ntfs][protect_hfs] * m[protect_ntfs][protect_hfs]);
printf("mean: %lfms, stddev: %lfms\n", m[protect_ntfs][protect_hfs] / (double)1e6, v[protect_ntfs][protect_hfs] / (double)1e6);
}
for (protect_ntfs = 0; protect_ntfs < 2; protect_ntfs++)
for (protect_hfs = 0; protect_hfs < 2; protect_hfs++)
printf("ntfs=%d/hfs=%d: %lf%% slower\n", protect_ntfs, protect_hfs, (m[protect_ntfs][protect_hfs] - m[0][0]) * 100 / m[0][0]);
return 0;
}
int cmd__path_utils(int argc, const char **argv)
{
if (argc == 3 && !strcmp(argv[1], "normalize_path_copy")) {
char *buf = xmallocz(strlen(argv[2]));
int rv = normalize_path_copy(buf, argv[2]);
puts(rv ? "++failed++" : buf);
free(buf);
return 0;
}
if (argc >= 2 && !strcmp(argv[1], "real_path")) {
struct strbuf realpath = STRBUF_INIT;
while (argc > 2) {
strbuf_realpath(&realpath, argv[2], 1);
puts(realpath.buf);
argc--;
argv++;
}
strbuf_release(&realpath);
return 0;
}
if (argc >= 2 && !strcmp(argv[1], "absolute_path")) {
while (argc > 2) {
puts(absolute_path(argv[2]));
argc--;
argv++;
}
return 0;
}
if (argc == 4 && !strcmp(argv[1], "longest_ancestor_length")) {
int len;
struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
longest_ancestor_length(): require prefix list entries to be normalized Move the responsibility for normalizing prefixes from longest_ancestor_length() to its callers. Use slightly different normalizations at the two callers: In setup_git_directory_gently_1(), use the old normalization, which ignores paths that are not usable. In the next commit we will change this caller to also resolve symlinks in the paths from GIT_CEILING_DIRECTORIES as part of the normalization. In "test-path-utils longest_ancestor_length", use the old normalization, but die() if any paths are unusable. Also change t0060 to only pass normalized paths to the test program (no empty entries or non-absolute paths, strip trailing slashes from the paths, and remove tests that thereby become redundant). The point of this change is to reduce the scope of the ancestor_length tests in t0060 from testing normalization+longest_prefix to testing only mostly longest_prefix. This is necessary because when setup_git_directory_gently_1() starts resolving symlinks as part of its normalization, it will not be reasonable to do the same in the test suite, because that would make the test results depend on the contents of the root directory of the filesystem on which the test is run. HOWEVER: under Windows, bash mangles arguments that look like absolute POSIX paths into DOS paths. So we have to retain the level of normalization done by normalize_path_copy() to convert the bash-mangled DOS paths (which contain backslashes) into paths that use forward slashes. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jeff King <peff@peff.net>
2012-10-28 17:16:25 +01:00
char *path = xstrdup(argv[2]);
longest_ancestor_length(): require prefix list entries to be normalized Move the responsibility for normalizing prefixes from longest_ancestor_length() to its callers. Use slightly different normalizations at the two callers: In setup_git_directory_gently_1(), use the old normalization, which ignores paths that are not usable. In the next commit we will change this caller to also resolve symlinks in the paths from GIT_CEILING_DIRECTORIES as part of the normalization. In "test-path-utils longest_ancestor_length", use the old normalization, but die() if any paths are unusable. Also change t0060 to only pass normalized paths to the test program (no empty entries or non-absolute paths, strip trailing slashes from the paths, and remove tests that thereby become redundant). The point of this change is to reduce the scope of the ancestor_length tests in t0060 from testing normalization+longest_prefix to testing only mostly longest_prefix. This is necessary because when setup_git_directory_gently_1() starts resolving symlinks as part of its normalization, it will not be reasonable to do the same in the test suite, because that would make the test results depend on the contents of the root directory of the filesystem on which the test is run. HOWEVER: under Windows, bash mangles arguments that look like absolute POSIX paths into DOS paths. So we have to retain the level of normalization done by normalize_path_copy() to convert the bash-mangled DOS paths (which contain backslashes) into paths that use forward slashes. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jeff King <peff@peff.net>
2012-10-28 17:16:25 +01:00
/*
* We have to normalize the arguments because under
* Windows, bash mangles arguments that look like
* absolute POSIX paths or colon-separate lists of
* absolute POSIX paths into DOS paths (e.g.,
* "/foo:/foo/bar" might be converted to
* "D:\Src\msysgit\foo;D:\Src\msysgit\foo\bar"),
* whereas longest_ancestor_length() requires paths
* that use forward slashes.
*/
if (normalize_path_copy(path, path))
die("Path \"%s\" could not be normalized", argv[2]);
string_list_split(&ceiling_dirs, argv[3], PATH_SEP, -1);
longest_ancestor_length(): require prefix list entries to be normalized Move the responsibility for normalizing prefixes from longest_ancestor_length() to its callers. Use slightly different normalizations at the two callers: In setup_git_directory_gently_1(), use the old normalization, which ignores paths that are not usable. In the next commit we will change this caller to also resolve symlinks in the paths from GIT_CEILING_DIRECTORIES as part of the normalization. In "test-path-utils longest_ancestor_length", use the old normalization, but die() if any paths are unusable. Also change t0060 to only pass normalized paths to the test program (no empty entries or non-absolute paths, strip trailing slashes from the paths, and remove tests that thereby become redundant). The point of this change is to reduce the scope of the ancestor_length tests in t0060 from testing normalization+longest_prefix to testing only mostly longest_prefix. This is necessary because when setup_git_directory_gently_1() starts resolving symlinks as part of its normalization, it will not be reasonable to do the same in the test suite, because that would make the test results depend on the contents of the root directory of the filesystem on which the test is run. HOWEVER: under Windows, bash mangles arguments that look like absolute POSIX paths into DOS paths. So we have to retain the level of normalization done by normalize_path_copy() to convert the bash-mangled DOS paths (which contain backslashes) into paths that use forward slashes. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jeff King <peff@peff.net>
2012-10-28 17:16:25 +01:00
filter_string_list(&ceiling_dirs, 0,
normalize_ceiling_entry, NULL);
len = longest_ancestor_length(path, &ceiling_dirs);
string_list_clear(&ceiling_dirs, 0);
longest_ancestor_length(): require prefix list entries to be normalized Move the responsibility for normalizing prefixes from longest_ancestor_length() to its callers. Use slightly different normalizations at the two callers: In setup_git_directory_gently_1(), use the old normalization, which ignores paths that are not usable. In the next commit we will change this caller to also resolve symlinks in the paths from GIT_CEILING_DIRECTORIES as part of the normalization. In "test-path-utils longest_ancestor_length", use the old normalization, but die() if any paths are unusable. Also change t0060 to only pass normalized paths to the test program (no empty entries or non-absolute paths, strip trailing slashes from the paths, and remove tests that thereby become redundant). The point of this change is to reduce the scope of the ancestor_length tests in t0060 from testing normalization+longest_prefix to testing only mostly longest_prefix. This is necessary because when setup_git_directory_gently_1() starts resolving symlinks as part of its normalization, it will not be reasonable to do the same in the test suite, because that would make the test results depend on the contents of the root directory of the filesystem on which the test is run. HOWEVER: under Windows, bash mangles arguments that look like absolute POSIX paths into DOS paths. So we have to retain the level of normalization done by normalize_path_copy() to convert the bash-mangled DOS paths (which contain backslashes) into paths that use forward slashes. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jeff King <peff@peff.net>
2012-10-28 17:16:25 +01:00
free(path);
printf("%d\n", len);
return 0;
}
if (argc >= 4 && !strcmp(argv[1], "prefix_path")) {
add an extra level of indirection to main() There are certain startup tasks that we expect every git process to do. In some cases this is just to improve the quality of the program (e.g., setting up gettext()). In others it is a requirement for using certain functions in libgit.a (e.g., system_path() expects that you have called git_extract_argv0_path()). Most commands are builtins and are covered by the git.c version of main(). However, there are still a few external commands that use their own main(). Each of these has to remember to include the correct startup sequence, and we are not always consistent. Rather than just fix the inconsistencies, let's make this harder to get wrong by providing a common main() that can run this standard startup. We basically have two options to do this: - the compat/mingw.h file already does something like this by adding a #define that replaces the definition of main with a wrapper that calls mingw_startup(). The upside is that the code in each program doesn't need to be changed at all; it's rewritten on the fly by the preprocessor. The downside is that it may make debugging of the startup sequence a bit more confusing, as the preprocessor is quietly inserting new code. - the builtin functions are all of the form cmd_foo(), and git.c's main() calls them. This is much more explicit, which may make things more obvious to somebody reading the code. It's also more flexible (because of course we have to figure out _which_ cmd_foo() to call). The downside is that each of the builtins must define cmd_foo(), instead of just main(). This patch chooses the latter option, preferring the more explicit approach, even though it is more invasive. We introduce a new file common-main.c, with the "real" main. It expects to call cmd_main() from whatever other objects it is linked against. We link common-main.o against anything that links against libgit.a, since we know that such programs will need to do this setup. Note that common-main.o can't actually go inside libgit.a, as the linker would not pick up its main() function automatically (it has no callers). The rest of the patch is just adjusting all of the various external programs (mostly in t/helper) to use cmd_main(). I've provided a global declaration for cmd_main(), which means that all of the programs also need to match its signature. In particular, many functions need to switch to "const char **" instead of "char **" for argv. This effect ripples out to a few other variables and functions, as well. This makes the patch even more invasive, but the end result is much better. We should be treating argv strings as const anyway, and now all programs conform to the same signature (which also matches the way builtins are defined). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01 07:58:58 +02:00
const char *prefix = argv[2];
int prefix_len = strlen(prefix);
int nongit_ok;
setup_git_directory_gently(&nongit_ok);
while (argc > 3) {
char *pfx = prefix_path(prefix, prefix_len, argv[3]);
puts(pfx);
free(pfx);
argc--;
argv++;
}
return 0;
}
if (argc == 4 && !strcmp(argv[1], "strip_path_suffix")) {
char *prefix = strip_path_suffix(argv[2], argv[3]);
printf("%s\n", prefix ? prefix : "(null)");
free(prefix);
return 0;
}
if (argc == 3 && !strcmp(argv[1], "print_path")) {
puts(argv[2]);
return 0;
}
if (argc == 4 && !strcmp(argv[1], "relative_path")) {
path.c: refactor relative_path(), not only strip prefix Original design of relative_path() is simple, just strip the prefix (*base) from the absolute path (*abs). In most cases, we need a real relative path, such as: ../foo, ../../bar. That's why there is another reimplementation (path_relative()) in quote.c. Borrow some codes from path_relative() in quote.c to refactor relative_path() in path.c, so that it could return real relative path, and user can reuse this function without reimplementing his/her own. The function path_relative() in quote.c will be substituted, and I would use the new relative_path() function when implementing the interactive git-clean later. Different results for relative_path() before and after this refactor: abs path base path relative (original) relative (refactor) ======== ========= =================== =================== /a/b /a/b . ./ /a/b/ /a/b . ./ /a /a/b/ /a ../ / /a/b/ / ../../ /a/c /a/b/ /a/c ../c /x/y /a/b/ /x/y ../../x/y a/b/ a/b/ . ./ a/b/ a/b . ./ a a/b a ../ x/y a/b/ x/y ../../x/y a/c a/b a/c ../c (empty) (null) (empty) ./ (empty) (empty) (empty) ./ (empty) /a/b (empty) ./ (null) (null) (null) ./ (null) (empty) (null) ./ (null) /a/b (segfault) ./ You may notice that return value "." has been changed to "./". It is because: * Function quote_path_relative() in quote.c will show the relative path as "./" if abs(in) and base(prefix) are the same. * Function relative_path() is called only once (in setup.c), and it will be OK for the return value as "./" instead of ".". Signed-off-by: Jiang Xin <worldhello.net@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-25 17:53:43 +02:00
struct strbuf sb = STRBUF_INIT;
const char *in, *prefix, *rel;
normalize_argv_string(&in, argv[2]);
normalize_argv_string(&prefix, argv[3]);
path.c: refactor relative_path(), not only strip prefix Original design of relative_path() is simple, just strip the prefix (*base) from the absolute path (*abs). In most cases, we need a real relative path, such as: ../foo, ../../bar. That's why there is another reimplementation (path_relative()) in quote.c. Borrow some codes from path_relative() in quote.c to refactor relative_path() in path.c, so that it could return real relative path, and user can reuse this function without reimplementing his/her own. The function path_relative() in quote.c will be substituted, and I would use the new relative_path() function when implementing the interactive git-clean later. Different results for relative_path() before and after this refactor: abs path base path relative (original) relative (refactor) ======== ========= =================== =================== /a/b /a/b . ./ /a/b/ /a/b . ./ /a /a/b/ /a ../ / /a/b/ / ../../ /a/c /a/b/ /a/c ../c /x/y /a/b/ /x/y ../../x/y a/b/ a/b/ . ./ a/b/ a/b . ./ a a/b a ../ x/y a/b/ x/y ../../x/y a/c a/b a/c ../c (empty) (null) (empty) ./ (empty) (empty) (empty) ./ (empty) /a/b (empty) ./ (null) (null) (null) ./ (null) (empty) (null) ./ (null) /a/b (segfault) ./ You may notice that return value "." has been changed to "./". It is because: * Function quote_path_relative() in quote.c will show the relative path as "./" if abs(in) and base(prefix) are the same. * Function relative_path() is called only once (in setup.c), and it will be OK for the return value as "./" instead of ".". Signed-off-by: Jiang Xin <worldhello.net@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-25 17:53:43 +02:00
rel = relative_path(in, prefix, &sb);
if (!rel)
puts("(null)");
else
puts(strlen(rel) > 0 ? rel : "(empty)");
path.c: refactor relative_path(), not only strip prefix Original design of relative_path() is simple, just strip the prefix (*base) from the absolute path (*abs). In most cases, we need a real relative path, such as: ../foo, ../../bar. That's why there is another reimplementation (path_relative()) in quote.c. Borrow some codes from path_relative() in quote.c to refactor relative_path() in path.c, so that it could return real relative path, and user can reuse this function without reimplementing his/her own. The function path_relative() in quote.c will be substituted, and I would use the new relative_path() function when implementing the interactive git-clean later. Different results for relative_path() before and after this refactor: abs path base path relative (original) relative (refactor) ======== ========= =================== =================== /a/b /a/b . ./ /a/b/ /a/b . ./ /a /a/b/ /a ../ / /a/b/ / ../../ /a/c /a/b/ /a/c ../c /x/y /a/b/ /x/y ../../x/y a/b/ a/b/ . ./ a/b/ a/b . ./ a a/b a ../ x/y a/b/ x/y ../../x/y a/c a/b a/c ../c (empty) (null) (empty) ./ (empty) (empty) (empty) ./ (empty) /a/b (empty) ./ (null) (null) (null) ./ (null) (empty) (null) ./ (null) /a/b (segfault) ./ You may notice that return value "." has been changed to "./". It is because: * Function quote_path_relative() in quote.c will show the relative path as "./" if abs(in) and base(prefix) are the same. * Function relative_path() is called only once (in setup.c), and it will be OK for the return value as "./" instead of ".". Signed-off-by: Jiang Xin <worldhello.net@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-25 17:53:43 +02:00
strbuf_release(&sb);
return 0;
}
if (argc == 2 && !strcmp(argv[1], "basename"))
return test_function(basename_data, posix_basename, argv[1]);
if (argc == 2 && !strcmp(argv[1], "dirname"))
return test_function(dirname_data, posix_dirname, argv[1]);
if (argc > 2 && !strcmp(argv[1], "is_dotgitmodules")) {
return check_dotfile("modules", argv + 2,
is_hfs_dotgitmodules,
is_ntfs_dotgitmodules);
}
if (argc > 2 && !strcmp(argv[1], "is_dotgitignore")) {
return check_dotfile("ignore", argv + 2,
is_hfs_dotgitignore,
is_ntfs_dotgitignore);
}
if (argc > 2 && !strcmp(argv[1], "is_dotgitattributes")) {
return check_dotfile("attributes", argv + 2,
is_hfs_dotgitattributes,
is_ntfs_dotgitattributes);
}
if (argc > 2 && !strcmp(argv[1], "is_dotmailmap")) {
return check_dotfile("mailmap", argv + 2,
is_hfs_dotmailmap,
is_ntfs_dotmailmap);
}
if (argc > 2 && !strcmp(argv[1], "file-size")) {
int res = 0, i;
struct stat st;
for (i = 2; i < argc; i++)
if (stat(argv[i], &st))
res = error_errno("Cannot stat '%s'", argv[i]);
else
printf("%"PRIuMAX"\n", (uintmax_t)st.st_size);
return !!res;
}
tests: include detailed trace logs with --write-junit-xml upon failure The JUnit XML format lends itself to be presented in a powerful UI, where you can drill down to the information you are interested in very quickly. For test failures, this usually means that you want to see the detailed trace of the failing tests. With Travis CI, we passed the `--verbose-log` option to get those traces. However, that seems excessive, as we do not need/use the logs in almost all of those cases: only when a test fails do we have a way to include the trace. So let's do something different when using Azure DevOps: let's run all the tests with `--quiet` first, and only if a failure is encountered, try to trace the commands as they are executed. Of course, we cannot turn on `--verbose-log` after the fact. So let's just re-run the test with all the same options, adding `--verbose-log`. And then munging the output file into the JUnit XML on the fly. Note: there is an off chance that re-running the test in verbose mode "fixes" the failures (and this does happen from time to time!). That is a possibility we should be able to live with. Ideally, we would label this as "Passed upon rerun", and Azure Pipelines even know about that outcome, but it is not available when using the JUnit XML format for now: https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 15:19:34 +01:00
if (argc == 4 && !strcmp(argv[1], "skip-n-bytes")) {
int fd = open(argv[2], O_RDONLY), offset = atoi(argv[3]);
char buffer[65536];
if (fd < 0)
die_errno("could not open '%s'", argv[2]);
if (lseek(fd, offset, SEEK_SET) < 0)
die_errno("could not skip %d bytes", offset);
for (;;) {
ssize_t count = read(fd, buffer, sizeof(buffer));
if (count < 0)
die_errno("could not read '%s'", argv[2]);
if (!count)
break;
if (write(1, buffer, count) < 0)
die_errno("could not write to stdout");
}
close(fd);
return 0;
}
ci: parallelize testing on Windows The fact that Git's test suite is implemented in Unix shell script that is as portable as we can muster, combined with the fact that Unix shell scripting is foreign to Windows (and therefore has to be emulated), results in pretty abysmal speed of the test suite on that platform, for pretty much no other reason than that language choice. For comparison: while the Linux build & test is typically done within about 8 minutes, the Windows build & test typically lasts about 80 minutes in Azure Pipelines. To help with that, let's use the Azure Pipeline feature where you can parallelize jobs, make jobs depend on each other, and pass artifacts between them. The tests are distributed using the following heuristic: listing all test scripts ordered by size in descending order (as a cheap way to estimate the overall run time), every Nth script is run (where N is the total number of parallel jobs), starting at the index corresponding to the parallel job. This slicing is performed by a new function that is added to the `test-tool`. To optimize the overall runtime of the entire Pipeline, we need to move the Windows jobs to the beginning (otherwise there would be a very decent chance for the Pipeline to be run only the Windows build, while all the parallel Windows test jobs wait for this single one). We use Azure Pipelines Artifacts for both the minimal Git for Windows SDK as well as the built executables, as deduplication and caching close to the agents makes that really fast. For comparison: while downloading and unpacking the minimal Git for Windows SDK via PowerShell takes only one minute (down from anywhere between 2.5 to 7 when using a shallow clone), uploading it as Pipeline Artifact takes less than 30s and downloading and unpacking less than 20s (sometimes even as little as only twelve seconds). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 15:19:38 +01:00
if (argc > 5 && !strcmp(argv[1], "slice-tests")) {
int res = 0;
long offset, stride, i;
struct string_list list = STRING_LIST_INIT_NODUP;
struct stat st;
offset = strtol(argv[2], NULL, 10);
stride = strtol(argv[3], NULL, 10);
if (stride < 1)
stride = 1;
for (i = 4; i < argc; i++)
if (stat(argv[i], &st))
res = error_errno("Cannot stat '%s'", argv[i]);
else
string_list_append(&list, argv[i])->util =
(void *)(intptr_t)st.st_size;
QSORT(list.items, list.nr, cmp_by_st_size);
for (i = offset; i < list.nr; i+= stride)
printf("%s\n", list.items[i].string);
return !!res;
}
if (argc > 1 && !strcmp(argv[1], "protect_ntfs_hfs"))
return !!protect_ntfs_hfs_benchmark(argc - 1, argv + 1);
mingw: refuse to access paths with trailing spaces or periods When creating a directory on Windows whose path ends in a space or a period (or chains thereof), the Win32 API "helpfully" trims those. For example, `mkdir("abc ");` will return success, but actually create a directory called `abc` instead. This stems back to the DOS days, when all file names had exactly 8 characters plus exactly 3 characters for the file extension, and the only way to have shorter names was by padding with spaces. Sadly, this "helpful" behavior is a bit inconsistent: after a successful `mkdir("abc ");`, a `mkdir("abc /def")` will actually _fail_ (because the directory `abc ` does not actually exist). Even if it would work, we now have a serious problem because a Git repository could contain directories `abc` and `abc `, and on Windows, they would be "merged" unintentionally. As these paths are illegal on Windows, anyway, let's disallow any accesses to such paths on that Operating System. For practical reasons, this behavior is still guarded by the config setting `core.protectNTFS`: it is possible (and at least two regression tests make use of it) to create commits without involving the worktree. In such a scenario, it is of course possible -- even on Windows -- to create such file names. Among other consequences, this patch disallows submodules' paths to end in spaces on Windows (which would formerly have confused Git enough to try to write into incorrect paths, anyway). While this patch does not fix a vulnerability on its own, it prevents an attack vector that was exploited in demonstrations of a number of recently-fixed security bugs. The regression test added to `t/t7417-submodule-path-url.sh` reflects that attack vector. Note that we have to adjust the test case "prevent git~1 squatting on Windows" in `t/t7415-submodule-names.sh` because of a very subtle issue. It tries to clone two submodules whose names differ only in a trailing period character, and as a consequence their git directories differ in the same way. Previously, when Git tried to clone the second submodule, it thought that the git directory already existed (because on Windows, when you create a directory with the name `b.` it actually creates `b`), but with this patch, the first submodule's clone will fail because of the illegal name of the git directory. Therefore, when cloning the second submodule, Git will take a different code path: a fresh clone (without an existing git directory). Both code paths fail to clone the second submodule, both because the the corresponding worktree directory exists and is not empty, but the error messages are worded differently. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-09-05 13:27:53 +02:00
if (argc > 1 && !strcmp(argv[1], "is_valid_path")) {
int res = 0, expect = 1, i;
for (i = 2; i < argc; i++)
if (!strcmp("--not", argv[i]))
expect = 0;
else if (expect != is_valid_path(argv[i]))
res = error("'%s' is%s a valid path",
argv[i], expect ? " not" : "");
else
fprintf(stderr,
"'%s' is%s a valid path\n",
argv[i], expect ? "" : " not");
return !!res;
}
fprintf(stderr, "%s: unknown function name: %s\n", argv[0],
argv[1] ? argv[1] : "(there was none)");
return 1;
}