mirror of
https://github.com/git/git.git
synced 2024-04-16 18:23:53 +02:00
Merge branch 'jh/builtin-fsmonitor-part2'
Built-in fsmonitor (part 2). * jh/builtin-fsmonitor-part2: (30 commits) t7527: test status with untracked-cache and fsmonitor--daemon fsmonitor: force update index after large responses fsmonitor--daemon: use a cookie file to sync with file system fsmonitor--daemon: periodically truncate list of modified files t/perf/p7519: add fsmonitor--daemon test cases t/perf/p7519: speed up test on Windows t/perf/p7519: fix coding style t/helper/test-chmtime: skip directories on Windows t/perf: avoid copying builtin fsmonitor files into test repo t7527: create test for fsmonitor--daemon t/helper/fsmonitor-client: create IPC client to talk to FSMonitor Daemon help: include fsmonitor--daemon feature flag in version info fsmonitor--daemon: implement handle_client callback compat/fsmonitor/fsm-listen-darwin: implement FSEvent listener on MacOS compat/fsmonitor/fsm-listen-darwin: add MacOS header files for FSEvent compat/fsmonitor/fsm-listen-win32: implement FSMonitor backend on Windows fsmonitor--daemon: create token-based changed path cache fsmonitor--daemon: define token-ids fsmonitor--daemon: add pathname classification fsmonitor--daemon: implement 'start' command ...
This commit is contained in:
Junio C Hamano
commit
439c1e6d5d
|
|
@ -72,6 +72,7 @@
|
|||
/git-format-patch
|
||||
/git-fsck
|
||||
/git-fsck-objects
|
||||
/git-fsmonitor--daemon
|
||||
/git-gc
|
||||
/git-get-tar-commit-id
|
||||
/git-grep
|
||||
|
|
|
|||
|
|
@ -62,22 +62,54 @@ core.protectNTFS::
|
|||
Defaults to `true` on Windows, and `false` elsewhere.
|
||||
|
||||
core.fsmonitor::
|
||||
If set, the value of this variable is used as a command which
|
||||
will identify all files that may have changed since the
|
||||
requested date/time. This information is used to speed up git by
|
||||
avoiding unnecessary processing of files that have not changed.
|
||||
See the "fsmonitor-watchman" section of linkgit:githooks[5].
|
||||
If set to true, enable the built-in file system monitor
|
||||
daemon for this working directory (linkgit:git-fsmonitor--daemon[1]).
|
||||
+
|
||||
Like hook-based file system monitors, the built-in file system monitor
|
||||
can speed up Git commands that need to refresh the Git index
|
||||
(e.g. `git status`) in a working directory with many files. The
|
||||
built-in monitor eliminates the need to install and maintain an
|
||||
external third-party tool.
|
||||
+
|
||||
The built-in file system monitor is currently available only on a
|
||||
limited set of supported platforms. Currently, this includes Windows
|
||||
and MacOS.
|
||||
+
|
||||
Otherwise, this variable contains the pathname of the "fsmonitor"
|
||||
hook command.
|
||||
+
|
||||
This hook command is used to identify all files that may have changed
|
||||
since the requested date/time. This information is used to speed up
|
||||
git by avoiding unnecessary scanning of files that have not changed.
|
||||
+
|
||||
See the "fsmonitor-watchman" section of linkgit:githooks[5].
|
||||
+
|
||||
Note that if you concurrently use multiple versions of Git, such
|
||||
as one version on the command line and another version in an IDE
|
||||
tool, that the definition of `core.fsmonitor` was extended to
|
||||
allow boolean values in addition to hook pathnames. Git versions
|
||||
2.35.1 and prior will not understand the boolean values and will
|
||||
consider the "true" or "false" values as hook pathnames to be
|
||||
invoked. Git versions 2.26 thru 2.35.1 default to hook protocol
|
||||
V2 and will fall back to no fsmonitor (full scan). Git versions
|
||||
prior to 2.26 default to hook protocol V1 and will silently
|
||||
assume there were no changes to report (no scan), so status
|
||||
commands may report incomplete results. For this reason, it is
|
||||
best to upgrade all of your Git versions before using the built-in
|
||||
file system monitor.
|
||||
|
||||
core.fsmonitorHookVersion::
|
||||
Sets the version of hook that is to be used when calling fsmonitor.
|
||||
There are currently versions 1 and 2. When this is not set,
|
||||
version 2 will be tried first and if it fails then version 1
|
||||
will be tried. Version 1 uses a timestamp as input to determine
|
||||
which files have changes since that time but some monitors
|
||||
like watchman have race conditions when used with a timestamp.
|
||||
Version 2 uses an opaque string so that the monitor can return
|
||||
something that can be used to determine what files have changed
|
||||
without race conditions.
|
||||
Sets the protocol version to be used when invoking the
|
||||
"fsmonitor" hook.
|
||||
+
|
||||
There are currently versions 1 and 2. When this is not set,
|
||||
version 2 will be tried first and if it fails then version 1
|
||||
will be tried. Version 1 uses a timestamp as input to determine
|
||||
which files have changes since that time but some monitors
|
||||
like Watchman have race conditions when used with a timestamp.
|
||||
Version 2 uses an opaque string so that the monitor can return
|
||||
something that can be used to determine what files have changed
|
||||
without race conditions.
|
||||
|
||||
core.trustctime::
|
||||
If false, the ctime differences between the index and the
|
||||
|
|
|
|||
|
|
@ -0,0 +1,75 @@
|
|||
git-fsmonitor--daemon(1)
|
||||
========================
|
||||
|
||||
NAME
|
||||
----
|
||||
git-fsmonitor--daemon - A Built-in File System Monitor
|
||||
|
||||
SYNOPSIS
|
||||
--------
|
||||
[verse]
|
||||
'git fsmonitor--daemon' start
|
||||
'git fsmonitor--daemon' run
|
||||
'git fsmonitor--daemon' stop
|
||||
'git fsmonitor--daemon' status
|
||||
|
||||
DESCRIPTION
|
||||
-----------
|
||||
|
||||
A daemon to watch the working directory for file and directory
|
||||
changes using platform-specific file system notification facilities.
|
||||
|
||||
This daemon communicates directly with commands like `git status`
|
||||
using the link:technical/api-simple-ipc.html[simple IPC] interface
|
||||
instead of the slower linkgit:githooks[5] interface.
|
||||
|
||||
This daemon is built into Git so that no third-party tools are
|
||||
required.
|
||||
|
||||
OPTIONS
|
||||
-------
|
||||
|
||||
start::
|
||||
Starts a daemon in the background.
|
||||
|
||||
run::
|
||||
Runs a daemon in the foreground.
|
||||
|
||||
stop::
|
||||
Stops the daemon running in the current working
|
||||
directory, if present.
|
||||
|
||||
status::
|
||||
Exits with zero status if a daemon is watching the
|
||||
current working directory.
|
||||
|
||||
REMARKS
|
||||
-------
|
||||
|
||||
This daemon is a long running process used to watch a single working
|
||||
directory and maintain a list of the recently changed files and
|
||||
directories. Performance of commands such as `git status` can be
|
||||
increased if they just ask for a summary of changes to the working
|
||||
directory and can avoid scanning the disk.
|
||||
|
||||
When `core.fsmonitor` is set to `true` (see linkgit:git-config[1])
|
||||
commands, such as `git status`, will ask the daemon for changes and
|
||||
automatically start it (if necessary).
|
||||
|
||||
For more information see the "File System Monitor" section in
|
||||
linkgit:git-update-index[1].
|
||||
|
||||
CAVEATS
|
||||
-------
|
||||
|
||||
The fsmonitor daemon does not currently know about submodules and does
|
||||
not know to filter out file system events that happen within a
|
||||
submodule. If fsmonitor daemon is watching a super repo and a file is
|
||||
modified within the working directory of a submodule, it will report
|
||||
the change (as happening against the super repo). However, the client
|
||||
will properly ignore these extra events, so performance may be affected
|
||||
but it will not cause an incorrect result.
|
||||
|
||||
GIT
|
||||
---
|
||||
Part of the linkgit:git[1] suite
|
||||
|
|
@ -527,7 +527,9 @@ FILE SYSTEM MONITOR
|
|||
This feature is intended to speed up git operations for repos that have
|
||||
large working directories.
|
||||
|
||||
It enables git to work together with a file system monitor (see the
|
||||
It enables git to work together with a file system monitor (see
|
||||
linkgit:git-fsmonitor--daemon[1]
|
||||
and the
|
||||
"fsmonitor-watchman" section of linkgit:githooks[5]) that can
|
||||
inform it as to what files have been modified. This enables git to avoid
|
||||
having to lstat() every file to find modified files.
|
||||
|
|
@ -538,8 +540,8 @@ looking for new files.
|
|||
|
||||
If you want to enable (or disable) this feature, it is easier to use
|
||||
the `core.fsmonitor` configuration variable (see
|
||||
linkgit:git-config[1]) than using the `--fsmonitor` option to
|
||||
`git update-index` in each repository, especially if you want to do so
|
||||
linkgit:git-config[1]) than using the `--fsmonitor` option to `git
|
||||
update-index` in each repository, especially if you want to do so
|
||||
across all repositories you use, because you can set the configuration
|
||||
variable in your `$HOME/.gitconfig` just once and have it affect all
|
||||
repositories you touch.
|
||||
|
|
|
|||
17
Makefile
17
Makefile
|
|
@ -475,6 +475,11 @@ include shared.mak
|
|||
# directory, and the JSON compilation database 'compile_commands.json' will be
|
||||
# created at the root of the repository.
|
||||
#
|
||||
# If your platform supports a built-in fsmonitor backend, set
|
||||
# FSMONITOR_DAEMON_BACKEND to the "<name>" of the corresponding
|
||||
# `compat/fsmonitor/fsm-listen-<name>.c` that implements the
|
||||
# `fsm_listen__*()` routines.
|
||||
#
|
||||
# Define DEVELOPER to enable more compiler warnings. Compiler version
|
||||
# and family are auto detected, but could be overridden by defining
|
||||
# COMPILER_FEATURES (see config.mak.dev). You can still set
|
||||
|
|
@ -716,6 +721,7 @@ TEST_BUILTINS_OBJS += test-dump-split-index.o
|
|||
TEST_BUILTINS_OBJS += test-dump-untracked-cache.o
|
||||
TEST_BUILTINS_OBJS += test-example-decorate.o
|
||||
TEST_BUILTINS_OBJS += test-fast-rebase.o
|
||||
TEST_BUILTINS_OBJS += test-fsmonitor-client.o
|
||||
TEST_BUILTINS_OBJS += test-genrandom.o
|
||||
TEST_BUILTINS_OBJS += test-genzeros.o
|
||||
TEST_BUILTINS_OBJS += test-getcwd.o
|
||||
|
|
@ -933,6 +939,8 @@ LIB_OBJS += fetch-pack.o
|
|||
LIB_OBJS += fmt-merge-msg.o
|
||||
LIB_OBJS += fsck.o
|
||||
LIB_OBJS += fsmonitor.o
|
||||
LIB_OBJS += fsmonitor-ipc.o
|
||||
LIB_OBJS += fsmonitor-settings.o
|
||||
LIB_OBJS += gettext.o
|
||||
LIB_OBJS += gpg-interface.o
|
||||
LIB_OBJS += graph.o
|
||||
|
|
@ -1139,6 +1147,7 @@ BUILTIN_OBJS += builtin/fmt-merge-msg.o
|
|||
BUILTIN_OBJS += builtin/for-each-ref.o
|
||||
BUILTIN_OBJS += builtin/for-each-repo.o
|
||||
BUILTIN_OBJS += builtin/fsck.o
|
||||
BUILTIN_OBJS += builtin/fsmonitor--daemon.o
|
||||
BUILTIN_OBJS += builtin/gc.o
|
||||
BUILTIN_OBJS += builtin/get-tar-commit-id.o
|
||||
BUILTIN_OBJS += builtin/grep.o
|
||||
|
|
@ -1992,6 +2001,11 @@ ifdef NEED_ACCESS_ROOT_HANDLER
|
|||
COMPAT_OBJS += compat/access.o
|
||||
endif
|
||||
|
||||
ifdef FSMONITOR_DAEMON_BACKEND
|
||||
COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND
|
||||
COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o
|
||||
endif
|
||||
|
||||
ifeq ($(TCLTK_PATH),)
|
||||
NO_TCLTK = NoThanks
|
||||
endif
|
||||
|
|
@ -2848,6 +2862,9 @@ GIT-BUILD-OPTIONS: FORCE
|
|||
@echo DC_SHA1=\''$(subst ','\'',$(subst ','\'',$(DC_SHA1)))'\' >>$@+
|
||||
@echo SANITIZE_LEAK=\''$(subst ','\'',$(subst ','\'',$(SANITIZE_LEAK)))'\' >>$@+
|
||||
@echo X=\'$(X)\' >>$@+
|
||||
ifdef FSMONITOR_DAEMON_BACKEND
|
||||
@echo FSMONITOR_DAEMON_BACKEND=\''$(subst ','\'',$(subst ','\'',$(FSMONITOR_DAEMON_BACKEND)))'\' >>$@+
|
||||
endif
|
||||
ifdef TEST_OUTPUT_DIRECTORY
|
||||
@echo TEST_OUTPUT_DIRECTORY=\''$(subst ','\'',$(subst ','\'',$(TEST_OUTPUT_DIRECTORY)))'\' >>$@+
|
||||
endif
|
||||
|
|
|
|||
|
|
@ -159,6 +159,7 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix);
|
|||
int cmd_for_each_repo(int argc, const char **argv, const char *prefix);
|
||||
int cmd_format_patch(int argc, const char **argv, const char *prefix);
|
||||
int cmd_fsck(int argc, const char **argv, const char *prefix);
|
||||
int cmd_fsmonitor__daemon(int argc, const char **argv, const char *prefix);
|
||||
int cmd_gc(int argc, const char **argv, const char *prefix);
|
||||
int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix);
|
||||
int cmd_grep(int argc, const char **argv, const char *prefix);
|
||||
|
|
|
|||
|
|
@ -0,0 +1,1479 @@
|
|||
#include "builtin.h"
|
||||
#include "config.h"
|
||||
#include "parse-options.h"
|
||||
#include "fsmonitor.h"
|
||||
#include "fsmonitor-ipc.h"
|
||||
#include "compat/fsmonitor/fsm-listen.h"
|
||||
#include "fsmonitor--daemon.h"
|
||||
#include "simple-ipc.h"
|
||||
#include "khash.h"
|
||||
#include "pkt-line.h"
|
||||
|
||||
static const char * const builtin_fsmonitor__daemon_usage[] = {
|
||||
N_("git fsmonitor--daemon start [<options>]"),
|
||||
N_("git fsmonitor--daemon run [<options>]"),
|
||||
N_("git fsmonitor--daemon stop"),
|
||||
N_("git fsmonitor--daemon status"),
|
||||
NULL
|
||||
};
|
||||
|
||||
#ifdef HAVE_FSMONITOR_DAEMON_BACKEND
|
||||
/*
|
||||
* Global state loaded from config.
|
||||
*/
|
||||
#define FSMONITOR__IPC_THREADS "fsmonitor.ipcthreads"
|
||||
static int fsmonitor__ipc_threads = 8;
|
||||
|
||||
#define FSMONITOR__START_TIMEOUT "fsmonitor.starttimeout"
|
||||
static int fsmonitor__start_timeout_sec = 60;
|
||||
|
||||
#define FSMONITOR__ANNOUNCE_STARTUP "fsmonitor.announcestartup"
|
||||
static int fsmonitor__announce_startup = 0;
|
||||
|
||||
static int fsmonitor_config(const char *var, const char *value, void *cb)
|
||||
{
|
||||
if (!strcmp(var, FSMONITOR__IPC_THREADS)) {
|
||||
int i = git_config_int(var, value);
|
||||
if (i < 1)
|
||||
return error(_("value of '%s' out of range: %d"),
|
||||
FSMONITOR__IPC_THREADS, i);
|
||||
fsmonitor__ipc_threads = i;
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (!strcmp(var, FSMONITOR__START_TIMEOUT)) {
|
||||
int i = git_config_int(var, value);
|
||||
if (i < 0)
|
||||
return error(_("value of '%s' out of range: %d"),
|
||||
FSMONITOR__START_TIMEOUT, i);
|
||||
fsmonitor__start_timeout_sec = i;
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (!strcmp(var, FSMONITOR__ANNOUNCE_STARTUP)) {
|
||||
int is_bool;
|
||||
int i = git_config_bool_or_int(var, value, &is_bool);
|
||||
if (i < 0)
|
||||
return error(_("value of '%s' not bool or int: %d"),
|
||||
var, i);
|
||||
fsmonitor__announce_startup = i;
|
||||
return 0;
|
||||
}
|
||||
|
||||
return git_default_config(var, value, cb);
|
||||
}
|
||||
|
||||
/*
|
||||
* Acting as a CLIENT.
|
||||
*
|
||||
* Send a "quit" command to the `git-fsmonitor--daemon` (if running)
|
||||
* and wait for it to shutdown.
|
||||
*/
|
||||
static int do_as_client__send_stop(void)
|
||||
{
|
||||
struct strbuf answer = STRBUF_INIT;
|
||||
int ret;
|
||||
|
||||
ret = fsmonitor_ipc__send_command("quit", &answer);
|
||||
|
||||
/* The quit command does not return any response data. */
|
||||
strbuf_release(&answer);
|
||||
|
||||
if (ret)
|
||||
return ret;
|
||||
|
||||
trace2_region_enter("fsm_client", "polling-for-daemon-exit", NULL);
|
||||
while (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING)
|
||||
sleep_millisec(50);
|
||||
trace2_region_leave("fsm_client", "polling-for-daemon-exit", NULL);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int do_as_client__status(void)
|
||||
{
|
||||
enum ipc_active_state state = fsmonitor_ipc__get_state();
|
||||
|
||||
switch (state) {
|
||||
case IPC_STATE__LISTENING:
|
||||
printf(_("fsmonitor-daemon is watching '%s'\n"),
|
||||
the_repository->worktree);
|
||||
return 0;
|
||||
|
||||
default:
|
||||
printf(_("fsmonitor-daemon is not watching '%s'\n"),
|
||||
the_repository->worktree);
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
|
||||
enum fsmonitor_cookie_item_result {
|
||||
FCIR_ERROR = -1, /* could not create cookie file ? */
|
||||
FCIR_INIT,
|
||||
FCIR_SEEN,
|
||||
FCIR_ABORT,
|
||||
};
|
||||
|
||||
struct fsmonitor_cookie_item {
|
||||
struct hashmap_entry entry;
|
||||
char *name;
|
||||
enum fsmonitor_cookie_item_result result;
|
||||
};
|
||||
|
||||
static int cookies_cmp(const void *data, const struct hashmap_entry *he1,
|
||||
const struct hashmap_entry *he2, const void *keydata)
|
||||
{
|
||||
const struct fsmonitor_cookie_item *a =
|
||||
container_of(he1, const struct fsmonitor_cookie_item, entry);
|
||||
const struct fsmonitor_cookie_item *b =
|
||||
container_of(he2, const struct fsmonitor_cookie_item, entry);
|
||||
|
||||
return strcmp(a->name, keydata ? keydata : b->name);
|
||||
}
|
||||
|
||||
static enum fsmonitor_cookie_item_result with_lock__wait_for_cookie(
|
||||
struct fsmonitor_daemon_state *state)
|
||||
{
|
||||
/* assert current thread holding state->main_lock */
|
||||
|
||||
int fd;
|
||||
struct fsmonitor_cookie_item *cookie;
|
||||
struct strbuf cookie_pathname = STRBUF_INIT;
|
||||
struct strbuf cookie_filename = STRBUF_INIT;
|
||||
enum fsmonitor_cookie_item_result result;
|
||||
int my_cookie_seq;
|
||||
|
||||
CALLOC_ARRAY(cookie, 1);
|
||||
|
||||
my_cookie_seq = state->cookie_seq++;
|
||||
|
||||
strbuf_addf(&cookie_filename, "%i-%i", getpid(), my_cookie_seq);
|
||||
|
||||
strbuf_addbuf(&cookie_pathname, &state->path_cookie_prefix);
|
||||
strbuf_addbuf(&cookie_pathname, &cookie_filename);
|
||||
|
||||
cookie->name = strbuf_detach(&cookie_filename, NULL);
|
||||
cookie->result = FCIR_INIT;
|
||||
hashmap_entry_init(&cookie->entry, strhash(cookie->name));
|
||||
|
||||
hashmap_add(&state->cookies, &cookie->entry);
|
||||
|
||||
trace_printf_key(&trace_fsmonitor, "cookie-wait: '%s' '%s'",
|
||||
cookie->name, cookie_pathname.buf);
|
||||
|
||||
/*
|
||||
* Create the cookie file on disk and then wait for a notification
|
||||
* that the listener thread has seen it.
|
||||
*/
|
||||
fd = open(cookie_pathname.buf, O_WRONLY | O_CREAT | O_EXCL, 0600);
|
||||
if (fd < 0) {
|
||||
error_errno(_("could not create fsmonitor cookie '%s'"),
|
||||
cookie->name);
|
||||
|
||||
cookie->result = FCIR_ERROR;
|
||||
goto done;
|
||||
}
|
||||
|
||||
/*
|
||||
* Technically, close() and unlink() can fail, but we don't
|
||||
* care here. We only created the file to trigger a watch
|
||||
* event from the FS to know that when we're up to date.
|
||||
*/
|
||||
close(fd);
|
||||
unlink(cookie_pathname.buf);
|
||||
|
||||
/*
|
||||
* Technically, this is an infinite wait (well, unless another
|
||||
* thread sends us an abort). I'd like to change this to
|
||||
* use `pthread_cond_timedwait()` and return an error/timeout
|
||||
* and let the caller do the trivial response thing, but we
|
||||
* don't have that routine in our thread-utils.
|
||||
*
|
||||
* After extensive beta testing I'm not really worried about
|
||||
* this. Also note that the above open() and unlink() calls
|
||||
* will cause at least two FS events on that path, so the odds
|
||||
* of getting stuck are pretty slim.
|
||||
*/
|
||||
while (cookie->result == FCIR_INIT)
|
||||
pthread_cond_wait(&state->cookies_cond,
|
||||
&state->main_lock);
|
||||
|
||||
done:
|
||||
hashmap_remove(&state->cookies, &cookie->entry, NULL);
|
||||
|
||||
result = cookie->result;
|
||||
|
||||
free(cookie->name);
|
||||
free(cookie);
|
||||
strbuf_release(&cookie_pathname);
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/*
|
||||
* Mark these cookies as _SEEN and wake up the corresponding client threads.
|
||||
*/
|
||||
static void with_lock__mark_cookies_seen(struct fsmonitor_daemon_state *state,
|
||||
const struct string_list *cookie_names)
|
||||
{
|
||||
/* assert current thread holding state->main_lock */
|
||||
|
||||
int k;
|
||||
int nr_seen = 0;
|
||||
|
||||
for (k = 0; k < cookie_names->nr; k++) {
|
||||
struct fsmonitor_cookie_item key;
|
||||
struct fsmonitor_cookie_item *cookie;
|
||||
|
||||
key.name = cookie_names->items[k].string;
|
||||
hashmap_entry_init(&key.entry, strhash(key.name));
|
||||
|
||||
cookie = hashmap_get_entry(&state->cookies, &key, entry, NULL);
|
||||
if (cookie) {
|
||||
trace_printf_key(&trace_fsmonitor, "cookie-seen: '%s'",
|
||||
cookie->name);
|
||||
cookie->result = FCIR_SEEN;
|
||||
nr_seen++;
|
||||
}
|
||||
}
|
||||
|
||||
if (nr_seen)
|
||||
pthread_cond_broadcast(&state->cookies_cond);
|
||||
}
|
||||
|
||||
/*
|
||||
* Set _ABORT on all pending cookies and wake up all client threads.
|
||||
*/
|
||||
static void with_lock__abort_all_cookies(struct fsmonitor_daemon_state *state)
|
||||
{
|
||||
/* assert current thread holding state->main_lock */
|
||||
|
||||
struct hashmap_iter iter;
|
||||
struct fsmonitor_cookie_item *cookie;
|
||||
int nr_aborted = 0;
|
||||
|
||||
hashmap_for_each_entry(&state->cookies, &iter, cookie, entry) {
|
||||
trace_printf_key(&trace_fsmonitor, "cookie-abort: '%s'",
|
||||
cookie->name);
|
||||
cookie->result = FCIR_ABORT;
|
||||
nr_aborted++;
|
||||
}
|
||||
|
||||
if (nr_aborted)
|
||||
pthread_cond_broadcast(&state->cookies_cond);
|
||||
}
|
||||
|
||||
/*
|
||||
* Requests to and from a FSMonitor Protocol V2 provider use an opaque
|
||||
* "token" as a virtual timestamp. Clients can request a summary of all
|
||||
* created/deleted/modified files relative to a token. In the response,
|
||||
* clients receive a new token for the next (relative) request.
|
||||
*
|
||||
*
|
||||
* Token Format
|
||||
* ============
|
||||
*
|
||||
* The contents of the token are private and provider-specific.
|
||||
*
|
||||
* For the built-in fsmonitor--daemon, we define a token as follows:
|
||||
*
|
||||
* "builtin" ":" <token_id> ":" <sequence_nr>
|
||||
*
|
||||
* The "builtin" prefix is used as a namespace to avoid conflicts
|
||||
* with other providers (such as Watchman).
|
||||
*
|
||||
* The <token_id> is an arbitrary OPAQUE string, such as a GUID,
|
||||
* UUID, or {timestamp,pid}. It is used to group all filesystem
|
||||
* events that happened while the daemon was monitoring (and in-sync
|
||||
* with the filesystem).
|
||||
*
|
||||
* Unlike FSMonitor Protocol V1, it is not defined as a timestamp
|
||||
* and does not define less-than/greater-than relationships.
|
||||
* (There are too many race conditions to rely on file system
|
||||
* event timestamps.)
|
||||
*
|
||||
* The <sequence_nr> is a simple integer incremented whenever the
|
||||
* daemon needs to make its state public. For example, if 1000 file
|
||||
* system events come in, but no clients have requested the data,
|
||||
* the daemon can continue to accumulate file changes in the same
|
||||
* bin and does not need to advance the sequence number. However,
|
||||
* as soon as a client does arrive, the daemon needs to start a new
|
||||
* bin and increment the sequence number.
|
||||
*
|
||||
* The sequence number serves as the boundary between 2 sets
|
||||
* of bins -- the older ones that the client has already seen
|
||||
* and the newer ones that it hasn't.
|
||||
*
|
||||
* When a new <token_id> is created, the <sequence_nr> is reset to
|
||||
* zero.
|
||||
*
|
||||
*
|
||||
* About Token Ids
|
||||
* ===============
|
||||
*
|
||||
* A new token_id is created:
|
||||
*
|
||||
* [1] each time the daemon is started.
|
||||
*
|
||||
* [2] any time that the daemon must re-sync with the filesystem
|
||||
* (such as when the kernel drops or we miss events on a very
|
||||
* active volume).
|
||||
*
|
||||
* [3] in response to a client "flush" command (for dropped event
|
||||
* testing).
|
||||
*
|
||||
* When a new token_id is created, the daemon is free to discard all
|
||||
* cached filesystem events associated with any previous token_ids.
|
||||
* Events associated with a non-current token_id will never be sent
|
||||
* to a client. A token_id change implicitly means that the daemon
|
||||
* has gap in its event history.
|
||||
*
|
||||
* Therefore, clients that present a token with a stale (non-current)
|
||||
* token_id will always be given a trivial response.
|
||||
*/
|
||||
struct fsmonitor_token_data {
|
||||
struct strbuf token_id;
|
||||
struct fsmonitor_batch *batch_head;
|
||||
struct fsmonitor_batch *batch_tail;
|
||||
uint64_t client_ref_count;
|
||||
};
|
||||
|
||||
struct fsmonitor_batch {
|
||||
struct fsmonitor_batch *next;
|
||||
uint64_t batch_seq_nr;
|
||||
const char **interned_paths;
|
||||
size_t nr, alloc;
|
||||
time_t pinned_time;
|
||||
};
|
||||
|
||||
static struct fsmonitor_token_data *fsmonitor_new_token_data(void)
|
||||
{
|
||||
static int test_env_value = -1;
|
||||
static uint64_t flush_count = 0;
|
||||
struct fsmonitor_token_data *token;
|
||||
struct fsmonitor_batch *batch;
|
||||
|
||||
CALLOC_ARRAY(token, 1);
|
||||
batch = fsmonitor_batch__new();
|
||||
|
||||
strbuf_init(&token->token_id, 0);
|
||||
token->batch_head = batch;
|
||||
token->batch_tail = batch;
|
||||
token->client_ref_count = 0;
|
||||
|
||||
if (test_env_value < 0)
|
||||
test_env_value = git_env_bool("GIT_TEST_FSMONITOR_TOKEN", 0);
|
||||
|
||||
if (!test_env_value) {
|
||||
struct timeval tv;
|
||||
struct tm tm;
|
||||
time_t secs;
|
||||
|
||||
gettimeofday(&tv, NULL);
|
||||
secs = tv.tv_sec;
|
||||
gmtime_r(&secs, &tm);
|
||||
|
||||
strbuf_addf(&token->token_id,
|
||||
"%"PRIu64".%d.%4d%02d%02dT%02d%02d%02d.%06ldZ",
|
||||
flush_count++,
|
||||
getpid(),
|
||||
tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday,
|
||||
tm.tm_hour, tm.tm_min, tm.tm_sec,
|
||||
(long)tv.tv_usec);
|
||||
} else {
|
||||
strbuf_addf(&token->token_id, "test_%08x", test_env_value++);
|
||||
}
|
||||
|
||||
/*
|
||||
* We created a new <token_id> and are starting a new series
|
||||
* of tokens with a zero <seq_nr>.
|
||||
*
|
||||
* Since clients cannot guess our new (non test) <token_id>
|
||||
* they will always receive a trivial response (because of the
|
||||
* mismatch on the <token_id>). The trivial response will
|
||||
* tell them our new <token_id> so that subsequent requests
|
||||
* will be relative to our new series. (And when sending that
|
||||
* response, we pin the current head of the batch list.)
|
||||
*
|
||||
* Even if the client correctly guesses the <token_id>, their
|
||||
* request of "builtin:<token_id>:0" asks for all changes MORE
|
||||
* RECENT than batch/bin 0.
|
||||
*
|
||||
* This implies that it is a waste to accumulate paths in the
|
||||
* initial batch/bin (because they will never be transmitted).
|
||||
*
|
||||
* So the daemon could be running for days and watching the
|
||||
* file system, but doesn't need to actually accumulate any
|
||||
* paths UNTIL we need to set a reference point for a later
|
||||
* relative request.
|
||||
*
|
||||
* However, it is very useful for testing to always have a
|
||||
* reference point set. Pin batch 0 to force early file system
|
||||
* events to accumulate.
|
||||
*/
|
||||
if (test_env_value)
|
||||
batch->pinned_time = time(NULL);
|
||||
|
||||
return token;
|
||||
}
|
||||
|
||||
struct fsmonitor_batch *fsmonitor_batch__new(void)
|
||||
{
|
||||
struct fsmonitor_batch *batch;
|
||||
|
||||
CALLOC_ARRAY(batch, 1);
|
||||
|
||||
return batch;
|
||||
}
|
||||
|
||||
void fsmonitor_batch__free_list(struct fsmonitor_batch *batch)
|
||||
{
|
||||
while (batch) {
|
||||
struct fsmonitor_batch *next = batch->next;
|
||||
|
||||
/*
|
||||
* The actual strings within the array of this batch
|
||||
* are interned, so we don't own them. We only own
|
||||
* the array.
|
||||
*/
|
||||
free(batch->interned_paths);
|
||||
free(batch);
|
||||
|
||||
batch = next;
|
||||
}
|
||||
}
|
||||
|
||||
void fsmonitor_batch__add_path(struct fsmonitor_batch *batch,
|
||||
const char *path)
|
||||
{
|
||||
const char *interned_path = strintern(path);
|
||||
|
||||
trace_printf_key(&trace_fsmonitor, "event: %s", interned_path);
|
||||
|
||||
ALLOC_GROW(batch->interned_paths, batch->nr + 1, batch->alloc);
|
||||
batch->interned_paths[batch->nr++] = interned_path;
|
||||
}
|
||||
|
||||
static void fsmonitor_batch__combine(struct fsmonitor_batch *batch_dest,
|
||||
const struct fsmonitor_batch *batch_src)
|
||||
{
|
||||
size_t k;
|
||||
|
||||
ALLOC_GROW(batch_dest->interned_paths,
|
||||
batch_dest->nr + batch_src->nr + 1,
|
||||
batch_dest->alloc);
|
||||
|
||||
for (k = 0; k < batch_src->nr; k++)
|
||||
batch_dest->interned_paths[batch_dest->nr++] =
|
||||
batch_src->interned_paths[k];
|
||||
}
|
||||
|
||||
/*
|
||||
* To keep the batch list from growing unbounded in response to filesystem
|
||||
* activity, we try to truncate old batches from the end of the list as
|
||||
* they become irrelevant.
|
||||
*
|
||||
* We assume that the .git/index will be updated with the most recent token
|
||||
* any time the index is updated. And future commands will only ask for
|
||||
* recent changes *since* that new token. So as tokens advance into the
|
||||
* future, older batch items will never be requested/needed. So we can
|
||||
* truncate them without loss of functionality.
|
||||
*
|
||||
* However, multiple commands may be talking to the daemon concurrently
|
||||
* or perform a slow command, so a little "token skew" is possible.
|
||||
* Therefore, we want this to be a little bit lazy and have a generous
|
||||
* delay.
|
||||
*
|
||||
* The current reader thread walked backwards in time from `token->batch_head`
|
||||
* back to `batch_marker` somewhere in the middle of the batch list.
|
||||
*
|
||||
* Let's walk backwards in time from that marker an arbitrary delay
|
||||
* and truncate the list there. Note that these timestamps are completely
|
||||
* artificial (based on when we pinned the batch item) and not on any
|
||||
* filesystem activity.
|
||||
*
|
||||
* Return the obsolete portion of the list after we have removed it from
|
||||
* the official list so that the caller can free it after leaving the lock.
|
||||
*/
|
||||
#define MY_TIME_DELAY_SECONDS (5 * 60) /* seconds */
|
||||
|
||||
static struct fsmonitor_batch *with_lock__truncate_old_batches(
|
||||
struct fsmonitor_daemon_state *state,
|
||||
const struct fsmonitor_batch *batch_marker)
|
||||
{
|
||||
/* assert current thread holding state->main_lock */
|
||||
|
||||
const struct fsmonitor_batch *batch;
|
||||
struct fsmonitor_batch *remainder;
|
||||
|
||||
if (!batch_marker)
|
||||
return NULL;
|
||||
|
||||
trace_printf_key(&trace_fsmonitor, "Truncate: mark (%"PRIu64",%"PRIu64")",
|
||||
batch_marker->batch_seq_nr,
|
||||
(uint64_t)batch_marker->pinned_time);
|
||||
|
||||
for (batch = batch_marker; batch; batch = batch->next) {
|
||||
time_t t;
|
||||
|
||||
if (!batch->pinned_time) /* an overflow batch */
|
||||
continue;
|
||||
|
||||
t = batch->pinned_time + MY_TIME_DELAY_SECONDS;
|
||||
if (t > batch_marker->pinned_time) /* too close to marker */
|
||||
continue;
|
||||
|
||||
goto truncate_past_here;
|
||||
}
|
||||
|
||||
return NULL;
|
||||
|
||||
truncate_past_here:
|
||||
state->current_token_data->batch_tail = (struct fsmonitor_batch *)batch;
|
||||
|
||||
remainder = ((struct fsmonitor_batch *)batch)->next;
|
||||
((struct fsmonitor_batch *)batch)->next = NULL;
|
||||
|
||||
return remainder;
|
||||
}
|
||||
|
||||
static void fsmonitor_free_token_data(struct fsmonitor_token_data *token)
|
||||
{
|
||||
if (!token)
|
||||
return;
|
||||
|
||||
assert(token->client_ref_count == 0);
|
||||
|
||||
strbuf_release(&token->token_id);
|
||||
|
||||
fsmonitor_batch__free_list(token->batch_head);
|
||||
|
||||
free(token);
|
||||
}
|
||||
|
||||
/*
|
||||
* Flush all of our cached data about the filesystem. Call this if we
|
||||
* lose sync with the filesystem and miss some notification events.
|
||||
*
|
||||
* [1] If we are missing events, then we no longer have a complete
|
||||
* history of the directory (relative to our current start token).
|
||||
* We should create a new token and start fresh (as if we just
|
||||
* booted up).
|
||||
*
|
||||
* [2] Some of those lost events may have been for cookie files. We
|
||||
* should assume the worst and abort them rather letting them starve.
|
||||
*
|
||||
* If there are no concurrent threads reading the current token data
|
||||
* series, we can free it now. Otherwise, let the last reader free
|
||||
* it.
|
||||
*
|
||||
* Either way, the old token data series is no longer associated with
|
||||
* our state data.
|
||||
*/
|
||||
static void with_lock__do_force_resync(struct fsmonitor_daemon_state *state)
|
||||
{
|
||||
/* assert current thread holding state->main_lock */
|
||||
|
||||
struct fsmonitor_token_data *free_me = NULL;
|
||||
struct fsmonitor_token_data *new_one = NULL;
|
||||
|
||||
new_one = fsmonitor_new_token_data();
|
||||
|
||||
if (state->current_token_data->client_ref_count == 0)
|
||||
free_me = state->current_token_data;
|
||||
state->current_token_data = new_one;
|
||||
|
||||
fsmonitor_free_token_data(free_me);
|
||||
|
||||
with_lock__abort_all_cookies(state);
|
||||
}
|
||||
|
||||
void fsmonitor_force_resync(struct fsmonitor_daemon_state *state)
|
||||
{
|
||||
pthread_mutex_lock(&state->main_lock);
|
||||
with_lock__do_force_resync(state);
|
||||
pthread_mutex_unlock(&state->main_lock);
|
||||
}
|
||||
|
||||
/*
|
||||
* Format an opaque token string to send to the client.
|
||||
*/
|
||||
static void with_lock__format_response_token(
|
||||
struct strbuf *response_token,
|
||||
const struct strbuf *response_token_id,
|
||||
const struct fsmonitor_batch *batch)
|
||||
{
|
||||
/* assert current thread holding state->main_lock */
|
||||
|
||||
strbuf_reset(response_token);
|
||||
strbuf_addf(response_token, "builtin:%s:%"PRIu64,
|
||||
response_token_id->buf, batch->batch_seq_nr);
|
||||
}
|
||||
|
||||
/*
|
||||
* Parse an opaque token from the client.
|
||||
* Returns -1 on error.
|
||||
*/
|
||||
static int fsmonitor_parse_client_token(const char *buf_token,
|
||||
struct strbuf *requested_token_id,
|
||||
uint64_t *seq_nr)
|
||||
{
|
||||
const char *p;
|
||||
char *p_end;
|
||||
|
||||
strbuf_reset(requested_token_id);
|
||||
*seq_nr = 0;
|
||||
|
||||
if (!skip_prefix(buf_token, "builtin:", &p))
|
||||
return -1;
|
||||
|
||||
while (*p && *p != ':')
|
||||
strbuf_addch(requested_token_id, *p++);
|
||||
if (!*p++)
|
||||
return -1;
|
||||
|
||||
*seq_nr = (uint64_t)strtoumax(p, &p_end, 10);
|
||||
if (*p_end)
|
||||
return -1;
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
KHASH_INIT(str, const char *, int, 0, kh_str_hash_func, kh_str_hash_equal)
|
||||
|
||||
static int do_handle_client(struct fsmonitor_daemon_state *state,
|
||||
const char *command,
|
||||
ipc_server_reply_cb *reply,
|
||||
struct ipc_server_reply_data *reply_data)
|
||||
{
|
||||
struct fsmonitor_token_data *token_data = NULL;
|
||||
struct strbuf response_token = STRBUF_INIT;
|
||||
struct strbuf requested_token_id = STRBUF_INIT;
|
||||
struct strbuf payload = STRBUF_INIT;
|
||||
uint64_t requested_oldest_seq_nr = 0;
|
||||
uint64_t total_response_len = 0;
|
||||
const char *p;
|
||||
const struct fsmonitor_batch *batch_head;
|
||||
const struct fsmonitor_batch *batch;
|
||||
struct fsmonitor_batch *remainder = NULL;
|
||||
intmax_t count = 0, duplicates = 0;
|
||||
kh_str_t *shown;
|
||||
int hash_ret;
|
||||
int do_trivial = 0;
|
||||
int do_flush = 0;
|
||||
int do_cookie = 0;
|
||||
enum fsmonitor_cookie_item_result cookie_result;
|
||||
|
||||
/*
|
||||
* We expect `command` to be of the form:
|
||||
*
|
||||
* <command> := quit NUL
|
||||
* | flush NUL
|
||||
* | <V1-time-since-epoch-ns> NUL
|
||||
* | <V2-opaque-fsmonitor-token> NUL
|
||||
*/
|
||||
|
||||
if (!strcmp(command, "quit")) {
|
||||
/*
|
||||
* A client has requested over the socket/pipe that the
|
||||
* daemon shutdown.
|
||||
*
|
||||
* Tell the IPC thread pool to shutdown (which completes
|
||||
* the await in the main thread (which can stop the
|
||||
* fsmonitor listener thread)).
|
||||
*
|
||||
* There is no reply to the client.
|
||||
*/
|
||||
return SIMPLE_IPC_QUIT;
|
||||
|
||||
} else if (!strcmp(command, "flush")) {
|
||||
/*
|
||||
* Flush all of our cached data and generate a new token
|
||||
* just like if we lost sync with the filesystem.
|
||||
*
|
||||
* Then send a trivial response using the new token.
|
||||
*/
|
||||
do_flush = 1;
|
||||
do_trivial = 1;
|
||||
|
||||
} else if (!skip_prefix(command, "builtin:", &p)) {
|
||||
/* assume V1 timestamp or garbage */
|
||||
|
||||
char *p_end;
|
||||
|
||||
strtoumax(command, &p_end, 10);
|
||||
trace_printf_key(&trace_fsmonitor,
|
||||
((*p_end) ?
|
||||
"fsmonitor: invalid command line '%s'" :
|
||||
"fsmonitor: unsupported V1 protocol '%s'"),
|
||||
command);
|
||||
do_trivial = 1;
|
||||
|
||||
} else {
|
||||
/* We have "builtin:*" */
|
||||
if (fsmonitor_parse_client_token(command, &requested_token_id,
|
||||
&requested_oldest_seq_nr)) {
|
||||
trace_printf_key(&trace_fsmonitor,
|
||||
"fsmonitor: invalid V2 protocol token '%s'",
|
||||
command);
|
||||
do_trivial = 1;
|
||||
|
||||
} else {
|
||||
/*
|
||||
* We have a V2 valid token:
|
||||
* "builtin:<token_id>:<seq_nr>"
|
||||
*/
|
||||
do_cookie = 1;
|
||||
}
|
||||
}
|
||||
|
||||
pthread_mutex_lock(&state->main_lock);
|
||||
|
||||
if (!state->current_token_data)
|
||||
BUG("fsmonitor state does not have a current token");
|
||||
|
||||
/*
|
||||
* Write a cookie file inside the directory being watched in
|
||||
* an effort to flush out existing filesystem events that we
|
||||
* actually care about. Suspend this client thread until we
|
||||
* see the filesystem events for this cookie file.
|
||||
*
|
||||
* Creating the cookie lets us guarantee that our FS listener
|
||||
* thread has drained the kernel queue and we are caught up
|
||||
* with the kernel.
|
||||
*
|
||||
* If we cannot create the cookie (or otherwise guarantee that
|
||||
* we are caught up), we send a trivial response. We have to
|
||||
* assume that there might be some very, very recent activity
|
||||
* on the FS still in flight.
|
||||
*/
|
||||
if (do_cookie) {
|
||||
cookie_result = with_lock__wait_for_cookie(state);
|
||||
if (cookie_result != FCIR_SEEN) {
|
||||
error(_("fsmonitor: cookie_result '%d' != SEEN"),
|
||||
cookie_result);
|
||||
do_trivial = 1;
|
||||
}
|
||||
}
|
||||
|
||||
if (do_flush)
|
||||
with_lock__do_force_resync(state);
|
||||
|
||||
/*
|
||||
* We mark the current head of the batch list as "pinned" so
|
||||
* that the listener thread will treat this item as read-only
|
||||
* (and prevent any more paths from being added to it) from
|
||||
* now on.
|
||||
*/
|
||||
token_data = state->current_token_data;
|
||||
batch_head = token_data->batch_head;
|
||||
((struct fsmonitor_batch *)batch_head)->pinned_time = time(NULL);
|
||||
|
||||
/*
|
||||
* FSMonitor Protocol V2 requires that we send a response header
|
||||
* with a "new current token" and then all of the paths that changed
|
||||
* since the "requested token". We send the seq_nr of the just-pinned
|
||||
* head batch so that future requests from a client will be relative
|
||||
* to it.
|
||||
*/
|
||||
with_lock__format_response_token(&response_token,
|
||||
&token_data->token_id, batch_head);
|
||||
|
||||
reply(reply_data, response_token.buf, response_token.len + 1);
|
||||
total_response_len += response_token.len + 1;
|
||||
|
||||
trace2_data_string("fsmonitor", the_repository, "response/token",
|
||||
response_token.buf);
|
||||
trace_printf_key(&trace_fsmonitor, "response token: %s",
|
||||
response_token.buf);
|
||||
|
||||
if (!do_trivial) {
|
||||
if (strcmp(requested_token_id.buf, token_data->token_id.buf)) {
|
||||
/*
|
||||
* The client last spoke to a different daemon
|
||||
* instance -OR- the daemon had to resync with
|
||||
* the filesystem (and lost events), so reject.
|
||||
*/
|
||||
trace2_data_string("fsmonitor", the_repository,
|
||||
"response/token", "different");
|
||||
do_trivial = 1;
|
||||
|
||||
} else if (requested_oldest_seq_nr <
|
||||
token_data->batch_tail->batch_seq_nr) {
|
||||
/*
|
||||
* The client wants older events than we have for
|
||||
* this token_id. This means that the end of our
|
||||
* batch list was truncated and we cannot give the
|
||||
* client a complete snapshot relative to their
|
||||
* request.
|
||||
*/
|
||||
trace_printf_key(&trace_fsmonitor,
|
||||
"client requested truncated data");
|
||||
do_trivial = 1;
|
||||
}
|
||||
}
|
||||
|
||||
if (do_trivial) {
|
||||
pthread_mutex_unlock(&state->main_lock);
|
||||
|
||||
reply(reply_data, "/", 2);
|
||||
|
||||
trace2_data_intmax("fsmonitor", the_repository,
|
||||
"response/trivial", 1);
|
||||
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
/*
|
||||
* We're going to hold onto a pointer to the current
|
||||
* token-data while we walk the list of batches of files.
|
||||
* During this time, we will NOT be under the lock.
|
||||
* So we ref-count it.
|
||||
*
|
||||
* This allows the listener thread to continue prepending
|
||||
* new batches of items to the token-data (which we'll ignore).
|
||||
*
|
||||
* AND it allows the listener thread to do a token-reset
|
||||
* (and install a new `current_token_data`).
|
||||
*/
|
||||
token_data->client_ref_count++;
|
||||
|
||||
pthread_mutex_unlock(&state->main_lock);
|
||||
|
||||
/*
|
||||
* The client request is relative to the token that they sent,
|
||||
* so walk the batch list backwards from the current head back
|
||||
* to the batch (sequence number) they named.
|
||||
*
|
||||
* We use khash to de-dup the list of pathnames.
|
||||
*
|
||||
* NEEDSWORK: each batch contains a list of interned strings,
|
||||
* so we only need to do pointer comparisons here to build the
|
||||
* hash table. Currently, we're still comparing the string
|
||||
* values.
|
||||
*/
|
||||
shown = kh_init_str();
|
||||
for (batch = batch_head;
|
||||
batch && batch->batch_seq_nr > requested_oldest_seq_nr;
|
||||
batch = batch->next) {
|
||||
size_t k;
|
||||
|
||||
for (k = 0; k < batch->nr; k++) {
|
||||
const char *s = batch->interned_paths[k];
|
||||
size_t s_len;
|
||||
|
||||
if (kh_get_str(shown, s) != kh_end(shown))
|
||||
duplicates++;
|
||||
else {
|
||||
kh_put_str(shown, s, &hash_ret);
|
||||
|
||||
trace_printf_key(&trace_fsmonitor,
|
||||
"send[%"PRIuMAX"]: %s",
|
||||
count, s);
|
||||
|
||||
/* Each path gets written with a trailing NUL */
|
||||
s_len = strlen(s) + 1;
|
||||
|
||||
if (payload.len + s_len >=
|
||||
LARGE_PACKET_DATA_MAX) {
|
||||
reply(reply_data, payload.buf,
|
||||
payload.len);
|
||||
total_response_len += payload.len;
|
||||
strbuf_reset(&payload);
|
||||
}
|
||||
|
||||
strbuf_add(&payload, s, s_len);
|
||||
count++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (payload.len) {
|
||||
reply(reply_data, payload.buf, payload.len);
|
||||
total_response_len += payload.len;
|
||||
}
|
||||
|
||||
kh_release_str(shown);
|
||||
|
||||
pthread_mutex_lock(&state->main_lock);
|
||||
|
||||
if (token_data->client_ref_count > 0)
|
||||
token_data->client_ref_count--;
|
||||
|
||||
if (token_data->client_ref_count == 0) {
|
||||
if (token_data != state->current_token_data) {
|
||||
/*
|
||||
* The listener thread did a token-reset while we were
|
||||
* walking the batch list. Therefore, this token is
|
||||
* stale and can be discarded completely. If we are
|
||||
* the last reader thread using this token, we own
|
||||
* that work.
|
||||
*/
|
||||
fsmonitor_free_token_data(token_data);
|
||||
} else if (batch) {
|
||||
/*
|
||||
* We are holding the lock and are the only
|
||||
* reader of the ref-counted portion of the
|
||||
* list, so we get the honor of seeing if the
|
||||
* list can be truncated to save memory.
|
||||
*
|
||||
* The main loop did not walk to the end of the
|
||||
* list, so this batch is the first item in the
|
||||
* batch-list that is older than the requested
|
||||
* end-point sequence number. See if the tail
|
||||
* end of the list is obsolete.
|
||||
*/
|
||||
remainder = with_lock__truncate_old_batches(state,
|
||||
batch);
|
||||
}
|
||||
}
|
||||
|
||||
pthread_mutex_unlock(&state->main_lock);
|
||||
|
||||
if (remainder)
|
||||
fsmonitor_batch__free_list(remainder);
|
||||
|
||||
trace2_data_intmax("fsmonitor", the_repository, "response/length", total_response_len);
|
||||
trace2_data_intmax("fsmonitor", the_repository, "response/count/files", count);
|
||||
trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates);
|
||||
|
||||
cleanup:
|
||||
strbuf_release(&response_token);
|
||||
strbuf_release(&requested_token_id);
|
||||
strbuf_release(&payload);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
static ipc_server_application_cb handle_client;
|
||||
|
||||
static int handle_client(void *data,
|
||||
const char *command, size_t command_len,
|
||||
ipc_server_reply_cb *reply,
|
||||
struct ipc_server_reply_data *reply_data)
|
||||
{
|
||||
struct fsmonitor_daemon_state *state = data;
|
||||
int result;
|
||||
|
||||
/*
|
||||
* The Simple IPC API now supports {char*, len} arguments, but
|
||||
* FSMonitor always uses proper null-terminated strings, so
|
||||
* we can ignore the command_len argument. (Trust, but verify.)
|
||||
*/
|
||||
if (command_len != strlen(command))
|
||||
BUG("FSMonitor assumes text messages");
|
||||
|
||||
trace_printf_key(&trace_fsmonitor, "requested token: %s", command);
|
||||
|
||||
trace2_region_enter("fsmonitor", "handle_client", the_repository);
|
||||
trace2_data_string("fsmonitor", the_repository, "request", command);
|
||||
|
||||
result = do_handle_client(state, command, reply, reply_data);
|
||||
|
||||
trace2_region_leave("fsmonitor", "handle_client", the_repository);
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
#define FSMONITOR_DIR "fsmonitor--daemon"
|
||||
#define FSMONITOR_COOKIE_DIR "cookies"
|
||||
#define FSMONITOR_COOKIE_PREFIX (FSMONITOR_DIR "/" FSMONITOR_COOKIE_DIR "/")
|
||||
|
||||
enum fsmonitor_path_type fsmonitor_classify_path_workdir_relative(
|
||||
const char *rel)
|
||||
{
|
||||
if (fspathncmp(rel, ".git", 4))
|
||||
return IS_WORKDIR_PATH;
|
||||
rel += 4;
|
||||
|
||||
if (!*rel)
|
||||
return IS_DOT_GIT;
|
||||
if (*rel != '/')
|
||||
return IS_WORKDIR_PATH; /* e.g. .gitignore */
|
||||
rel++;
|
||||
|
||||
if (!fspathncmp(rel, FSMONITOR_COOKIE_PREFIX,
|
||||
strlen(FSMONITOR_COOKIE_PREFIX)))
|
||||
return IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX;
|
||||
|
||||
return IS_INSIDE_DOT_GIT;
|
||||
}
|
||||
|
||||
enum fsmonitor_path_type fsmonitor_classify_path_gitdir_relative(
|
||||
const char *rel)
|
||||
{
|
||||
if (!fspathncmp(rel, FSMONITOR_COOKIE_PREFIX,
|
||||
strlen(FSMONITOR_COOKIE_PREFIX)))
|
||||
return IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX;
|
||||
|
||||
return IS_INSIDE_GITDIR;
|
||||
}
|
||||
|
||||
static enum fsmonitor_path_type try_classify_workdir_abs_path(
|
||||
struct fsmonitor_daemon_state *state,
|
||||
const char *path)
|
||||
{
|
||||
const char *rel;
|
||||
|
||||
if (fspathncmp(path, state->path_worktree_watch.buf,
|
||||
state->path_worktree_watch.len))
|
||||
return IS_OUTSIDE_CONE;
|
||||
|
||||
rel = path + state->path_worktree_watch.len;
|
||||
|
||||
if (!*rel)
|
||||
return IS_WORKDIR_PATH; /* it is the root dir exactly */
|
||||
if (*rel != '/')
|
||||
return IS_OUTSIDE_CONE;
|
||||
rel++;
|
||||
|
||||
return fsmonitor_classify_path_workdir_relative(rel);
|
||||
}
|
||||
|
||||
enum fsmonitor_path_type fsmonitor_classify_path_absolute(
|
||||
struct fsmonitor_daemon_state *state,
|
||||
const char *path)
|
||||
{
|
||||
const char *rel;
|
||||
enum fsmonitor_path_type t;
|
||||
|
||||
t = try_classify_workdir_abs_path(state, path);
|
||||
if (state->nr_paths_watching == 1)
|
||||
return t;
|
||||
if (t != IS_OUTSIDE_CONE)
|
||||
return t;
|
||||
|
||||
if (fspathncmp(path, state->path_gitdir_watch.buf,
|
||||
state->path_gitdir_watch.len))
|
||||
return IS_OUTSIDE_CONE;
|
||||
|
||||
rel = path + state->path_gitdir_watch.len;
|
||||
|
||||
if (!*rel)
|
||||
return IS_GITDIR; /* it is the <gitdir> exactly */
|
||||
if (*rel != '/')
|
||||
return IS_OUTSIDE_CONE;
|
||||
rel++;
|
||||
|
||||
return fsmonitor_classify_path_gitdir_relative(rel);
|
||||
}
|
||||
|
||||
/*
|
||||
* We try to combine small batches at the front of the batch-list to avoid
|
||||
* having a long list. This hopefully makes it a little easier when we want
|
||||
* to truncate and maintain the list. However, we don't want the paths array
|
||||
* to just keep growing and growing with realloc, so we insert an arbitrary
|
||||
* limit.
|
||||
*/
|
||||
#define MY_COMBINE_LIMIT (1024)
|
||||
|
||||
void fsmonitor_publish(struct fsmonitor_daemon_state *state,
|
||||
struct fsmonitor_batch *batch,
|
||||
const struct string_list *cookie_names)
|
||||
{
|
||||
if (!batch && !cookie_names->nr)
|
||||
return;
|
||||
|
||||
pthread_mutex_lock(&state->main_lock);
|
||||
|
||||
if (batch) {
|
||||
struct fsmonitor_batch *head;
|
||||
|
||||
head = state->current_token_data->batch_head;
|
||||
if (!head) {
|
||||
BUG("token does not have batch");
|
||||
} else if (head->pinned_time) {
|
||||
/*
|
||||
* We cannot alter the current batch list
|
||||
* because:
|
||||
*
|
||||
* [a] it is being transmitted to at least one
|
||||
* client and the handle_client() thread has a
|
||||
* ref-count, but not a lock on the batch list
|
||||
* starting with this item.
|
||||
*
|
||||
* [b] it has been transmitted in the past to
|
||||
* at least one client such that future
|
||||
* requests are relative to this head batch.
|
||||
*
|
||||
* So, we can only prepend a new batch onto
|
||||
* the front of the list.
|
||||
*/
|
||||
batch->batch_seq_nr = head->batch_seq_nr + 1;
|
||||
batch->next = head;
|
||||
state->current_token_data->batch_head = batch;
|
||||
} else if (!head->batch_seq_nr) {
|
||||
/*
|
||||
* Batch 0 is unpinned. See the note in
|
||||
* `fsmonitor_new_token_data()` about why we
|
||||
* don't need to accumulate these paths.
|
||||
*/
|
||||
fsmonitor_batch__free_list(batch);
|
||||
} else if (head->nr + batch->nr > MY_COMBINE_LIMIT) {
|
||||
/*
|
||||
* The head batch in the list has never been
|
||||
* transmitted to a client, but folding the
|
||||
* contents of the new batch onto it would
|
||||
* exceed our arbitrary limit, so just prepend
|
||||
* the new batch onto the list.
|
||||
*/
|
||||
batch->batch_seq_nr = head->batch_seq_nr + 1;
|
||||
batch->next = head;
|
||||
state->current_token_data->batch_head = batch;
|
||||
} else {
|
||||
/*
|
||||
* We are free to add the paths in the given
|
||||
* batch onto the end of the current head batch.
|
||||
*/
|
||||
fsmonitor_batch__combine(head, batch);
|
||||
fsmonitor_batch__free_list(batch);
|
||||
}
|
||||
}
|
||||
|
||||
if (cookie_names->nr)
|
||||
with_lock__mark_cookies_seen(state, cookie_names);
|
||||
|
||||
pthread_mutex_unlock(&state->main_lock);
|
||||
}
|
||||
|
||||
static void *fsm_listen__thread_proc(void *_state)
|
||||
{
|
||||
struct fsmonitor_daemon_state *state = _state;
|
||||
|
||||
trace2_thread_start("fsm-listen");
|
||||
|
||||
trace_printf_key(&trace_fsmonitor, "Watching: worktree '%s'",
|
||||
state->path_worktree_watch.buf);
|
||||
if (state->nr_paths_watching > 1)
|
||||
trace_printf_key(&trace_fsmonitor, "Watching: gitdir '%s'",
|
||||
state->path_gitdir_watch.buf);
|
||||
|
||||
fsm_listen__loop(state);
|
||||
|
||||
pthread_mutex_lock(&state->main_lock);
|
||||
if (state->current_token_data &&
|
||||
state->current_token_data->client_ref_count == 0)
|
||||
fsmonitor_free_token_data(state->current_token_data);
|
||||
state->current_token_data = NULL;
|
||||
pthread_mutex_unlock(&state->main_lock);
|
||||
|
||||
trace2_thread_exit();
|
||||
return NULL;
|
||||
}
|
||||
|
||||
static int fsmonitor_run_daemon_1(struct fsmonitor_daemon_state *state)
|
||||
{
|
||||
struct ipc_server_opts ipc_opts = {
|
||||
.nr_threads = fsmonitor__ipc_threads,
|
||||
|
||||
/*
|
||||
* We know that there are no other active threads yet,
|
||||
* so we can let the IPC layer temporarily chdir() if
|
||||
* it needs to when creating the server side of the
|
||||
* Unix domain socket.
|
||||
*/
|
||||
.uds_disallow_chdir = 0
|
||||
};
|
||||
|
||||
/*
|
||||
* Start the IPC thread pool before the we've started the file
|
||||
* system event listener thread so that we have the IPC handle
|
||||
* before we need it.
|
||||
*/
|
||||
if (ipc_server_run_async(&state->ipc_server_data,
|
||||
fsmonitor_ipc__get_path(), &ipc_opts,
|
||||
handle_client, state))
|
||||
return error_errno(
|
||||
_("could not start IPC thread pool on '%s'"),
|
||||
fsmonitor_ipc__get_path());
|
||||
|
||||
/*
|
||||
* Start the fsmonitor listener thread to collect filesystem
|
||||
* events.
|
||||
*/
|
||||
if (pthread_create(&state->listener_thread, NULL,
|
||||
fsm_listen__thread_proc, state) < 0) {
|
||||
ipc_server_stop_async(state->ipc_server_data);
|
||||
ipc_server_await(state->ipc_server_data);
|
||||
|
||||
return error(_("could not start fsmonitor listener thread"));
|
||||
}
|
||||
|
||||
/*
|
||||
* The daemon is now fully functional in background threads.
|
||||
* Wait for the IPC thread pool to shutdown (whether by client
|
||||
* request or from filesystem activity).
|
||||
*/
|
||||
ipc_server_await(state->ipc_server_data);
|
||||
|
||||
/*
|
||||
* The fsmonitor listener thread may have received a shutdown
|
||||
* event from the IPC thread pool, but it doesn't hurt to tell
|
||||
* it again. And wait for it to shutdown.
|
||||
*/
|
||||
fsm_listen__stop_async(state);
|
||||
pthread_join(state->listener_thread, NULL);
|
||||
|
||||
return state->error_code;
|
||||
}
|
||||
|
||||
static int fsmonitor_run_daemon(void)
|
||||
{
|
||||
struct fsmonitor_daemon_state state;
|
||||
int err;
|
||||
|
||||
memset(&state, 0, sizeof(state));
|
||||
|
||||
hashmap_init(&state.cookies, cookies_cmp, NULL, 0);
|
||||
pthread_mutex_init(&state.main_lock, NULL);
|
||||
pthread_cond_init(&state.cookies_cond, NULL);
|
||||
state.error_code = 0;
|
||||
state.current_token_data = fsmonitor_new_token_data();
|
||||
|
||||
/* Prepare to (recursively) watch the <worktree-root> directory. */
|
||||
strbuf_init(&state.path_worktree_watch, 0);
|
||||
strbuf_addstr(&state.path_worktree_watch, absolute_path(get_git_work_tree()));
|
||||
state.nr_paths_watching = 1;
|
||||
|
||||
/*
|
||||
* We create and delete cookie files somewhere inside the .git
|
||||
* directory to help us keep sync with the file system. If
|
||||
* ".git" is not a directory, then <gitdir> is not inside the
|
||||
* cone of <worktree-root>, so set up a second watch to watch
|
||||
* the <gitdir> so that we get events for the cookie files.
|
||||
*/
|
||||
strbuf_init(&state.path_gitdir_watch, 0);
|
||||
strbuf_addbuf(&state.path_gitdir_watch, &state.path_worktree_watch);
|
||||
strbuf_addstr(&state.path_gitdir_watch, "/.git");
|
||||
if (!is_directory(state.path_gitdir_watch.buf)) {
|
||||
strbuf_reset(&state.path_gitdir_watch);
|
||||
strbuf_addstr(&state.path_gitdir_watch, absolute_path(get_git_dir()));
|
||||
state.nr_paths_watching = 2;
|
||||
}
|
||||
|
||||
/*
|
||||
* We will write filesystem syncing cookie files into
|
||||
* <gitdir>/<fsmonitor-dir>/<cookie-dir>/<pid>-<seq>.
|
||||
*
|
||||
* The extra layers of subdirectories here keep us from
|
||||
* changing the mtime on ".git/" or ".git/foo/" when we create
|
||||
* or delete cookie files.
|
||||
*
|
||||
* There have been problems with some IDEs that do a
|
||||
* non-recursive watch of the ".git/" directory and run a
|
||||
* series of commands any time something happens.
|
||||
*
|
||||
* For example, if we place our cookie files directly in
|
||||
* ".git/" or ".git/foo/" then a `git status` (or similar
|
||||
* command) from the IDE will cause a cookie file to be
|
||||
* created in one of those dirs. This causes the mtime of
|
||||
* those dirs to change. This triggers the IDE's watch
|
||||
* notification. This triggers the IDE to run those commands
|
||||
* again. And the process repeats and the machine never goes
|
||||
* idle.
|
||||
*
|
||||
* Adding the extra layers of subdirectories prevents the
|
||||
* mtime of ".git/" and ".git/foo" from changing when a
|
||||
* cookie file is created.
|
||||
*/
|
||||
strbuf_init(&state.path_cookie_prefix, 0);
|
||||
strbuf_addbuf(&state.path_cookie_prefix, &state.path_gitdir_watch);
|
||||
|
||||
strbuf_addch(&state.path_cookie_prefix, '/');
|
||||
strbuf_addstr(&state.path_cookie_prefix, FSMONITOR_DIR);
|
||||
mkdir(state.path_cookie_prefix.buf, 0777);
|
||||
|
||||
strbuf_addch(&state.path_cookie_prefix, '/');
|
||||
strbuf_addstr(&state.path_cookie_prefix, FSMONITOR_COOKIE_DIR);
|
||||
mkdir(state.path_cookie_prefix.buf, 0777);
|
||||
|
||||
strbuf_addch(&state.path_cookie_prefix, '/');
|
||||
|
||||
/*
|
||||
* Confirm that we can create platform-specific resources for the
|
||||
* filesystem listener before we bother starting all the threads.
|
||||
*/
|
||||
if (fsm_listen__ctor(&state)) {
|
||||
err = error(_("could not initialize listener thread"));
|
||||
goto done;
|
||||
}
|
||||
|
||||
err = fsmonitor_run_daemon_1(&state);
|
||||
|
||||
done:
|
||||
pthread_cond_destroy(&state.cookies_cond);
|
||||
pthread_mutex_destroy(&state.main_lock);
|
||||
fsm_listen__dtor(&state);
|
||||
|
||||
ipc_server_free(state.ipc_server_data);
|
||||
|
||||
strbuf_release(&state.path_worktree_watch);
|
||||
strbuf_release(&state.path_gitdir_watch);
|
||||
strbuf_release(&state.path_cookie_prefix);
|
||||
|
||||
return err;
|
||||
}
|
||||
|
||||
static int try_to_run_foreground_daemon(int detach_console)
|
||||
{
|
||||
/*
|
||||
* Technically, we don't need to probe for an existing daemon
|
||||
* process, since we could just call `fsmonitor_run_daemon()`
|
||||
* and let it fail if the pipe/socket is busy.
|
||||
*
|
||||
* However, this method gives us a nicer error message for a
|
||||
* common error case.
|
||||
*/
|
||||
if (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING)
|
||||
die(_("fsmonitor--daemon is already running '%s'"),
|
||||
the_repository->worktree);
|
||||
|
||||
if (fsmonitor__announce_startup) {
|
||||
fprintf(stderr, _("running fsmonitor-daemon in '%s'\n"),
|
||||
the_repository->worktree);
|
||||
fflush(stderr);
|
||||
}
|
||||
|
||||
#ifdef GIT_WINDOWS_NATIVE
|
||||
if (detach_console)
|
||||
FreeConsole();
|
||||
#endif
|
||||
|
||||
return !!fsmonitor_run_daemon();
|
||||
}
|
||||
|
||||
static start_bg_wait_cb bg_wait_cb;
|
||||
|
||||
static int bg_wait_cb(const struct child_process *cp, void *cb_data)
|
||||
{
|
||||
enum ipc_active_state s = fsmonitor_ipc__get_state();
|
||||
|
||||
switch (s) {
|
||||
case IPC_STATE__LISTENING:
|
||||
/* child is "ready" */
|
||||
return 0;
|
||||
|
||||
case IPC_STATE__NOT_LISTENING:
|
||||
case IPC_STATE__PATH_NOT_FOUND:
|
||||
/* give child more time */
|
||||
return 1;
|
||||
|
||||
default:
|
||||
case IPC_STATE__INVALID_PATH:
|
||||
case IPC_STATE__OTHER_ERROR:
|
||||
/* all the time in world won't help */
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
|
||||
static int try_to_start_background_daemon(void)
|
||||
{
|
||||
struct child_process cp = CHILD_PROCESS_INIT;
|
||||
enum start_bg_result sbgr;
|
||||
|
||||
/*
|
||||
* Before we try to create a background daemon process, see
|
||||
* if a daemon process is already listening. This makes it
|
||||
* easier for us to report an already-listening error to the
|
||||
* console, since our spawn/daemon can only report the success
|
||||
* of creating the background process (and not whether it
|
||||
* immediately exited).
|
||||
*/
|
||||
if (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING)
|
||||
die(_("fsmonitor--daemon is already running '%s'"),
|
||||
the_repository->worktree);
|
||||
|
||||
if (fsmonitor__announce_startup) {
|
||||
fprintf(stderr, _("starting fsmonitor-daemon in '%s'\n"),
|
||||
the_repository->worktree);
|
||||
fflush(stderr);
|
||||
}
|
||||
|
||||
cp.git_cmd = 1;
|
||||
|
||||
strvec_push(&cp.args, "fsmonitor--daemon");
|
||||
strvec_push(&cp.args, "run");
|
||||
strvec_push(&cp.args, "--detach");
|
||||
strvec_pushf(&cp.args, "--ipc-threads=%d", fsmonitor__ipc_threads);
|
||||
|
||||
cp.no_stdin = 1;
|
||||
cp.no_stdout = 1;
|
||||
cp.no_stderr = 1;
|
||||
|
||||
sbgr = start_bg_command(&cp, bg_wait_cb, NULL,
|
||||
fsmonitor__start_timeout_sec);
|
||||
|
||||
switch (sbgr) {
|
||||
case SBGR_READY:
|
||||
return 0;
|
||||
|
||||
default:
|
||||
case SBGR_ERROR:
|
||||
case SBGR_CB_ERROR:
|
||||
return error(_("daemon failed to start"));
|
||||
|
||||
case SBGR_TIMEOUT:
|
||||
return error(_("daemon not online yet"));
|
||||
|
||||
case SBGR_DIED:
|
||||
return error(_("daemon terminated"));
|
||||
}
|
||||
}
|
||||
|
||||
int cmd_fsmonitor__daemon(int argc, const char **argv, const char *prefix)
|
||||
{
|
||||
const char *subcmd;
|
||||
int detach_console = 0;
|
||||
|
||||
struct option options[] = {
|
||||
OPT_BOOL(0, "detach", &detach_console, N_("detach from console")),
|
||||
OPT_INTEGER(0, "ipc-threads",
|
||||
&fsmonitor__ipc_threads,
|
||||
N_("use <n> ipc worker threads")),
|
||||
OPT_INTEGER(0, "start-timeout",
|
||||
&fsmonitor__start_timeout_sec,
|
||||
N_("max seconds to wait for background daemon startup")),
|
||||
|
||||
OPT_END()
|
||||
};
|
||||
|
||||
git_config(fsmonitor_config, NULL);
|
||||
|
||||
argc = parse_options(argc, argv, prefix, options,
|
||||
builtin_fsmonitor__daemon_usage, 0);
|
||||
if (argc != 1)
|
||||
usage_with_options(builtin_fsmonitor__daemon_usage, options);
|
||||
subcmd = argv[0];
|
||||
|
||||
if (fsmonitor__ipc_threads < 1)
|
||||
die(_("invalid 'ipc-threads' value (%d)"),
|
||||
fsmonitor__ipc_threads);
|
||||
|
||||
if (!strcmp(subcmd, "start"))
|
||||
return !!try_to_start_background_daemon();
|
||||
|
||||
if (!strcmp(subcmd, "run"))
|
||||
return !!try_to_run_foreground_daemon(detach_console);
|
||||
|
||||
if (!strcmp(subcmd, "stop"))
|
||||
return !!do_as_client__send_stop();
|
||||
|
||||
if (!strcmp(subcmd, "status"))
|
||||
return !!do_as_client__status();
|
||||
|
||||
die(_("Unhandled subcommand '%s'"), subcmd);
|
||||
}
|
||||
|
||||
#else
|
||||
int cmd_fsmonitor__daemon(int argc, const char **argv, const char *prefix)
|
||||
{
|
||||
struct option options[] = {
|
||||
OPT_END()
|
||||
};
|
||||
|
||||
if (argc == 2 && !strcmp(argv[1], "-h"))
|
||||
usage_with_options(builtin_fsmonitor__daemon_usage, options);
|
||||
|
||||
die(_("fsmonitor--daemon not supported on this platform"));
|
||||
}
|
||||
#endif
|
||||
|
|
@ -1236,14 +1236,17 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)
|
|||
}
|
||||
|
||||
if (fsmonitor > 0) {
|
||||
if (git_config_get_fsmonitor() == 0)
|
||||
enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r);
|
||||
if (fsm_mode == FSMONITOR_MODE_DISABLED) {
|
||||
warning(_("core.fsmonitor is unset; "
|
||||
"set it if you really want to "
|
||||
"enable fsmonitor"));
|
||||
}
|
||||
add_fsmonitor(&the_index);
|
||||
report(_("fsmonitor enabled"));
|
||||
} else if (!fsmonitor) {
|
||||
if (git_config_get_fsmonitor() == 1)
|
||||
enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r);
|
||||
if (fsm_mode > FSMONITOR_MODE_DISABLED)
|
||||
warning(_("core.fsmonitor is set; "
|
||||
"remove it if you really want to "
|
||||
"disable fsmonitor"));
|
||||
|
|
|
|||
1
cache.h
1
cache.h
|
|
@ -1048,7 +1048,6 @@ extern int core_preload_index;
|
|||
extern int precomposed_unicode;
|
||||
extern int protect_hfs;
|
||||
extern int protect_ntfs;
|
||||
extern const char *core_fsmonitor;
|
||||
|
||||
extern int core_apply_sparse_checkout;
|
||||
extern int core_sparse_checkout_cone;
|
||||
|
|
|
|||
|
|
@ -0,0 +1,92 @@
|
|||
#ifndef FSM_DARWIN_GCC_H
|
||||
#define FSM_DARWIN_GCC_H
|
||||
|
||||
#ifndef __clang__
|
||||
/*
|
||||
* It is possible to #include CoreFoundation/CoreFoundation.h when compiling
|
||||
* with clang, but not with GCC as of time of writing.
|
||||
*
|
||||
* See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93082 for details.
|
||||
*/
|
||||
typedef unsigned int FSEventStreamCreateFlags;
|
||||
#define kFSEventStreamEventFlagNone 0x00000000
|
||||
#define kFSEventStreamEventFlagMustScanSubDirs 0x00000001
|
||||
#define kFSEventStreamEventFlagUserDropped 0x00000002
|
||||
#define kFSEventStreamEventFlagKernelDropped 0x00000004
|
||||
#define kFSEventStreamEventFlagEventIdsWrapped 0x00000008
|
||||
#define kFSEventStreamEventFlagHistoryDone 0x00000010
|
||||
#define kFSEventStreamEventFlagRootChanged 0x00000020
|
||||
#define kFSEventStreamEventFlagMount 0x00000040
|
||||
#define kFSEventStreamEventFlagUnmount 0x00000080
|
||||
#define kFSEventStreamEventFlagItemCreated 0x00000100
|
||||
#define kFSEventStreamEventFlagItemRemoved 0x00000200
|
||||
#define kFSEventStreamEventFlagItemInodeMetaMod 0x00000400
|
||||
#define kFSEventStreamEventFlagItemRenamed 0x00000800
|
||||
#define kFSEventStreamEventFlagItemModified 0x00001000
|
||||
#define kFSEventStreamEventFlagItemFinderInfoMod 0x00002000
|
||||
#define kFSEventStreamEventFlagItemChangeOwner 0x00004000
|
||||
#define kFSEventStreamEventFlagItemXattrMod 0x00008000
|
||||
#define kFSEventStreamEventFlagItemIsFile 0x00010000
|
||||
#define kFSEventStreamEventFlagItemIsDir 0x00020000
|
||||
#define kFSEventStreamEventFlagItemIsSymlink 0x00040000
|
||||
#define kFSEventStreamEventFlagOwnEvent 0x00080000
|
||||
#define kFSEventStreamEventFlagItemIsHardlink 0x00100000
|
||||
#define kFSEventStreamEventFlagItemIsLastHardlink 0x00200000
|
||||
#define kFSEventStreamEventFlagItemCloned 0x00400000
|
||||
|
||||
typedef struct __FSEventStream *FSEventStreamRef;
|
||||
typedef const FSEventStreamRef ConstFSEventStreamRef;
|
||||
|
||||
typedef unsigned int CFStringEncoding;
|
||||
#define kCFStringEncodingUTF8 0x08000100
|
||||
|
||||
typedef const struct __CFString *CFStringRef;
|
||||
typedef const struct __CFArray *CFArrayRef;
|
||||
typedef const struct __CFRunLoop *CFRunLoopRef;
|
||||
|
||||
struct FSEventStreamContext {
|
||||
long long version;
|
||||
void *cb_data, *retain, *release, *copy_description;
|
||||
};
|
||||
|
||||
typedef struct FSEventStreamContext FSEventStreamContext;
|
||||
typedef unsigned int FSEventStreamEventFlags;
|
||||
#define kFSEventStreamCreateFlagNoDefer 0x02
|
||||
#define kFSEventStreamCreateFlagWatchRoot 0x04
|
||||
#define kFSEventStreamCreateFlagFileEvents 0x10
|
||||
|
||||
typedef unsigned long long FSEventStreamEventId;
|
||||
#define kFSEventStreamEventIdSinceNow 0xFFFFFFFFFFFFFFFFULL
|
||||
|
||||
typedef void (*FSEventStreamCallback)(ConstFSEventStreamRef streamRef,
|
||||
void *context,
|
||||
__SIZE_TYPE__ num_of_events,
|
||||
void *event_paths,
|
||||
const FSEventStreamEventFlags event_flags[],
|
||||
const FSEventStreamEventId event_ids[]);
|
||||
typedef double CFTimeInterval;
|
||||
FSEventStreamRef FSEventStreamCreate(void *allocator,
|
||||
FSEventStreamCallback callback,
|
||||
FSEventStreamContext *context,
|
||||
CFArrayRef paths_to_watch,
|
||||
FSEventStreamEventId since_when,
|
||||
CFTimeInterval latency,
|
||||
FSEventStreamCreateFlags flags);
|
||||
CFStringRef CFStringCreateWithCString(void *allocator, const char *string,
|
||||
CFStringEncoding encoding);
|
||||
CFArrayRef CFArrayCreate(void *allocator, const void **items, long long count,
|
||||
void *callbacks);
|
||||
void CFRunLoopRun(void);
|
||||
void CFRunLoopStop(CFRunLoopRef run_loop);
|
||||
CFRunLoopRef CFRunLoopGetCurrent(void);
|
||||
extern CFStringRef kCFRunLoopDefaultMode;
|
||||
void FSEventStreamScheduleWithRunLoop(FSEventStreamRef stream,
|
||||
CFRunLoopRef run_loop,
|
||||
CFStringRef run_loop_mode);
|
||||
unsigned char FSEventStreamStart(FSEventStreamRef stream);
|
||||
void FSEventStreamStop(FSEventStreamRef stream);
|
||||
void FSEventStreamInvalidate(FSEventStreamRef stream);
|
||||
void FSEventStreamRelease(FSEventStreamRef stream);
|
||||
|
||||
#endif /* !clang */
|
||||
#endif /* FSM_DARWIN_GCC_H */
|
||||
|
|
@ -0,0 +1,427 @@
|
|||
#ifndef __clang__
|
||||
#include "fsm-darwin-gcc.h"
|
||||
#else
|
||||
#include <CoreFoundation/CoreFoundation.h>
|
||||
#include <CoreServices/CoreServices.h>
|
||||
|
||||
#ifndef AVAILABLE_MAC_OS_X_VERSION_10_13_AND_LATER
|
||||
/*
|
||||
* This enum value was added in 10.13 to:
|
||||
*
|
||||
* /Applications/Xcode.app/Contents/Developer/Platforms/ \
|
||||
* MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/ \
|
||||
* Library/Frameworks/CoreServices.framework/Frameworks/ \
|
||||
* FSEvents.framework/Versions/Current/Headers/FSEvents.h
|
||||
*
|
||||
* If we're compiling against an older SDK, this symbol won't be
|
||||
* present. Silently define it here so that we don't have to ifdef
|
||||
* the logging or masking below. This should be harmless since older
|
||||
* versions of macOS won't ever emit this FS event anyway.
|
||||
*/
|
||||
#define kFSEventStreamEventFlagItemCloned 0x00400000
|
||||
#endif
|
||||
#endif
|
||||
|
||||
#include "cache.h"
|
||||
#include "fsmonitor.h"
|
||||
#include "fsm-listen.h"
|
||||
#include "fsmonitor--daemon.h"
|
||||
|
||||
struct fsmonitor_daemon_backend_data
|
||||
{
|
||||
CFStringRef cfsr_worktree_path;
|
||||
CFStringRef cfsr_gitdir_path;
|
||||
|
||||
CFArrayRef cfar_paths_to_watch;
|
||||
int nr_paths_watching;
|
||||
|
||||
FSEventStreamRef stream;
|
||||
|
||||
CFRunLoopRef rl;
|
||||
|
||||
enum shutdown_style {
|
||||
SHUTDOWN_EVENT = 0,
|
||||
FORCE_SHUTDOWN,
|
||||
FORCE_ERROR_STOP,
|
||||
} shutdown_style;
|
||||
|
||||
unsigned int stream_scheduled:1;
|
||||
unsigned int stream_started:1;
|
||||
};
|
||||
|
||||
static void log_flags_set(const char *path, const FSEventStreamEventFlags flag)
|
||||
{
|
||||
struct strbuf msg = STRBUF_INIT;
|
||||
|
||||
if (flag & kFSEventStreamEventFlagMustScanSubDirs)
|
||||
strbuf_addstr(&msg, "MustScanSubDirs|");
|
||||
if (flag & kFSEventStreamEventFlagUserDropped)
|
||||
strbuf_addstr(&msg, "UserDropped|");
|
||||
if (flag & kFSEventStreamEventFlagKernelDropped)
|
||||
strbuf_addstr(&msg, "KernelDropped|");
|
||||
if (flag & kFSEventStreamEventFlagEventIdsWrapped)
|
||||
strbuf_addstr(&msg, "EventIdsWrapped|");
|
||||
if (flag & kFSEventStreamEventFlagHistoryDone)
|
||||
strbuf_addstr(&msg, "HistoryDone|");
|
||||
if (flag & kFSEventStreamEventFlagRootChanged)
|
||||
strbuf_addstr(&msg, "RootChanged|");
|
||||
if (flag & kFSEventStreamEventFlagMount)
|
||||
strbuf_addstr(&msg, "Mount|");
|
||||
if (flag & kFSEventStreamEventFlagUnmount)
|
||||
strbuf_addstr(&msg, "Unmount|");
|
||||
if (flag & kFSEventStreamEventFlagItemChangeOwner)
|
||||
strbuf_addstr(&msg, "ItemChangeOwner|");
|
||||
if (flag & kFSEventStreamEventFlagItemCreated)
|
||||
strbuf_addstr(&msg, "ItemCreated|");
|
||||
if (flag & kFSEventStreamEventFlagItemFinderInfoMod)
|
||||
strbuf_addstr(&msg, "ItemFinderInfoMod|");
|
||||
if (flag & kFSEventStreamEventFlagItemInodeMetaMod)
|
||||
strbuf_addstr(&msg, "ItemInodeMetaMod|");
|
||||
if (flag & kFSEventStreamEventFlagItemIsDir)
|
||||
strbuf_addstr(&msg, "ItemIsDir|");
|
||||
if (flag & kFSEventStreamEventFlagItemIsFile)
|
||||
strbuf_addstr(&msg, "ItemIsFile|");
|
||||
if (flag & kFSEventStreamEventFlagItemIsHardlink)
|
||||
strbuf_addstr(&msg, "ItemIsHardlink|");
|
||||
if (flag & kFSEventStreamEventFlagItemIsLastHardlink)
|
||||
strbuf_addstr(&msg, "ItemIsLastHardlink|");
|
||||
if (flag & kFSEventStreamEventFlagItemIsSymlink)
|
||||
strbuf_addstr(&msg, "ItemIsSymlink|");
|
||||
if (flag & kFSEventStreamEventFlagItemModified)
|
||||
strbuf_addstr(&msg, "ItemModified|");
|
||||
if (flag & kFSEventStreamEventFlagItemRemoved)
|
||||
strbuf_addstr(&msg, "ItemRemoved|");
|
||||
if (flag & kFSEventStreamEventFlagItemRenamed)
|
||||
strbuf_addstr(&msg, "ItemRenamed|");
|
||||
if (flag & kFSEventStreamEventFlagItemXattrMod)
|
||||
strbuf_addstr(&msg, "ItemXattrMod|");
|
||||
if (flag & kFSEventStreamEventFlagOwnEvent)
|
||||
strbuf_addstr(&msg, "OwnEvent|");
|
||||
if (flag & kFSEventStreamEventFlagItemCloned)
|
||||
strbuf_addstr(&msg, "ItemCloned|");
|
||||
|
||||
trace_printf_key(&trace_fsmonitor, "fsevent: '%s', flags=%u %s",
|
||||
path, flag, msg.buf);
|
||||
|
||||
strbuf_release(&msg);
|
||||
}
|
||||
|
||||
static int ef_is_root_delete(const FSEventStreamEventFlags ef)
|
||||
{
|
||||
return (ef & kFSEventStreamEventFlagItemIsDir &&
|
||||
ef & kFSEventStreamEventFlagItemRemoved);
|
||||
}
|
||||
|
||||
static int ef_is_root_renamed(const FSEventStreamEventFlags ef)
|
||||
{
|
||||
return (ef & kFSEventStreamEventFlagItemIsDir &&
|
||||
ef & kFSEventStreamEventFlagItemRenamed);
|
||||
}
|
||||
|
||||
static int ef_is_dropped(const FSEventStreamEventFlags ef)
|
||||
{
|
||||
return (ef & kFSEventStreamEventFlagMustScanSubDirs ||
|
||||
ef & kFSEventStreamEventFlagKernelDropped ||
|
||||
ef & kFSEventStreamEventFlagUserDropped);
|
||||
}
|
||||
|
||||
static void fsevent_callback(ConstFSEventStreamRef streamRef,
|
||||
void *ctx,
|
||||
size_t num_of_events,
|
||||
void *event_paths,
|
||||
const FSEventStreamEventFlags event_flags[],
|
||||
const FSEventStreamEventId event_ids[])
|
||||
{
|
||||
struct fsmonitor_daemon_state *state = ctx;
|
||||
struct fsmonitor_daemon_backend_data *data = state->backend_data;
|
||||
char **paths = (char **)event_paths;
|
||||
struct fsmonitor_batch *batch = NULL;
|
||||
struct string_list cookie_list = STRING_LIST_INIT_DUP;
|
||||
const char *path_k;
|
||||
const char *slash;
|
||||
int k;
|
||||
struct strbuf tmp = STRBUF_INIT;
|
||||
|
||||
/*
|
||||
* Build a list of all filesystem changes into a private/local
|
||||
* list and without holding any locks.
|
||||
*/
|
||||
for (k = 0; k < num_of_events; k++) {
|
||||
/*
|
||||
* On Mac, we receive an array of absolute paths.
|
||||
*/
|
||||
path_k = paths[k];
|
||||
|
||||
/*
|
||||
* If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
|
||||
* Please don't log them to Trace2.
|
||||
*
|
||||
* trace_printf_key(&trace_fsmonitor, "Path: '%s'", path_k);
|
||||
*/
|
||||
|
||||
/*
|
||||
* If event[k] is marked as dropped, we assume that we have
|
||||
* lost sync with the filesystem and should flush our cached
|
||||
* data. We need to:
|
||||
*
|
||||
* [1] Abort/wake any client threads waiting for a cookie and
|
||||
* flush the cached state data (the current token), and
|
||||
* create a new token.
|
||||
*
|
||||
* [2] Discard the batch that we were locally building (since
|
||||
* they are conceptually relative to the just flushed
|
||||
* token).
|
||||
*/
|
||||
if (ef_is_dropped(event_flags[k])) {
|
||||
if (trace_pass_fl(&trace_fsmonitor))
|
||||
log_flags_set(path_k, event_flags[k]);
|
||||
|
||||
fsmonitor_force_resync(state);
|
||||
fsmonitor_batch__free_list(batch);
|
||||
string_list_clear(&cookie_list, 0);
|
||||
|
||||
/*
|
||||
* We assume that any events that we received
|
||||
* in this callback after this dropped event
|
||||
* may still be valid, so we continue rather
|
||||
* than break. (And just in case there is a
|
||||
* delete of ".git" hiding in there.)
|
||||
*/
|
||||
continue;
|
||||
}
|
||||
|
||||
switch (fsmonitor_classify_path_absolute(state, path_k)) {
|
||||
|
||||
case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX:
|
||||
case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX:
|
||||
/* special case cookie files within .git or gitdir */
|
||||
|
||||
/* Use just the filename of the cookie file. */
|
||||
slash = find_last_dir_sep(path_k);
|
||||
string_list_append(&cookie_list,
|
||||
slash ? slash + 1 : path_k);
|
||||
break;
|
||||
|
||||
case IS_INSIDE_DOT_GIT:
|
||||
case IS_INSIDE_GITDIR:
|
||||
/* ignore all other paths inside of .git or gitdir */
|
||||
break;
|
||||
|
||||
case IS_DOT_GIT:
|
||||
case IS_GITDIR:
|
||||
/*
|
||||
* If .git directory is deleted or renamed away,
|
||||
* we have to quit.
|
||||
*/
|
||||
if (ef_is_root_delete(event_flags[k])) {
|
||||
trace_printf_key(&trace_fsmonitor,
|
||||
"event: gitdir removed");
|
||||
goto force_shutdown;
|
||||
}
|
||||
if (ef_is_root_renamed(event_flags[k])) {
|
||||
trace_printf_key(&trace_fsmonitor,
|
||||
"event: gitdir renamed");
|
||||
goto force_shutdown;
|
||||
}
|
||||
break;
|
||||
|
||||
case IS_WORKDIR_PATH:
|
||||
/* try to queue normal pathnames */
|
||||
|
||||
if (trace_pass_fl(&trace_fsmonitor))
|
||||
log_flags_set(path_k, event_flags[k]);
|
||||
|
||||
/*
|
||||
* Because of the implicit "binning" (the
|
||||
* kernel calls us at a given frequency) and
|
||||
* de-duping (the kernel is free to combine
|
||||
* multiple events for a given pathname), an
|
||||
* individual fsevent could be marked as both
|
||||
* a file and directory. Add it to the queue
|
||||
* with both spellings so that the client will
|
||||
* know how much to invalidate/refresh.
|
||||
*/
|
||||
|
||||
if (event_flags[k] & kFSEventStreamEventFlagItemIsFile) {
|
||||
const char *rel = path_k +
|
||||
state->path_worktree_watch.len + 1;
|
||||
|
||||
if (!batch)
|
||||
batch = fsmonitor_batch__new();
|
||||
fsmonitor_batch__add_path(batch, rel);
|
||||
}
|
||||
|
||||
if (event_flags[k] & kFSEventStreamEventFlagItemIsDir) {
|
||||
const char *rel = path_k +
|
||||
state->path_worktree_watch.len + 1;
|
||||
|
||||
strbuf_reset(&tmp);
|
||||
strbuf_addstr(&tmp, rel);
|
||||
strbuf_addch(&tmp, '/');
|
||||
|
||||
if (!batch)
|
||||
batch = fsmonitor_batch__new();
|
||||
fsmonitor_batch__add_path(batch, tmp.buf);
|
||||
}
|
||||
|
||||
break;
|
||||
|
||||
case IS_OUTSIDE_CONE:
|
||||
default:
|
||||
trace_printf_key(&trace_fsmonitor,
|
||||
"ignoring '%s'", path_k);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
fsmonitor_publish(state, batch, &cookie_list);
|
||||
string_list_clear(&cookie_list, 0);
|
||||
strbuf_release(&tmp);
|
||||
return;
|
||||
|
||||
force_shutdown:
|
||||
fsmonitor_batch__free_list(batch);
|
||||
string_list_clear(&cookie_list, 0);
|
||||
|
||||
data->shutdown_style = FORCE_SHUTDOWN;
|
||||
CFRunLoopStop(data->rl);
|
||||
strbuf_release(&tmp);
|
||||
return;
|
||||
}
|
||||
|
||||
/*
|
||||
* In the call to `FSEventStreamCreate()` to setup our watch, the
|
||||
* `latency` argument determines the frequency of calls to our callback
|
||||
* with new FS events. Too slow and events get dropped; too fast and
|
||||
* we burn CPU unnecessarily. Since it is rather obscure, I don't
|
||||
* think this needs to be a config setting. I've done extensive
|
||||
* testing on my systems and chosen the value below. It gives good
|
||||
* results and I've not seen any dropped events.
|
||||
*
|
||||
* With a latency of 0.1, I was seeing lots of dropped events during
|
||||
* the "touch 100000" files test within t/perf/p7519, but with a
|
||||
* latency of 0.001 I did not see any dropped events. So I'm going
|
||||
* to assume that this is the "correct" value.
|
||||
*
|
||||
* https://developer.apple.com/documentation/coreservices/1443980-fseventstreamcreate
|
||||
*/
|
||||
|
||||
int fsm_listen__ctor(struct fsmonitor_daemon_state *state)
|
||||
{
|
||||
FSEventStreamCreateFlags flags = kFSEventStreamCreateFlagNoDefer |
|
||||
kFSEventStreamCreateFlagWatchRoot |
|
||||
kFSEventStreamCreateFlagFileEvents;
|
||||
FSEventStreamContext ctx = {
|
||||
0,
|
||||
state,
|
||||
NULL,
|
||||
NULL,
|
||||
NULL
|
||||
};
|
||||
struct fsmonitor_daemon_backend_data *data;
|
||||
const void *dir_array[2];
|
||||
|
||||
CALLOC_ARRAY(data, 1);
|
||||
state->backend_data = data;
|
||||
|
||||
data->cfsr_worktree_path = CFStringCreateWithCString(
|
||||
NULL, state->path_worktree_watch.buf, kCFStringEncodingUTF8);
|
||||
dir_array[data->nr_paths_watching++] = data->cfsr_worktree_path;
|
||||
|
||||
if (state->nr_paths_watching > 1) {
|
||||
data->cfsr_gitdir_path = CFStringCreateWithCString(
|
||||
NULL, state->path_gitdir_watch.buf,
|
||||
kCFStringEncodingUTF8);
|
||||
dir_array[data->nr_paths_watching++] = data->cfsr_gitdir_path;
|
||||
}
|
||||
|
||||
data->cfar_paths_to_watch = CFArrayCreate(NULL, dir_array,
|
||||
data->nr_paths_watching,
|
||||
NULL);
|
||||
data->stream = FSEventStreamCreate(NULL, fsevent_callback, &ctx,
|
||||
data->cfar_paths_to_watch,
|
||||
kFSEventStreamEventIdSinceNow,
|
||||
0.001, flags);
|
||||
if (data->stream == NULL)
|
||||
goto failed;
|
||||
|
||||
/*
|
||||
* `data->rl` needs to be set inside the listener thread.
|
||||
*/
|
||||
|
||||
return 0;
|
||||
|
||||
failed:
|
||||
error(_("Unable to create FSEventStream."));
|
||||
|
||||
FREE_AND_NULL(state->backend_data);
|
||||
return -1;
|
||||
}
|
||||
|
||||
void fsm_listen__dtor(struct fsmonitor_daemon_state *state)
|
||||
{
|
||||
struct fsmonitor_daemon_backend_data *data;
|
||||
|
||||
if (!state || !state->backend_data)
|
||||
return;
|
||||
|
||||
data = state->backend_data;
|
||||
|
||||
if (data->stream) {
|
||||
if (data->stream_started)
|
||||
FSEventStreamStop(data->stream);
|
||||
if (data->stream_scheduled)
|
||||
FSEventStreamInvalidate(data->stream);
|
||||
FSEventStreamRelease(data->stream);
|
||||
}
|
||||
|
||||
FREE_AND_NULL(state->backend_data);
|
||||
}
|
||||
|
||||
void fsm_listen__stop_async(struct fsmonitor_daemon_state *state)
|
||||
{
|
||||
struct fsmonitor_daemon_backend_data *data;
|
||||
|
||||
data = state->backend_data;
|
||||
data->shutdown_style = SHUTDOWN_EVENT;
|
||||
|
||||
CFRunLoopStop(data->rl);
|
||||
}
|
||||
|
||||
void fsm_listen__loop(struct fsmonitor_daemon_state *state)
|
||||
{
|
||||
struct fsmonitor_daemon_backend_data *data;
|
||||
|
||||
data = state->backend_data;
|
||||
|
||||
data->rl = CFRunLoopGetCurrent();
|
||||
|
||||
FSEventStreamScheduleWithRunLoop(data->stream, data->rl, kCFRunLoopDefaultMode);
|
||||
data->stream_scheduled = 1;
|
||||
|
||||
if (!FSEventStreamStart(data->stream)) {
|
||||
error(_("Failed to start the FSEventStream"));
|
||||
goto force_error_stop_without_loop;
|
||||
}
|
||||
data->stream_started = 1;
|
||||
|
||||
CFRunLoopRun();
|
||||
|
||||
switch (data->shutdown_style) {
|
||||
case FORCE_ERROR_STOP:
|
||||
state->error_code = -1;
|
||||
/* fall thru */
|
||||
case FORCE_SHUTDOWN:
|
||||
ipc_server_stop_async(state->ipc_server_data);
|
||||
/* fall thru */
|
||||
case SHUTDOWN_EVENT:
|
||||
default:
|
||||
break;
|
||||
}
|
||||
return;
|
||||
|
||||
force_error_stop_without_loop:
|
||||
state->error_code = -1;
|
||||
ipc_server_stop_async(state->ipc_server_data);
|
||||
return;
|
||||
}
|
||||
|
|
@ -0,0 +1,586 @@
|
|||
#include "cache.h"
|
||||
#include "config.h"
|
||||
#include "fsmonitor.h"
|
||||
#include "fsm-listen.h"
|
||||
#include "fsmonitor--daemon.h"
|
||||
|
||||
/*
|
||||
* The documentation of ReadDirectoryChangesW() states that the maximum
|
||||
* buffer size is 64K when the monitored directory is remote.
|
||||
*
|
||||
* Larger buffers may be used when the monitored directory is local and
|
||||
* will help us receive events faster from the kernel and avoid dropped
|
||||
* events.
|
||||
*
|
||||
* So we try to use a very large buffer and silently fallback to 64K if
|
||||
* we get an error.
|
||||
*/
|
||||
#define MAX_RDCW_BUF_FALLBACK (65536)
|
||||
#define MAX_RDCW_BUF (65536 * 8)
|
||||
|
||||
struct one_watch
|
||||
{
|
||||
char buffer[MAX_RDCW_BUF];
|
||||
DWORD buf_len;
|
||||
DWORD count;
|
||||
|
||||
struct strbuf path;
|
||||
HANDLE hDir;
|
||||
HANDLE hEvent;
|
||||
OVERLAPPED overlapped;
|
||||
|
||||
/*
|
||||
* Is there an active ReadDirectoryChangesW() call pending. If so, we
|
||||
* need to later call GetOverlappedResult() and possibly CancelIoEx().
|
||||
*/
|
||||
BOOL is_active;
|
||||
};
|
||||
|
||||
struct fsmonitor_daemon_backend_data
|
||||
{
|
||||
struct one_watch *watch_worktree;
|
||||
struct one_watch *watch_gitdir;
|
||||
|
||||
HANDLE hEventShutdown;
|
||||
|
||||
HANDLE hListener[3]; /* we don't own these handles */
|
||||
#define LISTENER_SHUTDOWN 0
|
||||
#define LISTENER_HAVE_DATA_WORKTREE 1
|
||||
#define LISTENER_HAVE_DATA_GITDIR 2
|
||||
int nr_listener_handles;
|
||||
};
|
||||
|
||||
/*
|
||||
* Convert the WCHAR path from the notification into UTF8 and
|
||||
* then normalize it.
|
||||
*/
|
||||
static int normalize_path_in_utf8(FILE_NOTIFY_INFORMATION *info,
|
||||
struct strbuf *normalized_path)
|
||||
{
|
||||
int reserve;
|
||||
int len = 0;
|
||||
|
||||
strbuf_reset(normalized_path);
|
||||
if (!info->FileNameLength)
|
||||
goto normalize;
|
||||
|
||||
/*
|
||||
* Pre-reserve enough space in the UTF8 buffer for
|
||||
* each Unicode WCHAR character to be mapped into a
|
||||
* sequence of 2 UTF8 characters. That should let us
|
||||
* avoid ERROR_INSUFFICIENT_BUFFER 99.9+% of the time.
|
||||
*/
|
||||
reserve = info->FileNameLength + 1;
|
||||
strbuf_grow(normalized_path, reserve);
|
||||
|
||||
for (;;) {
|
||||
len = WideCharToMultiByte(CP_UTF8, 0, info->FileName,
|
||||
info->FileNameLength / sizeof(WCHAR),
|
||||
normalized_path->buf,
|
||||
strbuf_avail(normalized_path) - 1,
|
||||
NULL, NULL);
|
||||
if (len > 0)
|
||||
goto normalize;
|
||||
if (GetLastError() != ERROR_INSUFFICIENT_BUFFER) {
|
||||
error(_("[GLE %ld] could not convert path to UTF-8: '%.*ls'"),
|
||||
GetLastError(),
|
||||
(int)(info->FileNameLength / sizeof(WCHAR)),
|
||||
info->FileName);
|
||||
return -1;
|
||||
}
|
||||
|
||||
strbuf_grow(normalized_path,
|
||||
strbuf_avail(normalized_path) + reserve);
|
||||
}
|
||||
|
||||
normalize:
|
||||
strbuf_setlen(normalized_path, len);
|
||||
return strbuf_normalize_path(normalized_path);
|
||||
}
|
||||
|
||||
void fsm_listen__stop_async(struct fsmonitor_daemon_state *state)
|
||||
{
|
||||
SetEvent(state->backend_data->hListener[LISTENER_SHUTDOWN]);
|
||||
}
|
||||
|
||||
static struct one_watch *create_watch(struct fsmonitor_daemon_state *state,
|
||||
const char *path)
|
||||
{
|
||||
struct one_watch *watch = NULL;
|
||||
DWORD desired_access = FILE_LIST_DIRECTORY;
|
||||
DWORD share_mode =
|
||||
FILE_SHARE_WRITE | FILE_SHARE_READ | FILE_SHARE_DELETE;
|
||||
HANDLE hDir;
|
||||
wchar_t wpath[MAX_PATH];
|
||||
|
||||
if (xutftowcs_path(wpath, path) < 0) {
|
||||
error(_("could not convert to wide characters: '%s'"), path);
|
||||
return NULL;
|
||||
}
|
||||
|
||||
hDir = CreateFileW(wpath,
|
||||
desired_access, share_mode, NULL, OPEN_EXISTING,
|
||||
FILE_FLAG_BACKUP_SEMANTICS | FILE_FLAG_OVERLAPPED,
|
||||
NULL);
|
||||
if (hDir == INVALID_HANDLE_VALUE) {
|
||||
error(_("[GLE %ld] could not watch '%s'"),
|
||||
GetLastError(), path);
|
||||
return NULL;
|
||||
}
|
||||
|
||||
CALLOC_ARRAY(watch, 1);
|
||||
|
||||
watch->buf_len = sizeof(watch->buffer); /* assume full MAX_RDCW_BUF */
|
||||
|
||||
strbuf_init(&watch->path, 0);
|
||||
strbuf_addstr(&watch->path, path);
|
||||
|
||||
watch->hDir = hDir;
|
||||
watch->hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
|
||||
|
||||
return watch;
|
||||
}
|
||||
|
||||
static void destroy_watch(struct one_watch *watch)
|
||||
{
|
||||
if (!watch)
|
||||
return;
|
||||
|
||||
strbuf_release(&watch->path);
|
||||
if (watch->hDir != INVALID_HANDLE_VALUE)
|
||||
CloseHandle(watch->hDir);
|
||||
if (watch->hEvent != INVALID_HANDLE_VALUE)
|
||||
CloseHandle(watch->hEvent);
|
||||
|
||||
free(watch);
|
||||
}
|
||||
|
||||
static int start_rdcw_watch(struct fsmonitor_daemon_backend_data *data,
|
||||
struct one_watch *watch)
|
||||
{
|
||||
DWORD dwNotifyFilter =
|
||||
FILE_NOTIFY_CHANGE_FILE_NAME |
|
||||
FILE_NOTIFY_CHANGE_DIR_NAME |
|
||||
FILE_NOTIFY_CHANGE_ATTRIBUTES |
|
||||
FILE_NOTIFY_CHANGE_SIZE |
|
||||
FILE_NOTIFY_CHANGE_LAST_WRITE |
|
||||
FILE_NOTIFY_CHANGE_CREATION;
|
||||
|
||||
ResetEvent(watch->hEvent);
|
||||
|
||||
memset(&watch->overlapped, 0, sizeof(watch->overlapped));
|
||||
watch->overlapped.hEvent = watch->hEvent;
|
||||
|
||||
/*
|
||||
* Queue an async call using Overlapped IO. This returns immediately.
|
||||
* Our event handle will be signalled when the real result is available.
|
||||
*
|
||||
* The return value here just means that we successfully queued it.
|
||||
* We won't know if the Read...() actually produces data until later.
|
||||
*/
|
||||
watch->is_active = ReadDirectoryChangesW(
|
||||
watch->hDir, watch->buffer, watch->buf_len, TRUE,
|
||||
dwNotifyFilter, &watch->count, &watch->overlapped, NULL);
|
||||
|
||||
if (watch->is_active)
|
||||
return 0;
|
||||
|
||||
error(_("ReadDirectoryChangedW failed on '%s' [GLE %ld]"),
|
||||
watch->path.buf, GetLastError());
|
||||
return -1;
|
||||
}
|
||||
|
||||
static int recv_rdcw_watch(struct one_watch *watch)
|
||||
{
|
||||
DWORD gle;
|
||||
|
||||
watch->is_active = FALSE;
|
||||
|
||||
/*
|
||||
* The overlapped result is ready. If the Read...() was successful
|
||||
* we finally receive the actual result into our buffer.
|
||||
*/
|
||||
if (GetOverlappedResult(watch->hDir, &watch->overlapped, &watch->count,
|
||||
TRUE))
|
||||
return 0;
|
||||
|
||||
gle = GetLastError();
|
||||
if (gle == ERROR_INVALID_PARAMETER &&
|
||||
/*
|
||||
* The kernel throws an invalid parameter error when our
|
||||
* buffer is too big and we are pointed at a remote
|
||||
* directory (and possibly for other reasons). Quietly
|
||||
* set it down and try again.
|
||||
*
|
||||
* See note about MAX_RDCW_BUF at the top.
|
||||
*/
|
||||
watch->buf_len > MAX_RDCW_BUF_FALLBACK) {
|
||||
watch->buf_len = MAX_RDCW_BUF_FALLBACK;
|
||||
return -2;
|
||||
}
|
||||
|
||||
/*
|
||||
* NEEDSWORK: If an external <gitdir> is deleted, the above
|
||||
* returns an error. I'm not sure that there's anything that
|
||||
* we can do here other than failing -- the <worktree>/.git
|
||||
* link file would be broken anyway. We might try to check
|
||||
* for that and return a better error message, but I'm not
|
||||
* sure it is worth it.
|
||||
*/
|
||||
|
||||
error(_("GetOverlappedResult failed on '%s' [GLE %ld]"),
|
||||
watch->path.buf, gle);
|
||||
return -1;
|
||||
}
|
||||
|
||||
static void cancel_rdcw_watch(struct one_watch *watch)
|
||||
{
|
||||
DWORD count;
|
||||
|
||||
if (!watch || !watch->is_active)
|
||||
return;
|
||||
|
||||
/*
|
||||
* The calls to ReadDirectoryChangesW() and GetOverlappedResult()
|
||||
* form a "pair" (my term) where we queue an IO and promise to
|
||||
* hang around and wait for the kernel to give us the result.
|
||||
*
|
||||
* If for some reason after we queue the IO, we have to quit
|
||||
* or otherwise not stick around for the second half, we must
|
||||
* tell the kernel to abort the IO. This prevents the kernel
|
||||
* from writing to our buffer and/or signalling our event
|
||||
* after we free them.
|
||||
*
|
||||
* (Ask me how much fun it was to track that one down).
|
||||
*/
|
||||
CancelIoEx(watch->hDir, &watch->overlapped);
|
||||
GetOverlappedResult(watch->hDir, &watch->overlapped, &count, TRUE);
|
||||
watch->is_active = FALSE;
|
||||
}
|
||||
|
||||
/*
|
||||
* Process filesystem events that happen anywhere (recursively) under the
|
||||
* <worktree> root directory. For a normal working directory, this includes
|
||||
* both version controlled files and the contents of the .git/ directory.
|
||||
*
|
||||
* If <worktree>/.git is a file, then we only see events for the file
|
||||
* itself.
|
||||
*/
|
||||
static int process_worktree_events(struct fsmonitor_daemon_state *state)
|
||||
{
|
||||
struct fsmonitor_daemon_backend_data *data = state->backend_data;
|
||||
struct one_watch *watch = data->watch_worktree;
|
||||
struct strbuf path = STRBUF_INIT;
|
||||
struct string_list cookie_list = STRING_LIST_INIT_DUP;
|
||||
struct fsmonitor_batch *batch = NULL;
|
||||
const char *p = watch->buffer;
|
||||
|
||||
/*
|
||||
* If the kernel gets more events than will fit in the kernel
|
||||
* buffer associated with our RDCW handle, it drops them and
|
||||
* returns a count of zero.
|
||||
*
|
||||
* Yes, the call returns WITHOUT error and with length zero.
|
||||
* This is the documented behavior. (My testing has confirmed
|
||||
* that it also sets the last error to ERROR_NOTIFY_ENUM_DIR,
|
||||
* but we do not rely on that since the function did not
|
||||
* return an error and it is not documented.)
|
||||
*
|
||||
* (The "overflow" case is not ambiguous with the "no data" case
|
||||
* because we did an INFINITE wait.)
|
||||
*
|
||||
* This means we have a gap in coverage. Tell the daemon layer
|
||||
* to resync.
|
||||
*/
|
||||
if (!watch->count) {
|
||||
trace2_data_string("fsmonitor", NULL, "fsm-listen/kernel",
|
||||
"overflow");
|
||||
fsmonitor_force_resync(state);
|
||||
return LISTENER_HAVE_DATA_WORKTREE;
|
||||
}
|
||||
|
||||
/*
|
||||
* On Windows, `info` contains an "array" of paths that are
|
||||
* relative to the root of whichever directory handle received
|
||||
* the event.
|
||||
*/
|
||||
for (;;) {
|
||||
FILE_NOTIFY_INFORMATION *info = (void *)p;
|
||||
const char *slash;
|
||||
enum fsmonitor_path_type t;
|
||||
|
||||
strbuf_reset(&path);
|
||||
if (normalize_path_in_utf8(info, &path) == -1)
|
||||
goto skip_this_path;
|
||||
|
||||
t = fsmonitor_classify_path_workdir_relative(path.buf);
|
||||
|
||||
switch (t) {
|
||||
case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX:
|
||||
/* special case cookie files within .git */
|
||||
|
||||
/* Use just the filename of the cookie file. */
|
||||
slash = find_last_dir_sep(path.buf);
|
||||
string_list_append(&cookie_list,
|
||||
slash ? slash + 1 : path.buf);
|
||||
break;
|
||||
|
||||
case IS_INSIDE_DOT_GIT:
|
||||
/* ignore everything inside of "<worktree>/.git/" */
|
||||
break;
|
||||
|
||||
case IS_DOT_GIT:
|
||||
/* "<worktree>/.git" was deleted (or renamed away) */
|
||||
if ((info->Action == FILE_ACTION_REMOVED) ||
|
||||
(info->Action == FILE_ACTION_RENAMED_OLD_NAME)) {
|
||||
trace2_data_string("fsmonitor", NULL,
|
||||
"fsm-listen/dotgit",
|
||||
"removed");
|
||||
goto force_shutdown;
|
||||
}
|
||||
break;
|
||||
|
||||
case IS_WORKDIR_PATH:
|
||||
/* queue normal pathname */
|
||||
if (!batch)
|
||||
batch = fsmonitor_batch__new();
|
||||
fsmonitor_batch__add_path(batch, path.buf);
|
||||
break;
|
||||
|
||||
case IS_GITDIR:
|
||||
case IS_INSIDE_GITDIR:
|
||||
case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX:
|
||||
default:
|
||||
BUG("unexpected path classification '%d' for '%s'",
|
||||
t, path.buf);
|
||||
}
|
||||
|
||||
skip_this_path:
|
||||
if (!info->NextEntryOffset)
|
||||
break;
|
||||
p += info->NextEntryOffset;
|
||||
}
|
||||
|
||||
fsmonitor_publish(state, batch, &cookie_list);
|
||||
batch = NULL;
|
||||
string_list_clear(&cookie_list, 0);
|
||||
strbuf_release(&path);
|
||||
return LISTENER_HAVE_DATA_WORKTREE;
|
||||
|
||||
force_shutdown:
|
||||
fsmonitor_batch__free_list(batch);
|
||||
string_list_clear(&cookie_list, 0);
|
||||
strbuf_release(&path);
|
||||
return LISTENER_SHUTDOWN;
|
||||
}
|
||||
|
||||
/*
|
||||
* Process filesystem events that happened anywhere (recursively) under the
|
||||
* external <gitdir> (such as non-primary worktrees or submodules).
|
||||
* We only care about cookie files that our client threads created here.
|
||||
*
|
||||
* Note that we DO NOT get filesystem events on the external <gitdir>
|
||||
* itself (it is not inside something that we are watching). In particular,
|
||||
* we do not get an event if the external <gitdir> is deleted.
|
||||
*/
|
||||
static int process_gitdir_events(struct fsmonitor_daemon_state *state)
|
||||
{
|
||||
struct fsmonitor_daemon_backend_data *data = state->backend_data;
|
||||
struct one_watch *watch = data->watch_gitdir;
|
||||
struct strbuf path = STRBUF_INIT;
|
||||
struct string_list cookie_list = STRING_LIST_INIT_DUP;
|
||||
const char *p = watch->buffer;
|
||||
|
||||
if (!watch->count) {
|
||||
trace2_data_string("fsmonitor", NULL, "fsm-listen/kernel",
|
||||
"overflow");
|
||||
fsmonitor_force_resync(state);
|
||||
return LISTENER_HAVE_DATA_GITDIR;
|
||||
}
|
||||
|
||||
for (;;) {
|
||||
FILE_NOTIFY_INFORMATION *info = (void *)p;
|
||||
const char *slash;
|
||||
enum fsmonitor_path_type t;
|
||||
|
||||
strbuf_reset(&path);
|
||||
if (normalize_path_in_utf8(info, &path) == -1)
|
||||
goto skip_this_path;
|
||||
|
||||
t = fsmonitor_classify_path_gitdir_relative(path.buf);
|
||||
|
||||
switch (t) {
|
||||
case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX:
|
||||
/* special case cookie files within gitdir */
|
||||
|
||||
/* Use just the filename of the cookie file. */
|
||||
slash = find_last_dir_sep(path.buf);
|
||||
string_list_append(&cookie_list,
|
||||
slash ? slash + 1 : path.buf);
|
||||
break;
|
||||
|
||||
case IS_INSIDE_GITDIR:
|
||||
goto skip_this_path;
|
||||
|
||||
default:
|
||||
BUG("unexpected path classification '%d' for '%s'",
|
||||
t, path.buf);
|
||||
}
|
||||
|
||||
skip_this_path:
|
||||
if (!info->NextEntryOffset)
|
||||
break;
|
||||
p += info->NextEntryOffset;
|
||||
}
|
||||
|
||||
fsmonitor_publish(state, NULL, &cookie_list);
|
||||
string_list_clear(&cookie_list, 0);
|
||||
strbuf_release(&path);
|
||||
return LISTENER_HAVE_DATA_GITDIR;
|
||||
}
|
||||
|
||||
void fsm_listen__loop(struct fsmonitor_daemon_state *state)
|
||||
{
|
||||
struct fsmonitor_daemon_backend_data *data = state->backend_data;
|
||||
DWORD dwWait;
|
||||
int result;
|
||||
|
||||
state->error_code = 0;
|
||||
|
||||
if (start_rdcw_watch(data, data->watch_worktree) == -1)
|
||||
goto force_error_stop;
|
||||
|
||||
if (data->watch_gitdir &&
|
||||
start_rdcw_watch(data, data->watch_gitdir) == -1)
|
||||
goto force_error_stop;
|
||||
|
||||
for (;;) {
|
||||
dwWait = WaitForMultipleObjects(data->nr_listener_handles,
|
||||
data->hListener,
|
||||
FALSE, INFINITE);
|
||||
|
||||
if (dwWait == WAIT_OBJECT_0 + LISTENER_HAVE_DATA_WORKTREE) {
|
||||
result = recv_rdcw_watch(data->watch_worktree);
|
||||
if (result == -1) {
|
||||
/* hard error */
|
||||
goto force_error_stop;
|
||||
}
|
||||
if (result == -2) {
|
||||
/* retryable error */
|
||||
if (start_rdcw_watch(data, data->watch_worktree) == -1)
|
||||
goto force_error_stop;
|
||||
continue;
|
||||
}
|
||||
|
||||
/* have data */
|
||||
if (process_worktree_events(state) == LISTENER_SHUTDOWN)
|
||||
goto force_shutdown;
|
||||
if (start_rdcw_watch(data, data->watch_worktree) == -1)
|
||||
goto force_error_stop;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (dwWait == WAIT_OBJECT_0 + LISTENER_HAVE_DATA_GITDIR) {
|
||||
result = recv_rdcw_watch(data->watch_gitdir);
|
||||
if (result == -1) {
|
||||
/* hard error */
|
||||
goto force_error_stop;
|
||||
}
|
||||
if (result == -2) {
|
||||
/* retryable error */
|
||||
if (start_rdcw_watch(data, data->watch_gitdir) == -1)
|
||||
goto force_error_stop;
|
||||
continue;
|
||||
}
|
||||
|
||||
/* have data */
|
||||
if (process_gitdir_events(state) == LISTENER_SHUTDOWN)
|
||||
goto force_shutdown;
|
||||
if (start_rdcw_watch(data, data->watch_gitdir) == -1)
|
||||
goto force_error_stop;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (dwWait == WAIT_OBJECT_0 + LISTENER_SHUTDOWN)
|
||||
goto clean_shutdown;
|
||||
|
||||
error(_("could not read directory changes [GLE %ld]"),
|
||||
GetLastError());
|
||||
goto force_error_stop;
|
||||
}
|
||||
|
||||
force_error_stop:
|
||||
state->error_code = -1;
|
||||
|
||||
force_shutdown:
|
||||
/*
|
||||
* Tell the IPC thead pool to stop (which completes the await
|
||||
* in the main thread (which will also signal this thread (if
|
||||
* we are still alive))).
|
||||
*/
|
||||
ipc_server_stop_async(state->ipc_server_data);
|
||||
|
||||
clean_shutdown:
|
||||
cancel_rdcw_watch(data->watch_worktree);
|
||||
cancel_rdcw_watch(data->watch_gitdir);
|
||||
}
|
||||
|
||||
int fsm_listen__ctor(struct fsmonitor_daemon_state *state)
|
||||
{
|
||||
struct fsmonitor_daemon_backend_data *data;
|
||||
|
||||
CALLOC_ARRAY(data, 1);
|
||||
|
||||
data->hEventShutdown = CreateEvent(NULL, TRUE, FALSE, NULL);
|
||||
|
||||
data->watch_worktree = create_watch(state,
|
||||
state->path_worktree_watch.buf);
|
||||
if (!data->watch_worktree)
|
||||
goto failed;
|
||||
|
||||
if (state->nr_paths_watching > 1) {
|
||||
data->watch_gitdir = create_watch(state,
|
||||
state->path_gitdir_watch.buf);
|
||||
if (!data->watch_gitdir)
|
||||
goto failed;
|
||||
}
|
||||
|
||||
data->hListener[LISTENER_SHUTDOWN] = data->hEventShutdown;
|
||||
data->nr_listener_handles++;
|
||||
|
||||
data->hListener[LISTENER_HAVE_DATA_WORKTREE] =
|
||||
data->watch_worktree->hEvent;
|
||||
data->nr_listener_handles++;
|
||||
|
||||
if (data->watch_gitdir) {
|
||||
data->hListener[LISTENER_HAVE_DATA_GITDIR] =
|
||||
data->watch_gitdir->hEvent;
|
||||
data->nr_listener_handles++;
|
||||
}
|
||||
|
||||
state->backend_data = data;
|
||||
return 0;
|
||||
|
||||
failed:
|
||||
CloseHandle(data->hEventShutdown);
|
||||
destroy_watch(data->watch_worktree);
|
||||
destroy_watch(data->watch_gitdir);
|
||||
|
||||
return -1;
|
||||
}
|
||||
|
||||
void fsm_listen__dtor(struct fsmonitor_daemon_state *state)
|
||||
{
|
||||
struct fsmonitor_daemon_backend_data *data;
|
||||
|
||||
if (!state || !state->backend_data)
|
||||
return;
|
||||
|
||||
data = state->backend_data;
|
||||
|
||||
CloseHandle(data->hEventShutdown);
|
||||
destroy_watch(data->watch_worktree);
|
||||
destroy_watch(data->watch_gitdir);
|
||||
|
||||
FREE_AND_NULL(state->backend_data);
|
||||
}
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
#ifndef FSM_LISTEN_H
|
||||
#define FSM_LISTEN_H
|
||||
|
||||
/* This needs to be implemented by each backend */
|
||||
|
||||
#ifdef HAVE_FSMONITOR_DAEMON_BACKEND
|
||||
|
||||
struct fsmonitor_daemon_state;
|
||||
|
||||
/*
|
||||
* Initialize platform-specific data for the fsmonitor listener thread.
|
||||
* This will be called from the main thread PRIOR to staring the
|
||||
* fsmonitor_fs_listener thread.
|
||||
*
|
||||
* Returns 0 if successful.
|
||||
* Returns -1 otherwise.
|
||||
*/
|
||||
int fsm_listen__ctor(struct fsmonitor_daemon_state *state);
|
||||
|
||||
/*
|
||||
* Cleanup platform-specific data for the fsmonitor listener thread.
|
||||
* This will be called from the main thread AFTER joining the listener.
|
||||
*/
|
||||
void fsm_listen__dtor(struct fsmonitor_daemon_state *state);
|
||||
|
||||
/*
|
||||
* The main body of the platform-specific event loop to watch for
|
||||
* filesystem events. This will run in the fsmonitor_fs_listen thread.
|
||||
*
|
||||
* It should call `ipc_server_stop_async()` if the listener thread
|
||||
* prematurely terminates (because of a filesystem error or if it
|
||||
* detects that the .git directory has been deleted). (It should NOT
|
||||
* do so if the listener thread receives a normal shutdown signal from
|
||||
* the IPC layer.)
|
||||
*
|
||||
* It should set `state->error_code` to -1 if the daemon should exit
|
||||
* with an error.
|
||||
*/
|
||||
void fsm_listen__loop(struct fsmonitor_daemon_state *state);
|
||||
|
||||
/*
|
||||
* Gently request that the fsmonitor listener thread shutdown.
|
||||
* It does not wait for it to stop. The caller should do a JOIN
|
||||
* to wait for it.
|
||||
*/
|
||||
void fsm_listen__stop_async(struct fsmonitor_daemon_state *state);
|
||||
|
||||
#endif /* HAVE_FSMONITOR_DAEMON_BACKEND */
|
||||
#endif /* FSM_LISTEN_H */
|
||||
14
config.c
14
config.c
|
|
@ -2735,20 +2735,6 @@ int git_config_get_max_percent_split_change(void)
|
|||
return -1; /* default value */
|
||||
}
|
||||
|
||||
int git_config_get_fsmonitor(void)
|
||||
{
|
||||
if (git_config_get_pathname("core.fsmonitor", &core_fsmonitor))
|
||||
core_fsmonitor = getenv("GIT_TEST_FSMONITOR");
|
||||
|
||||
if (core_fsmonitor && !*core_fsmonitor)
|
||||
core_fsmonitor = NULL;
|
||||
|
||||
if (core_fsmonitor)
|
||||
return 1;
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
int git_config_get_index_threads(int *dest)
|
||||
{
|
||||
int is_bool, val;
|
||||
|
|
|
|||
1
config.h
1
config.h
|
|
@ -597,7 +597,6 @@ int git_config_get_pathname(const char *key, const char **dest);
|
|||
int git_config_get_index_threads(int *dest);
|
||||
int git_config_get_split_index(void);
|
||||
int git_config_get_max_percent_split_change(void);
|
||||
int git_config_get_fsmonitor(void);
|
||||
|
||||
/* This dies if the configured or default date is in the future */
|
||||
int git_config_get_expiry(const char *key, const char **output);
|
||||
|
|
|
|||
|
|
@ -158,6 +158,16 @@ ifeq ($(uname_S),Darwin)
|
|||
MSGFMT = /usr/local/opt/gettext/bin/msgfmt
|
||||
endif
|
||||
endif
|
||||
|
||||
# The builtin FSMonitor on MacOS builds upon Simple-IPC. Both require
|
||||
# Unix domain sockets and PThreads.
|
||||
ifndef NO_PTHREADS
|
||||
ifndef NO_UNIX_SOCKETS
|
||||
FSMONITOR_DAEMON_BACKEND = darwin
|
||||
endif
|
||||
endif
|
||||
|
||||
BASIC_LDFLAGS += -framework CoreServices
|
||||
endif
|
||||
ifeq ($(uname_S),SunOS)
|
||||
NEEDS_SOCKET = YesPlease
|
||||
|
|
@ -436,6 +446,11 @@ ifeq ($(uname_S),Windows)
|
|||
# so we don't need this:
|
||||
#
|
||||
# SNPRINTF_RETURNS_BOGUS = YesPlease
|
||||
|
||||
# The builtin FSMonitor requires Named Pipes and Threads on Windows.
|
||||
# These are always available, so we do not have to conditionally
|
||||
# support it.
|
||||
FSMONITOR_DAEMON_BACKEND = win32
|
||||
NO_SVN_TESTS = YesPlease
|
||||
RUNTIME_PREFIX = YesPlease
|
||||
HAVE_WPGMPTR = YesWeDo
|
||||
|
|
@ -621,6 +636,11 @@ ifeq ($(uname_S),MINGW)
|
|||
NO_STRTOUMAX = YesPlease
|
||||
NO_MKDTEMP = YesPlease
|
||||
NO_SVN_TESTS = YesPlease
|
||||
|
||||
# The builtin FSMonitor requires Named Pipes and Threads on Windows.
|
||||
# These are always available, so we do not have to conditionally
|
||||
# support it.
|
||||
FSMONITOR_DAEMON_BACKEND = win32
|
||||
RUNTIME_PREFIX = YesPlease
|
||||
HAVE_WPGMPTR = YesWeDo
|
||||
NO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease
|
||||
|
|
|
|||
|
|
@ -293,6 +293,16 @@ else()
|
|||
endif()
|
||||
endif()
|
||||
|
||||
if(SUPPORTS_SIMPLE_IPC)
|
||||
if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
|
||||
add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
|
||||
list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c)
|
||||
elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin")
|
||||
add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
|
||||
list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c)
|
||||
endif()
|
||||
endif()
|
||||
|
||||
set(EXE_EXTENSION ${CMAKE_EXECUTABLE_SUFFIX})
|
||||
|
||||
#header checks
|
||||
|
|
|
|||
|
|
@ -87,7 +87,6 @@ int protect_hfs = PROTECT_HFS_DEFAULT;
|
|||
#define PROTECT_NTFS_DEFAULT 1
|
||||
#endif
|
||||
int protect_ntfs = PROTECT_NTFS_DEFAULT;
|
||||
const char *core_fsmonitor;
|
||||
|
||||
/*
|
||||
* The character that begins a commented line in user-editable file
|
||||
|
|
|
|||
|
|
@ -0,0 +1,166 @@
|
|||
#ifndef FSMONITOR_DAEMON_H
|
||||
#define FSMONITOR_DAEMON_H
|
||||
|
||||
#ifdef HAVE_FSMONITOR_DAEMON_BACKEND
|
||||
|
||||
#include "cache.h"
|
||||
#include "dir.h"
|
||||
#include "run-command.h"
|
||||
#include "simple-ipc.h"
|
||||
#include "thread-utils.h"
|
||||
|
||||
struct fsmonitor_batch;
|
||||
struct fsmonitor_token_data;
|
||||
|
||||
/*
|
||||
* Create a new batch of path(s). The returned batch is considered
|
||||
* private and not linked into the fsmonitor daemon state. The caller
|
||||
* should fill this batch with one or more paths and then publish it.
|
||||
*/
|
||||
struct fsmonitor_batch *fsmonitor_batch__new(void);
|
||||
|
||||
/*
|
||||
* Free the list of batches starting with this one.
|
||||
*/
|
||||
void fsmonitor_batch__free_list(struct fsmonitor_batch *batch);
|
||||
|
||||
/*
|
||||
* Add this path to this batch of modified files.
|
||||
*
|
||||
* The batch should be private and NOT (yet) linked into the fsmonitor
|
||||
* daemon state and therefore not yet visible to worker threads and so
|
||||
* no locking is required.
|
||||
*/
|
||||
void fsmonitor_batch__add_path(struct fsmonitor_batch *batch, const char *path);
|
||||
|
||||
struct fsmonitor_daemon_backend_data; /* opaque platform-specific data */
|
||||
|
||||
struct fsmonitor_daemon_state {
|
||||
pthread_t listener_thread;
|
||||
pthread_mutex_t main_lock;
|
||||
|
||||
struct strbuf path_worktree_watch;
|
||||
struct strbuf path_gitdir_watch;
|
||||
int nr_paths_watching;
|
||||
|
||||
struct fsmonitor_token_data *current_token_data;
|
||||
|
||||
struct strbuf path_cookie_prefix;
|
||||
pthread_cond_t cookies_cond;
|
||||
int cookie_seq;
|
||||
struct hashmap cookies;
|
||||
|
||||
int error_code;
|
||||
struct fsmonitor_daemon_backend_data *backend_data;
|
||||
|
||||
struct ipc_server_data *ipc_server_data;
|
||||
};
|
||||
|
||||
/*
|
||||
* Pathname classifications.
|
||||
*
|
||||
* The daemon classifies the pathnames that it receives from file
|
||||
* system notification events into the following categories and uses
|
||||
* that to decide whether clients are told about them. (And to watch
|
||||
* for file system synchronization events.)
|
||||
*
|
||||
* The daemon only collects and reports on the set of modified paths
|
||||
* within the working directory (proper).
|
||||
*
|
||||
* The client should only care about paths within the working
|
||||
* directory proper (inside the working directory and not ".git" nor
|
||||
* inside of ".git/"). That is, the client has read the index and is
|
||||
* asking for a list of any paths in the working directory that have
|
||||
* been modified since the last token. The client does not care about
|
||||
* file system changes within the ".git/" directory (such as new loose
|
||||
* objects or packfiles). So the client will only receive paths that
|
||||
* are classified as IS_WORKDIR_PATH.
|
||||
*
|
||||
* Note that ".git" is usually a directory and is therefore inside
|
||||
* the cone of the FS watch that we have on the working directory root,
|
||||
* so we will also get FS events for disk activity on and within ".git/"
|
||||
* that we need to respond to or filter from the client.
|
||||
*
|
||||
* But Git also allows ".git" to be a *file* that points to a GITDIR
|
||||
* outside of the working directory. When this happens, we need to
|
||||
* create FS watches on both the working directory root *and* on the
|
||||
* (external) GITDIR root. (The latter is required because we put
|
||||
* cookie files inside it and use them to sync with the FS event
|
||||
* stream.)
|
||||
*
|
||||
* Note that in the context of this discussion, I'm using "GITDIR"
|
||||
* to only mean an external GITDIR referenced by a ".git" file.
|
||||
*
|
||||
* The platform FS event backends will receive watch-specific
|
||||
* relative paths (except for those OS's that always emit absolute
|
||||
* paths). We use the following enum and routines to classify each
|
||||
* path so that we know how to handle it. There is a slight asymmetry
|
||||
* here because ".git/" is inside the working directory and the
|
||||
* (external) GITDIR is not, and therefore how we handle events may
|
||||
* vary slightly, so I have different enums for "IS...DOT_GIT..." and
|
||||
* "IS...GITDIR...".
|
||||
*
|
||||
* The daemon uses the IS_DOT_GIT and IS_GITDIR internally to mean the
|
||||
* exact ".git" file/directory or GITDIR directory. If the daemon
|
||||
* receives a delete event for either of these paths, it will
|
||||
* automatically shutdown, for example.
|
||||
*
|
||||
* Note that the daemon DOES NOT explicitly watch nor special case the
|
||||
* index. The daemon does not read the index nor have any internal
|
||||
* index-relative state, so there are no "IS...INDEX..." enum values.
|
||||
*/
|
||||
enum fsmonitor_path_type {
|
||||
IS_WORKDIR_PATH = 0,
|
||||
|
||||
IS_DOT_GIT,
|
||||
IS_INSIDE_DOT_GIT,
|
||||
IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX,
|
||||
|
||||
IS_GITDIR,
|
||||
IS_INSIDE_GITDIR,
|
||||
IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX,
|
||||
|
||||
IS_OUTSIDE_CONE,
|
||||
};
|
||||
|
||||
/*
|
||||
* Classify a pathname relative to the root of the working directory.
|
||||
*/
|
||||
enum fsmonitor_path_type fsmonitor_classify_path_workdir_relative(
|
||||
const char *relative_path);
|
||||
|
||||
/*
|
||||
* Classify a pathname relative to a <gitdir> that is external to the
|
||||
* worktree directory.
|
||||
*/
|
||||
enum fsmonitor_path_type fsmonitor_classify_path_gitdir_relative(
|
||||
const char *relative_path);
|
||||
|
||||
/*
|
||||
* Classify an absolute pathname received from a filesystem event.
|
||||
*/
|
||||
enum fsmonitor_path_type fsmonitor_classify_path_absolute(
|
||||
struct fsmonitor_daemon_state *state,
|
||||
const char *path);
|
||||
|
||||
/*
|
||||
* Prepend the this batch of path(s) onto the list of batches associated
|
||||
* with the current token. This makes the batch visible to worker threads.
|
||||
*
|
||||
* The caller no longer owns the batch and must not free it.
|
||||
*
|
||||
* Wake up the client threads waiting on these cookies.
|
||||
*/
|
||||
void fsmonitor_publish(struct fsmonitor_daemon_state *state,
|
||||
struct fsmonitor_batch *batch,
|
||||
const struct string_list *cookie_names);
|
||||
|
||||
/*
|
||||
* If the platform-specific layer loses sync with the filesystem,
|
||||
* it should call this to invalidate cached data and abort waiting
|
||||
* threads.
|
||||
*/
|
||||
void fsmonitor_force_resync(struct fsmonitor_daemon_state *state);
|
||||
|
||||
#endif /* HAVE_FSMONITOR_DAEMON_BACKEND */
|
||||
#endif /* FSMONITOR_DAEMON_H */
|
||||
|
|
@ -0,0 +1,171 @@
|
|||
#include "cache.h"
|
||||
#include "fsmonitor.h"
|
||||
#include "simple-ipc.h"
|
||||
#include "fsmonitor-ipc.h"
|
||||
#include "run-command.h"
|
||||
#include "strbuf.h"
|
||||
#include "trace2.h"
|
||||
|
||||
#ifndef HAVE_FSMONITOR_DAEMON_BACKEND
|
||||
|
||||
/*
|
||||
* A trivial implementation of the fsmonitor_ipc__ API for unsupported
|
||||
* platforms.
|
||||
*/
|
||||
|
||||
int fsmonitor_ipc__is_supported(void)
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
|
||||
const char *fsmonitor_ipc__get_path(void)
|
||||
{
|
||||
return NULL;
|
||||
}
|
||||
|
||||
enum ipc_active_state fsmonitor_ipc__get_state(void)
|
||||
{
|
||||
return IPC_STATE__OTHER_ERROR;
|
||||
}
|
||||
|
||||
int fsmonitor_ipc__send_query(const char *since_token,
|
||||
struct strbuf *answer)
|
||||
{
|
||||
return -1;
|
||||
}
|
||||
|
||||
int fsmonitor_ipc__send_command(const char *command,
|
||||
struct strbuf *answer)
|
||||
{
|
||||
return -1;
|
||||
}
|
||||
|
||||
#else
|
||||
|
||||
int fsmonitor_ipc__is_supported(void)
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
|
||||
GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
|
||||
|
||||
enum ipc_active_state fsmonitor_ipc__get_state(void)
|
||||
{
|
||||
return ipc_get_active_state(fsmonitor_ipc__get_path());
|
||||
}
|
||||
|
||||
static int spawn_daemon(void)
|
||||
{
|
||||
const char *args[] = { "fsmonitor--daemon", "start", NULL };
|
||||
|
||||
return run_command_v_opt_tr2(args, RUN_COMMAND_NO_STDIN | RUN_GIT_CMD,
|
||||
"fsmonitor");
|
||||
}
|
||||
|
||||
int fsmonitor_ipc__send_query(const char *since_token,
|
||||
struct strbuf *answer)
|
||||
{
|
||||
int ret = -1;
|
||||
int tried_to_spawn = 0;
|
||||
enum ipc_active_state state = IPC_STATE__OTHER_ERROR;
|
||||
struct ipc_client_connection *connection = NULL;
|
||||
struct ipc_client_connect_options options
|
||||
= IPC_CLIENT_CONNECT_OPTIONS_INIT;
|
||||
const char *tok = since_token ? since_token : "";
|
||||
size_t tok_len = since_token ? strlen(since_token) : 0;
|
||||
|
||||
options.wait_if_busy = 1;
|
||||
options.wait_if_not_found = 0;
|
||||
|
||||
trace2_region_enter("fsm_client", "query", NULL);
|
||||
trace2_data_string("fsm_client", NULL, "query/command", tok);
|
||||
|
||||
try_again:
|
||||
state = ipc_client_try_connect(fsmonitor_ipc__get_path(), &options,
|
||||
&connection);
|
||||
|
||||
switch (state) {
|
||||
case IPC_STATE__LISTENING:
|
||||
ret = ipc_client_send_command_to_connection(
|
||||
connection, tok, tok_len, answer);
|
||||
ipc_client_close_connection(connection);
|
||||
|
||||
trace2_data_intmax("fsm_client", NULL,
|
||||
"query/response-length", answer->len);
|
||||
goto done;
|
||||
|
||||
case IPC_STATE__NOT_LISTENING:
|
||||
case IPC_STATE__PATH_NOT_FOUND:
|
||||
if (tried_to_spawn)
|
||||
goto done;
|
||||
|
||||
tried_to_spawn++;
|
||||
if (spawn_daemon())
|
||||
goto done;
|
||||
|
||||
/*
|
||||
* Try again, but this time give the daemon a chance to
|
||||
* actually create the pipe/socket.
|
||||
*
|
||||
* Granted, the daemon just started so it can't possibly have
|
||||
* any FS cached yet, so we'll always get a trivial answer.
|
||||
* BUT the answer should include a new token that can serve
|
||||
* as the basis for subsequent requests.
|
||||
*/
|
||||
options.wait_if_not_found = 1;
|
||||
goto try_again;
|
||||
|
||||
case IPC_STATE__INVALID_PATH:
|
||||
ret = error(_("fsmonitor_ipc__send_query: invalid path '%s'"),
|
||||
fsmonitor_ipc__get_path());
|
||||
goto done;
|
||||
|
||||
case IPC_STATE__OTHER_ERROR:
|
||||
default:
|
||||
ret = error(_("fsmonitor_ipc__send_query: unspecified error on '%s'"),
|
||||
fsmonitor_ipc__get_path());
|
||||
goto done;
|
||||
}
|
||||
|
||||
done:
|
||||
trace2_region_leave("fsm_client", "query", NULL);
|
||||
|
||||
return ret;
|
||||
}
|
||||
|
||||
int fsmonitor_ipc__send_command(const char *command,
|
||||
struct strbuf *answer)
|
||||
{
|
||||
struct ipc_client_connection *connection = NULL;
|
||||
struct ipc_client_connect_options options
|
||||
= IPC_CLIENT_CONNECT_OPTIONS_INIT;
|
||||
int ret;
|
||||
enum ipc_active_state state;
|
||||
const char *c = command ? command : "";
|
||||
size_t c_len = command ? strlen(command) : 0;
|
||||
|
||||
strbuf_reset(answer);
|
||||
|
||||
options.wait_if_busy = 1;
|
||||
options.wait_if_not_found = 0;
|
||||
|
||||
state = ipc_client_try_connect(fsmonitor_ipc__get_path(), &options,
|
||||
&connection);
|
||||
if (state != IPC_STATE__LISTENING) {
|
||||
die(_("fsmonitor--daemon is not running"));
|
||||
return -1;
|
||||
}
|
||||
|
||||
ret = ipc_client_send_command_to_connection(connection, c, c_len,
|
||||
answer);
|
||||
ipc_client_close_connection(connection);
|
||||
|
||||
if (ret == -1) {
|
||||
die(_("could not send '%s' command to fsmonitor--daemon"), c);
|
||||
return -1;
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
#endif
|
||||
|
|
@ -0,0 +1,48 @@
|
|||
#ifndef FSMONITOR_IPC_H
|
||||
#define FSMONITOR_IPC_H
|
||||
|
||||
#include "simple-ipc.h"
|
||||
|
||||
/*
|
||||
* Returns true if built-in file system monitor daemon is defined
|
||||
* for this platform.
|
||||
*/
|
||||
int fsmonitor_ipc__is_supported(void);
|
||||
|
||||
/*
|
||||
* Returns the pathname to the IPC named pipe or Unix domain socket
|
||||
* where a `git-fsmonitor--daemon` process will listen. This is a
|
||||
* per-worktree value.
|
||||
*
|
||||
* Returns NULL if the daemon is not supported on this platform.
|
||||
*/
|
||||
const char *fsmonitor_ipc__get_path(void);
|
||||
|
||||
/*
|
||||
* Try to determine whether there is a `git-fsmonitor--daemon` process
|
||||
* listening on the IPC pipe/socket.
|
||||
*/
|
||||
enum ipc_active_state fsmonitor_ipc__get_state(void);
|
||||
|
||||
/*
|
||||
* Connect to a `git-fsmonitor--daemon` process via simple-ipc
|
||||
* and ask for the set of changed files since the given token.
|
||||
*
|
||||
* Spawn a daemon process in the background if necessary.
|
||||
*
|
||||
* Returns -1 on error; 0 on success.
|
||||
*/
|
||||
int fsmonitor_ipc__send_query(const char *since_token,
|
||||
struct strbuf *answer);
|
||||
|
||||
/*
|
||||
* Connect to a `git-fsmonitor--daemon` process via simple-ipc and
|
||||
* send a command verb. If no daemon is available, we DO NOT try to
|
||||
* start one.
|
||||
*
|
||||
* Returns -1 on error; 0 on success.
|
||||
*/
|
||||
int fsmonitor_ipc__send_command(const char *command,
|
||||
struct strbuf *answer);
|
||||
|
||||
#endif /* FSMONITOR_IPC_H */
|
||||
|
|
@ -0,0 +1,114 @@
|
|||
#include "cache.h"
|
||||
#include "config.h"
|
||||
#include "repository.h"
|
||||
#include "fsmonitor-settings.h"
|
||||
|
||||
/*
|
||||
* We keep this structure defintion private and have getters
|
||||
* for all fields so that we can lazy load it as needed.
|
||||
*/
|
||||
struct fsmonitor_settings {
|
||||
enum fsmonitor_mode mode;
|
||||
char *hook_path;
|
||||
};
|
||||
|
||||
static void lookup_fsmonitor_settings(struct repository *r)
|
||||
{
|
||||
struct fsmonitor_settings *s;
|
||||
const char *const_str;
|
||||
int bool_value;
|
||||
|
||||
if (r->settings.fsmonitor)
|
||||
return;
|
||||
|
||||
CALLOC_ARRAY(s, 1);
|
||||
s->mode = FSMONITOR_MODE_DISABLED;
|
||||
|
||||
r->settings.fsmonitor = s;
|
||||
|
||||
/*
|
||||
* Overload the existing "core.fsmonitor" config setting (which
|
||||
* has historically been either unset or a hook pathname) to
|
||||
* now allow a boolean value to enable the builtin FSMonitor
|
||||
* or to turn everything off. (This does imply that you can't
|
||||
* use a hook script named "true" or "false", but that's OK.)
|
||||
*/
|
||||
switch (repo_config_get_maybe_bool(r, "core.fsmonitor", &bool_value)) {
|
||||
|
||||
case 0: /* config value was set to <bool> */
|
||||
if (bool_value)
|
||||
fsm_settings__set_ipc(r);
|
||||
return;
|
||||
|
||||
case 1: /* config value was unset */
|
||||
const_str = getenv("GIT_TEST_FSMONITOR");
|
||||
break;
|
||||
|
||||
case -1: /* config value set to an arbitrary string */
|
||||
if (repo_config_get_pathname(r, "core.fsmonitor", &const_str))
|
||||
return; /* should not happen */
|
||||
break;
|
||||
|
||||
default: /* should not happen */
|
||||
return;
|
||||
}
|
||||
|
||||
if (!const_str || !*const_str)
|
||||
return;
|
||||
|
||||
fsm_settings__set_hook(r, const_str);
|
||||
}
|
||||
|
||||
enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
|
||||
{
|
||||
if (!r)
|
||||
r = the_repository;
|
||||
|
||||
lookup_fsmonitor_settings(r);
|
||||
|
||||
return r->settings.fsmonitor->mode;
|
||||
}
|
||||
|
||||
const char *fsm_settings__get_hook_path(struct repository *r)
|
||||
{
|
||||
if (!r)
|
||||
r = the_repository;
|
||||
|
||||
lookup_fsmonitor_settings(r);
|
||||
|
||||
return r->settings.fsmonitor->hook_path;
|
||||
}
|
||||
|
||||
void fsm_settings__set_ipc(struct repository *r)
|
||||
{
|
||||
if (!r)
|
||||
r = the_repository;
|
||||
|
||||
lookup_fsmonitor_settings(r);
|
||||
|
||||
r->settings.fsmonitor->mode = FSMONITOR_MODE_IPC;
|
||||
FREE_AND_NULL(r->settings.fsmonitor->hook_path);
|
||||
}
|
||||
|
||||
void fsm_settings__set_hook(struct repository *r, const char *path)
|
||||
{
|
||||
if (!r)
|
||||
r = the_repository;
|
||||
|
||||
lookup_fsmonitor_settings(r);
|
||||
|
||||
r->settings.fsmonitor->mode = FSMONITOR_MODE_HOOK;
|
||||
FREE_AND_NULL(r->settings.fsmonitor->hook_path);
|
||||
r->settings.fsmonitor->hook_path = strdup(path);
|
||||
}
|
||||
|
||||
void fsm_settings__set_disabled(struct repository *r)
|
||||
{
|
||||
if (!r)
|
||||
r = the_repository;
|
||||
|
||||
lookup_fsmonitor_settings(r);
|
||||
|
||||
r->settings.fsmonitor->mode = FSMONITOR_MODE_DISABLED;
|
||||
FREE_AND_NULL(r->settings.fsmonitor->hook_path);
|
||||
}
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
#ifndef FSMONITOR_SETTINGS_H
|
||||
#define FSMONITOR_SETTINGS_H
|
||||
|
||||
struct repository;
|
||||
|
||||
enum fsmonitor_mode {
|
||||
FSMONITOR_MODE_DISABLED = 0,
|
||||
FSMONITOR_MODE_HOOK = 1, /* core.fsmonitor=<hook_path> */
|
||||
FSMONITOR_MODE_IPC = 2, /* core.fsmonitor=<true> */
|
||||
};
|
||||
|
||||
void fsm_settings__set_ipc(struct repository *r);
|
||||
void fsm_settings__set_hook(struct repository *r, const char *path);
|
||||
void fsm_settings__set_disabled(struct repository *r);
|
||||
|
||||
enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);
|
||||
const char *fsm_settings__get_hook_path(struct repository *r);
|
||||
|
||||
struct fsmonitor_settings;
|
||||
|
||||
#endif /* FSMONITOR_SETTINGS_H */
|
||||
216
fsmonitor.c
216
fsmonitor.c
|
|
@ -3,6 +3,7 @@
|
|||
#include "dir.h"
|
||||
#include "ewah/ewok.h"
|
||||
#include "fsmonitor.h"
|
||||
#include "fsmonitor-ipc.h"
|
||||
#include "run-command.h"
|
||||
#include "strbuf.h"
|
||||
|
||||
|
|
@ -148,15 +149,18 @@ void write_fsmonitor_extension(struct strbuf *sb, struct index_state *istate)
|
|||
/*
|
||||
* Call the query-fsmonitor hook passing the last update token of the saved results.
|
||||
*/
|
||||
static int query_fsmonitor(int version, const char *last_update, struct strbuf *query_result)
|
||||
static int query_fsmonitor_hook(struct repository *r,
|
||||
int version,
|
||||
const char *last_update,
|
||||
struct strbuf *query_result)
|
||||
{
|
||||
struct child_process cp = CHILD_PROCESS_INIT;
|
||||
int result;
|
||||
|
||||
if (!core_fsmonitor)
|
||||
if (fsm_settings__get_mode(r) != FSMONITOR_MODE_HOOK)
|
||||
return -1;
|
||||
|
||||
strvec_push(&cp.args, core_fsmonitor);
|
||||
strvec_push(&cp.args, fsm_settings__get_hook_path(r));
|
||||
strvec_pushf(&cp.args, "%d", version);
|
||||
strvec_pushf(&cp.args, "%s", last_update);
|
||||
cp.use_shell = 1;
|
||||
|
|
@ -168,29 +172,15 @@ static int query_fsmonitor(int version, const char *last_update, struct strbuf *
|
|||
|
||||
if (result)
|
||||
trace2_data_intmax("fsm_hook", NULL, "query/failed", result);
|
||||
else {
|
||||
else
|
||||
trace2_data_intmax("fsm_hook", NULL, "query/response-length",
|
||||
query_result->len);
|
||||
|
||||
if (fsmonitor_is_trivial_response(query_result))
|
||||
trace2_data_intmax("fsm_hook", NULL,
|
||||
"query/trivial-response", 1);
|
||||
}
|
||||
|
||||
trace2_region_leave("fsm_hook", "query", NULL);
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
int fsmonitor_is_trivial_response(const struct strbuf *query_result)
|
||||
{
|
||||
static char trivial_response[3] = { '\0', '/', '\0' };
|
||||
|
||||
return query_result->len >= 3 &&
|
||||
!memcmp(trivial_response,
|
||||
&query_result->buf[query_result->len - 3], 3);
|
||||
}
|
||||
|
||||
static void fsmonitor_refresh_callback(struct index_state *istate, char *name)
|
||||
{
|
||||
int i, len = strlen(name);
|
||||
|
|
@ -229,6 +219,43 @@ static void fsmonitor_refresh_callback(struct index_state *istate, char *name)
|
|||
untracked_cache_invalidate_path(istate, name, 0);
|
||||
}
|
||||
|
||||
/*
|
||||
* The number of pathnames that we need to receive from FSMonitor
|
||||
* before we force the index to be updated.
|
||||
*
|
||||
* Note that any pathname within the set of received paths MAY cause
|
||||
* cache-entry or istate flag bits to be updated and thus cause the
|
||||
* index to be updated on disk.
|
||||
*
|
||||
* However, the response may contain many paths (such as ignored
|
||||
* paths) that will not update any flag bits. And thus not force the
|
||||
* index to be updated. (This is fine and normal.) It also means
|
||||
* that the token will not be updated in the FSMonitor index
|
||||
* extension. So the next Git command will find the same token in the
|
||||
* index, make the same token-relative request, and receive the same
|
||||
* response (plus any newly changed paths). If this response is large
|
||||
* (and continues to grow), performance could be impacted.
|
||||
*
|
||||
* For example, if the user runs a build and it writes 100K object
|
||||
* files but doesn't modify any source files, the index would not need
|
||||
* to be updated. The FSMonitor response (after the build and
|
||||
* relative to a pre-build token) might be 5MB. Each subsequent Git
|
||||
* command will receive that same 100K/5MB response until something
|
||||
* causes the index to be updated. And `refresh_fsmonitor()` will
|
||||
* have to iterate over those 100K paths each time.
|
||||
*
|
||||
* Performance could be improved if we optionally force update the
|
||||
* index after a very large response and get an updated token into
|
||||
* the FSMonitor index extension. This should allow subsequent
|
||||
* commands to get smaller and more current responses.
|
||||
*
|
||||
* The value chosen here does not need to be precise. The index
|
||||
* will be updated automatically the first time the user touches
|
||||
* a tracked file and causes a command like `git status` to
|
||||
* update an mtime to be updated and/or set a flag bit.
|
||||
*/
|
||||
static int fsmonitor_force_update_threshold = 100;
|
||||
|
||||
void refresh_fsmonitor(struct index_state *istate)
|
||||
{
|
||||
struct strbuf query_result = STRBUF_INIT;
|
||||
|
|
@ -238,17 +265,62 @@ void refresh_fsmonitor(struct index_state *istate)
|
|||
struct strbuf last_update_token = STRBUF_INIT;
|
||||
char *buf;
|
||||
unsigned int i;
|
||||
int is_trivial = 0;
|
||||
struct repository *r = istate->repo ? istate->repo : the_repository;
|
||||
enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r);
|
||||
|
||||
if (!core_fsmonitor || istate->fsmonitor_has_run_once)
|
||||
if (fsm_mode <= FSMONITOR_MODE_DISABLED ||
|
||||
istate->fsmonitor_has_run_once)
|
||||
return;
|
||||
|
||||
hook_version = fsmonitor_hook_version();
|
||||
|
||||
istate->fsmonitor_has_run_once = 1;
|
||||
|
||||
trace_printf_key(&trace_fsmonitor, "refresh fsmonitor");
|
||||
|
||||
if (fsm_mode == FSMONITOR_MODE_IPC) {
|
||||
query_success = !fsmonitor_ipc__send_query(
|
||||
istate->fsmonitor_last_update ?
|
||||
istate->fsmonitor_last_update : "builtin:fake",
|
||||
&query_result);
|
||||
if (query_success) {
|
||||
/*
|
||||
* The response contains a series of nul terminated
|
||||
* strings. The first is the new token.
|
||||
*
|
||||
* Use `char *buf` as an interlude to trick the CI
|
||||
* static analysis to let us use `strbuf_addstr()`
|
||||
* here (and only copy the token) rather than
|
||||
* `strbuf_addbuf()`.
|
||||
*/
|
||||
buf = query_result.buf;
|
||||
strbuf_addstr(&last_update_token, buf);
|
||||
bol = last_update_token.len + 1;
|
||||
is_trivial = query_result.buf[bol] == '/';
|
||||
if (is_trivial)
|
||||
trace2_data_intmax("fsm_client", NULL,
|
||||
"query/trivial-response", 1);
|
||||
} else {
|
||||
/*
|
||||
* The builtin daemon is not available on this
|
||||
* platform -OR- we failed to get a response.
|
||||
*
|
||||
* Generate a fake token (rather than a V1
|
||||
* timestamp) for the index extension. (If
|
||||
* they switch back to the hook API, we don't
|
||||
* want ambiguous state.)
|
||||
*/
|
||||
strbuf_addstr(&last_update_token, "builtin:fake");
|
||||
}
|
||||
|
||||
goto apply_results;
|
||||
}
|
||||
|
||||
assert(fsm_mode == FSMONITOR_MODE_HOOK);
|
||||
|
||||
hook_version = fsmonitor_hook_version();
|
||||
|
||||
/*
|
||||
* This could be racy so save the date/time now and query_fsmonitor
|
||||
* This could be racy so save the date/time now and query_fsmonitor_hook
|
||||
* should be inclusive to ensure we don't miss potential changes.
|
||||
*/
|
||||
last_update = getnanotime();
|
||||
|
|
@ -256,13 +328,14 @@ void refresh_fsmonitor(struct index_state *istate)
|
|||
strbuf_addf(&last_update_token, "%"PRIu64"", last_update);
|
||||
|
||||
/*
|
||||
* If we have a last update token, call query_fsmonitor for the set of
|
||||
* If we have a last update token, call query_fsmonitor_hook for the set of
|
||||
* changes since that token, else assume everything is possibly dirty
|
||||
* and check it all.
|
||||
*/
|
||||
if (istate->fsmonitor_last_update) {
|
||||
if (hook_version == -1 || hook_version == HOOK_INTERFACE_VERSION2) {
|
||||
query_success = !query_fsmonitor(HOOK_INTERFACE_VERSION2,
|
||||
query_success = !query_fsmonitor_hook(
|
||||
r, HOOK_INTERFACE_VERSION2,
|
||||
istate->fsmonitor_last_update, &query_result);
|
||||
|
||||
if (query_success) {
|
||||
|
|
@ -283,6 +356,7 @@ void refresh_fsmonitor(struct index_state *istate)
|
|||
query_success = 0;
|
||||
} else {
|
||||
bol = last_update_token.len + 1;
|
||||
is_trivial = query_result.buf[bol] == '/';
|
||||
}
|
||||
} else if (hook_version < 0) {
|
||||
hook_version = HOOK_INTERFACE_VERSION1;
|
||||
|
|
@ -292,37 +366,83 @@ void refresh_fsmonitor(struct index_state *istate)
|
|||
}
|
||||
|
||||
if (hook_version == HOOK_INTERFACE_VERSION1) {
|
||||
query_success = !query_fsmonitor(HOOK_INTERFACE_VERSION1,
|
||||
query_success = !query_fsmonitor_hook(
|
||||
r, HOOK_INTERFACE_VERSION1,
|
||||
istate->fsmonitor_last_update, &query_result);
|
||||
if (query_success)
|
||||
is_trivial = query_result.buf[0] == '/';
|
||||
}
|
||||
|
||||
trace_performance_since(last_update, "fsmonitor process '%s'", core_fsmonitor);
|
||||
trace_printf_key(&trace_fsmonitor, "fsmonitor process '%s' returned %s",
|
||||
core_fsmonitor, query_success ? "success" : "failure");
|
||||
if (is_trivial)
|
||||
trace2_data_intmax("fsm_hook", NULL,
|
||||
"query/trivial-response", 1);
|
||||
|
||||
trace_performance_since(last_update, "fsmonitor process '%s'",
|
||||
fsm_settings__get_hook_path(r));
|
||||
trace_printf_key(&trace_fsmonitor,
|
||||
"fsmonitor process '%s' returned %s",
|
||||
fsm_settings__get_hook_path(r),
|
||||
query_success ? "success" : "failure");
|
||||
}
|
||||
|
||||
/* a fsmonitor process can return '/' to indicate all entries are invalid */
|
||||
if (query_success && query_result.buf[bol] != '/') {
|
||||
/* Mark all entries returned by the monitor as dirty */
|
||||
apply_results:
|
||||
/*
|
||||
* The response from FSMonitor (excluding the header token) is
|
||||
* either:
|
||||
*
|
||||
* [a] a (possibly empty) list of NUL delimited relative
|
||||
* pathnames of changed paths. This list can contain
|
||||
* files and directories. Directories have a trailing
|
||||
* slash.
|
||||
*
|
||||
* [b] a single '/' to indicate the provider had no
|
||||
* information and that we should consider everything
|
||||
* invalid. We call this a trivial response.
|
||||
*/
|
||||
trace2_region_enter("fsmonitor", "apply_results", istate->repo);
|
||||
|
||||
if (query_success && !is_trivial) {
|
||||
/*
|
||||
* Mark all pathnames returned by the monitor as dirty.
|
||||
*
|
||||
* This updates both the cache-entries and the untracked-cache.
|
||||
*/
|
||||
int count = 0;
|
||||
|
||||
buf = query_result.buf;
|
||||
for (i = bol; i < query_result.len; i++) {
|
||||
if (buf[i] != '\0')
|
||||
continue;
|
||||
fsmonitor_refresh_callback(istate, buf + bol);
|
||||
bol = i + 1;
|
||||
count++;
|
||||
}
|
||||
if (bol < query_result.len)
|
||||
if (bol < query_result.len) {
|
||||
fsmonitor_refresh_callback(istate, buf + bol);
|
||||
count++;
|
||||
}
|
||||
|
||||
/* Now mark the untracked cache for fsmonitor usage */
|
||||
if (istate->untracked)
|
||||
istate->untracked->use_fsmonitor = 1;
|
||||
} else {
|
||||
|
||||
/* We only want to run the post index changed hook if we've actually changed entries, so keep track
|
||||
* if we actually changed entries or not */
|
||||
if (count > fsmonitor_force_update_threshold)
|
||||
istate->cache_changed |= FSMONITOR_CHANGED;
|
||||
|
||||
trace2_data_intmax("fsmonitor", istate->repo, "apply_count",
|
||||
count);
|
||||
|
||||
} else {
|
||||
/*
|
||||
* We failed to get a response or received a trivial response,
|
||||
* so invalidate everything.
|
||||
*
|
||||
* We only want to run the post index changed hook if
|
||||
* we've actually changed entries, so keep track if we
|
||||
* actually changed entries or not.
|
||||
*/
|
||||
int is_cache_changed = 0;
|
||||
/* Mark all entries invalid */
|
||||
|
||||
for (i = 0; i < istate->cache_nr; i++) {
|
||||
if (istate->cache[i]->ce_flags & CE_FSMONITOR_VALID) {
|
||||
is_cache_changed = 1;
|
||||
|
|
@ -330,13 +450,18 @@ void refresh_fsmonitor(struct index_state *istate)
|
|||
}
|
||||
}
|
||||
|
||||
/* If we're going to check every file, ensure we save the results */
|
||||
/*
|
||||
* If we're going to check every file, ensure we save
|
||||
* the results.
|
||||
*/
|
||||
if (is_cache_changed)
|
||||
istate->cache_changed |= FSMONITOR_CHANGED;
|
||||
|
||||
if (istate->untracked)
|
||||
istate->untracked->use_fsmonitor = 0;
|
||||
}
|
||||
trace2_region_leave("fsmonitor", "apply_results", istate->repo);
|
||||
|
||||
strbuf_release(&query_result);
|
||||
|
||||
/* Now that we've updated istate, save the last_update_token */
|
||||
|
|
@ -411,7 +536,8 @@ void remove_fsmonitor(struct index_state *istate)
|
|||
void tweak_fsmonitor(struct index_state *istate)
|
||||
{
|
||||
unsigned int i;
|
||||
int fsmonitor_enabled = git_config_get_fsmonitor();
|
||||
int fsmonitor_enabled = (fsm_settings__get_mode(istate->repo)
|
||||
> FSMONITOR_MODE_DISABLED);
|
||||
|
||||
if (istate->fsmonitor_dirty) {
|
||||
if (fsmonitor_enabled) {
|
||||
|
|
@ -431,16 +557,8 @@ void tweak_fsmonitor(struct index_state *istate)
|
|||
istate->fsmonitor_dirty = NULL;
|
||||
}
|
||||
|
||||
switch (fsmonitor_enabled) {
|
||||
case -1: /* keep: do nothing */
|
||||
break;
|
||||
case 0: /* false */
|
||||
remove_fsmonitor(istate);
|
||||
break;
|
||||
case 1: /* true */
|
||||
if (fsmonitor_enabled)
|
||||
add_fsmonitor(istate);
|
||||
break;
|
||||
default: /* unknown value: do nothing */
|
||||
break;
|
||||
}
|
||||
else
|
||||
remove_fsmonitor(istate);
|
||||
}
|
||||
|
|
|
|||
15
fsmonitor.h
15
fsmonitor.h
|
|
@ -3,6 +3,7 @@
|
|||
|
||||
#include "cache.h"
|
||||
#include "dir.h"
|
||||
#include "fsmonitor-settings.h"
|
||||
|
||||
extern struct trace_key trace_fsmonitor;
|
||||
|
||||
|
|
@ -57,7 +58,10 @@ int fsmonitor_is_trivial_response(const struct strbuf *query_result);
|
|||
*/
|
||||
static inline int is_fsmonitor_refreshed(const struct index_state *istate)
|
||||
{
|
||||
return !core_fsmonitor || istate->fsmonitor_has_run_once;
|
||||
enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(istate->repo);
|
||||
|
||||
return fsm_mode <= FSMONITOR_MODE_DISABLED ||
|
||||
istate->fsmonitor_has_run_once;
|
||||
}
|
||||
|
||||
/*
|
||||
|
|
@ -67,7 +71,10 @@ static inline int is_fsmonitor_refreshed(const struct index_state *istate)
|
|||
*/
|
||||
static inline void mark_fsmonitor_valid(struct index_state *istate, struct cache_entry *ce)
|
||||
{
|
||||
if (core_fsmonitor && !(ce->ce_flags & CE_FSMONITOR_VALID)) {
|
||||
enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(istate->repo);
|
||||
|
||||
if (fsm_mode > FSMONITOR_MODE_DISABLED &&
|
||||
!(ce->ce_flags & CE_FSMONITOR_VALID)) {
|
||||
istate->cache_changed = 1;
|
||||
ce->ce_flags |= CE_FSMONITOR_VALID;
|
||||
trace_printf_key(&trace_fsmonitor, "mark_fsmonitor_clean '%s'", ce->name);
|
||||
|
|
@ -83,7 +90,9 @@ static inline void mark_fsmonitor_valid(struct index_state *istate, struct cache
|
|||
*/
|
||||
static inline void mark_fsmonitor_invalid(struct index_state *istate, struct cache_entry *ce)
|
||||
{
|
||||
if (core_fsmonitor) {
|
||||
enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(istate->repo);
|
||||
|
||||
if (fsm_mode > FSMONITOR_MODE_DISABLED) {
|
||||
ce->ce_flags &= ~CE_FSMONITOR_VALID;
|
||||
untracked_cache_invalidate_path(istate, ce->name, 1);
|
||||
trace_printf_key(&trace_fsmonitor, "mark_fsmonitor_invalid '%s'", ce->name);
|
||||
|
|
|
|||
1
git.c
1
git.c
|
|
@ -537,6 +537,7 @@ static struct cmd_struct commands[] = {
|
|||
{ "format-patch", cmd_format_patch, RUN_SETUP },
|
||||
{ "fsck", cmd_fsck, RUN_SETUP },
|
||||
{ "fsck-objects", cmd_fsck, RUN_SETUP },
|
||||
{ "fsmonitor--daemon", cmd_fsmonitor__daemon, RUN_SETUP },
|
||||
{ "gc", cmd_gc, RUN_SETUP },
|
||||
{ "get-tar-commit-id", cmd_get_tar_commit_id, NO_PARSEOPT },
|
||||
{ "grep", cmd_grep, RUN_SETUP_GENTLY },
|
||||
|
|
|
|||
4
help.c
4
help.c
|
|
@ -12,6 +12,7 @@
|
|||
#include "refs.h"
|
||||
#include "parse-options.h"
|
||||
#include "prompt.h"
|
||||
#include "fsmonitor-ipc.h"
|
||||
|
||||
struct category_description {
|
||||
uint32_t category;
|
||||
|
|
@ -714,6 +715,9 @@ void get_version_info(struct strbuf *buf, int show_build_options)
|
|||
strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
|
||||
strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
|
||||
/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
|
||||
|
||||
if (fsmonitor_ipc__is_supported())
|
||||
strbuf_addstr(buf, "feature: fsmonitor--daemon\n");
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@
|
|||
#include "config.h"
|
||||
#include "repository.h"
|
||||
#include "midx.h"
|
||||
#include "compat/fsmonitor/fsm-listen.h"
|
||||
|
||||
static void repo_cfg_bool(struct repository *r, const char *key, int *dest,
|
||||
int def)
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@
|
|||
#include "path.h"
|
||||
|
||||
struct config_set;
|
||||
struct fsmonitor_settings;
|
||||
struct git_hash_algo;
|
||||
struct index_state;
|
||||
struct lock_file;
|
||||
|
|
@ -35,6 +36,8 @@ struct repo_settings {
|
|||
int command_requires_full_index;
|
||||
int sparse_index;
|
||||
|
||||
struct fsmonitor_settings *fsmonitor; /* lazily loaded */
|
||||
|
||||
int index_version;
|
||||
enum untracked_cache_setting core_untracked_cache;
|
||||
|
||||
|
|
|
|||
4
t/README
4
t/README
|
|
@ -405,8 +405,8 @@ every 'git commit-graph write', as if the `--changed-paths` option was
|
|||
passed in.
|
||||
|
||||
GIT_TEST_FSMONITOR=$PWD/t7519/fsmonitor-all exercises the fsmonitor
|
||||
code path for utilizing a file system monitor to speed up detecting
|
||||
new or changed files.
|
||||
code paths for utilizing a (hook based) file system monitor to speed up
|
||||
detecting new or changed files.
|
||||
|
||||
GIT_TEST_INDEX_VERSION=<n> exercises the index read/write code path
|
||||
for the index version specified. Can be set to any valid version
|
||||
|
|
|
|||
|
|
@ -134,6 +134,21 @@ int cmd__chmtime(int argc, const char **argv)
|
|||
}
|
||||
|
||||
if (utb.modtime != sb.st_mtime && utime(argv[i], &utb) < 0) {
|
||||
#ifdef GIT_WINDOWS_NATIVE
|
||||
if (S_ISDIR(sb.st_mode)) {
|
||||
/*
|
||||
* NEEDSWORK: The Windows version of `utime()`
|
||||
* (aka `mingw_utime()`) does not correctly
|
||||
* handle directory arguments, since it uses
|
||||
* `_wopen()`. Ignore it for now since this
|
||||
* is just a test.
|
||||
*/
|
||||
fprintf(stderr,
|
||||
("Failed to modify time on directory %s. "
|
||||
"Skipping\n"), argv[i]);
|
||||
continue;
|
||||
}
|
||||
#endif
|
||||
fprintf(stderr, "Failed to modify time on %s: %s\n",
|
||||
argv[i], strerror(errno));
|
||||
return 1;
|
||||
|
|
|
|||
|
|
@ -0,0 +1,116 @@
|
|||
/*
|
||||
* test-fsmonitor-client.c: client code to send commands/requests to
|
||||
* a `git fsmonitor--daemon` daemon.
|
||||
*/
|
||||
|
||||
#include "test-tool.h"
|
||||
#include "cache.h"
|
||||
#include "parse-options.h"
|
||||
#include "fsmonitor-ipc.h"
|
||||
|
||||
#ifndef HAVE_FSMONITOR_DAEMON_BACKEND
|
||||
int cmd__fsmonitor_client(int argc, const char **argv)
|
||||
{
|
||||
die("fsmonitor--daemon not available on this platform");
|
||||
}
|
||||
#else
|
||||
|
||||
/*
|
||||
* Read the `.git/index` to get the last token written to the
|
||||
* FSMonitor Index Extension.
|
||||
*/
|
||||
static const char *get_token_from_index(void)
|
||||
{
|
||||
struct index_state *istate = the_repository->index;
|
||||
|
||||
if (do_read_index(istate, the_repository->index_file, 0) < 0)
|
||||
die("unable to read index file");
|
||||
if (!istate->fsmonitor_last_update)
|
||||
die("index file does not have fsmonitor extension");
|
||||
|
||||
return istate->fsmonitor_last_update;
|
||||
}
|
||||
|
||||
/*
|
||||
* Send an IPC query to a `git-fsmonitor--daemon` daemon and
|
||||
* ask for the changes since the given token or from the last
|
||||
* token in the index extension.
|
||||
*
|
||||
* This will implicitly start a daemon process if necessary. The
|
||||
* daemon process will persist after we exit.
|
||||
*/
|
||||
static int do_send_query(const char *token)
|
||||
{
|
||||
struct strbuf answer = STRBUF_INIT;
|
||||
int ret;
|
||||
|
||||
if (!token || !*token)
|
||||
token = get_token_from_index();
|
||||
|
||||
ret = fsmonitor_ipc__send_query(token, &answer);
|
||||
if (ret < 0)
|
||||
die("could not query fsmonitor--daemon");
|
||||
|
||||
write_in_full(1, answer.buf, answer.len);
|
||||
strbuf_release(&answer);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
/*
|
||||
* Send a "flush" command to the `git-fsmonitor--daemon` (if running)
|
||||
* and tell it to flush its cache.
|
||||
*
|
||||
* This feature is primarily used by the test suite to simulate a loss of
|
||||
* sync with the filesystem where we miss kernel events.
|
||||
*/
|
||||
static int do_send_flush(void)
|
||||
{
|
||||
struct strbuf answer = STRBUF_INIT;
|
||||
int ret;
|
||||
|
||||
ret = fsmonitor_ipc__send_command("flush", &answer);
|
||||
if (ret)
|
||||
return ret;
|
||||
|
||||
write_in_full(1, answer.buf, answer.len);
|
||||
strbuf_release(&answer);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
int cmd__fsmonitor_client(int argc, const char **argv)
|
||||
{
|
||||
const char *subcmd;
|
||||
const char *token = NULL;
|
||||
|
||||
const char * const fsmonitor_client_usage[] = {
|
||||
"test-tool fsmonitor-client query [<token>]",
|
||||
"test-tool fsmonitor-client flush",
|
||||
NULL,
|
||||
};
|
||||
|
||||
struct option options[] = {
|
||||
OPT_STRING(0, "token", &token, "token",
|
||||
"command token to send to the server"),
|
||||
OPT_END()
|
||||
};
|
||||
|
||||
argc = parse_options(argc, argv, NULL, options, fsmonitor_client_usage, 0);
|
||||
|
||||
if (argc != 1)
|
||||
usage_with_options(fsmonitor_client_usage, options);
|
||||
|
||||
subcmd = argv[0];
|
||||
|
||||
setup_git_directory();
|
||||
|
||||
if (!strcmp(subcmd, "query"))
|
||||
return !!do_send_query(token);
|
||||
|
||||
if (!strcmp(subcmd, "flush"))
|
||||
return !!do_send_flush();
|
||||
|
||||
die("Unhandled subcommand: '%s'", subcmd);
|
||||
}
|
||||
#endif
|
||||
|
|
@ -32,6 +32,7 @@ static struct test_cmd cmds[] = {
|
|||
{ "dump-untracked-cache", cmd__dump_untracked_cache },
|
||||
{ "example-decorate", cmd__example_decorate },
|
||||
{ "fast-rebase", cmd__fast_rebase },
|
||||
{ "fsmonitor-client", cmd__fsmonitor_client },
|
||||
{ "genrandom", cmd__genrandom },
|
||||
{ "genzeros", cmd__genzeros },
|
||||
{ "getcwd", cmd__getcwd },
|
||||
|
|
|
|||
|
|
@ -23,6 +23,7 @@ int cmd__dump_untracked_cache(int argc, const char **argv);
|
|||
int cmd__dump_reftable(int argc, const char **argv);
|
||||
int cmd__example_decorate(int argc, const char **argv);
|
||||
int cmd__fast_rebase(int argc, const char **argv);
|
||||
int cmd__fsmonitor_client(int argc, const char **argv);
|
||||
int cmd__genrandom(int argc, const char **argv);
|
||||
int cmd__genzeros(int argc, const char **argv);
|
||||
int cmd__getcwd(int argc, const char **argv);
|
||||
|
|
|
|||
|
|
@ -72,7 +72,7 @@ then
|
|||
fi
|
||||
fi
|
||||
|
||||
trace_start() {
|
||||
trace_start () {
|
||||
if test -n "$GIT_PERF_7519_TRACE"
|
||||
then
|
||||
name="$1"
|
||||
|
|
@ -91,13 +91,20 @@ trace_start() {
|
|||
fi
|
||||
}
|
||||
|
||||
trace_stop() {
|
||||
trace_stop () {
|
||||
if test -n "$GIT_PERF_7519_TRACE"
|
||||
then
|
||||
unset GIT_TRACE2_PERF
|
||||
fi
|
||||
}
|
||||
|
||||
touch_files () {
|
||||
n=$1 &&
|
||||
d="$n"_files &&
|
||||
|
||||
(cd $d && test_seq 1 $n | xargs touch )
|
||||
}
|
||||
|
||||
test_expect_success "one time repo setup" '
|
||||
# set untrackedCache depending on the environment
|
||||
if test -n "$GIT_PERF_7519_UNTRACKED_CACHE"
|
||||
|
|
@ -119,10 +126,11 @@ test_expect_success "one time repo setup" '
|
|||
fi &&
|
||||
|
||||
mkdir 1_file 10_files 100_files 1000_files 10000_files &&
|
||||
for i in $(test_seq 1 10); do touch 10_files/$i || return 1; done &&
|
||||
for i in $(test_seq 1 100); do touch 100_files/$i || return 1; done &&
|
||||
for i in $(test_seq 1 1000); do touch 1000_files/$i || return 1; done &&
|
||||
for i in $(test_seq 1 10000); do touch 10000_files/$i || return 1; done &&
|
||||
: 1_file directory should be left empty &&
|
||||
touch_files 10 &&
|
||||
touch_files 100 &&
|
||||
touch_files 1000 &&
|
||||
touch_files 10000 &&
|
||||
git add 1_file 10_files 100_files 1000_files 10000_files &&
|
||||
git commit -qm "Add files" &&
|
||||
|
||||
|
|
@ -133,7 +141,7 @@ test_expect_success "one time repo setup" '
|
|||
fi
|
||||
'
|
||||
|
||||
setup_for_fsmonitor() {
|
||||
setup_for_fsmonitor_hook () {
|
||||
# set INTEGRATION_SCRIPT depending on the environment
|
||||
if test -n "$INTEGRATION_PATH"
|
||||
then
|
||||
|
|
@ -173,8 +181,12 @@ test_perf_w_drop_caches () {
|
|||
test_perf "$@"
|
||||
}
|
||||
|
||||
test_fsmonitor_suite() {
|
||||
if test -n "$INTEGRATION_SCRIPT"; then
|
||||
test_fsmonitor_suite () {
|
||||
if test -n "$USE_FSMONITOR_DAEMON"
|
||||
then
|
||||
DESC="builtin fsmonitor--daemon"
|
||||
elif test -n "$INTEGRATION_SCRIPT"
|
||||
then
|
||||
DESC="fsmonitor=$(basename $INTEGRATION_SCRIPT)"
|
||||
else
|
||||
DESC="fsmonitor=disabled"
|
||||
|
|
@ -199,15 +211,15 @@ test_fsmonitor_suite() {
|
|||
|
||||
# Update the mtimes on upto 100k files to make status think
|
||||
# that they are dirty. For simplicity, omit any files with
|
||||
# LFs (i.e. anything that ls-files thinks it needs to dquote).
|
||||
# Then fully backslash-quote the paths to capture any
|
||||
# whitespace so that they pass thru xargs properly.
|
||||
# LFs (i.e. anything that ls-files thinks it needs to dquote)
|
||||
# and any files with whitespace so that they pass thru xargs
|
||||
# properly.
|
||||
#
|
||||
test_perf_w_drop_caches "status (dirty) ($DESC)" '
|
||||
git ls-files | \
|
||||
head -100000 | \
|
||||
grep -v \" | \
|
||||
sed '\''s/\(.\)/\\\1/g'\'' | \
|
||||
grep -v " ." | \
|
||||
xargs test-tool chmtime -300 &&
|
||||
git status
|
||||
'
|
||||
|
|
@ -253,11 +265,11 @@ test_fsmonitor_suite() {
|
|||
trace_start fsmonitor-watchman
|
||||
if test -n "$GIT_PERF_7519_FSMONITOR"; then
|
||||
for INTEGRATION_PATH in $GIT_PERF_7519_FSMONITOR; do
|
||||
test_expect_success "setup for fsmonitor $INTEGRATION_PATH" 'setup_for_fsmonitor'
|
||||
test_expect_success "setup for fsmonitor $INTEGRATION_PATH" 'setup_for_fsmonitor_hook'
|
||||
test_fsmonitor_suite
|
||||
done
|
||||
else
|
||||
test_expect_success "setup for fsmonitor" 'setup_for_fsmonitor'
|
||||
test_expect_success "setup for fsmonitor hook" 'setup_for_fsmonitor_hook'
|
||||
test_fsmonitor_suite
|
||||
fi
|
||||
|
||||
|
|
@ -285,4 +297,30 @@ test_expect_success "setup without fsmonitor" '
|
|||
test_fsmonitor_suite
|
||||
trace_stop
|
||||
|
||||
#
|
||||
# Run a full set of perf tests using the built-in fsmonitor--daemon.
|
||||
# It does not use the Hook API, so it has a different setup.
|
||||
# Explicitly start the daemon here and before we start client commands
|
||||
# so that we can later add custom tracing.
|
||||
#
|
||||
if test_have_prereq FSMONITOR_DAEMON
|
||||
then
|
||||
USE_FSMONITOR_DAEMON=t
|
||||
|
||||
test_expect_success "setup for builtin fsmonitor" '
|
||||
trace_start fsmonitor--daemon--server &&
|
||||
git fsmonitor--daemon start &&
|
||||
|
||||
trace_start fsmonitor--daemon--client &&
|
||||
|
||||
git config core.fsmonitor true &&
|
||||
git update-index --fsmonitor
|
||||
'
|
||||
|
||||
test_fsmonitor_suite
|
||||
|
||||
git fsmonitor--daemon stop
|
||||
trace_stop
|
||||
fi
|
||||
|
||||
test_done
|
||||
|
|
|
|||
|
|
@ -78,7 +78,7 @@ test_perf_copy_repo_contents () {
|
|||
for stuff in "$1"/*
|
||||
do
|
||||
case "$stuff" in
|
||||
*/objects|*/hooks|*/config|*/commondir|*/gitdir|*/worktrees)
|
||||
*/objects|*/hooks|*/config|*/commondir|*/gitdir|*/worktrees|*/fsmonitor--daemon*)
|
||||
;;
|
||||
*)
|
||||
cp -R "$stuff" "$repo/.git/" || exit 1
|
||||
|
|
|
|||
|
|
@ -0,0 +1,609 @@
|
|||
#!/bin/sh
|
||||
|
||||
test_description='built-in file system watcher'
|
||||
|
||||
. ./test-lib.sh
|
||||
|
||||
if ! test_have_prereq FSMONITOR_DAEMON
|
||||
then
|
||||
skip_all="fsmonitor--daemon is not supported on this platform"
|
||||
test_done
|
||||
fi
|
||||
|
||||
stop_daemon_delete_repo () {
|
||||
r=$1 &&
|
||||
test_might_fail git -C $r fsmonitor--daemon stop &&
|
||||
rm -rf $1
|
||||
}
|
||||
|
||||
start_daemon () {
|
||||
r= tf= t2= tk= &&
|
||||
|
||||
while test "$#" -ne 0
|
||||
do
|
||||
case "$1" in
|
||||
-C)
|
||||
r="-C ${2?}"
|
||||
shift
|
||||
;;
|
||||
--tf)
|
||||
tf="${2?}"
|
||||
shift
|
||||
;;
|
||||
--t2)
|
||||
t2="${2?}"
|
||||
shift
|
||||
;;
|
||||
--tk)
|
||||
tk="${2?}"
|
||||
shift
|
||||
;;
|
||||
-*)
|
||||
BUG "error: unknown option: '$1'"
|
||||
;;
|
||||
*)
|
||||
BUG "error: unbound argument: '$1'"
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done &&
|
||||
|
||||
(
|
||||
if test -n "$tf"
|
||||
then
|
||||
GIT_TRACE_FSMONITOR="$tf"
|
||||
export GIT_TRACE_FSMONITOR
|
||||
fi &&
|
||||
|
||||
if test -n "$t2"
|
||||
then
|
||||
GIT_TRACE2_PERF="$t2"
|
||||
export GIT_TRACE2_PERF
|
||||
fi &&
|
||||
|
||||
if test -n "$tk"
|
||||
then
|
||||
GIT_TEST_FSMONITOR_TOKEN="$tk"
|
||||
export GIT_TEST_FSMONITOR_TOKEN
|
||||
fi &&
|
||||
|
||||
git $r fsmonitor--daemon start &&
|
||||
git $r fsmonitor--daemon status
|
||||
)
|
||||
}
|
||||
|
||||
# Is a Trace2 data event present with the given catetory and key?
|
||||
# We do not care what the value is.
|
||||
#
|
||||
have_t2_data_event () {
|
||||
c=$1 &&
|
||||
k=$2 &&
|
||||
|
||||
grep -e '"event":"data".*"category":"'"$c"'".*"key":"'"$k"'"'
|
||||
}
|
||||
|
||||
test_expect_success 'explicit daemon start and stop' '
|
||||
test_when_finished "stop_daemon_delete_repo test_explicit" &&
|
||||
|
||||
git init test_explicit &&
|
||||
start_daemon -C test_explicit &&
|
||||
|
||||
git -C test_explicit fsmonitor--daemon stop &&
|
||||
test_must_fail git -C test_explicit fsmonitor--daemon status
|
||||
'
|
||||
|
||||
test_expect_success 'implicit daemon start' '
|
||||
test_when_finished "stop_daemon_delete_repo test_implicit" &&
|
||||
|
||||
git init test_implicit &&
|
||||
test_must_fail git -C test_implicit fsmonitor--daemon status &&
|
||||
|
||||
# query will implicitly start the daemon.
|
||||
#
|
||||
# for test-script simplicity, we send a V1 timestamp rather than
|
||||
# a V2 token. either way, the daemon response to any query contains
|
||||
# a new V2 token. (the daemon may complain that we sent a V1 request,
|
||||
# but this test case is only concerned with whether the daemon was
|
||||
# implicitly started.)
|
||||
|
||||
GIT_TRACE2_EVENT="$PWD/.git/trace" \
|
||||
test-tool -C test_implicit fsmonitor-client query --token 0 >actual &&
|
||||
nul_to_q <actual >actual.filtered &&
|
||||
grep "builtin:" actual.filtered &&
|
||||
|
||||
# confirm that a daemon was started in the background.
|
||||
#
|
||||
# since the mechanism for starting the background daemon is platform
|
||||
# dependent, just confirm that the foreground command received a
|
||||
# response from the daemon.
|
||||
|
||||
have_t2_data_event fsm_client query/response-length <.git/trace &&
|
||||
|
||||
git -C test_implicit fsmonitor--daemon status &&
|
||||
git -C test_implicit fsmonitor--daemon stop &&
|
||||
test_must_fail git -C test_implicit fsmonitor--daemon status
|
||||
'
|
||||
|
||||
test_expect_success 'implicit daemon stop (delete .git)' '
|
||||
test_when_finished "stop_daemon_delete_repo test_implicit_1" &&
|
||||
|
||||
git init test_implicit_1 &&
|
||||
|
||||
start_daemon -C test_implicit_1 &&
|
||||
|
||||
# deleting the .git directory will implicitly stop the daemon.
|
||||
rm -rf test_implicit_1/.git &&
|
||||
|
||||
# [1] Create an empty .git directory so that the following Git
|
||||
# command will stay relative to the `-C` directory.
|
||||
#
|
||||
# Without this, the Git command will override the requested
|
||||
# -C argument and crawl out to the containing Git source tree.
|
||||
# This would make the test result dependent upon whether we
|
||||
# were using fsmonitor on our development worktree.
|
||||
#
|
||||
sleep 1 &&
|
||||
mkdir test_implicit_1/.git &&
|
||||
|
||||
test_must_fail git -C test_implicit_1 fsmonitor--daemon status
|
||||
'
|
||||
|
||||
test_expect_success 'implicit daemon stop (rename .git)' '
|
||||
test_when_finished "stop_daemon_delete_repo test_implicit_2" &&
|
||||
|
||||
git init test_implicit_2 &&
|
||||
|
||||
start_daemon -C test_implicit_2 &&
|
||||
|
||||
# renaming the .git directory will implicitly stop the daemon.
|
||||
mv test_implicit_2/.git test_implicit_2/.xxx &&
|
||||
|
||||
# See [1] above.
|
||||
#
|
||||
sleep 1 &&
|
||||
mkdir test_implicit_2/.git &&
|
||||
|
||||
test_must_fail git -C test_implicit_2 fsmonitor--daemon status
|
||||
'
|
||||
|
||||
test_expect_success 'cannot start multiple daemons' '
|
||||
test_when_finished "stop_daemon_delete_repo test_multiple" &&
|
||||
|
||||
git init test_multiple &&
|
||||
|
||||
start_daemon -C test_multiple &&
|
||||
|
||||
test_must_fail git -C test_multiple fsmonitor--daemon start 2>actual &&
|
||||
grep "fsmonitor--daemon is already running" actual &&
|
||||
|
||||
git -C test_multiple fsmonitor--daemon stop &&
|
||||
test_must_fail git -C test_multiple fsmonitor--daemon status
|
||||
'
|
||||
|
||||
# These tests use the main repo in the trash directory
|
||||
|
||||
test_expect_success 'setup' '
|
||||
>tracked &&
|
||||
>modified &&
|
||||
>delete &&
|
||||
>rename &&
|
||||
mkdir dir1 &&
|
||||
>dir1/tracked &&
|
||||
>dir1/modified &&
|
||||
>dir1/delete &&
|
||||
>dir1/rename &&
|
||||
mkdir dir2 &&
|
||||
>dir2/tracked &&
|
||||
>dir2/modified &&
|
||||
>dir2/delete &&
|
||||
>dir2/rename &&
|
||||
mkdir dirtorename &&
|
||||
>dirtorename/a &&
|
||||
>dirtorename/b &&
|
||||
|
||||
cat >.gitignore <<-\EOF &&
|
||||
.gitignore
|
||||
expect*
|
||||
actual*
|
||||
flush*
|
||||
trace*
|
||||
EOF
|
||||
|
||||
git -c core.fsmonitor=false add . &&
|
||||
test_tick &&
|
||||
git -c core.fsmonitor=false commit -m initial &&
|
||||
|
||||
git config core.fsmonitor true
|
||||
'
|
||||
|
||||
# The test already explicitly stopped (or tried to stop) the daemon.
|
||||
# This is here in case something else fails first.
|
||||
#
|
||||
redundant_stop_daemon () {
|
||||
test_might_fail git fsmonitor--daemon stop
|
||||
}
|
||||
|
||||
test_expect_success 'update-index implicitly starts daemon' '
|
||||
test_when_finished redundant_stop_daemon &&
|
||||
|
||||
test_must_fail git fsmonitor--daemon status &&
|
||||
|
||||
GIT_TRACE2_EVENT="$PWD/.git/trace_implicit_1" \
|
||||
git update-index --fsmonitor &&
|
||||
|
||||
git fsmonitor--daemon status &&
|
||||
test_might_fail git fsmonitor--daemon stop &&
|
||||
|
||||
# Confirm that the trace2 log contains a record of the
|
||||
# daemon starting.
|
||||
test_subcommand git fsmonitor--daemon start <.git/trace_implicit_1
|
||||
'
|
||||
|
||||
test_expect_success 'status implicitly starts daemon' '
|
||||
test_when_finished redundant_stop_daemon &&
|
||||
|
||||
test_must_fail git fsmonitor--daemon status &&
|
||||
|
||||
GIT_TRACE2_EVENT="$PWD/.git/trace_implicit_2" \
|
||||
git status >actual &&
|
||||
|
||||
git fsmonitor--daemon status &&
|
||||
test_might_fail git fsmonitor--daemon stop &&
|
||||
|
||||
# Confirm that the trace2 log contains a record of the
|
||||
# daemon starting.
|
||||
test_subcommand git fsmonitor--daemon start <.git/trace_implicit_2
|
||||
'
|
||||
|
||||
edit_files () {
|
||||
echo 1 >modified &&
|
||||
echo 2 >dir1/modified &&
|
||||
echo 3 >dir2/modified &&
|
||||
>dir1/untracked
|
||||
}
|
||||
|
||||
delete_files () {
|
||||
rm -f delete &&
|
||||
rm -f dir1/delete &&
|
||||
rm -f dir2/delete
|
||||
}
|
||||
|
||||
create_files () {
|
||||
echo 1 >new &&
|
||||
echo 2 >dir1/new &&
|
||||
echo 3 >dir2/new
|
||||
}
|
||||
|
||||
rename_files () {
|
||||
mv rename renamed &&
|
||||
mv dir1/rename dir1/renamed &&
|
||||
mv dir2/rename dir2/renamed
|
||||
}
|
||||
|
||||
file_to_directory () {
|
||||
rm -f delete &&
|
||||
mkdir delete &&
|
||||
echo 1 >delete/new
|
||||
}
|
||||
|
||||
directory_to_file () {
|
||||
rm -rf dir1 &&
|
||||
echo 1 >dir1
|
||||
}
|
||||
|
||||
# The next few test cases confirm that our fsmonitor daemon sees each type
|
||||
# of OS filesystem notification that we care about. At this layer we just
|
||||
# ensure we are getting the OS notifications and do not try to confirm what
|
||||
# is reported by `git status`.
|
||||
#
|
||||
# We run a simple query after modifying the filesystem just to introduce
|
||||
# a bit of a delay so that the trace logging from the daemon has time to
|
||||
# get flushed to disk.
|
||||
#
|
||||
# We `reset` and `clean` at the bottom of each test (and before stopping the
|
||||
# daemon) because these commands might implicitly restart the daemon.
|
||||
|
||||
clean_up_repo_and_stop_daemon () {
|
||||
git reset --hard HEAD &&
|
||||
git clean -fd &&
|
||||
test_might_fail git fsmonitor--daemon stop &&
|
||||
rm -f .git/trace
|
||||
}
|
||||
|
||||
test_expect_success 'edit some files' '
|
||||
test_when_finished clean_up_repo_and_stop_daemon &&
|
||||
|
||||
start_daemon --tf "$PWD/.git/trace" &&
|
||||
|
||||
edit_files &&
|
||||
|
||||
test-tool fsmonitor-client query --token 0 &&
|
||||
|
||||
grep "^event: dir1/modified$" .git/trace &&
|
||||
grep "^event: dir2/modified$" .git/trace &&
|
||||
grep "^event: modified$" .git/trace &&
|
||||
grep "^event: dir1/untracked$" .git/trace
|
||||
'
|
||||
|
||||
test_expect_success 'create some files' '
|
||||
test_when_finished clean_up_repo_and_stop_daemon &&
|
||||
|
||||
start_daemon --tf "$PWD/.git/trace" &&
|
||||
|
||||
create_files &&
|
||||
|
||||
test-tool fsmonitor-client query --token 0 &&
|
||||
|
||||
grep "^event: dir1/new$" .git/trace &&
|
||||
grep "^event: dir2/new$" .git/trace &&
|
||||
grep "^event: new$" .git/trace
|
||||
'
|
||||
|
||||
test_expect_success 'delete some files' '
|
||||
test_when_finished clean_up_repo_and_stop_daemon &&
|
||||
|
||||
start_daemon --tf "$PWD/.git/trace" &&
|
||||
|
||||
delete_files &&
|
||||
|
||||
test-tool fsmonitor-client query --token 0 &&
|
||||
|
||||
grep "^event: dir1/delete$" .git/trace &&
|
||||
grep "^event: dir2/delete$" .git/trace &&
|
||||
grep "^event: delete$" .git/trace
|
||||
'
|
||||
|
||||
test_expect_success 'rename some files' '
|
||||
test_when_finished clean_up_repo_and_stop_daemon &&
|
||||
|
||||
start_daemon --tf "$PWD/.git/trace" &&
|
||||
|
||||
rename_files &&
|
||||
|
||||
test-tool fsmonitor-client query --token 0 &&
|
||||
|
||||
grep "^event: dir1/rename$" .git/trace &&
|
||||
grep "^event: dir2/rename$" .git/trace &&
|
||||
grep "^event: rename$" .git/trace &&
|
||||
grep "^event: dir1/renamed$" .git/trace &&
|
||||
grep "^event: dir2/renamed$" .git/trace &&
|
||||
grep "^event: renamed$" .git/trace
|
||||
'
|
||||
|
||||
test_expect_success 'rename directory' '
|
||||
test_when_finished clean_up_repo_and_stop_daemon &&
|
||||
|
||||
start_daemon --tf "$PWD/.git/trace" &&
|
||||
|
||||
mv dirtorename dirrenamed &&
|
||||
|
||||
test-tool fsmonitor-client query --token 0 &&
|
||||
|
||||
grep "^event: dirtorename/*$" .git/trace &&
|
||||
grep "^event: dirrenamed/*$" .git/trace
|
||||
'
|
||||
|
||||
test_expect_success 'file changes to directory' '
|
||||
test_when_finished clean_up_repo_and_stop_daemon &&
|
||||
|
||||
start_daemon --tf "$PWD/.git/trace" &&
|
||||
|
||||
file_to_directory &&
|
||||
|
||||
test-tool fsmonitor-client query --token 0 &&
|
||||
|
||||
grep "^event: delete$" .git/trace &&
|
||||
grep "^event: delete/new$" .git/trace
|
||||
'
|
||||
|
||||
test_expect_success 'directory changes to a file' '
|
||||
test_when_finished clean_up_repo_and_stop_daemon &&
|
||||
|
||||
start_daemon --tf "$PWD/.git/trace" &&
|
||||
|
||||
directory_to_file &&
|
||||
|
||||
test-tool fsmonitor-client query --token 0 &&
|
||||
|
||||
grep "^event: dir1$" .git/trace
|
||||
'
|
||||
|
||||
# The next few test cases exercise the token-resync code. When filesystem
|
||||
# drops events (because of filesystem velocity or because the daemon isn't
|
||||
# polling fast enough), we need to discard the cached data (relative to the
|
||||
# current token) and start collecting events under a new token.
|
||||
#
|
||||
# the 'test-tool fsmonitor-client flush' command can be used to send a
|
||||
# "flush" message to a running daemon and ask it to do a flush/resync.
|
||||
|
||||
test_expect_success 'flush cached data' '
|
||||
test_when_finished "stop_daemon_delete_repo test_flush" &&
|
||||
|
||||
git init test_flush &&
|
||||
|
||||
start_daemon -C test_flush --tf "$PWD/.git/trace_daemon" --tk true &&
|
||||
|
||||
# The daemon should have an initial token with no events in _0 and
|
||||
# then a few (probably platform-specific number of) events in _1.
|
||||
# These should both have the same <token_id>.
|
||||
|
||||
test-tool -C test_flush fsmonitor-client query --token "builtin:test_00000001:0" >actual_0 &&
|
||||
nul_to_q <actual_0 >actual_q0 &&
|
||||
|
||||
>test_flush/file_1 &&
|
||||
>test_flush/file_2 &&
|
||||
|
||||
test-tool -C test_flush fsmonitor-client query --token "builtin:test_00000001:0" >actual_1 &&
|
||||
nul_to_q <actual_1 >actual_q1 &&
|
||||
|
||||
grep "file_1" actual_q1 &&
|
||||
|
||||
# Force a flush. This will change the <token_id>, reset the <seq_nr>, and
|
||||
# flush the file data. Then create some events and ensure that the file
|
||||
# again appears in the cache. It should have the new <token_id>.
|
||||
|
||||
test-tool -C test_flush fsmonitor-client flush >flush_0 &&
|
||||
nul_to_q <flush_0 >flush_q0 &&
|
||||
grep "^builtin:test_00000002:0Q/Q$" flush_q0 &&
|
||||
|
||||
test-tool -C test_flush fsmonitor-client query --token "builtin:test_00000002:0" >actual_2 &&
|
||||
nul_to_q <actual_2 >actual_q2 &&
|
||||
|
||||
grep "^builtin:test_00000002:0Q$" actual_q2 &&
|
||||
|
||||
>test_flush/file_3 &&
|
||||
|
||||
test-tool -C test_flush fsmonitor-client query --token "builtin:test_00000002:0" >actual_3 &&
|
||||
nul_to_q <actual_3 >actual_q3 &&
|
||||
|
||||
grep "file_3" actual_q3
|
||||
'
|
||||
|
||||
# The next few test cases create repos where the .git directory is NOT
|
||||
# inside the one of the working directory. That is, where .git is a file
|
||||
# that points to a directory elsewhere. This happens for submodules and
|
||||
# non-primary worktrees.
|
||||
|
||||
test_expect_success 'setup worktree base' '
|
||||
git init wt-base &&
|
||||
echo 1 >wt-base/file1 &&
|
||||
git -C wt-base add file1 &&
|
||||
git -C wt-base commit -m "c1"
|
||||
'
|
||||
|
||||
test_expect_success 'worktree with .git file' '
|
||||
git -C wt-base worktree add ../wt-secondary &&
|
||||
|
||||
start_daemon -C wt-secondary \
|
||||
--tf "$PWD/trace_wt_secondary" \
|
||||
--t2 "$PWD/trace2_wt_secondary" &&
|
||||
|
||||
git -C wt-secondary fsmonitor--daemon stop &&
|
||||
test_must_fail git -C wt-secondary fsmonitor--daemon status
|
||||
'
|
||||
|
||||
# NEEDSWORK: Repeat one of the "edit" tests on wt-secondary and
|
||||
# confirm that we get the same events and behavior -- that is, that
|
||||
# fsmonitor--daemon correctly watches BOTH the working directory and
|
||||
# the external GITDIR directory and behaves the same as when ".git"
|
||||
# is a directory inside the working directory.
|
||||
|
||||
test_expect_success 'cleanup worktrees' '
|
||||
stop_daemon_delete_repo wt-secondary &&
|
||||
stop_daemon_delete_repo wt-base
|
||||
'
|
||||
|
||||
# The next few tests perform arbitrary/contrived file operations and
|
||||
# confirm that status is correct. That is, that the data (or lack of
|
||||
# data) from fsmonitor doesn't cause incorrect results. And doesn't
|
||||
# cause incorrect results when the untracked-cache is enabled.
|
||||
|
||||
test_lazy_prereq UNTRACKED_CACHE '
|
||||
git update-index --test-untracked-cache
|
||||
'
|
||||
|
||||
test_expect_success 'Matrix: setup for untracked-cache,fsmonitor matrix' '
|
||||
test_unconfig core.fsmonitor &&
|
||||
git update-index --no-fsmonitor &&
|
||||
test_might_fail git fsmonitor--daemon stop
|
||||
'
|
||||
|
||||
matrix_clean_up_repo () {
|
||||
git reset --hard HEAD &&
|
||||
git clean -fd
|
||||
}
|
||||
|
||||
matrix_try () {
|
||||
uc=$1 &&
|
||||
fsm=$2 &&
|
||||
fn=$3 &&
|
||||
|
||||
if test $uc = true && test $fsm = false
|
||||
then
|
||||
# The untracked-cache is buggy when FSMonitor is
|
||||
# DISABLED, so skip the tests for this matrix
|
||||
# combination.
|
||||
#
|
||||
# We've observed random, occasional test failures on
|
||||
# Windows and MacOS when the UC is turned on and FSM
|
||||
# is turned off. These are rare, but they do happen
|
||||
# indicating that it is probably a race condition within
|
||||
# the untracked cache itself.
|
||||
#
|
||||
# It usually happens when a test does F/D trickery and
|
||||
# then the NEXT test fails because of extra status
|
||||
# output from stale UC data from the previous test.
|
||||
#
|
||||
# Since FSMonitor is not involved in the error, skip
|
||||
# the tests for this matrix combination.
|
||||
#
|
||||
return 0
|
||||
fi &&
|
||||
|
||||
test_expect_success "Matrix[uc:$uc][fsm:$fsm] $fn" '
|
||||
matrix_clean_up_repo &&
|
||||
$fn &&
|
||||
if test $uc = false && test $fsm = false
|
||||
then
|
||||
git status --porcelain=v1 >.git/expect.$fn
|
||||
else
|
||||
git status --porcelain=v1 >.git/actual.$fn &&
|
||||
test_cmp .git/expect.$fn .git/actual.$fn
|
||||
fi
|
||||
'
|
||||
}
|
||||
|
||||
uc_values="false"
|
||||
test_have_prereq UNTRACKED_CACHE && uc_values="false true"
|
||||
for uc_val in $uc_values
|
||||
do
|
||||
if test $uc_val = false
|
||||
then
|
||||
test_expect_success "Matrix[uc:$uc_val] disable untracked cache" '
|
||||
git config core.untrackedcache false &&
|
||||
git update-index --no-untracked-cache
|
||||
'
|
||||
else
|
||||
test_expect_success "Matrix[uc:$uc_val] enable untracked cache" '
|
||||
git config core.untrackedcache true &&
|
||||
git update-index --untracked-cache
|
||||
'
|
||||
fi
|
||||
|
||||
fsm_values="false true"
|
||||
for fsm_val in $fsm_values
|
||||
do
|
||||
if test $fsm_val = false
|
||||
then
|
||||
test_expect_success "Matrix[uc:$uc_val][fsm:$fsm_val] disable fsmonitor" '
|
||||
test_unconfig core.fsmonitor &&
|
||||
git update-index --no-fsmonitor &&
|
||||
test_might_fail git fsmonitor--daemon stop
|
||||
'
|
||||
else
|
||||
test_expect_success "Matrix[uc:$uc_val][fsm:$fsm_val] enable fsmonitor" '
|
||||
git config core.fsmonitor true &&
|
||||
git fsmonitor--daemon start &&
|
||||
git update-index --fsmonitor
|
||||
'
|
||||
fi
|
||||
|
||||
matrix_try $uc_val $fsm_val edit_files
|
||||
matrix_try $uc_val $fsm_val delete_files
|
||||
matrix_try $uc_val $fsm_val create_files
|
||||
matrix_try $uc_val $fsm_val rename_files
|
||||
matrix_try $uc_val $fsm_val file_to_directory
|
||||
matrix_try $uc_val $fsm_val directory_to_file
|
||||
|
||||
if test $fsm_val = true
|
||||
then
|
||||
test_expect_success "Matrix[uc:$uc_val][fsm:$fsm_val] disable fsmonitor at end" '
|
||||
test_unconfig core.fsmonitor &&
|
||||
git update-index --no-fsmonitor &&
|
||||
test_might_fail git fsmonitor--daemon stop
|
||||
'
|
||||
fi
|
||||
done
|
||||
done
|
||||
|
||||
test_done
|
||||
|
|
@ -1851,3 +1851,10 @@ test_lazy_prereq SHA1 '
|
|||
# Tests that verify the scheduler integration must set this locally
|
||||
# to avoid errors.
|
||||
GIT_TEST_MAINT_SCHEDULER="none:exit 1"
|
||||
|
||||
# Does this platform support `git fsmonitor--daemon`
|
||||
#
|
||||
test_lazy_prereq FSMONITOR_DAEMON '
|
||||
git version --build-options >output &&
|
||||
grep "feature: fsmonitor--daemon" output
|
||||
'
|
||||
|
|
|
|||
Loading…
Reference in New Issue