Most of the submodule operations work on a set of submodules.
Calculating and using this set is usually done via:
module_list "$@" | {
while read mode sha1 stage sm_path
do
# the actual operation
done
}
Currently the function `module_list` is implemented in the
git-submodule.sh as a shell script wrapping a perl script.
The rewrite is in C, such that it is faster and can later be
easily adapted when other functions are rewritten in C.
git-submodule.sh, similar to the builtin commands, will navigate
to the top-most directory of the repository and keep the
subdirectory as a variable. As the helper is called from
within the git-submodule.sh script, we are already navigated
to the root level, but the path arguments are still relative
to the subdirectory we were in when calling git-submodule.sh.
That's why there is a `--prefix` option pointing to an alternative
path which to anchor relative path arguments.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* hv/submodule-config:
submodule: allow erroneous values for the fetchRecurseSubmodules option
submodule: use new config API for worktree configurations
submodule: extract functions for config set and lookup
submodule: implement a config API for lookup of .gitmodules values
We should not die when reading the submodule config cache since the
user might not be able to get out of that situation when the
configuration is part of the history.
We should handle this condition later when the value is about to be
used.
Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We remove the extracted functions and directly parse into and read out
of the cache. This allows us to have one unified way of accessing
submodule configuration values specific to single submodules. Regardless
whether we need to access a configuration from history or from the
worktree.
Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is one step towards using the new configuration API. We just
extract these functions to make replacing the actual code easier.
Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a superproject some commands need to interact with submodules. They
need to query values from the .gitmodules file either from the worktree
of from certain revisions. At the moment this is quite hard since a
caller would need to read the .gitmodules file from the history and then
parse the values. We want to provide an API for this so we have one
place to get values from .gitmodules from any revision (including the
worktree).
The API is realized as a cache which allows us to lazily read
.gitmodules configurations by commit into a runtime cache which can then
be used to easily lookup values from it. Currently only the values for
path or name are stored but it can be extended for any value needed.
It is expected that .gitmodules files do not change often between
commits. Thats why we lookup the .gitmodules sha1 from a commit and then
either lookup an already parsed configuration or parse and cache an
unknown one for each sha1. The cache is lazily build on demand for each
requested commit.
This cache can be used for all purposes which need knowledge about
submodule configurations. Example use cases are:
* Recursive submodule checkout needs to lookup a submodule name from
its path when a submodule first appears. This needs be done before
this configuration exists in the worktree.
* The implementation of submodule support for 'git archive' needs to
lookup the submodule name to generate the archive when given a
revision that is not checked out.
* 'git fetch' when given the --recurse-submodules=on-demand option (or
configuration) needs to lookup submodule names by path from the
database rather than reading from the worktree. For new submodule it
needs to lookup the name from its path to allow cloning new
submodules into the .git folder so they can be checked out without
any network interaction when the user does a checkout of that
revision.
Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
One of the most common uses of git_path() is to pass a
constant, like git_path("MERGE_MSG"). This has two
drawbacks:
1. The return value is a static buffer, and the lifetime
is dependent on other calls to git_path, etc.
2. There's no compile-time checking of the pathname. This
is OK for a one-off (after all, we have to spell it
correctly at least once), but many of these constant
strings appear throughout the code.
This patch introduces a series of functions to "memoize"
these strings, which are essentially globals for the
lifetime of the program. We compute the value once, take
ownership of the buffer, and return the cached value for
subsequent calls. cache.h provides a helper macro for
defining these functions as one-liners, and defines a few
common ones for global use.
Using a macro is a little bit gross, but it does nicely
document the purpose of the functions. If we need to touch
them all later (e.g., because we learned how to change the
git_dir variable at runtime, and need to invalidate all of
the stored values), it will be much easier to have the
complete list.
Note that the shared-global functions have separate, manual
declarations. We could do something clever with the macros
(e.g., expand it to a declaration in some places, and a
declaration _and_ a definition in path.c). But there aren't
that many, and it's probably better to stay away from
too-magical macros.
Likewise, if we abandon the C preprocessor in favor of
generating these with a script, we could get much fancier.
E.g., normalizing "FOO/BAR-BAZ" into "git_path_foo_bar_baz".
But the small amount of saved typing is probably not worth
the resulting confusion to readers who want to grep for the
function's definition.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The get_repo_path function calls mkpath() and then does some
non-trivial operations on it, like calling
is_git_directory() and read_gitfile(). These are actually
OK (they do not use more pathname static buffers
themselves), but it takes a fair bit of work to verify.
Let's use our own strbuf to store the path, and we can
simply reuse it for each iteration of the loop (we can even
avoid rewriting the beginning part, since we are trying a
series of suffixes).
To make the strbuf cleanup easier, we split out a thin
wrapper. As a bonus, this wrapper can factor out the
canonicalization that happens in all of the early-return
code paths.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The find_hook function returns the results of git_path,
which is a static buffer shared by other path-related calls.
Returning such a buffer is slightly dangerous, because it
can be overwritten by seemingly unrelated functions.
Let's at least keep our _own_ static buffer, so you can
only get in trouble by calling find_hook in quick
succession, which is less likely to happen and more obvious
to notice.
While we're at it, let's add some documentation of the
function's limitations.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The first thing we do in this function is copy the input
into a strbuf. Of the 4 callers, 3 of them already have a
strbuf we could use. Let's just take the strbuf, and convert
the remaining caller to use a strbuf, rather than a raw
git_path. This is safer, anyway, as remove_dir_recursively
is a non-trivial function that might use the pathname
buffers itself (this is _probably_ OK, as the likely culprit
would be calling resolve_gitlink_ref, but we do not pass the
proper flags to ask it to avoid blowing away gitlinks).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Assigning the result of git_path is a bad pattern, because
it's not immediately obvious how long you expect the content
to stay valid (and it may be overwritten by subsequent
calls). Let's use a function-local strbuf here instead,
which we know is safe (we just have to remember to free it
in all code paths).
As a bonus, we get rid of a confusing variable-reuse
("ref_file" is used for two distinct purposes).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Because it's not safe to store the static-buffer results of
git_path for a long time, we end up formatting the same
filename over and over. We can fix this by using a
function-local strbuf to store the formatted pathname and
avoid repeating ourselves.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 1a83c24 (git_snpath(): retire and replace with
strbuf_git_path(), 2014-11-30) taught log_ref_setup and
log_ref_write_1 to take a strbuf parameter, rather than a
bare string. It then makes an alias to the strbuf's "buf"
field under the original name.
This made the original diff much shorter, but the resulting
code is more complicated that it needs to be. Since we've
aliased the pointer, we drop our reference to the strbuf to
ensure we don't accidentally change it. But if we simply
drop our alias and use "logfile.buf" directly, we do not
have to worry about this aliasing. It's a larger diff, but
the resulting code is simpler.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are no callers of the slightly-dangerous static-buffer
git_path_submodule left. Let's drop it.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In iterating over the loose refs in "refs/foo/", we keep a
running strbuf with "refs/foo/one", "refs/foo/two", etc. But
we also need to access these files in the filesystem, as
".git/refs/foo/one", etc. For this latter purpose, we make a
series of independent calls to git_path(). These are safe
(we only use the result to call stat()), but assigning the
result of git_path is a suspicious pattern that we'd rather
avoid.
This patch keeps a running buffer with ".git/refs/foo/", and
we can just append/reset each directory element as we loop.
This matches how we handle the refnames. It should also be
more efficient, as we do not keep formatting the same
".git/refs/foo" prefix (which can be arbitrarily deep).
Technically we are dropping a call to strbuf_cleanup() on
each generated filename, but that's OK; it wasn't doing
anything, as we are putting in single-level names we read
from the filesystem (so it could not possibly be cleaning up
cruft like "./" in this instance).
A clever reader may also note that the running refname
buffer ("refs/foo/") is actually a subset of the filesystem
path buffer (".git/refs/foo/"). We could get by with one
buffer, indexing the length of $GIT_DIR when we want the
refname. However, having tried this, the resulting code
actually ends up a little more confusing, and the efficiency
improvement is tiny (and almost certainly dwarfed by the
system calls we are making).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It's an anti-pattern to assign the result of git_path to a
variable, since other calls may reuse our buffer. In this
case, we feed the result to unlink_or_warn immediately
afterwards, so it's OK. However, it's nice to avoid
assignment entirely, which makes it more obvious that
there's no bug.
We can just pass the result directly to unlink_or_warn,
which is a known-simple function. As a bonus, the code flow
is a little more obvious, as we eliminate an extra
conditional (a reader does not have to wonder any more
"under which circumstances is 'path' set?").
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As with the previous commit to git_path, assigning the
result of mkpath is suspicious, since it is not clear
whether we will still depend on the value after it may have
been overwritten by subsequent calls. This patch converts
low-hanging fruit to use mkpathdup instead of mkpath (with
the downside that we must remember to free the result).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Because git_path uses a static buffer that is shared with
calls to git_path, mkpath, etc, it can be dangerous to
assign the result to a variable or pass it to a non-trivial
function. The value may change unexpectedly due to other
calls.
None of the cases changed here has a known bug, but they're
worth converting away from git_path because:
1. It's easy to use git_pathdup in these cases.
2. They use constructs (like assignment) that make it
hard to tell whether they're safe or not.
The extra malloc overhead should be trivial, as an
allocation should be an order of magnitude cheaper than a
system call (which we are clearly about to make, since we
are constructing a filename). The real cost is that we must
remember to free the result.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The add_to_alternates_file function blindly uses
hold_lock_file_for_append to copy the existing contents, and
then adds the new line to it. This has two minor problems:
1. We might add duplicate entries, which are ugly and
inefficient.
2. We do not check that the file ends with a newline, in
which case we would bogusly append to the final line.
This is quite unlikely in practice, though, as we call
this function only from git-clone, so presumably we are
the only writers of the file (and we always add a
newline).
Instead of using hold_lock_file_for_append, let's copy the
file line by line, which ensures all records are properly
terminated. If we see an extra line, we can simply abort the
update (there is no point in even copying the rest, as we
know that it would be identical to the original).
As a bonus, we also get rid of some calls to the
static-buffer mkpath and git_path functions.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The early part of this test is rather old, and does not
follow our usual style guidelines. In particular:
- the tests liberally chdir, and expect out-of-test "cd"
commands to return them to a sane state
- test commands aren't indented at all
- there are a lot of minor formatting nits, like the
opening quote of the test block on the wrong line,
spaces after ">", etc
This patch fixes the style issues, and uses a few helper
functions, along with subshells and "git -C", to avoid
changing the cwd of the main script.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The git_path function has "git_pathdup" and
"strbuf_git_path" variants, but git_submodule_path only
comes in the dangerous, static-buffer variant. That makes
refactoring callers to use the safer functions hard (since
they don't exist).
Since we're already using a strbuf behind the scenes, it's
easy to expose all three of these interfaces with thin
wrappers.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The comment above these functions actually describes
sha1_file_name, and comes from the very first revision of
git. Commit 723c31f (Add "git_path()" and "head_ref()"
helper functions., 2005-07-05) added git_path, pushing the
comment away from the function it describes; later commits
added more functions in this block.
Let's fix the comment to describe these related functions in
more detail. Let's also make sure to point out their safer
alternatives (and move those alternatives below, which makes
more sense when reading the file).
Note that we do not need to move the existing comment to
sha1_file_name. Commit d40d535 (sha1_file.c: document a
bunch of functions defined in the file, 2014-02-21) already
added a much more descriptive comment to it.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* es/doc-clean-outdated-tools:
Documentation/git-tools: retire manually-maintained list
Documentation/git-tools: drop references to defunct tools
Documentation/git-tools: fix item text formatting
Documentation/git-tools: improve discoverability of Git wiki
Documentation/git: drop outdated Cogito reference
Allow an asterisk as a substring (as opposed to the entirety) of
a path component for both side of a refspec, e.g.
"refs/heads/o*:refs/remotes/heads/i*".
* jk/refspec-parse-wildcard:
refs: loosen restriction on wildcard "*" refspecs
refs: cleanup comments regarding check_refname_component()
"git subtree" (in contrib/) depended on "git log" output to be
stable, which was a no-no. Apply a workaround to force a
particular date format.
* da/subtree-date-confusion:
contrib/subtree: ignore log.date configuration
An attempt to delete a ref by pushing into a repositorywhose HEAD
symbolic reference points at an unborn branch that cannot be
created due to ref D/F conflict (e.g. refs/heads/a/b exists, HEAD
points at refs/heads/a) failed.
* jx/do-not-crash-receive-pack-wo-head:
receive-pack: crash when checking with non-exist HEAD
The low-level "git send-pack" did not honor 'user.signingkey'
configuration variable when sending a signed-push.
* db/send-pack-user-signingkey:
builtin/send-pack.c: respect user.signingkey
In preparation for allowing different "backends" to store the refs
in a way different from the traditional "one ref per file in $GIT_DIR
or in a $GIT_DIR/packed-refs file" filesystem storage, reduce
direct filesystem access to ref-like things like CHERRY_PICK_HEAD
from scripts and programs.
* dt/refs-backend-preamble:
git-stash: use update-ref --create-reflog instead of creating files
update-ref and tag: add --create-reflog arg
refs: add REF_FORCE_CREATE_REFLOG flag
git-reflog: add exists command
refs: new public ref function: safe_create_reflog
refs: break out check for reflog autocreation
refs.c: add err arguments to reflog functions
"sparse checkout" misbehaved for a path that is excluded from the
checkout when switching between branches that differ at the path.
* as/sparse-checkout-removal:
unpack-trees: don't update files with CE_WT_REMOVE set
Teach "git log" and friends a new "--date=format:..." option to
format timestamps using system's strftime(3).
* jk/date-mode-format:
strbuf: make strbuf_addftime more robust
introduce "format" date-mode
convert "enum date_mode" into a struct
show-branch: use DATE_RELATIVE instead of magic number
* pt/am-tests:
t3901: test git-am encoding conversion
t3418: non-interactive rebase --continue with rerere enabled
t4150: tests for am --[no-]scissors
t4150: am with post-applypatch hook
t4150: am with pre-applypatch hook
t4150: am with applypatch-msg hook
t4150: am --resolved fails if index has unmerged entries
t4150: am --resolved fails if index has no changes
t4150: am refuses patches when paused
t4151: am --abort will keep dirty index intact
t4150: am fails if index is dirty
t4150: am.messageid really adds the message id
Optimize computation of untracked status indicator by bash prompt
script (in contrib/).
* sg/bash-prompt-untracked-optim:
bash prompt: faster untracked status indicator with untracked directories
bash prompt: test untracked files status indicator with untracked dirs
An experimental "untracked cache" feature used uname(2) in a
slightly unportable way.
* cb/uname-in-untracked:
untracked: fix detection of uname(2) failure
A "rebase" replays changes of the local branch on top of something
else, as such they are placed in stage #3 and referred to as
"theirs", while the changes in the new base, typically a foreign
work, are placed in stage #2 and referred to as "ours". Clarify
the "checkout --ours/--theirs".
* se/doc-checkout-ours-theirs:
checkout: document subtlety around --ours/--theirs
The "rev-parse --parseopt" mode parsed the option specification
and the argument hint in a strange way to allow '=' and other
special characters in the option name while forbidding them from
the argument hint. This made it impossible to define an option
like "--pair <key>=<value>" with "pair=key=value" specification,
which instead would have defined a "--pair=key <value>" option.
* ib/scripted-parse-opt-better-hint-string:
rev-parse --parseopt: allow [*=?!] in argument hints
Often a fast-import stream builds a new commit on top of the
previous commit it built, and it often unconditionally emits a
"from" command to specify the first parent, which can be omitted in
such a case. This caused fast-import to forget the tree of the
previous commit and then re-read it from scratch, which was
inefficient. Optimize for this common case.
* mh/fast-import-optimize-current-from:
fast-import: do less work when given "from" matches current branch head
"git fast-import" learned to respond to the get-mark command via
its cat-blob-fd interface.
* mh/fast-import-get-mark:
fast-import: add a get-mark command
Add "drop commit-object-name subject" command as another way to
skip replaying of a commit in "rebase -i", and then punish those
who do not use it (and instead just remove the lines) by throwing
a warning.
* gr/rebase-i-drop-warn:
git rebase -i: add static check for commands and SHA-1
git rebase -i: warn about removed commits
git-rebase -i: add command "drop" to remove a commit
Memory use reduction when commit-slab facility is used to annotate
sparsely (which is not recommended in the first place).
* jc/commit-slab:
commit-slab: introduce slabname##_peek() function
Add a new configuration variable to enable "--follow" automatically
when "git log" is run with one pathspec argument.
* dt/log-follow-config:
log: add "log.follow" configuration variable
Teach "git status" to show a more detailed information regarding
the "rebase -i" session in progress.
* gp/status-rebase-i-info:
status: add new tests for status during rebase -i
status: give more information during rebase -i
status: differentiate interactive from non-interactive rebases
status: factor two rebase-related messages together
"cat-file" learned "--batch-all-objects" option to enumerate all
available objects in the repository more quickly than "rev-list
--all --objects" (the output includes unreachable objects, though).
* jk/cat-file-batch-all:
cat-file: sort and de-dup output of --batch-all-objects
cat-file: add --batch-all-objects option
cat-file: split batch_one_object into two stages
cat-file: stop returning value from batch_one_object
cat-file: add --buffer option
cat-file: move batch_options definition to top of file
cat-file: minor style fix in options list
Allow ignoring fsck errors on specific set of known-to-be-bad
objects, and also tweaking warning level of various kinds of non
critical breakages reported.
* js/fsck-opt:
fsck: support ignoring objects in `git fsck` via fsck.skiplist
fsck: git receive-pack: support excluding objects from fsck'ing
fsck: introduce `git fsck --connectivity-only`
fsck: support demoting errors to warnings
fsck: document the new receive.fsck.<msg-id> options
fsck: allow upgrading fsck warnings to errors
fsck: optionally ignore specific fsck issues completely
fsck: disallow demoting grave fsck errors to warnings
fsck: add a simple test for receive.fsck.<msg-id>
fsck: make fsck_tag() warn-friendly
fsck: handle multiple authors in commits specially
fsck: make fsck_commit() warn-friendly
fsck: make fsck_ident() warn-friendly
fsck: report the ID of the error/warning
fsck (receive-pack): allow demoting errors to warnings
fsck: offer a function to demote fsck errors to warnings
fsck: provide a function to parse fsck message IDs
fsck: introduce identifiers for fsck messages
fsck: introduce fsck options
Clean up refs API and make "git clone" less intimate with the
implementation detail.
* mh/init-delete-refs-api:
delete_ref(): use the usual convention for old_sha1
cmd_update_ref(): make logic more straightforward
update_ref(): don't read old reference value before delete
check_branch_commit(): make first parameter const
refs.h: add some parameter names to function declarations
refs: move the remaining ref module declarations to refs.h
initial_ref_transaction_commit(): check for ref D/F conflicts
initial_ref_transaction_commit(): check for duplicate refs
refs: remove some functions from the module's public interface
initial_ref_transaction_commit(): function for initial ref creation
repack_without_refs(): make function private
prune_refs(): use delete_refs()
prune_remote(): use delete_refs()
delete_refs(): bail early if the packed-refs file cannot be rewritten
delete_refs(): make error message more generic
delete_refs(): new function for the refs API
delete_ref(): handle special case more explicitly
remove_branches(): remove temporary
delete_ref(): move declaration to refs.h
Reimplement 'git pull' in C.
* pt/pull-builtin:
pull: remove redirection to git-pull.sh
pull --rebase: error on no merge candidate cases
pull --rebase: exit early when the working directory is dirty
pull: configure --rebase via branch.<name>.rebase or pull.rebase
pull: teach git pull about --rebase
pull: set reflog message
pull: implement pulling into an unborn branch
pull: fast-forward working tree if head is updated
pull: check if in unresolved merge state
pull: support pull.ff config
pull: error on no merge candidates
pull: pass git-fetch's options to git-fetch
pull: pass git-merge's options to git-merge
pull: pass verbosity, --progress flags to fetch and merge
pull: implement fetch + merge
pull: implement skeletal builtin pull
argv-array: implement argv_array_pushv()
parse-options-cb: implement parse_opt_passthru_argv()
parse-options-cb: implement parse_opt_passthru()
Enhance packet tracing machinery to allow capturing an incoming
pack data to a file for debugging.
* jk/pkt-log-pack:
pkt-line: support tracing verbatim pack contents
pkt-line: tighten sideband PACK check when tracing
pkt-line: simplify starts_with checks in packet tracing
