PEP8 requires that binary operators such as assignment and comparison
operators should always be surrounded by a pair of single spaces, and
recommends that all other binary operators should typically be surround
by single spaces.
The recommendation is given here in the "Other Recommendations"
section
https://www.python.org/dev/peps/pep-0008/#other-recommendations
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
PEP8 recommends that every comment should begin with a single '#'
character.
This guideline is described here:
https://www.python.org/dev/peps/pep-0008/#comments
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
PEP8 makes no specific recommendation about spaces preceding colons in
dictionary declarations, but all the code examples contained with it
declare dictionaries with a single space after the colon, and none
before.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
PEP8 recommends that backslash line continuations should only be used
for line-breaks outside parentheses.
This recommendation is described in the "Maximum Line Length" section:
https://www.python.org/dev/peps/pep-0008/#maximum-line-length
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
PEP8 recommends that there should be no spaces before function arguments
in the in the "Pet Peeves" section:
https://www.python.org/dev/peps/pep-0008/#pet-peeves
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch improves consistency across git-p4 by ensuring all command
separated arguments to function invocations, tuples and lists are
separated by commas with a single space following.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In several places, git-p4 contains code of the form:
(a, b) = foo()
In each case, multiple values are returned through a tuple or a list and
bound into multiple values.
The brackets around the assigned variables are redundant and can be
removed:
a, b = foo()
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
PEP8 recommends that there should be no spaces around the = sign of
default argument values of functions.
This guideline is described here:
https://www.python.org/dev/peps/pep-0008/#other-recommendations
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
PEP8 discourages use of extraneous padding inside any parenthesis,
brackets or braces in the "Pet Peeves" section:
https://www.python.org/dev/peps/pep-0008/#pet-peeves
This patch removes all cases of these.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-p4 contains configuration commands for pylint embedded in the header
comment.
Previously, these were combined onto single lines and not alphabetically
sorted. This patch breaks each disable command onto a separate line to
give cleaner diffs, removed duplicate entries, and sorts the list
alphabetically.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Previously, the script contained commented code including Python 2 print
statements. Presumably, these were used as a developer aid at some point
in history. However, the commented code is generally undesirable, and
this commented code serves no useful purpose. Therefore this patch
removes it.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Previously, a small number of functions, methods and classes were
documented using comments. This patch improves consistency by converting
these into docstrings similar to those that already exist in the script.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch attempts to improve the consistency of the docstrings by
making the following changes:
- Rewraps all docstrings to a 79-character column limit.
- Adds a full stop at the end of every docstring.
- Removes any spaces after the opening triple-quotes of all
docstrings.
- Sets the hanging indent of multi-line docstrings to 3-spaces.
- Ensures that the closing triple-quotes of multi-line docstrings are
always on a new line indented by a 3-space indent.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
PEP8 recommends that all code should be indented in 4-space units. This
guideline is described here:
https://www.python.org/dev/peps/pep-0008/#indentation
Previously git-p4 had multiple cases where code was indented with a
non-multiple of 4-spaces. This patch fixes each of these.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Python allows the usage of compound statements where multiple statements
are written on a single line separared by semicolons. It is also
possible to add a semicolon after a single statement, however this is
generally considered to be untidy, and is unnecessary.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In the PEP8 style guidelines, top-level functions and class definitions
should be separated by two blank lines. Methods should be surrounded by
a single blank line.
This guideline is described here in the "Blank Lines" section:
https://www.python.org/dev/peps/pep-0008/#blank-lines
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
More "config-based hooks".
* ab/config-based-hooks-2:
run-command: remove old run_hook_{le,ve}() hook API
receive-pack: convert push-to-checkout hook to hook.h
read-cache: convert post-index-change to use hook.h
commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
git-p4: use 'git hook' to run hooks
send-email: use 'git hook run' for 'sendemail-validate'
git hook run: add an --ignore-missing flag
hooks: convert worktree 'post-checkout' hook to hook library
hooks: convert non-worktree 'post-checkout' hook to hook library
merge: convert post-merge to use hook.h
am: convert applypatch-msg to use hook.h
rebase: convert pre-rebase to use hook.h
hook API: add a run_hooks_l() wrapper
am: convert {pre,post}-applypatch to use hook.h
gc: use hook library for pre-auto-gc hook
hook API: add a run_hooks() wrapper
hook: add 'run' subcommand
* jh/p4-spawning-external-commands-cleanup:
git-p4: don't print shell commands as python lists
git-p4: pass command arguments as lists instead of using shell
git-p4: don't select shell mode using the type of the command argument
The way "git p4" shows file sizes in its output has been updated to
use human-readable units.
* jh/p4-human-unit-numbers:
git-p4: show progress as an integer
git-p4: print size values in appropriate units
Instead of duplicating the behavior of run-command.h:run_hook_le() in
Python, we can directly call 'git hook run'. We emulate the existence
check with the --ignore-missing flag.
We're dropping the "verbose" handling added in 9f59ca4d6a (git-p4:
create new function run_git_hook, 2020-02-11), those who want
diagnostic output about how hooks are run are now able to get that via
e.g. the trace2 facility and GIT_TRACE=1.
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Acked-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
CalledProcessError is an exception class from the subprocess namespace.
When raising this exception, git-p4 would instantiate CalledProcessError
objects without properly referencing the subprocess namespace causing
the script to fail.
Resolves the issue by replacing CalledProcessError with
subprocess.CalledProcessError.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Previously the git-p4 script would log commands as stringified
representations of the command parameter, leading to output such as
this:
Reading pipe: ['git', 'config', '--bool', 'git-p4.useclientspec']
Now that all commands are list objects, this patch instead joins the
elements of the list into a single string so the output now looks more
readable:
Reading pipe: git config --bool git-p4.useclientspec
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In the majority of the subprocess calls where shell=True was used, it
was only needed to parse command arguments by spaces. In each of these
cases, the commands are now being passed in as lists instead of strings.
This change aids the comprehensibility of the code. Constucting commands
and arguments using strings risks bugs from unsanitized inputs, and the
attendant complexity of properly quoting and escaping command arguments.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Previously, the script would invoke subprocess functions setting the
shell argument True if the command argument was a string, setting it
False otherwise.
This patch replaces this implicit type-driven behaviour with explicit
shell arguments specified by the caller.
The apparent motive for the implict behaviour is that the subprocess
functions do not divide command strings into args. Invoking
subprocess.call("echo hello") will attempt to execute a program by the
name "echo hello". With subprocess.call("echo hello", shell=True), sh
-c "echo hello" will be executed instead, which will cause the command
and args to be divided by spaces.
Eventually, all usage of shell=True, that is not necessary for some
purpose beyond parsing command strings, should be removed. For now,
this patch makes the usage of shells explicit.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "rollback" verb implements a simple algorithm which takes the set of
remote perforce tracker branches, or optionally, the complete collection
of local branches in a git repository, and deletes commits from these
branches until there are no commits left with a perforce change number
greater than than a user-specified change number. If the base of a git
branch has a newer change number than the user-specified maximum, then
the branch is deleted.
In future, there might be an argument for the addition of some kind of
"reset this branch back to a given perforce change number" verb for
git-p4. However, in its current form it is unlikely to be useful to
users for the following reasons:
* The verb is completely undocumented. The only description provided
contains the following text: "A tool to debug the multi-branch
import. Don't use :)".
* The verb has a very narrow purpose in that it applies the rollback
operation to fixed sets of branches - either all remote p4 branches,
or all local branches. There is no way for users to specify branches
with more granularity, for example, allowing users to specify a
single branch or a set of branches. The utility of the current
implementation is therefore a niche within a niche.
Given these shortcomings, this patch removes the verb from git-p4.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The git-p4 "debug" verb is described as "A tool to debug the output of
p4 -G".
The verb is not documented in any detail, but implements a function
which executes an arbitrary p4 command with the -G flag, which causes
perforce to format all output as marshalled Python dictionary objects.
The verb was implemented early in the history of git-p4, and may once
have served a useful purpose to the authors in the early stages of
development. However, the "debug" verb is no longer being used by the
current developers (and users) of git-p4, and whatever purpose the verb
previously offered is easily replaced by invoking p4 directly.
This patch therefore removes the verb from git-p4.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When importing files from Perforce, git-p4 periodically logs the
progress of file transfers as a percentage. However, the value is
printed as a float with an excessive number of decimal places.
For example a typical update might contain the following message:
Importing revision 12345 (26.199617677553135%)
This patch simply rounds the value down to the nearest integer
percentage value, greatly improving readability.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The git-p4 script reports file sizes in various log messages.
Previously, in each case the script would print them as the number of
bytes divided by 1048576 i.e. the size in mebibytes, rounded down to an
integer. This resulted in small files being described as having a size
of "0 MB".
This patch replaces the existing behaviour with a new helper function:
format_size_human_readable, which takes a number of bytes (or any other
quantity), and computes the appropriate prefix to use: none, Ki, Mi, Gi,
Ti, Pi, Ei, Zi, Yi.
For example, a size of 123456 will now be printed as "120.6 KiB" greatly
improving the readability of the log output.
Large valued prefixes such as pebi, exbi, zebi and yobi are included for
completeness, though they not expected to appear in any real-world
Perforce repository!
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
RCS keywords are strings that are replaced with information from
Perforce. Examples include $Date$, $Author$, $File$, $Change$ etc.
Perforce resolves these by expanding them with their expanded values
when files are synced, but Git's data model requires these expanded
values to be converted back into their unexpanded form.
Previously, git-p4.py would implement this behaviour through the use of
regular expressions. However, the regular expression substitution was
applied using decoded strings i.e. the content of incoming commit diffs
was first decoded from bytes into UTF-8, processed with regular
expressions, then converted back to bytes.
Not only is this behaviour inefficient, but it is also a cause of a
common issue caused by text files containing invalid UTF-8 data. For
files created in Windows, CP1252 Smart Quote Characters (0x93 and 0x94)
are seen fairly frequently. These codes are invalid in UTF-8, so if the
script encountered any file containing them, on Python 2 the symbols
will be corrupted, and on Python 3 the script will fail with an
exception.
This patch replaces this decoding/encoding with bytes object regular
expressions, so that the substitution is performed directly upon the
source data with no conversions.
A test for smart quote handling has been added to the
t9810-git-p4-rcs.sh test suite.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The patchRCSKeywords method creates a temporary file in which to store
the patched output data. Previously this file was opened in "w+" mode
(write and read), but the code never reads the contents of the file
while open, so it only needs to be opened in "w" mode (write-only).
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Previously the read_lines function always decoded the result lines. In
order to improve support for non-decoded binary processing of data in
git-p4.py, this patch adds a raw option to the function that allows
decoding to be disabled.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Previously git-p4.py would compile one of two regular expressions for
ever RCS keyword-enabled file. This patch improves simplifies the code
by pre-compiling the two regular expressions when the script first
loads.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Python with statements are used to wrap the execution of a block of code
so that an object can be safely released when execution leaves the
scope.
They are desirable for improving code tidyness, and to ensure that
objects are properly destroyed even when exceptions are thrown.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If the submit contain binary files, it will throw exception and stop submit when try to append diff line description.
This commit will skip non-text data files when exception UnicodeDecodeError thrown.
The skip will not affect actual submit files in the resulting cl,
the diff line description will only appear in submit template,
so you can review what changed before actully submit to p4.
I don't know if add any message here will be helpful for users,
so I choose to just skip binary content, since it already append filename previously.
Signed-off-by: dorgon.chang <dorgonman@hotmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
For every new branch that git-p4 imports, it needs to find the commit
where it branched off its parent branch. While p4 doesn't record this
information explicitly, the first changelist on a branch is usually an
identical copy of the parent branch.
The method searchParent() tries to find a commit in the history of the
given "parent" branch whose tree exactly matches the initial changelist
of the new branch, "target". The code iterates through the parent
commits and compares each of them to this initial changelist using
diff-tree.
Since we already know the tree object name we are looking for, spawning
diff-tree for each commit is wasteful.
Use the "--format" option of "rev-list" to find out the tree object name
of each commit in the history, and find the tree whose name is exactly
the same as the tree of the target commit to optimize this.
This results in a considerable speed-up, at least on Windows. On one
Windows machine with a fairly large repository of about 16000 commits in
the parent branch, the current code takes over 7 minutes, while the new
code only takes just over 10 seconds for the same changelist:
Before:
$ time git p4 sync
Importing from/into multiple branches
Depot paths: //depot
Importing revision 31274 (100.0%)
Updated branches: b1
real 7m41.458s
user 0m0.000s
sys 0m0.077s
After:
$ time git p4 sync
Importing from/into multiple branches
Depot paths: //depot
Importing revision 31274 (100.0%)
Updated branches: b1
real 0m10.235s
user 0m0.000s
sys 0m0.062s
Signed-off-by: Joachim Kuebart <joachim.kuebart@gmail.com>
Helped-by: Junio C Hamano <gitster@pobox.com>
Helped-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Example of pattern file type: text+k
Text filtered through the p4 pattern regexp must be converted from
string back to bytes, otherwise 'data' command for the fast-import
will receive extra invalid characters, followed by the fast-import
process error.
CC: Yang Zhao <yang.zhao@skyboxlabs.com>
Signed-off-by: Daniel Levin <dendy.ua@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In `git p4 clone`, we hard-code the branch name `master` instead of
looking what the _actual_ initial branch name is. Let's fix that.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Fix misspelled "specified" and "occurred" in documentation and
comments.
Signed-off-by: Marlon Rac Cambasis <marlonrc08@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "--prepare-p4-only" option is supposed to stop after replaying
one changeset, but kept going (by mistake?)
* bk/p4-prepare-p4-only-fix:
git-p4.py: fix --prepare-p4-only error with multiple commits
When using git p4 submit with the --prepare-p4-only option, the program
should prepare a single p4 changelist and notify the user that more
commits are pending and then stop processing.
A bug has been introduced by the p4-changelist hook feature that
causes the program to continue to try and process all pending
changelists at the same time.
The function applyCommit returns True when applying the commit
was successful and the program should continue. However, when the
optional flag --prepare-p4-only is set, the program should stop
after the first application.
Change the logic in the run method for P4Submit to check for the
flag --prepare-p4-only after successfully completing the applyCommit
method.
Be aware - this change will fix the existing test error in t9807.23
for --prepare-p4-only. However there is insufficent coverage for
this flag. If more than 1 commit is pending submission to P4, the
method will properly prepare the P4 changelist, however it will
still exit the application with an exitcode of 1.
The current documentation does not define what the exit code should be
in this condition.
(See: https://git-scm.com/docs/git-p4#Documentation/git-p4.txt---prepare-p4-only)
Signed-off-by: Ben Keene <seraphire@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Perforce allows you commit files and directories with the same name,
so you could have files //depot/foo and //depot/foo/bar both checked
in. A p4 sync of a repository in this state fails. Deleting one of
the files recovers the repository.
When this happens we want git-p4 to recover in the same way as
perforce.
Note that Perforce has this change in their 2017.1 version:
Bugs fixed in 2017.1
#1489051 (Job #2170) **
Submitting a file with the same name as an existing depot
directory path (or vice versa) will now be rejected.
so people hopefully will not creating damaged Perforce repos
anymore, but "git p4" needs to be able to interact with already
corrupt ones.
Signed-off-by: Andrew Oakley <andrew@adoakley.name>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git p4" learned four new hooks and also "--no-verify" option to
bypass them (and the existing "p4-pre-submit" hook).
* bk/p4-pre-edit-changelist:
git-p4: add RCS keyword status message
git-p4: add p4 submit hooks
git-p4: restructure code in submit
git-p4: add --no-verify option
git-p4: add p4-pre-submit exit text
git-p4: create new function run_git_hook
git-p4: rewrite prompt to be Windows compatible
Update "git p4" to work with Python 3.
* yz/p4-py3:
ci: use python3 in linux-gcc and osx-gcc and python2 elsewhere
git-p4: use python3's input() everywhere
git-p4: simplify regex pattern generation for parsing diff-tree
git-p4: use dict.items() iteration for python3 compatibility
git-p4: use functools.reduce instead of reduce
git-p4: fix freezing while waiting for fast-import progress
git-p4: use marshal format version 2 when sending to p4
git-p4: open .gitp4-usercache.txt in text mode
git-p4: convert path to unicode before processing them
git-p4: encode/decode communication with git for python3
git-p4: encode/decode communication with p4 for python3
git-p4: remove string type aliasing
git-p4: change the expansion test from basestring to list
git-p4: make python2.7 the oldest supported version
During the p4 submit process, git-p4 will attempt to apply a patch
to the files found in the p4 workspace. However, if P4 uses RCS
keyword expansion, this patch may fail.
When the patch fails, the user is alerted to the failure and that git-p4
will attempt to clear the expanded text from the files and re-apply
the patch. The current version of git-p4 does not tell the user the
result of the re-apply attempt after the RCS expansion has been removed
which can be confusing.
Add a new print statement after the git patch has been successfully
applied when the RCS keywords have been cleansed.
Signed-off-by: Ben Keene <seraphire@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The git command "commit" supports a number of hooks that support
changing the behavior of the commit command. The git-p4.py program only
has one existing hook, "p4-pre-submit". This command occurs early in
the process. There are no hooks in the process flow for modifying
the P4 changelist text programmatically.
Adds 3 new hooks to git-p4.py to the submit option.
The new hooks are:
* p4-prepare-changelist - Execute this hook after the changelist file
has been created. The hook will be executed even if the
--prepare-p4-only option is set. This hook ignores the --no-verify
option in keeping with the existing behavior of git commit.
* p4-changelist - Execute this hook after the user has edited the
changelist. Do not execute this hook if the user has selected the
--prepare-p4-only option. This hook will honor the --no-verify,
following the conventions of git commit.
* p4-post-changelist - Execute this hook after the P4 submission process
has completed successfully. This hook takes no parameters and is
executed regardless of the --no-verify option. It's return value will
not be checked.
The calls to the new hooks: p4-prepare-changelist, p4-changelist,
and p4-post-changelist should all be called inside the try-finally
block.
Signed-off-by: Ben Keene <seraphire@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Joel Holdsworth
Junio C Hamano
Emily Shaffer
dorgon.chang
Joachim Kuebart
Daniel Levin
Johannes Schindelin
Marlon Rac Cambasis
Luke Diamand
Ben Keene
Andrew Oakley