mirror/ gitea
1
1
Fork 1
mirror of https://github.com/go-gitea/gitea.git synced 2024-05-13 16:06:16 +02:00
Code Issues
14,480 Commits 17 Branches 210 Tags 575 MiB
Commit Graph

27 Commits

Author SHA1 Message Date
wxiaoguang 6bc3079c00
Refactor git command package to improve security and maintainability (#22678) 
This PR follows #21535 (and replace #22592)

## Review without space diff

https://github.com/go-gitea/gitea/pull/22678/files?diff=split&w=1

## Purpose of this PR

1. Make git module command completely safe (risky user inputs won't be
passed as argument option anymore)
2. Avoid low-level mistakes like
https://github.com/go-gitea/gitea/pull/22098#discussion_r1045234918
3. Remove deprecated and dirty `CmdArgCheck` function, hide the `CmdArg`
type
4. Simplify code when using git command

## The main idea of this PR

* Move the `git.CmdArg` to the `internal` package, then no other package
except `git` could use it. Then developers could never do
`AddArguments(git.CmdArg(userInput))` any more.
* Introduce `git.ToTrustedCmdArgs`, it's for user-provided and already
trusted arguments. It's only used in a few cases, for example: use git
arguments from config file, help unit test with some arguments.
* Introduce `AddOptionValues` and `AddOptionFormat`, they make code more
clear and simple:
    * Before: `AddArguments("-m").AddDynamicArguments(message)`
    * After: `AddOptionValues("-m", message)`
    * -
* Before: `AddArguments(git.CmdArg(fmt.Sprintf("--author='%s <%s>'",
sig.Name, sig.Email)))`
* After: `AddOptionFormat("--author='%s <%s>'", sig.Name, sig.Email)`

## FAQ

### Why these changes were not done in #21535 ?

#21535 is mainly a search&replace, it did its best to not change too
much logic.

Making the framework better needs a lot of changes, so this separate PR
is needed as the second step.


### The naming of `AddOptionXxx`

According to git's manual, the `--xxx` part is called `option`.

### How can it guarantee that `internal.CmdArg` won't be not misused?

Go's specification guarantees that. Trying to access other package's
internal package causes compilation error.

And, `golangci-lint` also denies the git/internal package. Only the
`git/command.go` can use it carefully.

### There is still a `ToTrustedCmdArgs`, will it still allow developers
to make mistakes and pass untrusted arguments?

Generally speaking, no. Because when using `ToTrustedCmdArgs`, the code
will be very complex (see the changes for examples). Then developers and
reviewers can know that something might be unreasonable.

### Why there was a `CmdArgCheck` and why it's removed?

At the moment of #21535, to reduce unnecessary changes, `CmdArgCheck`
was introduced as a hacky patch. Now, almost all code could be written
as `cmd := NewCommand(); cmd.AddXxx(...)`, then there is no need for
`CmdArgCheck` anymore.


### Why many codes for `signArg == ""` is deleted?

Because in the old code, `signArg` could never be empty string, it's
either `-S[key-id]` or `--no-gpg-sign`. So the `signArg == ""` is just
dead code.

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
 2023-02-04 10:30:43 +08:00
Jason Song 7adc2de464
Use context parameter in models/git (#22367) 
After #22362, we can feel free to use transactions without
`db.DefaultContext`.

And there are still lots of models using `db.DefaultContext`, I think we
should refactor them carefully and one by one.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
 2023-01-09 11:50:54 +08:00
flynnnnnnnnnn e81ccc406b
Implement FSFE REUSE for golang files (#21840) 
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
 2022-11-27 18:20:29 +00:00
zeripath c772934ff6
Adjust gitea doctor --run storages to check all storage types (#21785) 
The doctor check `storages` currently only checks the attachment
storage. This PR adds some basic garbage collection functionality for
the other types of storage.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
 2022-11-15 16:08:59 +08:00
delvh 0ebb45cfe7
Replace all instances of fmt.Errorf(%v) with fmt.Errorf(%w) (#21551) 
Found using
`find . -type f -name '*.go' -print -exec vim {} -c
':%s/fmt\.Errorf(\(.*\)%v\(.*\)err/fmt.Errorf(\1%w\2err/g' -c ':wq' \;`

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
 2022-10-24 20:29:17 +01:00
wxiaoguang dcd9fc7ee8
Refactor git command arguments and make all arguments to be safe to be used (#21535) 
Follow #21464

Make all git command arguments strictly safe. Most changes are one-to-one replacing, keep all existing logic.
 2022-10-23 22:44:45 +08:00
KN4CK3R 0e57ff7eee
Add generic set type (#21408) 
This PR adds a generic set type to get rid of maps used as sets.

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
 2022-10-12 13:18:26 +08:00
zeripath 99efa02edf
Switch Unicode Escaping to a VSCode-like system (#19990) 
This PR rewrites the invisible unicode detection algorithm to more
closely match that of the Monaco editor on the system. It provides a
technique for detecting ambiguous characters and relaxes the detection
of combining marks.

Control characters are in addition detected as invisible in this
implementation whereas they are not on monaco but this is related to
font issues.

Close #19913

Signed-off-by: Andrew Thornton <art27@cantab.net>
 2022-08-13 19:32:34 +01:00
Lunny Xiao 110fc57cbc
Move some code into models/git (#19879) 
* Move access and repo permission to models/perm/access

* fix test

* Move some git related files into sub package models/git

* Fix build

* fix git test

* move lfs to sub package

* move more git related functions to models/git

* Move functions sequence

* Some improvements per @KN4CK3R and @delvh
 2022-06-12 23:51:54 +08:00
wxiaoguang a0051634b9
Refactor git module, make Gitea use internal git config (#19732) 
* Refactor git module, make Gitea use internal git config, add safe.directory config

* introduce git.InitSimple and git.InitWithConfigSync, make serv cmd use gitconfig

* use HOME instead of GIT_CONFIG_GLOBAL, because git always needs a correct HOME

* fix cmd env in cmd/serv.go

* fine tune error message

* Fix a incorrect test case

* fix configAddNonExist

* fix configAddNonExist logic, add `--fixed-value` flag, add tests

* add configSetNonExist function in case it's needed.

* use configSetNonExist for `user.name` and `user.email`

* add some comments

* Update cmd/serv.go

Co-authored-by: zeripath <art27@cantab.net>

* Update cmd/serv.go

Co-authored-by: zeripath <art27@cantab.net>

* Update modules/git/git.go

Co-authored-by: zeripath <art27@cantab.net>

* Update modules/setting/setting.go

Co-authored-by: zeripath <art27@cantab.net>

* Update modules/git/repo_attribute.go

Co-authored-by: zeripath <art27@cantab.net>

* fix spaces in messages

* use `configSet("core.protectNTFS", ...)` instead of `globalCommandArgs`

* remove GIT_CONFIG_NOSYSTEM, continue to use system's git config

* Update cmd/serv.go

Co-authored-by: zeripath <art27@cantab.net>

* fix merge

* remove code for safe.directory

* separate git.CommonEnvs to CommonGitCmdEnvs and CommonCmdServEnvs

* avoid Golang's data race error

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
 2022-06-10 09:57:49 +08:00
Lunny Xiao 4ca1d7547a
Move some helper files out of models (#19355) 
* Move some helper files out of models

* Some improvements

Co-authored-by: delvh <dev.lh@web.de>
 2022-05-08 18:46:32 +02:00
6543 3e88af898a
Make git.OpenRepository accept Context (#19260) 
* OpenRepositoryCtx -> OpenRepository
* OpenRepository -> openRepositoryWithDefaultContext, only for internal usage
 2022-03-30 03:13:41 +08:00
KN4CK3R 80fd25524e
Renamed ctx.User to ctx.Doer. (#19161) 
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
 2022-03-22 15:03:22 +08:00
Lunny Xiao 49db87a035
Fix lfs bug (#19072) 
* Fix lfs bug
 2022-03-14 16:18:27 +01:00
Gusted 6d55a132a5
Uncapitalize errors (#18915) 
- See: https://github.com/golang/go/wiki/CodeReviewComments#error-strings

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
 2022-02-26 20:15:32 +08:00
6543 54e9ee37a7
format with gofumpt (#18184) 
* gofumpt -w -l .

* gofumpt -w -l -extra .

* Add linter

* manual fix

* change make fmt
 2022-01-20 18:46:10 +01:00
zeripath 5cb0c9aa0d
Propagate context and ensure git commands run in request context (#17868) 
This PR continues the work in #17125 by progressively ensuring that git
commands run within the request context.

This now means that the if there is a git repo already open in the context it will be used instead of reopening it.

Signed-off-by: Andrew Thornton <art27@cantab.net>
 2022-01-19 23:26:57 +00:00
zeripath 21ed4fd8da
Add warning for BIDI characters in page renders and in diffs (#17562) 
Fix #17514

Given the comments I've adjusted this somewhat. The numbers of characters detected are increased and include things like the use of U+300 to make à instead of à and non-breaking spaces.

There is a button which can be used to escape the content to show it.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Gwyneth Morgan <gwymor@tilde.club>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
 2022-01-07 02:18:52 +01:00
wxiaoguang 385dc6a992
Allow admin to associate missing LFS objects for repositories (#18143) 
This PR reworked the Find pointer files feature in Settings -> LFS page.

When a LFS object is missing from database but exists in LFS content store, admin can associate it to the repository by clicking the Associate button.

This PR is not perfect (because the LFS module itself should be improved too), it's just a nice-to-have feature to help users recover their LFS repositories (eg: database was lost / table was truncated)
 2022-01-01 17:05:31 +08:00
Lunny Xiao 719bddcd76
Move repository model into models/repo (#17933) 
* Some refactors related repository model

* Move more methods out of repository

* Move repository into models/repo

* Fix test

* Fix test

* some improvements

* Remove unnecessary function
 2021-12-10 09:27:50 +08:00
Lunny Xiao a666829a37
Move user related model into models/user (#17781) 
* Move user related model into models/user

* Fix lint for windows

* Fix windows lint

* Fix windows lint

* Move some tests in models

* Merge
 2021-11-24 17:49:20 +08:00
zeripath bbffcc3aec
Multiple Escaping Improvements (#17551) 
There are multiple places where Gitea does not properly escape URLs that it is building and there are multiple places where it builds urls when there is already a simpler function available to use this.
    
This is an extensive PR attempting to fix these issues.

1. The first commit in this PR looks through all href, src and links in the Gitea codebase and has attempted to catch all the places where there is potentially incomplete escaping.
2. Whilst doing this we will prefer to use functions that create URLs over recreating them by hand.
3. All uses of strings should be directly escaped - even if they are not currently expected to contain escaping characters. The main benefit to doing this will be that we can consider relaxing the constraints on user names and reponames in future. 
4. The next commit looks at escaping in the wiki and re-considers the urls that are used there. Using the improved escaping here wiki files containing '/'. (This implementation will currently still place all of the wiki files the root directory of the repo but this would not be difficult to change.)
5. The title generation in feeds is now properly escaped.
6. EscapePound is no longer needed - urls should be PathEscaped / QueryEscaped as necessary but then re-escaped with Escape when creating html with locales Signed-off-by: Andrew Thornton <art27@cantab.net>

Signed-off-by: Andrew Thornton <art27@cantab.net>
 2021-11-16 18:18:25 +00:00
KN4CK3R f99d50fc9f
Read expected buffer size (#17409) 
* Read expected buffer size.

* Changed name.
 2021-10-24 22:12:43 +01:00
Eng Zer Jun f2e7d5477f
refactor: move from io/ioutil to io and os package (#17109) 
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
 2021-09-22 13:38:34 +08:00
6543 c4d70a0325
Rename ctx.Form() to ctx.FormString() and move code into own file (#16571) 
Followup from #16562 prepare for #16567

* Rename ctx.Form() to ctx.FormString()
* Reimplement FormX func to need less code and cpu cycles
* Move code into own file
 2021-08-11 02:31:13 +02:00
Lunny Xiao 33e0b38287
Rename context.Query to context.Form (#16562) 2021-07-29 03:42:15 +02:00
Lunny Xiao 1bfb0a24d8
Refactor routers directory (#15800) 
* refactor routers directory

* move func used for web and api to common

* make corsHandler a function to prohibit side efects

* rm unused func

Co-authored-by: 6543 <6543@obermui.de>
 2021-06-09 01:33:54 +02:00