mirror/ gitea
1
1
Fork 1
mirror of https://github.com/go-gitea/gitea.git synced 2024-05-13 11:06:37 +02:00
Code Issues
14,480 Commits 17 Branches 210 Tags 574 MiB
Commit Graph

61 Commits

Author SHA1 Message Date
wxiaoguang 6bc3079c00
Refactor git command package to improve security and maintainability (#22678) 
This PR follows #21535 (and replace #22592)

## Review without space diff

https://github.com/go-gitea/gitea/pull/22678/files?diff=split&w=1

## Purpose of this PR

1. Make git module command completely safe (risky user inputs won't be
passed as argument option anymore)
2. Avoid low-level mistakes like
https://github.com/go-gitea/gitea/pull/22098#discussion_r1045234918
3. Remove deprecated and dirty `CmdArgCheck` function, hide the `CmdArg`
type
4. Simplify code when using git command

## The main idea of this PR

* Move the `git.CmdArg` to the `internal` package, then no other package
except `git` could use it. Then developers could never do
`AddArguments(git.CmdArg(userInput))` any more.
* Introduce `git.ToTrustedCmdArgs`, it's for user-provided and already
trusted arguments. It's only used in a few cases, for example: use git
arguments from config file, help unit test with some arguments.
* Introduce `AddOptionValues` and `AddOptionFormat`, they make code more
clear and simple:
    * Before: `AddArguments("-m").AddDynamicArguments(message)`
    * After: `AddOptionValues("-m", message)`
    * -
* Before: `AddArguments(git.CmdArg(fmt.Sprintf("--author='%s <%s>'",
sig.Name, sig.Email)))`
* After: `AddOptionFormat("--author='%s <%s>'", sig.Name, sig.Email)`

## FAQ

### Why these changes were not done in #21535 ?

#21535 is mainly a search&replace, it did its best to not change too
much logic.

Making the framework better needs a lot of changes, so this separate PR
is needed as the second step.


### The naming of `AddOptionXxx`

According to git's manual, the `--xxx` part is called `option`.

### How can it guarantee that `internal.CmdArg` won't be not misused?

Go's specification guarantees that. Trying to access other package's
internal package causes compilation error.

And, `golangci-lint` also denies the git/internal package. Only the
`git/command.go` can use it carefully.

### There is still a `ToTrustedCmdArgs`, will it still allow developers
to make mistakes and pass untrusted arguments?

Generally speaking, no. Because when using `ToTrustedCmdArgs`, the code
will be very complex (see the changes for examples). Then developers and
reviewers can know that something might be unreasonable.

### Why there was a `CmdArgCheck` and why it's removed?

At the moment of #21535, to reduce unnecessary changes, `CmdArgCheck`
was introduced as a hacky patch. Now, almost all code could be written
as `cmd := NewCommand(); cmd.AddXxx(...)`, then there is no need for
`CmdArgCheck` anymore.


### Why many codes for `signArg == ""` is deleted?

Because in the old code, `signArg` could never be empty string, it's
either `-S[key-id]` or `--no-gpg-sign`. So the `signArg == ""` is just
dead code.

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
 2023-02-04 10:30:43 +08:00
Brecht Van Lommel b5b3e0714e
Pull request yaml template support for including commit body in a field (#22629) 
When using a markdown template the commit body is prepended to text in
the description. This adds the same functionality for yaml templates,
adding the commit message in a field with id "body".

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
 2023-01-30 22:39:07 +08:00
Jason Song 7adc2de464
Use context parameter in models/git (#22367) 
After #22362, we can feel free to use transactions without
`db.DefaultContext`.

And there are still lots of models using `db.DefaultContext`, I think we
should refactor them carefully and one by one.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
 2023-01-09 11:50:54 +08:00
Lunny Xiao 68704532c2
Rename almost all Ctx functions (#22071) 2022-12-10 10:46:31 +08:00
Lunny Xiao 0a7d3ff786
refactor some functions to support ctx as first parameter (#21878) 
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Lauris BH <lauris@nix.lv>
 2022-12-03 10:48:26 +08:00
flynnnnnnnnnn e81ccc406b
Implement FSFE REUSE for golang files (#21840) 
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
 2022-11-27 18:20:29 +00:00
KN4CK3R 044c754ea5
Add `context.Context` to more methods (#21546) 
This PR adds a context parameter to a bunch of methods. Some helper
`xxxCtx()` methods got replaced with the normal name now.

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
 2022-11-19 16:12:33 +08:00
wxiaoguang dcd9fc7ee8
Refactor git command arguments and make all arguments to be safe to be used (#21535) 
Follow #21464

Make all git command arguments strictly safe. Most changes are one-to-one replacing, keep all existing logic.
 2022-10-23 22:44:45 +08:00
KN4CK3R 43c10def68
Fix CSV diff for added/deleted files (#21189) 
Fixes #21184
Regression of #19552

Instead of using `GetBlobByPath` I use the already existing instances.

We need more information from #19530 if that error is still present.
 2022-09-17 10:45:32 +08:00
Jason Song 84447df4d3
Support Issue forms and PR forms (#20987) 
* feat: extend issue template for yaml

* feat: support yaml template

* feat: render form to markdown

* feat: support yaml template for pr

* chore: rename to Fields

* feat: template unmarshal

* feat: split template

* feat: render to markdown

* feat: use full name as template file name

* chore: remove useless file

* feat: use dropdown of fomantic ui

* feat: update input style

* docs: more comments

* fix: render text without render

* chore: fix lint error

* fix: support use description as about in markdown

* fix: add field class in form

* chore: generate swagger

* feat: validate template

* feat: support is_nummber and regex

* test: fix broken unit tests

* fix: ignore empty body of md template

* fix: make multiple easymde editors work in one page

* feat: better UI

* fix: js error in pr form

* chore: generate swagger

* feat: support regex validation

* chore: generate swagger

* fix: refresh each markdown editor

* chore: give up required validation

* fix: correct issue template candidates

* fix: correct checkboxes style

* chore: ignore .hugo_build.lock in docs

* docs: separate out a new doc for merge templates

* docs: introduce syntax of yaml template

* feat: show a alert for invalid templates

* test: add case for a valid template

* fix: correct attributes of required checkbox

* fix: add class not-under-easymde for dropzone

* fix: use more back-quotes

* chore: remove translation in zh-CN

* fix EasyMDE statusbar margin

* fix: remove repeated blocks

* fix: reuse regex for quotes

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
 2022-09-02 15:58:49 +08:00
Lunny Xiao 1d8543e7db
Move some files into models' sub packages (#20262) 
* Move some files into models' sub packages

* Move functions

* merge main branch

* Fix check

* fix check

* Fix some tests

* Fix lint

* Fix lint

* Revert lint changes

* Fix error comments

* Fix lint

Co-authored-by: 6543 <6543@obermui.de>
 2022-08-25 10:31:57 +08:00
Gusted 9cf0352f14
Prepend commit message to template content (#20429) 
- When a repository has a pull request template, it will always override
the current content. With this PR it will prepend content to the
template content when appropriate. This is similar how GitHub(and GitLab
I presume) does it and it saves developers time to not go open their
commit and copy paste their will written commit message.
 2022-07-24 04:45:33 +01:00
Lunny Xiao b01dce2a6e
Allow render HTML with css/js external links (#19017) 
* Allow render HTML with css/js external links

* Fix bug because of filename escape chars

* Fix lint

* Update docs about new configuration item

* Fix bug of render HTML in sub directory

* Add CSP head for displaying iframe in rendering file

* Fix test

* Apply suggestions from code review

Co-authored-by: delvh <dev.lh@web.de>

* Some improvements

* some improvement

* revert change in SanitizerDisabled of external renderer

* Add sandbox for iframe and support allow-scripts and allow-same-origin

* refactor

* fix

* fix lint

* fine tune

* use single option RENDER_CONTENT_MODE, use sandbox=allow-scripts

* fine tune CSP

* Apply suggestions from code review

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
 2022-06-16 11:33:23 +08:00
Lunny Xiao 1a9821f57a
Move issues related files into models/issues (#19931) 
* Move access and repo permission to models/perm/access

* fix test

* fix git test

* Move functions sequence

* Some improvements per @KN4CK3R and @delvh

* Move issues related code to models/issues

* Move some issues related sub package

* Merge

* Fix test

* Fix test

* Fix test

* Fix test

* Rename some files
 2022-06-13 17:37:59 +08:00
Lunny Xiao 110fc57cbc
Move some code into models/git (#19879) 
* Move access and repo permission to models/perm/access

* fix test

* Move some git related files into sub package models/git

* Fix build

* fix git test

* move lfs to sub package

* move more git related functions to models/git

* Move functions sequence

* Some improvements per @KN4CK3R and @delvh
 2022-06-12 23:51:54 +08:00
Lunny Xiao 26095115f4
Move some repository related code into sub package (#19711) 
* Move some repository related code into sub package

* Move more repository functions out of models

* Fix lint

* Some performance optimization for webhooks and others

* some refactors

* Fix lint

* Fix

* Update modules/repository/delete.go

Co-authored-by: delvh <dev.lh@web.de>

* Fix test

* Merge

* Fix test

* Fix test

* Fix test

* Fix test

Co-authored-by: delvh <dev.lh@web.de>
 2022-06-06 16:01:49 +08:00
Lunny Xiao fd7d83ace6
Move almost all functions' parameter db.Engine to context.Context (#19748) 
* Move almost all functions' parameter db.Engine to context.Context
* remove some unnecessary wrap functions
 2022-05-20 22:08:52 +08:00
Lunny Xiao cbd45471b1
Move access and repo permission to models/perm/access (#19350) 
* Move access and repo permission to models/perm/access

* Remove unnecessary code
 2022-05-11 12:09:36 +02:00
KN4CK3R a9ca4b4100
Calculate filename hash only once (#19654) 
* Calculate hash only once.

* remove unused Sha1 template helper function, use ctx.Data["FileNameHash"]

* fix unit tests
 2022-05-09 00:29:50 +02:00
99rgosse 438646e0b5
Handle the error of a missing blob object fix #19530 (#19552) 
* Handle the error of a missing blob object

* Show error in logs

* as per @zeripath

* Add missing error check

* Add missing error check

* Update compare.go

* Use formal code

* Update compare.go

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
 2022-05-02 18:46:50 +02:00
qwerty287 8eb1cd9264
Add "Allow edits from maintainer" feature (#18002) 
Adds a feature [like GitHub has](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork) (step 7).
If you create a new PR from a forked repo, you can select (and change later, but only if you are the PR creator/poster) the "Allow edits from maintainers" option.
Then users with write access to the base branch get more permissions on this branch:
* use the update pull request button
* push directly from the command line (`git push`)
* edit/delete/upload files via web UI
* use related API endpoints

You can't merge PRs to this branch with this enabled, you'll need "full" code write permissions.

This feature has a pretty big impact on the permission system. I might forgot changing some things or didn't find security vulnerabilities. In this case, please leave a review or comment on this PR.

Closes #17728

Co-authored-by: 6543 <6543@obermui.de>
 2022-04-28 17:45:33 +02:00
6543 06e4687cec
more context for models (#19511) 
make more usage of context, to have more db transaction in one session

(make diff of  #9307 smaller)
 2022-04-28 13:48:48 +02:00
6543 3e88af898a
Make git.OpenRepository accept Context (#19260) 
* OpenRepositoryCtx -> OpenRepository
* OpenRepository -> openRepositoryWithDefaultContext, only for internal usage
 2022-03-30 03:13:41 +08:00
a1012112796 0eff23dae0
Fix compare link in active feeds for new branch (#19149) 
When a new branch is pushed the old SHA is always listed as the empty sha and thus the compare link that is created does not work correctly. 

Therefore when creating the compare link for new branches:

1. Attempt to get the parent of the first commit and use that as the basis 
for the compare link.
2. If this is not possible make a comparison to the default branch
3. Finally if that is not possible simply do not show a compare link.

However, there are multiple broken compare links remaining therefore, in order for these to not break we will simply make the compare link redirect to the default branch.

Fix #19144

Signed-off-by: a1012112796 <1012112796@qq.com>
Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Andrew Thornton <art27@cantab.net>
 2022-03-23 13:40:12 +00:00
wxiaoguang 7a550b3af2
Use `ctx` instead of `db.DefaultContext` in some packages(routers/services/modules) (#19163) 
* Remove `db.DefaultContext` usage in routers, use `ctx` directly

* Use `ctx` directly if there is one, remove some `db.DefaultContext` in `services`

* Use ctx instead of db.DefaultContext for `cmd` and some `modules` packages

* fix incorrect context usage
 2022-03-22 16:22:54 +01:00
KN4CK3R 80fd25524e
Renamed ctx.User to ctx.Doer. (#19161) 
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
 2022-03-22 15:03:22 +08:00
Jimmy Praet e4ef61ee0f
BeforeSourcePath should point to base commit (#18799) 
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
 2022-02-25 14:46:15 +08:00
zeripath 933e819165
Ensure that blob-excerpt links work for wiki (#18587) 
It appears that the blob-excerpt links do not work on the wiki - likely since their
introduction.

This PR adds support for the wiki on these links.

Signed-off-by: Andrew Thornton <art27@cantab.net>
 2022-02-05 18:26:12 +00:00
6543 54e9ee37a7
format with gofumpt (#18184) 
* gofumpt -w -l .

* gofumpt -w -l -extra .

* Add linter

* manual fix

* change make fmt
 2022-01-20 18:46:10 +01:00
zeripath 5cb0c9aa0d
Propagate context and ensure git commands run in request context (#17868) 
This PR continues the work in #17125 by progressively ensuring that git
commands run within the request context.

This now means that the if there is a git repo already open in the context it will be used instead of reopening it.

Signed-off-by: Andrew Thornton <art27@cantab.net>
 2022-01-19 23:26:57 +00:00
wxiaoguang a38ba634a4
Load EasyMDE/CodeMirror dynamically, remove RequireEasyMDE (#18069) 
This PR makes frontend load EasyMDE/CodeMirror dynamically, and removes `RequireEasyMDE`.
 2022-01-05 13:17:25 +01:00
Lunny Xiao d7770539f6
Improve interface when comparing a branch which has created a pull request (#17911) 
* Improve interface when comparing a branch which has created a pull request

* Take the note back
 2021-12-24 12:14:42 +00:00
Gusted ff2fd08228
Simplify parameter types (#18006) 
Remove repeated type declarations in function definitions.
 2021-12-20 04:41:31 +00:00
Lunny Xiao b300e3f485
Support open compare page directly (#17975) 
* Support open compare page directly

* simple code

* Some improvements

Co-authored-by: 6543 <6543@obermui.de>
 2021-12-17 22:20:27 +00:00
zeripath 8354670708
Prevent hang in git cat-file if repository is not a valid repository and other fixes (#17991) 
This PR contains multiple fixes. The most important of which is:

* Prevent hang in git cat-file if the repository is not a valid repository 
    
    Unfortunately it appears that if git cat-file is run in an invalid
    repository it will hang until stdin is closed. This will result in
    deadlocked /pulls pages and dangling git cat-file calls if a broken
    repository is tried to be reviewed or pulls exists for a broken
    repository.

    Fix #14734
    Fix #9271
    Fix #16113

Otherwise there are a few small other fixes included which this PR was initially intending to fix:

* Fix panic on partial compares due to missing PullRequestWorkInProgressPrefixes
* Fix links on pulls pages  due to regression from #17551 - by making most /issues routes match /pulls too - Fix #17983
* Fix links on feeds pages due to another regression from #17551 but also fix issue with syncing tags - Fix #17943
* Add missing locale entries for oauth group claims
* Prevent NPEs if ColorFormat is called on nil users, repos or teams.
 2021-12-16 19:01:14 +00:00
Lunny Xiao 5723240490
Some repository refactors (#17950) 
* some repository refactors

* remove unnecessary code

* Fix test

* Remove unnecessary banner
 2021-12-12 23:48:20 +08:00
wxiaoguang 0a9fcf63a4
Clean legacy SimpleMDE code (#17926) 
Since we are using EasyMDE now, we do not need to keep the SimpleMDE code anymore.

This PR removes all legacy SimpleMDE code, and makes some related changes:
* `createCommentEasyMDE` can accept native DOM element, and it doesn't need `jQuery.data` to store EasyMDE editor object (as discussed about the frontend guideline).
* introduce `getAttachedEasyMDE` to get the attached EasyMDE editor object, it's easier to find all the usage of EasyMDE.
* rename variable names from `$simplemde` to `easyMDE`, the `$` was incorrect because it is a EasyMDE editor, not a jQuery object.

With this PR, it will be easier to do more refactoring or replacing EasyMDE with other editors.
 2021-12-10 10:51:27 +08:00
Lunny Xiao 719bddcd76
Move repository model into models/repo (#17933) 
* Some refactors related repository model

* Move more methods out of repository

* Move repository into models/repo

* Fix test

* Fix test

* some improvements

* Remove unnecessary function
 2021-12-10 09:27:50 +08:00
zeripath 9e6e1dc950
Improve checkBranchName (#17901) 
The current implementation of checkBranchName is highly inefficient
involving opening the repository, the listing all of the branch names
checking them individually before then using using opened repo to get
the tags.

This PR avoids this by simply walking the references from show-ref
instead of opening the repository (in the nogogit case).

Signed-off-by: Andrew Thornton <art27@cantab.net>
 2021-12-08 19:08:16 +00:00
John Olheiser eb69c7ec8c
Allow default branch to be inferred on compare page (#17908) 
* Allow default branch to be inferred

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Add test for inferred default branch

Signed-off-by: jolheiser <john.olheiser@gmail.com>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
 2021-12-06 11:04:07 -06:00
Lunny Xiao a666829a37
Move user related model into models/user (#17781) 
* Move user related model into models/user

* Fix lint for windows

* Fix windows lint

* Fix windows lint

* Move some tests in models

* Merge
 2021-11-24 17:49:20 +08:00
Lunny Xiao baed01f247
Remove unnecessary attributes of User struct (#17745) 
* Remove unnecessary functions of User struct

* Move more database methods out of user struct

* Move more database methods out of user struct

* Fix template failure

* Fix bug

* Remove finished FIXME

* remove unnecessary code
 2021-11-22 23:21:55 +08:00
zeripath 8511eec4d4
Allow Loading of Diffs that are too large (#17739) 
* Allow Loading of Diffs that are too large

This PR allows the loading of diffs that are suppressed because the file
is too large. It does not handle diffs of files which have lines which
are too long.

Fix #17738

Signed-off-by: Andrew Thornton <art27@cantab.net>
 2021-11-22 00:51:08 +08:00
Gusted 00448ebe91
Remove unused `user` paramater (#17723) 
As title
 2021-11-19 08:37:13 +00:00
zeripath bbffcc3aec
Multiple Escaping Improvements (#17551) 
There are multiple places where Gitea does not properly escape URLs that it is building and there are multiple places where it builds urls when there is already a simpler function available to use this.
    
This is an extensive PR attempting to fix these issues.

1. The first commit in this PR looks through all href, src and links in the Gitea codebase and has attempted to catch all the places where there is potentially incomplete escaping.
2. Whilst doing this we will prefer to use functions that create URLs over recreating them by hand.
3. All uses of strings should be directly escaped - even if they are not currently expected to contain escaping characters. The main benefit to doing this will be that we can consider relaxing the constraints on user names and reponames in future. 
4. The next commit looks at escaping in the wiki and re-considers the urls that are used there. Using the improved escaping here wiki files containing '/'. (This implementation will currently still place all of the wiki files the root directory of the repo but this would not be difficult to change.)
5. The title generation in feeds is now properly escaped.
6. EscapePound is no longer needed - urls should be PathEscaped / QueryEscaped as necessary but then re-escaped with Escape when creating html with locales Signed-off-by: Andrew Thornton <art27@cantab.net>

Signed-off-by: Andrew Thornton <art27@cantab.net>
 2021-11-16 18:18:25 +00:00
Lunny Xiao 99b2858e62
Move unit into models/unit/ (#17576) 
* Move unit into models/unit/

* Rename unit.UnitType as unit.Type
 2021-11-09 20:57:58 +01:00
Richard Mahn 40c8451b7d
Properly determine CSV delimiter (#17459) 
* Fixes #16558 CSV delimiter determiner

* Fixes #16558 - properly determine CSV delmiiter

* Moves quoteString to a new function

* Adds big test with lots of commas for tab delimited csv

* Adds comments

* Shortens the text of the test

* Removes single quotes from regexp as only double quotes need to be searched

* Fixes spelling

* Fixes check of length as it probalby will only be 1e4, not greater

* Makes sample size a const, properly removes truncated line

* Makes sample size a const, properly removes truncated line

* Fixes comment

* Fixes comment

* tests for FormatError() function

* Adds logic to find the limiter before or after a quoted value

* Simplifies regex

* Error tests

* Error tests

* Update modules/csv/csv.go

Co-authored-by: delvh <dev.lh@web.de>

* Update modules/csv/csv.go

Co-authored-by: delvh <dev.lh@web.de>

* Adds comments

* Update modules/csv/csv.go

Co-authored-by: delvh <dev.lh@web.de>

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
 2021-10-30 23:50:40 +08:00
KN4CK3R 44f2c27d28
Fix CSV render error (#17406) 
closed #17378 

Both errors from #17378 were caused by  #15175.

Problem 1 (error with added file):
`ToUTF8WithFallbackReader` creates a `MultiReader` from a `byte[2048]` and the remaining reader. `CreateReaderAndGuessDelimiter` tries to read 10000 bytes from this reader but only gets 2048 because that's the first reader in the `MultiReader`. Then the `if size < 1e4` thinks the input is at EOF and just returns that.

Problem 2 (error with changed file):
The blob reader gets defer closed. That was fine because the old version reads the whole file into memory. Now with the streaming version the close needs to defer after the method.
 2021-10-25 00:42:32 +02:00
zeripath a889d0cc8c
Add buttons to allow loading of incomplete diffs (#16829) 
This PR adds two buttons to the stats and the end of the diffs list to load the (some of) the remaining incomplete diff sections.

Contains #16775
    
Signed-off-by: Andrew Thornton <art27@cantab.net>


## Screenshots

### Show more button at the end of the diff
![Screenshot from 2021-09-04 11-12-37](https://user-images.githubusercontent.com/1824502/132091009-b1f6113e-2c04-4be5-8a04-b8ecea56887b.png)

### Show more button at the end of the diff stats box
![Screenshot from 2021-09-04 11-14-54](https://user-images.githubusercontent.com/1824502/132091063-86da5a6d-6628-4b82-bea9-3655cd9f40f6.png)
 2021-10-15 17:05:33 +01:00
zeripath 206ea10b45
Prevent NPE on invalid diff (#17197) 
* Prevent NPE on invalid diff

If ParseCompareInfo returns a nil compare info the defer function needs to ensure
that it does not attempt to close the HeadGitRepo.

Fix #17193

Signed-off-by: Andrew Thornton <art27@cantab.net>

* add TEST

Co-authored-by: 6543 <6543@obermui.de>
 2021-09-30 21:31:02 +02:00