wanderer/ gitea
1
0
Fork 0
forked from mirror/gitea
Code Issues Pull Requests Releases Activity
17,087 Commits 2 Branches 0 Tags 162 MiB
Commit Graph

64 Commits

Author SHA1 Message Date
Lunny Xiao 5f82ead13c
Simplify how git repositories are opened (#28937) 
## Purpose
This is a refactor toward building an abstraction over managing git
repositories.
Afterwards, it does not matter anymore if they are stored on the local
disk or somewhere remote.

## What this PR changes
We used `git.OpenRepository` everywhere previously.
Now, we should split them into two distinct functions:

Firstly, there are temporary repositories which do not change:

```go
git.OpenRepository(ctx, diskPath)
```

Gitea managed repositories having a record in the database in the
`repository` table are moved into the new package `gitrepo`:

```go
gitrepo.OpenRepository(ctx, repo_model.Repo)
```

Why is `repo_model.Repository` the second parameter instead of file
path?
Because then we can easily adapt our repository storage strategy.
The repositories can be stored locally, however, they could just as well
be stored on a remote server.

## Further changes in other PRs
- A Git Command wrapper on package `gitrepo` could be created. i.e.
`NewCommand(ctx, repo_model.Repository, commands...)`. `git.RunOpts{Dir:
repo.RepoPath()}`, the directory should be empty before invoking this
method and it can be filled in the function only. #28940
- Remove the `RepoPath()`/`WikiPath()` functions to reduce the
possibility of mistakes.

---------

Co-authored-by: delvh <dev.lh@web.de>
 2024-01-27 21:09:51 +01:00
JakobDev 4567a3a1ad
Allow to sync tags from admin dashboard (#28045) 
Inspired by #28043

This PR adds a option to the Admin Dashboard to sync all tags to the
database.


![grafik](https://github.com/go-gitea/gitea/assets/15185051/26ac51ef-82a4-4fd9-a6a6-5aefec612ff6)
 2024-01-24 03:02:04 +00:00
delvh 778ad795fd
Refactor deletion (#28610) 
Introduce the new generic deletion methods
- `func DeleteByID[T any](ctx context.Context, id int64) (int64, error)`
- `func DeleteByIDs[T any](ctx context.Context, ids ...int64) error`
- `func Delete[T any](ctx context.Context, opts FindOptions) (int64,
error)`

So, we no longer need any specific deletion method and can just use
the generic ones instead.

Replacement of #28450

Closes #28450

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
 2023-12-25 21:25:29 +01:00
Lunny Xiao 408a484224
Adjust object format interface (#28469) 
- Remove `ObjectFormatID`
- Remove function `ObjectFormatFromID`.
- Use `Sha1ObjectFormat` directly but not a pointer because it's an
empty struct.
- Store `ObjectFormatName` in `repository` struct
 2023-12-17 11:56:08 +00:00
Adam Majer cbf923e87b
Abstract hash function usage (#28138) 
Refactor Hash interfaces and centralize hash function. This will allow
easier introduction of different hash function later on.

This forms the "no-op" part of the SHA256 enablement patch.
 2023-12-13 21:02:00 +00:00
Lunny Xiao 882e502327
Fix comment permissions (#28213) 
This PR will fix some missed checks for private repositories' data on
web routes and API routes.
 2023-11-25 17:21:21 +00:00
JakobDev 7047df36d4
Another round of `db.DefaultContext` refactor (#27103) 
Part of #27065

---------

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
 2023-09-25 13:17:37 +00:00
JakobDev 28f9b313ba
Updates to the API for archived repos (#27149) 2023-09-21 23:43:29 +00:00
Lunny Xiao 540bf9fa6d
Move notification interface to services layer (#26915) 
Extract from #22266
 2023-09-05 18:37:47 +00:00
6543 8995046110
Less naked returns (#25713) 
just a step towards  #25655

and some related refactoring
 2023-07-07 05:31:56 +00:00
Lunny Xiao f9cfd6ce5b
Use the type RefName for all the needed places and fix pull mirror sync bugs (#24634) 
This PR replaces all string refName as a type `git.RefName` to make the
code more maintainable.

Fix #15367
Replaces #23070 
It also fixed a bug that tags are not sync because `git remote --prune
origin` will not remove local tags if remote removed.

We in fact should use `git fetch --prune --tags origin` but not `git
remote update origin` to do the sync.

Some answer from ChatGPT as ref.

> If the git fetch --prune --tags command is not working as expected,
there could be a few reasons why. Here are a few things to check:
> 
>Make sure that you have the latest version of Git installed on your
system. You can check the version by running git --version in your
terminal. If you have an outdated version, try updating Git and see if
that resolves the issue.
> 
>Check that your Git repository is properly configured to track the
remote repository's tags. You can check this by running git config
--get-all remote.origin.fetch and verifying that it includes
+refs/tags/*:refs/tags/*. If it does not, you can add it by running git
config --add remote.origin.fetch "+refs/tags/*:refs/tags/*".
> 
>Verify that the tags you are trying to prune actually exist on the
remote repository. You can do this by running git ls-remote --tags
origin to list all the tags on the remote repository.
> 
>Check if any local tags have been created that match the names of tags
on the remote repository. If so, these local tags may be preventing the
git fetch --prune --tags command from working properly. You can delete
local tags using the git tag -d command.

---------

Co-authored-by: delvh <dev.lh@web.de>
 2023-05-26 01:04:48 +00:00
Lunny Xiao e5dcd6aada
Delete deleted release attachments immediately from storage (#23913) 
Previously, deleted release attachments were kept forever on the
external storage.
Note: It may be very slow now if there are many attachments to be
deleted on this release.

Fix #23728
 2023-04-06 18:01:36 -04:00
Lunny Xiao bd820aa9c5
Add context cache as a request level cache (#22294) 
To avoid duplicated load of the same data in an HTTP request, we can set
a context cache to do that. i.e. Some pages may load a user from a
database with the same id in different areas on the same page. But the
code is hidden in two different deep logic. How should we share the
user? As a result of this PR, now if both entry functions accept
`context.Context` as the first parameter and we just need to refactor
`GetUserByID` to reuse the user from the context cache. Then it will not
be loaded twice on an HTTP request.

But of course, sometimes we would like to reload an object from the
database, that's why `RemoveContextData` is also exposed.

The core context cache is here. It defines a new context
```go
type cacheContext struct {
	ctx  context.Context
	data map[any]map[any]any
        lock sync.RWMutex
}

var cacheContextKey = struct{}{}

func WithCacheContext(ctx context.Context) context.Context {
	return context.WithValue(ctx, cacheContextKey, &cacheContext{
		ctx:  ctx,
		data: make(map[any]map[any]any),
	})
}
```

Then you can use the below 4 methods to read/write/del the data within
the same context.

```go
func GetContextData(ctx context.Context, tp, key any) any
func SetContextData(ctx context.Context, tp, key, value any)
func RemoveContextData(ctx context.Context, tp, key any)
func GetWithContextCache[T any](ctx context.Context, cacheGroupKey string, cacheTargetID any, f func() (T, error)) (T, error)
```

Then let's take a look at how `system.GetString` implement it.

```go
func GetSetting(ctx context.Context, key string) (string, error) {
	return cache.GetWithContextCache(ctx, contextCacheKey, key, func() (string, error) {
		return cache.GetString(genSettingCacheKey(key), func() (string, error) {
			res, err := GetSettingNoCache(ctx, key)
			if err != nil {
				return "", err
			}
			return res.SettingValue, nil
		})
	})
}
```

First, it will check if context data include the setting object with the
key. If not, it will query from the global cache which may be memory or
a Redis cache. If not, it will get the object from the database. In the
end, if the object gets from the global cache or database, it will be
set into the context cache.

An object stored in the context cache will only be destroyed after the
context disappeared.
 2023-02-15 21:37:34 +08:00
KN4CK3R 3c59d31bc6
Add API management for issue/pull and comment attachments (#21783) 
Close #14601
Fix #3690

Revive of #14601.
Updated to current code, cleanup and added more read/write checks.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Signed-off-by: Andre Bruch <ab@andrebruch.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Norwin <git@nroo.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
 2022-12-09 14:35:56 +08:00
Lunny Xiao 0a7d3ff786
refactor some functions to support ctx as first parameter (#21878) 
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Lauris BH <lauris@nix.lv>
 2022-12-03 10:48:26 +08:00
flynnnnnnnnnn e81ccc406b
Implement FSFE REUSE for golang files (#21840) 
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
 2022-11-27 18:20:29 +00:00
KN4CK3R 044c754ea5
Add `context.Context` to more methods (#21546) 
This PR adds a context parameter to a bunch of methods. Some helper
`xxxCtx()` methods got replaced with the normal name now.

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
 2022-11-19 16:12:33 +08:00
Lunny Xiao 34283a74e8
Allow detect whether it's in a database transaction for a context.Context (#21756) 
Fix #19513

This PR introduce a new db method `InTransaction(context.Context)`,
and also builtin check on `db.TxContext` and `db.WithTx`.
There is also a new method `db.AutoTx` has been introduced but could be used by other PRs.

`WithTx` will always open a new transaction, if a transaction exist in context, return an error.
`AutoTx` will try to open a new transaction if no transaction exist in context.
That means it will always enter a transaction if there is no error.

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: 6543 <6543@obermui.de>
 2022-11-12 21:18:50 +01:00
techknowlogick 3f7cab42dc
Revert "Do not send notifications for draft releases (#21451)" (#21594) 
This reverts commit a37e8b275d / #21451

Temporarily revert this PR to be able to continue discussion, and
potentially get it into 1.19.0
 2022-10-26 00:00:00 +02:00
delvh 0ebb45cfe7
Replace all instances of fmt.Errorf(%v) with fmt.Errorf(%w) (#21551) 
Found using
`find . -type f -name '*.go' -print -exec vim {} -c
':%s/fmt\.Errorf(\(.*\)%v\(.*\)err/fmt.Errorf(\1%w\2err/g' -c ':wq' \;`

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
 2022-10-24 20:29:17 +01:00
wxiaoguang dcd9fc7ee8
Refactor git command arguments and make all arguments to be safe to be used (#21535) 
Follow #21464

Make all git command arguments strictly safe. Most changes are one-to-one replacing, keep all existing logic.
 2022-10-23 22:44:45 +08:00
KN4CK3R a37e8b275d
Do not send notifications for draft releases (#21451) 
Fixes #21448

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
 2022-10-17 17:33:27 -04:00
KN4CK3R 0e57ff7eee
Add generic set type (#21408) 
This PR adds a generic set type to get rid of maps used as sets.

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
 2022-10-12 13:18:26 +08:00
zeripath e6b3be4608
Add more checks in migration code (#21011) 
When migrating add several more important sanity checks:

* SHAs must be SHAs
* Refs must be valid Refs
* URLs must be reasonable

Signed-off-by: Andrew Thornton <art27@cantab.net>

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <matti@mdranta.net>
 2022-09-04 13:47:56 +03:00
Lunny Xiao 1d8543e7db
Move some files into models' sub packages (#20262) 
* Move some files into models' sub packages

* Move functions

* merge main branch

* Fix check

* fix check

* Fix some tests

* Fix lint

* Fix lint

* Revert lint changes

* Fix error comments

* Fix lint

Co-authored-by: 6543 <6543@obermui.de>
 2022-08-25 10:31:57 +08:00
a1012112796 e3e06d13af
fix permission check for delete tag (#19985) 
fix #19970

by the way, fix some error response about protected tags.

Signed-off-by: a1012112796 <1012112796@qq.com>
 2022-06-16 22:03:03 +02:00
Lunny Xiao 110fc57cbc
Move some code into models/git (#19879) 
* Move access and repo permission to models/perm/access

* fix test

* Move some git related files into sub package models/git

* Fix build

* fix git test

* move lfs to sub package

* move more git related functions to models/git

* Move functions sequence

* Some improvements per @KN4CK3R and @delvh
 2022-06-12 23:51:54 +08:00
Lunny Xiao e09fb30d34
improvement some release related code (#19867) 2022-06-03 02:13:58 -04:00
wxiaoguang 124b072f0b
Remove `git.Command.Run` and `git.Command.RunInDir*` (#19280) 
Follows #19266, #8553, Close #18553, now there are only three `Run..(&RunOpts{})` functions.
 * before: `stdout, err := RunInDir(path)`
 * now: `stdout, _, err := RunStdString(&git.RunOpts{Dir:path})`
 2022-04-01 10:55:30 +08:00
wxiaoguang 7a550b3af2
Use `ctx` instead of `db.DefaultContext` in some packages(routers/services/modules) (#19163) 
* Remove `db.DefaultContext` usage in routers, use `ctx` directly

* Use `ctx` directly if there is one, remove some `db.DefaultContext` in `services`

* Use ctx instead of db.DefaultContext for `cmd` and some `modules` packages

* fix incorrect context usage
 2022-03-22 16:22:54 +01:00
silverwind 2b9df564b8
Lock gofumpt to v0.3.0 and run it (#18866) 
We can't depend on `latest` version of gofumpt because the output will
not be stable across versions. Lock it down to the latest version
released yesterday and run it again.
 2022-02-23 20:16:07 +00:00
6543 3043eb36bf
Delete old git.NewCommand() and use it as git.NewCommandContext() (#18552) 2022-02-06 20:01:47 +01:00
6543 54e9ee37a7
format with gofumpt (#18184) 
* gofumpt -w -l .

* gofumpt -w -l -extra .

* Add linter

* manual fix

* change make fmt
 2022-01-20 18:46:10 +01:00
zeripath 5cb0c9aa0d
Propagate context and ensure git commands run in request context (#17868) 
This PR continues the work in #17125 by progressively ensuring that git
commands run within the request context.

This now means that the if there is a git repo already open in the context it will be used instead of reopening it.

Signed-off-by: Andrew Thornton <art27@cantab.net>
 2022-01-19 23:26:57 +00:00
KN4CK3R 8fa97a25f0
Set HeadCommit when creating tags. (#18116) 
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
 2021-12-29 19:40:57 +08:00
Lunny Xiao 719bddcd76
Move repository model into models/repo (#17933) 
* Some refactors related repository model

* Move more methods out of repository

* Move repository into models/repo

* Fix test

* Fix test

* some improvements

* Remove unnecessary function
 2021-12-10 09:27:50 +08:00
Lunny Xiao a666829a37
Move user related model into models/user (#17781) 
* Move user related model into models/user

* Fix lint for windows

* Fix windows lint

* Fix windows lint

* Move some tests in models

* Merge
 2021-11-24 17:49:20 +08:00
Lunny Xiao fc3d082609
Move attachment into models/repo/ (#17650) 
* Move attachment into models/repo/

* Fix test

* Fix bug
 2021-11-19 21:39:57 +08:00
zeripath 9302eba971
DBContext is just a Context (#17100) 
* DBContext is just a Context

This PR removes some of the specialness from the DBContext and makes it context
This allows us to simplify the GetEngine code to wrap around any context in future
and means that we can change our loadRepo(e Engine) functions to simply take contexts.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix unit tests

Signed-off-by: Andrew Thornton <art27@cantab.net>

* another place that needs to set the initial context

Signed-off-by: Andrew Thornton <art27@cantab.net>

* avoid race

Signed-off-by: Andrew Thornton <art27@cantab.net>

* change attachment error

Signed-off-by: Andrew Thornton <art27@cantab.net>
 2021-09-23 23:45:36 +08:00
Lunny Xiao a4bfef265d
Move db related basic functions to models/db (#17075) 
* Move db related basic functions to models/db

* Fix lint

* Fix lint

* Fix test

* Fix lint

* Fix lint

* revert unnecessary change

* Fix test

* Fix wrong replace string

* Use *Context

* Correct committer spelling and fix wrong replaced words

Co-authored-by: zeripath <art27@cantab.net>
 2021-09-19 19:49:59 +08:00
Lunny Xiao 8d7704b5a2
Fix dump and restore respository (#16698) 
* Fix dump and restore

* return different error message for get commit

* Fix missing delete release attachment when deleting repository

* Fix ci and add some comments

Co-authored-by: zeripath <art27@cantab.net>
 2021-08-30 21:18:15 -04:00
KN4CK3R 44b8b07631
Add tag protection (#15629) 
* Added tag protection in hook.

* Prevent UI tag creation if protected.

* Added settings page.

* Added tests.

* Added suggestions.

* Moved tests.

* Use individual errors.

* Removed unneeded methods.

* Switched delete selector.

* Changed method names.

* No reason to be unique.

* Allow editing of protected tags.

* Removed unique key from migration.

* Added docs page.

* Changed date.

* Respond with 404 to not found tags.

* Replaced glob with regex pattern.

* Added support for glob and regex pattern.

* Updated documentation.

* Changed white* to allow*.

* Fixed edit button link.

* Added cancel button.

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
 2021-06-25 16:28:55 +02:00
Lunny Xiao 8567cba0d9
Implement delete release attachments and update release attachments' name (#14130) 
* Implement delete release attachment

* Add attachments on release edit page

* Fix bug

* Finish del release attachments

* Fix frontend lint

* Fix tests

* Support edit release attachments

* Added tests

* Remove the unnecessary parameter isCreate from UpdateReleaseOrCreatReleaseFromTag

* Rename UpdateReleaseOrCreatReleaseFromTag to UpdateRelease

* Fix middle align
 2021-03-23 00:09:51 +08:00
a1012112796 83cf1a894e
Create tag on ui (#13467) 
Support create single tag directly

support create tag with message from create release ui

Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
 2021-02-28 19:57:45 +00:00
Lunny Xiao 8c086bae6e
Fix wrong publisher id (#14212) 
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: zeripath <art27@cantab.net>
 2021-01-01 16:03:31 -05:00
Cirno the Strongest 53308de0bf
Send webhook when tag is removed via Web UI (#14015) 
* Send webhook when tag is removed via Web UI

* Stray code
 2020-12-16 10:00:46 -06:00
a1012112796 b687707014
Add the tag list page to the release page (#12096) 
* Add the tag list page to the release page

* Add the tags list view
* Add the delete tag way on ui
* Not delete tag and clear message when delete a release

Signed-off-by: a1012112796 <1012112796@qq.com>

* Apply suggestions from code review

Co-authored-by: Cirno the Strongest <1447794+CirnoT@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: zeripath <art27@cantab.net>

* fix ui

* Add to header

* ui

* permission

* ui

* Update options/locale/locale_en-US.ini

Co-authored-by: Lauris BH <lauris@nix.lv>

* tweaks

* monospace commit hashes

* Add read permission check

* fix permission

* fix nit

* Update web_src/less/_base.less

Co-authored-by: silverwind <me@silverwind.io>

* ui fixes

* title tweaks

* fix lint

* fix test

* fix test and some ui nits

* Apply suggestions from code review

Co-authored-by: silverwind <me@silverwind.io>

Co-authored-by: Cirno the Strongest <1447794+CirnoT@users.noreply.github.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: silverwind <me@silverwind.io>
 2020-11-02 18:10:22 -05:00
Lunny Xiao 4df2ed29f2
Refactor: Move PushUpdateOptions (#13363) 
Co-authored-by: Antoine GIRARD <sapk@users.noreply.github.com>
 2020-10-30 22:59:02 +01:00
Lanre Adelowo e918636283
Enhance release list (#6025) 
* show author for releases created outside Gitea UI.

Also show the number of commits behind the default branch for tags created outside the UI

don't show the tag date again for tags pushed to the repo. Since it is already on the sidebar and looks like duplication

* add migration for already existing tags

* update as per review

* fix build

* add space

* fix import statments

* Update models/migrations/v113.go

Co-Authored-By: zeripath <art27@cantab.net>

* Update models/migrations/v114.go

Co-authored-by: 6543 <6543@obermui.de>

* Update services/release/release.go

Co-authored-by: 6543 <6543@obermui.de>

* impruve

* remove dependency on models package

* Close the gitrepos in a defer to ensure that they are closed.

* gofmt

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
 2020-10-21 20:55:25 -04:00
Lunny Xiao 62e6c9bc6c
Add a storage layer for attachments (#11387) 
* Add a storage layer for attachments

* Fix some bug

* fix test

* Fix copyright head and lint

* Fix bug

* Add setting for minio and flags for migrate-storage

* Add documents

* fix lint

* Add test for minio store type on attachments

* fix test

* fix test

* Apply suggestions from code review

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Add warning when storage migrated successfully

* Fix drone

* fix test

* rebase

* Fix test

* display the error on console

* Move minio test to amd64 since minio docker don't support arm64

* refactor the codes

* add trace

* Fix test

* remove log on xorm

* Fi download bug

* Add a storage layer for attachments

* Add setting for minio and flags for migrate-storage

* fix lint

* Add test for minio store type on attachments

* Apply suggestions from code review

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Fix drone

* fix test

* Fix test

* display the error on console

* Move minio test to amd64 since minio docker don't support arm64

* refactor the codes

* add trace

* Fix test

* Add URL function to serve attachments directly from S3/Minio

* Add ability to enable/disable redirection in attachment configuration

* Fix typo

* Add a storage layer for attachments

* Add setting for minio and flags for migrate-storage

* fix lint

* Add test for minio store type on attachments

* Apply suggestions from code review

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Fix drone

* fix test

* Fix test

* display the error on console

* Move minio test to amd64 since minio docker don't support arm64

* don't change unrelated files

* Fix lint

* Fix build

* update go.mod and go.sum

* Use github.com/minio/minio-go/v6

* Remove unused function

* Upgrade minio to v7 and some other improvements

* fix lint

* Fix go mod

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: Tyler <tystuyfzand@gmail.com>
 2020-08-18 12:23:45 +08:00