Backport #31829 by @lunny
Fix #31730
This PR rewrote the function `PublicKeysAreExternallyManaged` with a
simple test. The new function removed the loop to make it more readable.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
(cherry picked from commit 5fa90ad9bc7fe800d657e909462e5e1caefc7193)
`log.Xxx("%v")` is not ideal, this PR adds necessary context messages.
Remove some unnecessary logs.
Co-authored-by: Giteabot <teabot@gitea.io>
(cherry picked from commit 83f83019ef3471b847a300f0821499b3896ec987)
Conflicts:
- modules/util/util.go
Conflict resolved by picking `util.Iif` from 654cfd1dfbd3f3f1d94addee50b6fe2b018a49c3
(cherry picked from commit 492d116b2a468991f44d6d37ec33f918ccbe4514)
Conflicts:
modules/util/util.go
trivial context conflict as the commit is picked from https://codeberg.org/forgejo/forgejo/pulls/3212
Fixes #30235
If the key id "front" byte has a single digit, `%X` is missing the 0
prefix.
` 38D1A3EADDBEA9C` instead of
`038D1A3EADDBEA9C`
When using the `IssuerFingerprint` slice `%X` is enough but I changed it
to `%016X` too to be consistent.
(cherry picked from commit eb505b128c7b9b2459f2a5d20b5740017125178b)
Conflicts:
- models/asymkey/gpg_key_commit_verification.go
Ported the change to models/asymkey/gpg_key_object_verification.go
(cherry picked from commit 63904e2f97)
Backport #30193 by @KN4CK3R
Fixes #30189
Can't verify subkeys if they are not loaded.
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
(cherry picked from commit e581efe238d1f735787b5dbdfcce9a1d73384ef3)
This is, in large part, a refactoring: we rename `CommitVerification` to
`ObjectVerification`, and adjust `ParseObjectWithSignature` (previously
`ParseCommitWithSignature`) to work on an object, rather than a commit.
This in turn, lets us implement `ParseTagWithSignature` on top of it, so
commit & tag signature verification will share most of the code.
Work sponsored by @glts.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
`CommitGPGSignature` was originally made to store information about a
commit's GPG signature. Nowadays, it is used to store information about
SSH signatures too, and not just commit signatures, but tag signatures
too.
As such, rename it to `ObjectSignature`, because that describes what it
does a whole lot better.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
- When someone really wants to avoid sharing their email, they could
configure git to use the noreply email for git commits. However if they
also wanted to use SSH signing, it would not show up as verified as the
noreply email address was technically not an activated email address for
the user.
- Add unit tests for the `ParseCommitWithSSHSignature` function.
- Resolves https://codeberg.org/Codeberg/Community/issues/946
(cherry picked from commit 1685de7eba)
(cherry picked from commit b1e8858de9)
(cherry picked from commit 1a6bf24d28)
(cherry picked from commit 0122943345)
(cherry picked from commit cc83614853)
(cherry picked from commit 429febe0dc)
(cherry picked from commit 58a9c2ebe9)
(cherry picked from commit fef94aff1c)
(cherry picked from commit 5c6ecd7579)
(cherry picked from commit ffa33a82bf)
(cherry picked from commit a97de1d5bb)
(cherry picked from commit 57ab2b4a40)
Introduce the new generic deletion methods
- `func DeleteByID[T any](ctx context.Context, id int64) (int64, error)`
- `func DeleteByIDs[T any](ctx context.Context, ids ...int64) error`
- `func Delete[T any](ctx context.Context, opts FindOptions) (int64,
error)`
So, we no longer need any specific deletion method and can just use
the generic ones instead.
Replacement of #28450
Closes #28450
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Windows-based shells will add a CRLF when piping the token into
ssh-keygen command resulting in
verification error. This resolves #21527.
---------
Co-authored-by: Heiko Besemann <heiko.besemann@qbeyond.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
The function `GetByBean` has an obvious defect that when the fields are
empty values, it will be ignored. Then users will get a wrong result
which is possibly used to make a security problem.
To avoid the possibility, this PR removed function `GetByBean` and all
references.
And some new generic functions have been introduced to be used.
The recommand usage like below.
```go
// if query an object according id
obj, err := db.GetByID[Object](ctx, id)
// query with other conditions
obj, err := db.Get[Object](ctx, builder.Eq{"a": a, "b":b})
```
assert.Fail() will continue to execute the code while assert.FailNow()
not. I thought those uses of assert.Fail() should exit immediately.
PS: perhaps it's a good idea to use
[require](https://pkg.go.dev/github.com/stretchr/testify/require)
somewhere because the assert package's default behavior does not exit
when an error occurs, which makes it difficult to find the root error
reason.
This PR removed `unittest.MainTest` the second parameter
`TestOptions.GiteaRoot`. Now it detects the root directory by current
working directory.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Part of #27065
This reduces the usage of `db.DefaultContext`. I think I've got enough
files for the first PR. When this is merged, I will continue working on
this.
Considering how many files this PR affect, I hope it won't take to long
to merge, so I don't end up in the merge conflict hell.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Without this patch, the setting SSH.StartBuiltinServer decides whether
the native (Go) implementation is used rather than calling 'ssh-keygen'.
It's possible for 'using ssh-keygen' and 'using the built-in server' to
be independent.
In fact, the gitea rootless container doesn't ship ssh-keygen and can be
configured to use the host's SSH server - which will cause the public
key parsing mechanism to break.
This commit changes the decision to be based on SSH.KeygenPath instead.
Any existing configurations with a custom KeygenPath set will continue
to function. The new default value of '' selects the native version. The
downside of this approach is that anyone who has relying on plain
'ssh-keygen' to have special properties will now be using the native
version instead.
I assume the exec-variant is only there because /x/crypto/ssh didn't
support ssh-ed25519 until 2016. I don't see any other reason for using
it so it might be an acceptable risk.
Fixes #23363
EDIT: this message was garbled when I tried to get the commit
description back in.. Trying to reconstruct it:
## ⚠️ BREAKING ⚠️ Users who don't have SSH.KeygenPath
explicitly set and rely on the ssh-keygen binary need to set
SSH.KeygenPath to 'ssh-keygen' in order to be able to continue using it
for public key parsing.
There was something else but I can't remember at the moment.
EDIT2: It was about `make test` and `make lint`. Can't get them to run.
To reproduce the issue, I installed `golang` in `docker.io/node:16` and
got:
```
...
go: mvdan.cc/xurls/v2@v2.4.0: unknown revision mvdan.cc/xurls/v2.4.0
go: gotest.tools/v3@v3.4.0: unknown revision gotest.tools/v3.4.0
...
go: gotest.tools/v3@v3.0.3: unknown revision gotest.tools/v3.0.3
...
go: error loading module requirements
```
Signed-off-by: Leon M. Busch-George <leon@georgemail.eu>
Some bugs caused by less unit tests in fundamental packages. This PR
refactor `setting` package so that create a unit test will be easier
than before.
- All `LoadFromXXX` files has been splited as two functions, one is
`InitProviderFromXXX` and `LoadCommonSettings`. The first functions will
only include the code to create or new a ini file. The second function
will load common settings.
- It also renames all functions in setting from `newXXXService` to
`loadXXXSetting` or `loadXXXFrom` to make the function name less
confusing.
- Move `XORMLog` to `SQLLog` because it's a better name for that.
Maybe we should finally move these `loadXXXSetting` into the `XXXInit`
function? Any idea?
---------
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: delvh <dev.lh@web.de>
Unfortunately #20896 does not completely prevent Data too long issues
and GPGKeyImport needs to be increased too.
Fix #22896
Signed-off-by: Andrew Thornton <art27@cantab.net>
To avoid duplicated load of the same data in an HTTP request, we can set
a context cache to do that. i.e. Some pages may load a user from a
database with the same id in different areas on the same page. But the
code is hidden in two different deep logic. How should we share the
user? As a result of this PR, now if both entry functions accept
`context.Context` as the first parameter and we just need to refactor
`GetUserByID` to reuse the user from the context cache. Then it will not
be loaded twice on an HTTP request.
But of course, sometimes we would like to reload an object from the
database, that's why `RemoveContextData` is also exposed.
The core context cache is here. It defines a new context
```go
type cacheContext struct {
ctx context.Context
data map[any]map[any]any
lock sync.RWMutex
}
var cacheContextKey = struct{}{}
func WithCacheContext(ctx context.Context) context.Context {
return context.WithValue(ctx, cacheContextKey, &cacheContext{
ctx: ctx,
data: make(map[any]map[any]any),
})
}
```
Then you can use the below 4 methods to read/write/del the data within
the same context.
```go
func GetContextData(ctx context.Context, tp, key any) any
func SetContextData(ctx context.Context, tp, key, value any)
func RemoveContextData(ctx context.Context, tp, key any)
func GetWithContextCache[T any](ctx context.Context, cacheGroupKey string, cacheTargetID any, f func() (T, error)) (T, error)
```
Then let's take a look at how `system.GetString` implement it.
```go
func GetSetting(ctx context.Context, key string) (string, error) {
return cache.GetWithContextCache(ctx, contextCacheKey, key, func() (string, error) {
return cache.GetString(genSettingCacheKey(key), func() (string, error) {
res, err := GetSettingNoCache(ctx, key)
if err != nil {
return "", err
}
return res.SettingValue, nil
})
})
}
```
First, it will check if context data include the setting object with the
key. If not, it will query from the global cache which may be memory or
a Redis cache. If not, it will get the object from the database. In the
end, if the object gets from the global cache or database, it will be
set into the context cache.
An object stored in the context cache will only be destroyed after the
context disappeared.
The error reported when a user passes a private ssh key as their ssh
public key is not very nice.
This PR improves this slightly.
Ref #22693
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
- Move the file `compare.go` and `slice.go` to `slice.go`.
- Fix `ExistsInSlice`, it's buggy
- It uses `sort.Search`, so it assumes that the input slice is sorted.
- It passes `func(i int) bool { return slice[i] == target })` to
`sort.Search`, that's incorrect, check the doc of `sort.Search`.
- Conbine `IsInt64InSlice(int64, []int64)` and `ExistsInSlice(string,
[]string)` to `SliceContains[T]([]T, T)`.
- Conbine `IsSliceInt64Eq([]int64, []int64)` and `IsEqualSlice([]string,
[]string)` to `SliceSortedEqual[T]([]T, T)`.
- Add `SliceEqual[T]([]T, T)` as a distinction from
`SliceSortedEqual[T]([]T, T)`.
- Redesign `RemoveIDFromList([]int64, int64) ([]int64, bool)` to
`SliceRemoveAll[T]([]T, T) []T`.
- Add `SliceContainsFunc[T]([]T, func(T) bool)` and
`SliceRemoveAllFunc[T]([]T, func(T) bool)` for general use.
- Add comments to explain why not `golang.org/x/exp/slices`.
- Add unit tests.
Change all license headers to comply with REUSE specification.
Fix #16132
Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
Fix #19513
This PR introduce a new db method `InTransaction(context.Context)`,
and also builtin check on `db.TxContext` and `db.WithTx`.
There is also a new method `db.AutoTx` has been introduced but could be used by other PRs.
`WithTx` will always open a new transaction, if a transaction exist in context, return an error.
`AutoTx` will try to open a new transaction if no transaction exist in context.
That means it will always enter a transaction if there is no error.
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: 6543 <6543@obermui.de>
A lot of our code is repeatedly testing if individual errors are
specific types of Not Exist errors. This is repetitative and unnecesary.
`Unwrap() error` provides a common way of labelling an error as a
NotExist error and we can/should use this.
This PR has chosen to use the common `io/fs` errors e.g.
`fs.ErrNotExist` for our errors. This is in some ways not completely
correct as these are not filesystem errors but it seems like a
reasonable thing to do and would allow us to simplify a lot of our code
to `errors.Is(err, fs.ErrNotExist)` instead of
`package.IsErr...NotExist(err)`
I am open to suggestions to use a different base error - perhaps
`models/db.ErrNotExist` if that would be felt to be better.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
Unfortunately some keys are too big to fix within the 65535 limit of TEXT on MySQL
this causes issues with these large keys.
Therefore increase these fields to MEDIUMTEXT.
Fix #20894
Signed-off-by: Andrew Thornton <art27@cantab.net>
