Renovate tried to update redis/go-redis, but failed because they changes
the interface, they added two new functions: `BitFieldRO` and
`ObjectFreq`.
Changes:
- Update redis/go-redis
- Run mockgen:
```
mockgen -package mock -destination ./modules/queue/mock/redisuniversalclient.go github.com/redis/go-redis/v9 UniversalClient
```
References:
- https://codeberg.org/forgejo/forgejo/pulls/4009
This updates the mapping definition of the elasticsearch issue indexer backend to use `long` instead of `integer`s wherever the go type is a `int64`. Without it larger instances could run into an issue.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/3982
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Mai-Lapyst <mai-lapyst@noreply.codeberg.org>
Co-committed-by: Mai-Lapyst <mai-lapyst@noreply.codeberg.org>
It is fine to use MockVariableValue to change a setting such as:
defer test.MockVariableValue(&setting.Mirror.Enabled, true)()
But when testing for errors and mocking a function, multiple variants
of the functions will be used, not just one. MockProtect a function
will make sure that when the test fails it always restores a sane
version of the function. For instance:
defer test.MockProtect(&mirror_service.AddPushMirrorRemote)()
mirror_service.AddPushMirrorRemote = mockOne
do some tests that may fail
mirror_service.AddPushMirrorRemote = mockTwo
do more tests that may fail
We wanted to be able to use the IAM role provided by the EC2 instance
metadata in order to access S3 via the Minio configuration. To do this,
a new credentials chain is added that will check the following locations
for credentials when an access key is not provided. In priority order,
they are:
1. MINIO_ prefixed environment variables
2. AWS_ prefixed environment variables
3. a minio credentials file
4. an aws credentials file
5. EC2 instance metadata
(cherry picked from commit c0880e7695346997c6a93f05cd01634cb3ad03ee)
Conflicts:
docs/content/administration/config-cheat-sheet.en-us.md
does not exist in Forgejo
This PR split the `Board` into two parts. One is the struct has been
renamed to `Column` and the second we have a `Template Type`.
But to make it easier to review, this PR will not change the database
schemas, they are just renames. The database schema changes could be in
future PRs.
---------
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: yp05327 <576951401@qq.com>
(cherry picked from commit 98751108b11dc748cc99230ca0fc1acfdf2c8929)
Conflicts:
docs/content/administration/config-cheat-sheet.en-us.md
docs/content/index.en-us.md
docs/content/installation/comparison.en-us.md
docs/content/usage/permissions.en-us.md
non existent files
options/locale/locale_en-US.ini
routers/web/web.go
templates/repo/header.tmpl
templates/repo/settings/options.tmpl
trivial context conflicts
Remove "EncodeSha1", it shouldn't be used as a general purpose hasher
(just like we have removed "EncodeMD5" in #28622)
Rewrite the "time-limited code" related code and write better tests, the
old code doesn't seem quite right.
(cherry picked from commit fb1ad920b769799aa1287441289d15477d9878c5)
Conflicts:
modules/git/utils_test.go
trivial context conflict because sha256 testing in Forgejo has diverged
regression from 767e9634d3. It changed
the parsing of the [admin] section from being derived from the content
of each key with mustMapSetting(rootCfg, "admin", &Admin) to
explicitly listing all keys in the code.
SEND_NOTIFICATION_EMAIL_ON_NEW_USER was not added and therefore
ignored. As a consequence notifications of newly registered users were
never sent.
Summary:
- Move existing test under a `testify` Suite as `baseRedisWithServerTestSuite`
- Those tests require real redis server.
- Add `go.uber.org/mock/mockgen@latest` as dependency
- as a tool (Makefile).
- in the `go.mod` file.
- Mock redis client lives under a `mock` directory under the queue module.
- That mock module has an extra hand-written mock in-memory redis-like struct.
- Add tests using the mock redis client.
- Changed the logic around queue provider creation.
- Now the `getNewQueue` returns a Queue provider directly, not an init
function to create it.
The whole Queue module is close to impossible to test properly because
everything is private, everything goes through a struct route. Because
of that, we can't test for example what keys are used for given queue.
To overcome this, as a first step I removed one step from that hard
route by allowing custom calls to create new queue provider. To achieve
this, I moved the creation logic into the `getNewQueue` (previously it
was `getNewQueueFn`). That changes nothing on that side, everything goes
as before, except the `newXXX` call happens directly in that function
and not outside that.
That made it possible to add extra provider specific parameters to those
function (`newXXX`). For example a client on redis. Calling it through
the `getNewQueue` function, it gets `nil`.
- If the provided client is not `nil`, it will use that instead of the
connection string.
- If it's `nil` (default behaviour), it creates a new redis client as it
did before, no changes to that.
The rest of the provider code is unchanged. All these changes were
required to make it possible to generate mock clients for providers and
use them.
For the tests, the existing two test cases are good with redis server,
and they need some extra helpers, for example to start a new redis
server if required, or waiting on a redis server to be ready to use.
These helpers are only required for test cases using real redis server.
For better isolation, moved existing test under a testify Suite, and
moved them into a new test file called `base_redis_with_server_test.go`
because, well they test the code with server. These tests do exactly the
same as before, calling the same sub-tests the same way as before, the
only change is the structure of the test (remove repetition, scope
server related helper functions).
Finally, we can create unit tests without redis server. The main focus of
this group of tests are higher level overview of operations. With the
mock redis client we can set up expectations about used queue names,
received values, return value to simulate faulty state.
These new unit test functions don't test all functionality, at least
it's not aimed for it now. It's more about the possibility of doing that
and add extra tests around parts we couldn't test before, for example
key.
What extra features can test the new unit test group:
- What is the received key for given queue? For example using `prefix`,
or if all the `SXxx` calls are expected to use `queue_unique` if
it's a unique queue.
- If it's not a unique queue, no `SXxx` functions are called, because
those sets are used only to check if a value is unique or not.
- `HasItem` return `false` always if it's a non-unique queue.
- All functions are called exactly `N` times, and we don't have any
unexpected calls to redis from the code.
Signed-off-by: Victoria Nadasdi <victoria@efertone.me>
For security reasons, scoping access to a redis server via ACL rules is
a good practice. Some parts of the codebase handles prefix like cache[^1]
and session[^2], but the queue module doesn't.
This patch adds this missing functionality to the queue module.
Note about relevant test:
I tried to keep the PR as small as possible (and reasonable), and not
change how the test runs. Updated the existing test to use the same
redis address and basically duplicated the test with the extra flag. It
does NOT test if the keys are correct, it ensures only it works as
expected. To make assertions about the keys, the whole test has to be
updated as the general wrapper doesn't allow the main test to check
anything provider (redis) specific property. That's not something I
wanted to take on now.
[^1]: e4c3c039be/modules/cache/cache_redis.go (L139-L150)
[^2]: e4c3c039be/modules/session/redis.go (L122-L129)
Signed-off-by: Victoria Nadasdi <victoria@efertone.me>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/3836
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Victoria Nadasdi <victoria@efertone.me>
Co-committed-by: Victoria Nadasdi <victoria@efertone.me>
Fix #30923
(cherry picked from commit effb405cae88474c27f5c8322a2627019af1cf64)
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Conflicts:
- modules/setting/oauth2.go
Conflicted due to different ways of logging. Since the log
message is removed anyway, resolved by removing it.
- modules/setting/oauth2_test.go
Manually copied the test added by Gitea.
- routers/install/install.go
Not a conflict per se, but adjusted to use NewJwtSecret().
Resolve #30917
Make the APIs for adding labels and replacing labels support both label
IDs and label names so the
[`actions/labeler`](https://github.com/actions/labeler) action can work
in Gitea.
<img width="600px"
src="https://github.com/go-gitea/gitea/assets/15528715/7835c771-f637-4c57-9ce5-e4fbf56fa0d3"
/>
(cherry picked from commit b3beaed147466739de0c24fd80206b5af8b71617)
Conflicts:
- modules/structs/issue_label.go
Resolved by applying the Gitea change by hand.
- tests/integration/api_issue_label_test.go
Resolved by copying the new tests.
Merging PR may fail because of various problems. The pull request may
have a dirty state because there is no transaction when merging a pull
request. ref
https://github.com/go-gitea/gitea/pull/25741#issuecomment-2074126393
This PR moves all database update operations to post-receive handler for
merging a pull request and having a database transaction. That means if
database operations fail, then the git merging will fail, the git client
will get a fail result.
There are already many tests for pull request merging, so we don't need
to add a new one.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit ebf0c969403d91ed80745ff5bd7dfbdb08174fc7)
Conflicts:
modules/private/hook.go
routers/private/hook_post_receive.go
trivial conflicts because
263a716cb5 * Performance optimization for git push (#30104)
was not cherry-picked and because of
998a431747 Do not update PRs based on events that happened before they existed
More about codespell: https://github.com/codespell-project/codespell .
I personally introduced it to dozens if not hundreds of projects already and so far only positive feedback.
```
❯ grep lint-spell Makefile
@echo " - lint-spell lint spelling"
@echo " - lint-spell-fix lint spelling and fix issues"
lint: lint-frontend lint-backend lint-spell
lint-fix: lint-frontend-fix lint-backend-fix lint-spell-fix
.PHONY: lint-spell
lint-spell: lint-codespell
.PHONY: lint-spell-fix
lint-spell-fix: lint-codespell-fix
❯ git grep lint- -- .forgejo/
.forgejo/workflows/testing.yml: - run: make --always-make -j$(nproc) lint-backend checks-backend # ensure the "go-licenses" make target runs
.forgejo/workflows/testing.yml: - run: make lint-frontend
```
so how would you like me to invoke `lint-codespell` on CI? (without that would be IMHO very suboptimal and let typos sneak in)
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/3270
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
Co-committed-by: Yaroslav Halchenko <debian@onerussian.com>
Fix #30807
reuse functions in services
(cherry picked from commit a50026e2f30897904704895362da0fb12c7e5b26)
Conflicts:
models/issues/issue_update.go
routers/api/v1/repo/issue.go
trivial context conflict because of 'allow setting the update date on issues and comments'
The test had a dependency on `https://api.pwnedpasswords.com` which
caused many failures on CI recently:
```
--- FAIL: TestPassword (2.37s)
pwn_test.go:41: Get "https://api.pwnedpasswords.com/range/e6b6a": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
FAIL
coverage: 82.9% of statements
```
(cherry picked from commit 9235442ba58524c8d12ae54865d583acfa1f439d)
Co-authored-by: silverwind <me@silverwind.io>
(cherry picked from commit be112c1fc30f87a248b30f48e891d1c8c18e8280)
Conflicts:
routers/web/web.go
trivial conflict because of https://codeberg.org/forgejo/forgejo/pulls/1533
Resolve all cases for `unused parameter` and `unnecessary type
arguments`
Related: #30729
---------
Co-authored-by: Giteabot <teabot@gitea.io>
(cherry picked from commit e80466f7349164ce4cf3c07bdac30d736d20f035)
Conflicts:
modules/markup/markdown/transform_codespan.go
modules/setting/incoming_email.go
routers/api/v1/admin/user_badge.go
routers/private/hook_pre_receive.go
tests/integration/repo_search_test.go
resolved by discarding the change, this is linting only and
for the sake of avoiding future conflicts
Adjust the `anyHashPattern` to match URL query parameters too, and
adjust `fullHashPatternProcessor` accordingly.
Includes a test case, and an update to an existing one to account for
the new capture group.
Fixes #3548.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
When mentioning a user, the markup post-processor did not handle the
case where the mentioned user did not exist well: it tried to skip to
the next node, which in turn, ended up skipping the rest of the line.
To fix this, lets skip just the mentioned, but non-existing user, and
continue processing the current node from there.
Fixes #3535.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Suggested by logs in #30729
- Remove `math/rand.Seed`
`rand.Seed is deprecated: As of Go 1.20 there is no reason to call Seed
with a random value.`
- Replace `math/rand.Read`
`rand.Read is deprecated: For almost all use cases, [crypto/rand.Read]
is more appropriate.`
- Replace `math/rand` with `math/rand/v2`, which is available since Go
1.22
(cherry picked from commit 7b8e418da1e082786b844562a05864ec1177ce97)
Should resolve #30642.
Before this commit, we were treating an empty `?sort=` query parameter
as the correct sorting type (which is to sort issues in descending order
by their created UNIX time). But when we perform `sort=latest`, we did
not include this as a type so we would sort by the most recently updated
when reaching the `default` switch statement block.
This commit fixes this by considering the empty string, "latest", and
just any other string that is not mentioned in the switch statement as
sorting by newest.
(cherry picked from commit 9b7af4340c36d3e1888788499d16f83feeb1601b)
Noteable additions:
- `redefines-builtin-id` forbid variable names that shadow go builtins
- `empty-lines` remove unnecessary empty lines that `gofumpt` does not
remove for some reason
- `superfluous-else` eliminate more superfluous `else` branches
Rules are also sorted alphabetically and I cleaned up various parts of
`.golangci.yml`.
(cherry picked from commit 74f0c84fa4245a20ce6fb87dac1faf2aeeded2a2)
Conflicts:
.golangci.yml
apply the linter recommendations to Forgejo code as well
