- Remove a unused dependency. This dependency was added to handle YAML
'frontmatter' meta, parsing them and converting them to a table or
details in the resulting HTML. As can be read in the issue that reported
the behavior of YAML frontmatter being rendered literally,
https://github.com/go-gitea/gitea/issues/5377.
- It's an unused dependency as the codebase since then moved on to do this YAML
parsing and rendering on their own, this was implemented in
812cfd0ad9.
- Adds unit tests that was related to this functionality, to proof the
codebase already handles this and to prevent regressions.
- Update the `github.com/buildkite/terminal-to-html/v3` dependency from
version v3.10.1 to v3.13.0.
- Version v3.12.0 introduced an incompatible change, the return type of
`AsHTML` changed from `[]byte` to `string`. That same version also
introduced streaming mode
https://github.com/buildkite/terminal-to-html/pull/126, which allows us
to avoid reading the whole input into memory.
- Closes #4313
Vulnerability #1: GO-2024-2947
Leak of sensitive information to log files in
github.com/hashicorp/go-retryablehttp
More info: https://pkg.go.dev/vuln/GO-2024-2947
Module: github.com/hashicorp/go-retryablehttp
Found in: github.com/hashicorp/go-retryablehttp@v0.7.5
Fixed in: github.com/hashicorp/go-retryablehttp@v0.7.7
Example traces found:
#1: services/migrations/gitlab.go:500:74: migrations.GitlabDownloader.GetComments calls gitlab.DiscussionsService.ListMergeRequestDiscussions, which eventually calls retryablehttp.Client.Do
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
| code.forgejo.org/f3/gof3/v3 | require | minor | `v3.3.1` -> `v3.4.0` |
---
### Configuration
📅 **Schedule**: Branch creation - "before 4am" (UTC), Automerge - "before 4am" (UTC).
🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
---
This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MDkuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQwOS4xIiwidGFyZ2V0QnJhbmNoIjoiZm9yZ2VqbyIsImxhYmVscyI6WyJkZXBlbmRlbmN5LXVwZ3JhZGUiXX0=-->
Co-authored-by: Twenty Panda <twenty-panda@posteo.com>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4196
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Renovate Bot <forgejo-renovate-action@forgejo.org>
Co-committed-by: Renovate Bot <forgejo-renovate-action@forgejo.org>
It is not possible to tell vulncheck that Forgejo is not affected by
CVE-2024-0406. Use a mirror of the repository to do that.
Refs: https://github.com/mholt/archiver/issues/404
Renovate tried to update redis/go-redis, but failed because they changes
the interface, they added two new functions: `BitFieldRO` and
`ObjectFreq`.
Changes:
- Update redis/go-redis
- Run mockgen:
```
mockgen -package mock -destination ./modules/queue/mock/redisuniversalclient.go github.com/redis/go-redis/v9 UniversalClient
```
References:
- https://codeberg.org/forgejo/forgejo/pulls/4009
Summary:
- Move existing test under a `testify` Suite as `baseRedisWithServerTestSuite`
- Those tests require real redis server.
- Add `go.uber.org/mock/mockgen@latest` as dependency
- as a tool (Makefile).
- in the `go.mod` file.
- Mock redis client lives under a `mock` directory under the queue module.
- That mock module has an extra hand-written mock in-memory redis-like struct.
- Add tests using the mock redis client.
- Changed the logic around queue provider creation.
- Now the `getNewQueue` returns a Queue provider directly, not an init
function to create it.
The whole Queue module is close to impossible to test properly because
everything is private, everything goes through a struct route. Because
of that, we can't test for example what keys are used for given queue.
To overcome this, as a first step I removed one step from that hard
route by allowing custom calls to create new queue provider. To achieve
this, I moved the creation logic into the `getNewQueue` (previously it
was `getNewQueueFn`). That changes nothing on that side, everything goes
as before, except the `newXXX` call happens directly in that function
and not outside that.
That made it possible to add extra provider specific parameters to those
function (`newXXX`). For example a client on redis. Calling it through
the `getNewQueue` function, it gets `nil`.
- If the provided client is not `nil`, it will use that instead of the
connection string.
- If it's `nil` (default behaviour), it creates a new redis client as it
did before, no changes to that.
The rest of the provider code is unchanged. All these changes were
required to make it possible to generate mock clients for providers and
use them.
For the tests, the existing two test cases are good with redis server,
and they need some extra helpers, for example to start a new redis
server if required, or waiting on a redis server to be ready to use.
These helpers are only required for test cases using real redis server.
For better isolation, moved existing test under a testify Suite, and
moved them into a new test file called `base_redis_with_server_test.go`
because, well they test the code with server. These tests do exactly the
same as before, calling the same sub-tests the same way as before, the
only change is the structure of the test (remove repetition, scope
server related helper functions).
Finally, we can create unit tests without redis server. The main focus of
this group of tests are higher level overview of operations. With the
mock redis client we can set up expectations about used queue names,
received values, return value to simulate faulty state.
These new unit test functions don't test all functionality, at least
it's not aimed for it now. It's more about the possibility of doing that
and add extra tests around parts we couldn't test before, for example
key.
What extra features can test the new unit test group:
- What is the received key for given queue? For example using `prefix`,
or if all the `SXxx` calls are expected to use `queue_unique` if
it's a unique queue.
- If it's not a unique queue, no `SXxx` functions are called, because
those sets are used only to check if a value is unique or not.
- `HasItem` return `false` always if it's a non-unique queue.
- All functions are called exactly `N` times, and we don't have any
unexpected calls to redis from the code.
Signed-off-by: Victoria Nadasdi <victoria@efertone.me>
replace reply with forgejos forked version
If plain text is selected as the message format in e.g. Apple Mail, the inline attachments are no longer at the end of the mail, but instead directly where they are in the mail. When parsing the mail, these inline attachments are replaced by "--". The new reply version no longer cuts the text at the first "--".
Tests for this are present in reply (7dc5750c6d).
Fixes https://codeberg.org/forgejo/forgejo/issues/3496#issuecomment-1798416
---
Additionally, I reduced the allocations for the inline attachments.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/3747
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Beowulf <beowulf@beocode.eu>
Co-committed-by: Beowulf <beowulf@beocode.eu>
The test had a dependency on `https://api.pwnedpasswords.com` which
caused many failures on CI recently:
```
--- FAIL: TestPassword (2.37s)
pwn_test.go:41: Get "https://api.pwnedpasswords.com/range/e6b6a": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
FAIL
coverage: 82.9% of statements
```
(cherry picked from commit 9235442ba58524c8d12ae54865d583acfa1f439d)
- It was only used to parse old U2F data to webauthn credentials. We
only used the public key and keyhandle. This functiontionality was
reworked to `parseU2FRegistration`.
- Tests are already present, `Test_RemigrateU2FCredentials`.
- Remove `gitea.com/lunny/dingtalk_webhook` as dependency, we only use
two structs which are small enough to be recreated in Forgejo and don't
need to rely on the dependency.
- Existing tests (thanks @oliverpool) prove that this has no effect.
This adds new lexers and includes some fixes. See
https://github.com/alecthomas/chroma/releases/tag/v2.13.0 for the full
changelog.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
(cherry picked from commit 3e7ae79f99ef0e5ba3d1201c38f491121ea2a156)
- Update github.com/bufbuild/connect-go to
https://github.com/connectrpc/connect-go.
- This is a fork that's actively maintained and is recommend by the
original library. Looking at the recent release notes, it looks like
going in the right direction what one would expect of a library, no
strange features being added, lots of improvements.
- There's still an indirect dependency by
`code.gitea.io/actions-proto-go` on a old version of `connect-go`.
- The [rupture](https://github.com/ethantkoenig/rupture) dependency was
essentially outdated in the sense it was using old version of
dependencies.
- The usage by Forgejo was rather a small portion, so that portion is
now vendored (with its tests).
- Removes old dependencies from go.sum (less dependencies is better for
reviewing what the heck we're importing). Just to note that they were
likely not being used by Go's build process (according to
https://go.dev/ref/mod#minimal-version-selection), so it's really a
matter of formal cleaning up dependencies we don't use and therefor
don't want to download and be in our go.sum.
(cherry picked from commit aa72a5f009)
Conflicts:
go.sum
https://codeberg.org/forgejo/forgejo/pulls/2148
(cherry picked from commit fbe8d65f0b)
(cherry picked from commit e18debcb6a)
Conflicts:
go.sum
https://codeberg.org/forgejo/forgejo/pulls/2245
(cherry picked from commit 8c43c2ada8)
Mainly for MySQL/MSSQL.
It is important for Gitea to use case-sensitive database charset
collation. If the database is using a case-insensitive collation, Gitea
will show startup error/warning messages, and show the errors/warnings
on the admin panel's Self-Check page.
Make `gitea doctor convert` work for MySQL to convert the collations of
database & tables & columns.
* Fix #28131
## ⚠️ BREAKING ⚠️
It is not quite breaking, but it's highly recommended to convert the
database&table&column to a consistent and case-sensitive collation.
Fix https://github.com/go-gitea/gitea/pull/28547#issuecomment-1867740842
Since https://gitea.com/xorm/xorm/pulls/2383 merged, xorm now supports
UPDATE JOIN.
To keep consistent from different databases, xorm use
`engine.Join().Update`, but the actural generated SQL are different
between different databases.
For MySQL, it's `UPDATE talbe1 JOIN table2 ON join_conditions SET xxx
Where xxx`.
For MSSQL, it's `UPDATE table1 SET xxx FROM TABLE1, TABLE2 WHERE
join_conditions`.
For SQLITE per https://www.sqlite.org/lang_update.html, sqlite support
`UPDATE table1 SET xxx FROM table2 WHERE join conditions` from
3.33.0(2020-8-14).
POSTGRES is the same as SQLITE.
Update golang.org/x/crypto for CVE-2023-48795 and update other packages.
`go-git` is not updated because it needs time to figure out why some
tests fail.
Replace `github.com/gogs/cron` with `github.com/go-co-op/gocron` as the
former package is not maintained for many years.
---------
Co-authored-by: delvh <dev.lh@web.de>
Replace #10912
And there are many new tests to cover the CLI behavior
There were some concerns about the "option order in hook scripts"
(https://github.com/go-gitea/gitea/pull/10912#issuecomment-1137543314),
it's not a problem now. Because the hook script uses `/gitea hook
--config=/app.ini pre-receive` format. The "config" is a global option,
it can appear anywhere.
----
## ⚠️ BREAKING ⚠️
This PR does it best to avoid breaking anything. The major changes are:
* `gitea` itself won't accept web's options: `--install-port` / `--pid`
/ `--port` / `--quiet` / `--verbose` .... They are `web` sub-command's
options.
* Use `./gitea web --pid ....` instead
* `./gitea` can still run the `web` sub-command as shorthand, with
default options
* The sub-command's options must follow the sub-command
* Before: `./gitea --sub-opt subcmd` might equal to `./gitea subcmd
--sub-opt` (well, might not ...)
* After: only `./gitea subcmd --sub-opt` could be used
* The global options like `--config` are not affected
The package `github.com/nfnt/resize` is deprecated and archived by the
author. `github.com/oliamb/cutter` is not maintained since 2018. We
could use `golang.org/x/image/draw` instead.
Bumping `github.com/golang-jwt/jwt` from v4 to v5.
`github.com/golang-jwt/jwt` v5 is bringing some breaking changes:
- standard `Valid()` method on claims is removed. It's replaced by
`ClaimsValidator` interface implementing `Validator()` method instead,
which is called after standard validation. Gitea doesn't seem to be
using this logic.
- `jwt.Token` has a field `Valid`, so it's checked in `ParseToken`
function in `services/auth/source/oauth2/token.go`
---------
Co-authored-by: Giteabot <teabot@gitea.io>
Regression: https://github.com/go-gitea/gitea/pull/24805
Closes: #25945
- Disallow `javascript`, `vbscript` and `data` (data uri images still
work) url schemes even if all other schemes are allowed
- Fixed older `cbthunderlink` tests
---------
Co-authored-by: delvh <dev.lh@web.de>
The recent change on xorm for `Sync` is it will not warn when database
have columns which is not listed on struct. So we just need this warn
logs when `Sync` the whole database but not in the migrations Sync.
This PR will remove almost unnecessary warning logs on migrations.
Now below logs in CI will disappear.
```log
2023/06/23 17:51:32 models/db/engine.go:191:InitEngineWithMigration() [W] Table gtestschema.project has column creator_id but struct has not related field
2023/06/23 17:51:32 models/db/engine.go:191:InitEngineWithMigration() [W] Table gtestschema.project has column is_closed but struct has not related field
2023/06/23 17:51:32 models/db/engine.go:191:InitEngineWithMigration() [W] Table gtestschema.project has column board_type but struct has not related field
2023/06/23 17:51:32 models/db/engine.go:191:InitEngineWithMigration() [W] Table gtestschema.project has column type but struct has not related field
2023/06/23 17:51:32 models/db/engine.go:191:InitEngineWithMigration() [W] Table gtestschema.project has column closed_date_unix but struct has not related field
2023/06/23 17:51:32 models/db/engine.go:191:InitEngineWithMigration() [W] Table gtestschema.project has column created_unix but struct has not related field
2023/06/23 17:51:32 models/db/engine.go:191:InitEngineWithMigration() [W] Table gtestschema.project has column updated_unix but struct has not related field
2023/06/23 17:51:32 models/db/engine.go:191:InitEngineWithMigration() [W] Table gtestschema.project has column card_type but struct has not related field
```
close #24540
related:
- Protocol: https://gitea.com/gitea/actions-proto-def/pulls/9
- Runner side: https://gitea.com/gitea/act_runner/pulls/201
changes:
- Add column of `labels` to table `action_runner`, and combine the value
of `agent_labels` and `custom_labels` column to `labels` column.
- Store `labels` when registering `act_runner`.
- Update `labels` when `act_runner` starting and calling `Declare`.
- Users cannot modify the `custom labels` in edit page any more.
other changes:
- Store `version` when registering `act_runner`.
- If runner is latest version, parse version from `Declare`. But older
version runner still parse version from request header.
Co-authored-by: @awkwardbunny
This PR adds a Debian package registry. You can follow [this
tutorial](https://www.baeldung.com/linux/create-debian-package) to build
a *.deb package for testing. Source packages are not supported at the
moment and I did not find documentation of the architecture "all" and
how these packages should be treated.
---------
Co-authored-by: Brian Hong <brian@hongs.me>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Users can now upload `webp` images.
Browsers supporting webp images then display this as the avatar of this
user (every major browser except IE).
---------
Co-authored-by: silverwind <me@silverwind.io>
None of the features of `unrolled/render` package is used.
The Golang builtin "html/template" just works well. Then we can improve
our HTML render to resolve the "$.root.locale.Tr" problem as much as
possible.
Next step: we can have a template render pool (by Clone), then we can
inject global functions with dynamic context to every `Execute` calls.
Then we can use `{{Locale.Tr ....}}` directly in all templates , no need
to pass the `$.root.locale` again and again.
Update replace:
```diff
- replace github.com/nektos/act => gitea.com/gitea/act v0.234.2-0.20230131074955-e46ede1b1744
+ replace github.com/nektos/act => gitea.com/gitea/act v0.243.1
```
Update require:
```diff
- github.com/nektos/act v0.0.0
+ github.com/nektos/act v0.2.43
```
Actually, `v0.2.43` doesn't work, it will be replaced by `gitea/act`, so
it's OK to put any version here. But `gitea/act` is based on
`nektos/act`, so keeping the right upstream version will make security
dependabot help.
BTW, the [security
report](https://github.com/go-gitea/gitea/security/dependabot/20) is
false positive, we don't use the artifact server in act, see #22738.
This PR does a bulk update of a lot of our go deps.
I have not included nektos/act and xorm for the following reasons:
* Xorm updates can sometimes be complex and I'd rather do that in a
separate PR
* I think people more update with the actions code should double check
that the latest nektos/act library works correctly.
---------
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
This notably brings support for GOARCH=loong64, among other fixes.
---------
Signed-off-by: WANG Xuerui <xen0n@gentoo.org>
Co-authored-by: zeripath <art27@cantab.net>
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.4.0 to
0.7.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="8e2b117aee"><code>8e2b117</code></a>
http2/hpack: avoid quadratic complexity in hpack decoding</li>
<li><a
href="547e7edf38"><code>547e7ed</code></a>
http2: avoid referencing ResponseWrite.Write parameter after
returning</li>
<li><a
href="39940adcaa"><code>39940ad</code></a>
html: parse comments per HTML spec</li>
<li><a
href="87ce33ecb4"><code>87ce33e</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="415cb6d518"><code>415cb6d</code></a>
all: fix some comments</li>
<li><a
href="7e3c19ca52"><code>7e3c19c</code></a>
all: correct typos in comments</li>
<li><a
href="296f09aa38"><code>296f09a</code></a>
http2: case insensitive handling for 100-continue</li>
<li><a
href="f8411da775"><code>f8411da</code></a>
nettest: fix tests on dragonfly and js/wasm</li>
<li><a
href="8e0e7d8d38"><code>8e0e7d8</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="7805fdc37d"><code>7805fdc</code></a>
http2: rewrite inbound flow control tracking</li>
<li>Additional commits viewable in <a
href="https://github.com/golang/net/compare/v0.4.0...v0.7.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: delvh <leon@kske.dev>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
#21937 implemented only basic events based on name because of `act`'s
limitation. So I sent a PR to parse all possible events details in
https://gitea.com/gitea/act/pulls/11 and it merged. The ref
documentation is
https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows
This PR depends on that and make more detail responses for `push` events
and `pull_request` events. And it lefts more events there for future
PRs.
---------
Co-authored-by: Jason Song <i@wolfogre.com>
This PR just consumes the
[hcaptcha](https://gitea.com/jolheiser/hcaptcha) and
[haveibeenpwned](https://gitea.com/jolheiser/pwn) modules directly into
Gitea.
Also let this serve as a notice that I'm fine with transferring my
license (which was already MIT) from my own name to "The Gitea Authors".
Signed-off-by: jolheiser <john.olheiser@gmail.com>
closes #13585
fixes #9067
fixes #2386
ref #6226
ref #6219
fixes #745
This PR adds support to process incoming emails to perform actions.
Currently I added handling of replies and unsubscribing from
issues/pulls. In contrast to #13585 the IMAP IDLE command is used
instead of polling which results (in my opinion 😉) in cleaner code.
Procedure:
- When sending an issue/pull reply email, a token is generated which is
present in the Reply-To and References header.
- IMAP IDLE waits until a new email arrives
- The token tells which action should be performed
A possible signature and/or reply gets stripped from the content.
I added a new service to the drone pipeline to test the receiving of
incoming mails. If we keep this in, we may test our outgoing emails too
in future.
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
