Functions modifying the labels in the database (DeleteIssueLabel,
NewIssueLabels, NewIssueLabel, ReplaceIssueLabels) need to force
reload them. Instead of:
issue.isLabelsLoaded = false
issue.Labels = nil
if err = issue.LoadLabels(ctx); err != nil {
return err
}
They can now use:
if err = issue.ReloadLabels(ctx); err != nil {
return err
}
When ReplaceIssueLabels calls issue.LoadLabels it was a noop because
issue.isLabelsLoaded is still set to true because of the call to
issue.LoadLabels that was done at the beginning of the function.
Closes https://github.com/go-gitea/gitea/issues/30296
- Adds a DB fixture for actions artifacts
- Adds artifacts test files
- Clears artifacts test files between each run
- Note: I initially initialized the artifacts only for artifacts tests,
but because the files are small it only takes ~8ms, so I changed it to
always run in test setup for simplicity
- Fix some otherwise flaky tests by making them not depend on previous
tests
(cherry picked from commit 66971e591e5dddd5b6dc1572ac48f4e4ab29b8e0)
Conflicts:
- tests/integration/api_actions_artifact_test.go
Conflict resolved by manually changing the tested artifact
name from "artifact" to "artifact-download"
- tests/integration/api_actions_artifact_v4_test.go
Conflict resolved by manually updating the tested artifact
names, and adjusting the test case only present in our tree.
- tests/test_utils.go
Resolved by manually copying the added function.
(cherry picked from commit f4d3aaeeb9e1b11c5495e4608a3f52f316c35758)
Conflicts:
- modules/charset/charset_test.go
Resolved by manually changing a `=` to `:=`, as per the
original patch. Conflict was due to `require.NoError`.
before if it was nonglob each load would try to glob it and the check
that is not glob ... now we only do that once and no future loading will
trigger it
---
*Sponsored by Kithara Software GmbH*
(cherry picked from commit 5d43801b72790ce5862aefdc4520edb06bb4cbba)
When a workflow has
on:
pull_request:
types:
- labeled
- unlabeled
The outcome of the workflow (success or failure) must be associated
with the head sha commit status. Otherwise it cannot be used as a
requirement for merging the pull request (branch protections).
Introduce "idNumber" for each migration, and clarify the difference
between the migration ID number and database version.
---
Conflict resolution: trivial
(cherry picked from commit d70af38447a759d4a935e315e18efa4dd625f655)
- Make use of `test.MockVariableValue` to override variables for the
duration of the test.
- Don't needlessly call `onGiteaRun`, its only needed when a HTTP server
needs to be called by the code.
- When `onGiteaRun` is used, make use of the passed parameters, such as
the passed `*testing.T` variable and `*url.URL` (this also avoids
needing to serve the routers in the test code again).
- Use `(*url.URL).JoinPath` to craft new URLs.
- Don't override `setting.AppURL` & `setting.Database.LogSQL` when its
does not affect the test.
- Add empty fixture files for `FederatedUser` & `FederationHost` so they
are truncated and do not persist between tests.
- Combine review requests comments similairy how labels comments are
combined. If review requests comments were made within 60 seconds of
each other they will be grouped.
- Integration and unit test added.
- Resolves #2774
- The driver being used for PostgreSQL doesn't handle interleaved
queries (you start a query, read some rows and start another query while
you didn't finish that query yet), this is the case with using
`.Iterate` from XORM.
- Switch to a variant of what exist in the current codebase of
`db.Iterate`, which is a simple buffered iteration and doesn't keep
queries open, which allow other database operations to happen.
- Unit test added. This doesn't cover that postgres does not error on
this case, as this is not run with a postgres database.
- Resolves #5696
- Add `recentupdated` to the `OrderByMap`.
- Add integration testing for organization and user repository sorting.
- Resolves #5612
- Regression from 12e23ee199 where the
`recentupdated` case was not added to the map, but was handled
seperately as a fallback. The regression came into affect when
5a0bc35799 also relied on this map but
didn't handle the `recentupdated` case.
(cherry picked from commit 6029d78ab5006e8fb4f42adb5a8c491f19fa7b0a)
Conflicts:
models/user/user.go
services/user/user_test.go
trivial context conflict
tests/integration/user_test.go
discarded entirely because dot may be allowed in Forgejo under
some conditions
- On editting a team, only update the units if the team isn't the
'Owners' team. Otherwise the 'Owners' team end up having all of their
unit access modes set to 'None'; because the request form doesn't send
over any units, as it's simply not shown in the UI.
- Adds a database inconstency check and fix for the case where the
'Owners' team is affected by this bug.
- Adds unit test.
- Adds integration test.
- Resolves #5528
- Regression of https://github.com/go-gitea/gitea/pull/24012
Port of https://github.com/go-gitea/gitea/pull/32204
(cherry picked from commit d6d3c96e6555fc91b3e2ef21f4d8d7475564bb3e)
Conflicts:
routers/api/v1/api.go
services/context/api.go
trivial context conflicts
The Issue and PullRequest list has 3 states:
- open: This lists all open Issues/PullRequests
- closed: This lists all closed Issues/PullRequests
- all: This lists all open and closed Issues/PullRequests
If you want to get to the all state, you need to click Open while in open state or Closed while in closed state, which is very unintuitive. This PR adss a third button to get to this state.
I'm not sure if the eye icon fits well, but I couldn't find a better one.
Tests will be added once #4124 is merged.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4125
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: JakobDev <jakobdev@gmx.de>
Co-committed-by: JakobDev <jakobdev@gmx.de>
Fix #30898
we have an option `SearchByEmail`, so enable it, then we can search user
by email.
Also added a test for it.
(cherry picked from commit 5d6d025c9b8d2abca9ec2bfdc795d1f0c1c6592d)
Fix #31423
(cherry picked from commit f4b8f6fc40ce2869135372a5c6ec6418d27ebfba)
Conflicts:
models/fixtures/comment.yml
comment fixtures have to be shifted because there is one more in Forgejo
I'm new to go and contributing to gitea, your guidance is much
appreciated.
This is meant to solve https://github.com/go-gitea/gitea/issues/13309
Previously, closed issues would not be shown under new issues in the
activity tab, even if they were newly created.
changes:
* Split out newlyCreatedIssues from issuesForActivityStatement to count
both currently open and closed issues.
* Use a seperate function to count active issues to prevent
double-counting issues after the above change.
Result is that new issues that have been closed are shown both under
"new" and "closed".
Signed-off-by: Timon van der Berg <tmnvanderberg@gmail.com>
(cherry picked from commit ebfde845294cc681de6b1fe1adcf27e35f61b89b)
Change word order for issue comment actions
- An attempt to address https://codeberg.org/forgejo/forgejo/issues/2650
Org team permissions improvements
- consistency: added missing dot
- clarity: explain what external units mean
- use dedicated keys to explain the permissions.
- split in read/write permissions
- use explicit labels for accessibility
- ext_wiki.desc and ext_issues.desc are no longer in use.
- For WebAuthn Credential level 3, the `backup_eligible` and
`backup_state` flags are checked if they are consistent with the values
given on login. Forgejo never stored this data, so add a database
migration that makes all webauthn credentials 'legacy' and on the next
first use capture the values of `backup_eligible` and `backup_state`.
As suggested in https://github.com/go-webauthn/webauthn/discussions/219#discussioncomment-10429662
- Adds unit tests.
- Add E2E test.
This adds a new configuration setting: `[quota.default].TOTAL`, which
will be used if no groups are configured for a particular user. The new
option makes it possible to entirely skip configuring quotas via the API
if all that one wants is a total size.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
- Continuation of https://github.com/go-gitea/gitea/pull/18835 (by
@Gusted, so it's fine to change copyright holder to Forgejo).
- Add the option to use SSH for push mirrors, this would allow for the
deploy keys feature to be used and not require tokens to be used which
cannot be limited to a specific repository. The private key is stored
encrypted (via the `keying` module) on the database and NEVER given to
the user, to avoid accidental exposure and misuse.
- CAVEAT: This does require the `ssh` binary to be present, which may
not be available in containerized environments, this could be solved by
adding a SSH client into forgejo itself and use the forgejo binary as
SSH command, but should be done in another PR.
- CAVEAT: Mirroring of LFS content is not supported, this would require
the previous stated problem to be solved due to LFS authentication (an
attempt was made at forgejo/forgejo#2544).
- Integration test added.
- Resolves #4416
- When a comment was updated or deleted and was part of an
pending/ongoing review, it would have triggered a notification, such as
a webhook.
- This patch checks if the comment is part of a pending review and then
does not fire a notification and, in the case of updating a comment,
does not save the content history because this is not necessary if it is
still a "draft" comment given it is a pending comment (there is no need
to see my embarrassing typos).
- Adds integration tests.
- Resolves https://codeberg.org/forgejo/forgejo/issues/4368
Fix #31730
This PR rewrote the function `PublicKeysAreExternallyManaged` with a
simple test. The new function removed the loop to make it more readable.
(cherry picked from commit b491b2104f83ee8fc4956c099c427b339291b3be)
Support compression for Actions logs to save storage space and
bandwidth. Inspired by
https://github.com/go-gitea/gitea/issues/24256#issuecomment-1521153015
The biggest challenge is that the compression format should support
[seekable](https://github.com/facebook/zstd/blob/dev/contrib/seekable_format/zstd_seekable_compression_format.md).
So when users are viewing a part of the log lines, Gitea doesn't need to
download the whole compressed file and decompress it.
That means gzip cannot help here. And I did research, there aren't too
many choices, like bgzip and xz, but I think zstd is the most popular
one. It has an implementation in Golang with
[zstd](https://github.com/klauspost/compress/tree/master/zstd) and
[zstd-seekable-format-go](https://github.com/SaveTheRbtz/zstd-seekable-format-go),
and what is better is that it has good compatibility: a seekable format
zstd file can be read by a regular zstd reader.
This PR introduces a new package `zstd` to combine and wrap the two
packages, to provide a unified and easy-to-use API.
And a new setting `LOG_COMPRESSION` is added to the config, although I
don't see any reason why not to use compression, I think's it's a good
idea to keep the default with `none` to be consistent with old versions.
`LOG_COMPRESSION` takes effect for only new log files, it adds `.zst` as
an extension to the file name, so Gitea can determine if it needs
decompression according to the file name when reading. Old files will
keep the format since it's not worth converting them, as they will be
cleared after #31735.
<img width="541" alt="image"
src="https://github.com/user-attachments/assets/e9598764-a4e0-4b68-8c2b-f769265183c9">
(cherry picked from commit 33cc5837a655ad544b936d4d040ca36d74092588)
Conflicts:
assets/go-licenses.json
go.mod
go.sum
resolved with make tidy
If the assign the pull request review to a team, it did not show the
members of the team in the "requested_reviewers" field, so the field was
null. As a solution, I added the team members to the array.
fix #31764
(cherry picked from commit 94cca8846e7d62c8a295d70c8199d706dfa60e5c)
Now that my colleague just posted a wonderful blog post https://blog.datalad.org/posts/forgejo-runner-podman-deployment/ on forgejo runner, some time I will try to add that damn codespell action to work on CI here ;) meanwhile some typos managed to sneak in and this PR should address them (one change might be functional in a test -- not sure if would cause a fail or not)
### Release notes
- [ ] I do not want this change to show in the release notes.
- [ ] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4857
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
Co-committed-by: Yaroslav Halchenko <debian@onerussian.com>
An instance-wide actor is required for outgoing signed requests that are
done on behalf of the instance, rather than on behalf of other actors.
Such things include updating profile information, or fetching public
keys.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Part of #24256.
Clear up old action logs to free up storage space.
Users will see a message indicating that the log has been cleared if
they view old tasks.
<img width="1361" alt="image"
src="https://github.com/user-attachments/assets/9f0f3a3a-bc5a-402f-90ca-49282d196c22">
Docs: https://gitea.com/gitea/docs/pulls/40
---------
Co-authored-by: silverwind <me@silverwind.io>
(cherry picked from commit 687c1182482ad9443a5911c068b317a91c91d586)
Conflicts:
custom/conf/app.example.ini
routers/web/repo/actions/view.go
trivial context conflict
Fixes #22722
Currently, it is not possible to force push to a branch with branch
protection rules in place. There are often times where this is necessary
(CI workflows/administrative tasks etc).
The current workaround is to rename/remove the branch protection,
perform the force push, and then reinstate the protections.
Provide an additional section in the branch protection rules to allow
users to specify which users with push access can also force push to the
branch. The default value of the rule will be set to `Disabled`, and the
UI is intuitive and very similar to the `Push` section.
It is worth noting in this implementation that allowing force push does
not override regular push access, and both will need to be enabled for a
user to force push.
This applies to manual force push to a remote, and also in Gitea UI
updating a PR by rebase (which requires force push)
This modifies the `BranchProtection` API structs to add:
- `enable_force_push bool`
- `enable_force_push_whitelist bool`
- `force_push_whitelist_usernames string[]`
- `force_push_whitelist_teams string[]`
- `force_push_whitelist_deploy_keys bool`
<img width="943" alt="image"
src="https://github.com/go-gitea/gitea/assets/79623665/7491899c-d816-45d5-be84-8512abd156bf">
branch `test` being a protected branch:
<img width="1038" alt="image"
src="https://github.com/go-gitea/gitea/assets/79623665/57ead13e-9006-459f-b83c-7079e6f4c654">
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 12cb1d2998f2a307713ce979f8d585711e92061c)
Fix #31657.
According to the
[doc](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#onschedule)
of GitHub Actions, The timezone for cron should be UTC, not the local
timezone. And Gitea Actions doesn't have any reasons to change this, so
I think it's a bug.
However, Gitea Actions has extended the syntax, as it supports
descriptors like `@weekly` and `@every 5m`, and supports specifying the
timezone like `TZ=UTC 0 10 * * *`. So we can make it use UTC only when
the timezone is not specified, to be compatible with GitHub Actions, and
also respect the user's specified.
It does break the feature because the times to run tasks would be
changed, and it may confuse users. So I don't think we should backport
this.
## ⚠️ BREAKING ⚠️
If the server's local time zone is not UTC, a scheduled task would run
at a different time after upgrading Gitea to this version.
(cherry picked from commit 21a73ae642b15982a911837775c9583deb47220c)
Fix #31707.
Also related to #31715.
Some Actions resources could has different types of ownership. It could
be:
- global: all repos and orgs/users can use it.
- org/user level: only the org/user can use it.
- repo level: only the repo can use it.
There are two ways to distinguish org/user level from repo level:
1. `{owner_id: 1, repo_id: 2}` for repo level, and `{owner_id: 1,
repo_id: 0}` for org level.
2. `{owner_id: 0, repo_id: 2}` for repo level, and `{owner_id: 1,
repo_id: 0}` for org level.
The first way seems more reasonable, but it may not be true. The point
is that although a resource, like a runner, belongs to a repo (it can be
used by the repo), the runner doesn't belong to the repo's org (other
repos in the same org cannot use the runner). So, the second method
makes more sense.
And the first way is not user-friendly to query, we must set the repo id
to zero to avoid wrong results.
So, #31715 should be right. And the most simple way to fix #31707 is
just:
```diff
- shared.GetRegistrationToken(ctx, ctx.Repo.Repository.OwnerID, ctx.Repo.Repository.ID)
+ shared.GetRegistrationToken(ctx, 0, ctx.Repo.Repository.ID)
```
However, it is quite intuitive to set both owner id and repo id since
the repo belongs to the owner. So I prefer to be compatible with it. If
we get both owner id and repo id not zero when creating or finding, it's
very clear that the caller want one with repo level, but set owner id
accidentally. So it's OK to accept it but fix the owner id to zero.
(cherry picked from commit a33e74d40d356e8f628ac06a131cb203a3609dec)
Fix #26685
If a commit status comes from Gitea Actions and the user cannot access
the repo's actions unit (the user does not have the permission or the
actions unit is disabled), a 404 page will occur after clicking the
"Details" link. We should hide the "Details" link in this case.
<img
src="https://github.com/go-gitea/gitea/assets/15528715/68361714-b784-4bb5-baab-efde4221f466"
width="400px" />
(cherry picked from commit 7dec8de9147b20c014d68bb1020afe28a263b95a)
Conflicts:
routers/web/repo/commit.go
trivial context commit
This is an implementation of a quota engine, and the API routes to
manage its settings. This does *not* contain any enforcement code: this
is just the bedrock, the engine itself.
The goal of the engine is to be flexible and future proof: to be nimble
enough to build on it further, without having to rewrite large parts of
it.
It might feel a little more complicated than necessary, because the goal
was to be able to support scenarios only very few Forgejo instances
need, scenarios the vast majority of mostly smaller instances simply do
not care about. The goal is to support both big and small, and for that,
we need a solid, flexible foundation.
There are thee big parts to the engine: counting quota use, setting
limits, and evaluating whether the usage is within the limits. Sounds
simple on paper, less so in practice!
Quota counting
==============
Quota is counted based on repo ownership, whenever possible, because
repo owners are in ultimate control over the resources they use: they
can delete repos, attachments, everything, even if they don't *own*
those themselves. They can clean up, and will always have the permission
and access required to do so. Would we count quota based on the owning
user, that could lead to situations where a user is unable to free up
space, because they uploaded a big attachment to a repo that has been
taken private since. It's both more fair, and much safer to count quota
against repo owners.
This means that if user A uploads an attachment to an issue opened
against organization O, that will count towards the quota of
organization O, rather than user A.
One's quota usage stats can be queried using the `/user/quota` API
endpoint. To figure out what's eating into it, the
`/user/repos?order_by=size`, `/user/quota/attachments`,
`/user/quota/artifacts`, and `/user/quota/packages` endpoints should be
consulted. There's also `/user/quota/check?subject=<...>` to check
whether the signed-in user is within a particular quota limit.
Quotas are counted based on sizes stored in the database.
Setting quota limits
====================
There are different "subjects" one can limit usage for. At this time,
only size-based limits are implemented, which are:
- `size:all`: As the name would imply, the total size of everything
Forgejo tracks.
- `size:repos:all`: The total size of all repositories (not including
LFS).
- `size:repos:public`: The total size of all public repositories (not
including LFS).
- `size:repos:private`: The total size of all private repositories (not
including LFS).
- `sizeall`: The total size of all git data (including all
repositories, and LFS).
- `sizelfs`: The size of all git LFS data (either in private or
public repos).
- `size:assets:all`: The size of all assets tracked by Forgejo.
- `size:assets:attachments:all`: The size of all kinds of attachments
tracked by Forgejo.
- `size:assets:attachments:issues`: Size of all attachments attached to
issues, including issue comments.
- `size:assets:attachments:releases`: Size of all attachments attached
to releases. This does *not* include automatically generated archives.
- `size:assets:artifacts`: Size of all Action artifacts.
- `size:assets:packages:all`: Size of all Packages.
- `size:wiki`: Wiki size
Wiki size is currently not tracked, and the engine will always deem it
within quota.
These subjects are built into Rules, which set a limit on *all* subjects
within a rule. Thus, we can create a rule that says: "1Gb limit on all
release assets, all packages, and git LFS, combined". For a rule to
stand, the total sum of all subjects must be below the rule's limit.
Rules are in turn collected into groups. A group is just a name, and a
list of rules. For a group to stand, all of its rules must stand. Thus,
if we have a group with two rules, one that sets a combined 1Gb limit on
release assets, all packages, and git LFS, and another rule that sets a
256Mb limit on packages, if the user has 512Mb of packages, the group
will not stand, because the second rule deems it over quota. Similarly,
if the user has only 128Mb of packages, but 900Mb of release assets, the
group will not stand, because the combined size of packages and release
assets is over the 1Gb limit of the first rule.
Groups themselves are collected into Group Lists. A group list stands
when *any* of the groups within stand. This allows an administrator to
set conservative defaults, but then place select users into additional
groups that increase some aspect of their limits.
To top it off, it is possible to set the default quota groups a user
belongs to in `app.ini`. If there's no explicit assignment, the engine
will use the default groups. This makes it possible to avoid having to
assign each and every user a list of quota groups, and only those need
to be explicitly assigned who need a different set of groups than the
defaults.
If a user has any quota groups assigned to them, the default list will
not be considered for them.
The management APIs
===================
This commit contains the engine itself, its unit tests, and the quota
management APIs. It does not contain any enforcement.
The APIs are documented in-code, and in the swagger docs, and the
integration tests can serve as an example on how to use them.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
- add package counter to repo/user/org overview pages
- add go unit tests for repo/user has/count packages
- add many more unit tests for packages model
- fix error for non-existing packages in DeletePackageByID and SetRepositoryLink
Replace a double select with a simple select.
The complication originates from the initial implementation which
deleted packages instead of selecting them. It was justified to
workaround a problem in MySQL. But it is just a waste of resources
when collecting a list of IDs.
- In the spirit of #4635
- Notify the owner when their account is getting enrolled into TOTP. The
message is changed according if they have security keys or not.
- Integration test added.
- Currently if the password, primary mail, TOTP or security keys are
changed, no notification is made of that and makes compromising an
account a bit easier as it's essentially undetectable until the original
person tries to log in. Although other changes should be made as
well (re-authing before allowing a password change), this should go a
long way of improving the account security in Forgejo.
- Adds a mail notification for password and primary mail changes. For
the primary mail change, a mail notification is sent to the old primary
mail.
- Add a mail notification when TOTP or a security keys is removed, if no
other 2FA method is configured the mail will also contain that 2FA is
no longer needed to log into their account.
- `MakeEmailAddressPrimary` is refactored to the user service package,
as it now involves calling the mailer service.
- Unit tests added.
- Integration tests added.
This leverages the existing `sync_external_users` cron job to
synchronize the `IsActive` flag on users who use an OAuth2 provider set
to synchronize. This synchronization is done by checking for expired
access tokens, and using the stored refresh token to request a new
access token. If the response back from the OAuth2 provider is the
`invalid_grant` error code, the user is marked as inactive. However, the
user is able to reactivate their account by logging in the web browser
through their OAuth2 flow.
Also changed to support this is that a linked `ExternalLoginUser` is
always created upon a login or signup via OAuth2.
Ideally, we would also refresh permissions from the configured OAuth
provider (e.g., admin, restricted and group mappings) to match the
implementation of LDAP. However, the OAuth library used for this `goth`,
doesn't seem to support issuing a session via refresh tokens. The
interface provides a [`RefreshToken`
method](https://github.com/markbates/goth/blob/master/provider.go#L20),
but the returned `oauth.Token` doesn't implement the `goth.Session` we
would need to call `FetchUser`. Due to specific implementations, we
would need to build a compatibility function for every provider, since
they cast to concrete types (e.g.
[Azure](https://github.com/markbates/goth/blob/master/providers/azureadv2/azureadv2.go#L132))
---------
Co-authored-by: Kyle D <kdumontnu@gmail.com>
(cherry picked from commit 416c36f3034e228a27258b5a8a15eec4e5e426ba)
Conflicts:
- tests/integration/auth_ldap_test.go
Trivial conflict resolved by manually applying the change.
- routers/web/auth/oauth.go
Technically not a conflict, but the original PR removed the
modules/util import, which in our version, is still in use. Added it
back.
