This PR do some performance optimzations.
- [x] Add `index` for the column `comment_id` of `Attachment` table to
accelerate query from the database.
- [x] Remove unnecessary database queries when viewing issues. Before
some conditions which id = 0 will be sent to the database
- [x] Remove duplicated load posters
- [x] Batch loading attachements, isread of comments on viewing issue
---------
Co-authored-by: Zettat123 <zettat123@gmail.com>
Conflicts:
models/issues/comment_code.go: function was renamed in Forgejo
models/migrations/migrations.go: migration already ported
Fixes #30235
If the key id "front" byte has a single digit, `%X` is missing the 0
prefix.
` 38D1A3EADDBEA9C` instead of
`038D1A3EADDBEA9C`
When using the `IssuerFingerprint` slice `%X` is enough but I changed it
to `%016X` too to be consistent.
(cherry picked from commit eb505b128c7b9b2459f2a5d20b5740017125178b)
Conflicts:
- models/asymkey/gpg_key_commit_verification.go
Ported the change to models/asymkey/gpg_key_object_verification.go
- Adds setting `EXTERNAL_USER_DISABLE_FEATURES` to disable any supported
user features when login type is not plain
- In general, this is necessary for SSO implementations to avoid
inconsistencies between the external account management and the linked
account
- Adds helper functions to encourage correct use
(cherry picked from commit 59d4aadba5c15d02f3b9f0e61abb7476870c20a5)
Conflicts:
- docs/content/administration/config-cheat-sheet.en-us.md
Removed.
- modules/setting/admin.go
Trivial resolution: pick the newly added struct member.
close #27801
---------
Co-authored-by: silverwind <me@silverwind.io>
(cherry picked from commit 62b073e6f31645e446c7e8d6b5a506f61b47924e)
Conflicts:
- modules/util/util.go
Trivial resolution, only picking the newly introduced function
- routers/api/v1/swagger/options.go
Trivial resolution. We don't have UserBadges, don't pick that part.
- templates/swagger/v1_json.tmpl
Regenerated.
- When the database consistency is being run it would check for any
OAuth2 applications that don't have an existing user. However there are
few special OAuth2 applications that don't have an user set, because
they are global applications.
- This was not taken into account by the database consistency checker
and were removed if the database consistency check was being run with
autofix enabled.
- Take into account to ignore these global OAuth2 applications when
running the database consistency check.
- Add unit tests.
- Ref: https://codeberg.org/Codeberg/Community/issues/1530
- Remove options that currently aren't set
on `GarbageCollectLFSMetaObjectsOptions` and
`IterateLFSMetaObjectsForRepoOptions`.
- Simplify `IterateRepositoryIDsWithLFSMetaObjects` and
`IterateLFSMetaObjectsForRepo`.
- `IterateLFSMetaObjectsForRepo` was previously able to get in a
loop (`gc-lfs` doctor check was able to reproduce this) because the code
expected that the records would be updated to not match the SQL query,
but that wasn't the case. Simply enforce that only records higher than
the latest `id` from the previous iteration are allowed.
- For `gc-lfs` doctor check this was because `UpdatedLessRecentlyThan`
option was not set, which caused that records just marked as active in
the iteration weren't being filtered.
- Add unit tests
- Most likely a regression from 2cc3a6381c.
- The bug with `gc-lfs` was found on Codeberg.
- It was only used to parse old U2F data to webauthn credentials. We
only used the public key and keyhandle. This functiontionality was
reworked to `parseU2FRegistration`.
- Tests are already present, `Test_RemigrateU2FCredentials`.
- If the database returns a error in integration tests, it should be
marked as a failure of the test.
- Ref: https://codeberg.org/forgejo/forgejo/issues/2962 (this should
help with logging the SQL that is resulting in the error).
This is a squashed result of conflict resolution for the following commits from Gitea:
- 36de5b299b
- 9a93b1816e
- 712e19fa6f
- 83850cc479
It is lacking CSS rule for archived labels, though.
Changes in this commit are authored by:
- 6543
- delvh
- silverwind
This is, in large part, a refactoring: we rename `CommitVerification` to
`ObjectVerification`, and adjust `ParseObjectWithSignature` (previously
`ParseCommitWithSignature`) to work on an object, rather than a commit.
This in turn, lets us implement `ParseTagWithSignature` on top of it, so
commit & tag signature verification will share most of the code.
Work sponsored by @glts.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Fix https://github.com/go-gitea/gitea/pull/29874#discussion_r1542227686
- The migration of v292 will miss many projects. These projects will
have no default board. This PR introduced a new migration number and
removed v292 migration.
- This PR also added the missed transactions on project-related
operations.
- Only `SetDefaultBoard` will remove duplicated defaults but not in
`GetDefaultBoard`
(cherry picked from commit 40cdc84b368cce8328b4b49ea5ecf1c5fa040300)
Resolves #29965.
---
Manually tested this by:
- Following the
[installation](https://docs.gitea.com/next/installation/install-with-docker#basics)
guide (but built a local Docker image instead)
- Creating 2 users, one who is the `Owner` of a newly-created repository
and the other a `Collaborator`
- Had the `Collaborator` create a PR that the `Owner` reviews
- `Collaborator` resolves conversation and `Owner` merges PR
And with this change we see that we can no longer see re-request review
button for the `Owner`:
<img width="1351" alt="Screenshot 2024-03-25 at 12 39 18 AM"
src="https://github.com/go-gitea/gitea/assets/60799661/bcd9c579-3cf7-474f-a51e-b436fe1a39a4">
(cherry picked from commit 242b331260925e604150346e61329097d5731e77)
This PR replaces the use of `max( id )`, and instead using ``max(
`index` )`` for determining the latest commit status. Building business
logic over an `auto_increment` primary key like `id` is risky and
there’re already plenty of discussions on the Internet.
There‘s no guarantee for `auto_increment` values to be monotonic,
especially upon failures or with a cluster. In the specific case, we met
the problem of commit statuses being outdated when using TiDB as the
database. As [being
documented](https://docs.pingcap.com/tidb/stable/auto-increment),
`auto_increment` values assigned to an `insert` statement will only be
monotonic on a per server (node) basis.
Closes #30074.
(cherry picked from commit 7443a10fc3d722d3326a0cb7b15b208f907c72d7)
On creation of an empty project (no template) a default board will be
created instead of falling back to the uneditable pseudo-board.
Every project now has to have exactly one default boards. As a
consequence, you cannot unset a board as default, instead you have to
set another board as default. Existing projects will be modified using a
cron job, additionally this check will run every midnight by default.
Deleting the default board is not allowed, you have to set another board
as default to do it.
Fixes #29873
Fixes #14679 along the way
Fixes #29853
Co-authored-by: delvh <dev.lh@web.de>
(cherry picked from commit e5160185ed65fd1c2bcb2fc7dc7e0b5514ddb299)
Conflicts:
options/locale/locale_en-US.ini
trivial conflict because Forgejo strings do not have
surrounding double quotes
Fix #30103
ps: comments has `LoadAttributes`, but maybe considering performance
problem, we don't call it.
(cherry picked from commit ce3c3512265df3b4940672be40065c4fb415ef95)
Help #29999, or its tests cannot pass.
Also, add some comments to clarify the usage of `TxContext`.
I don't check all usages of `TxContext` because there are too many
(almost 140+). It's a better idea to replace them with `WithTx` instead
of checking them one by one. However, that may be another refactoring
PR.
(cherry picked from commit c6c4d66004c70b24abc8048b39b660b8361a0395)
This PR uses `db.ListOptions` instead of `Paginor` to make the code
simpler.
And it also fixed the performance problem when viewing /pulls or
/issues. Before the counting in fact will also do the search.
---------
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: silverwind <me@silverwind.io>
(cherry picked from commit 3f26fe2fa2c7141c9e622297e50a70f3e0003e4d)
- `user_model.DeleteInactiveEmailAddresses` related code was added in
Gogs as part to delete inactive users, however since then the related
code to delete users has changed and this code now already delete email
addresses of the user, it's therefore not needed anymore to
`DeleteInactiveEmailAddresses`.
- The call to `DeleteInactiveEmailAddresses` can actually cause issues.
As the associated user might not have been deleted, because it
was not older than the specified `olderThan` argument. Therefore causing
a database inconsistency and lead to internal server errors if the user
tries to activate their account.
- Adds unit test to verify correct behavior (fails without this patch).
- Currently protected branch rules do not apply to admins, however in
some cases (like in the case of Forgejo project) you might also want to
apply these rules to admins to avoid accidental merges.
- Add new option to configure this on a per-rule basis.
- Adds integration tests.
- Resolves #65
Fixes https://github.com/go-gitea/gitea/issues/28297
This PR also fixed a problem that it needs a database transaction when
removing the WIP title.
---
Resolves #2771
Also partially ports gitea#29783
(cherry picked from commit 17d7ab5ad4ce3d0fbc1251572c22687c237a30b1)
There is no code change. Just move notification list related
structs/functions from one file to another.
---
Resolves #2772
Simply move the moderation code to the new function (which wasn't
changed).
(cherry picked from commit b25eec41eb4d7058be808daefd6fd47eed61c7d3)
The fix against the race incorrectly assumes the sha of the commit being
pushed belongs to the base repository. It finds the highest possible
pull request ID from the head repository instead of looking it up in
the base repository.
Figuring out if a PR was created in the future based on the highest
index of the base repository would require collecting all of them
because there is no way to know in advance which repository may be
involved in the race.
Fixing this race can be done either by:
* Introducing a new field in the pull_request table https://codeberg.org/forgejo/forgejo/pulls/2842
which feels more like a hack than a real solution
* Refactoring the logic
which would be a significant undertaking
The race has been in the codebase for a very long time and manifests
itself in the CI, when events happen in quick succession. The only
concrete manifestation was however fixed by https://codeberg.org/forgejo/forgejo/issues/2009
Since this race now only exists in theory and not in practice, let's
revert this bugous commit until a proper solution is implemented.
Fixes: https://codeberg.org/forgejo/forgejo/issues/2817
This reverts commit 036f1eddc5.
Conflicts:
services/pull/pull.go
This PR adds `setting.Service.DefaultOrgMemberVisible` value to dataset
of user when the initial org creator is being added to the created org.
Fixes #30012.
(cherry picked from commit e3e08dcc5184cdbdac5023fabaafba123a995c3e)
This PR will avoid load pullrequest.Issue twice in pull request list
page. It will reduce x times database queries for those WIP pull
requests.
Partially fix #29585
---------
Co-authored-by: Giteabot <teabot@gitea.io>
(cherry picked from commit 62f8174aa2fae1481c7e17a6afcb731a5b178cd0)
Conflicts:
models/activities/notification_list.go
moved to models/activities/notification.go
`CommitGPGSignature` was originally made to store information about a
commit's GPG signature. Nowadays, it is used to store information about
SSH signatures too, and not just commit signatures, but tag signatures
too.
As such, rename it to `ObjectSignature`, because that describes what it
does a whole lot better.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Because the `git` module did not recognize SSH signed tags, those
signatures ended up in the `notes` column of the `release` table. While
future signatures will not end up there, Forgejo should clean up the old
ones.
This migration does just that: finds all releases that have an SSH
signature, and removes those signatures, preserving the rest of the
note (if any).
While this may seem like an expensive operation, it's only done once,
and even on the largest known Forgejo instance as of this
writing (Codeberg), the number of affected rows are just over a hundred,
a tiny amount all things considered.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Fix #28157
This PR fix the possible bugs about actions schedule.
- Move `UpdateRepositoryUnit` and `SetRepoDefaultBranch` from models to
service layer
- Remove schedules plan from database and cancel waiting & running
schedules tasks in this repository when actions unit has been disabled
or global disabled.
- Remove schedules plan from database and cancel waiting & running
schedules tasks in this repository when default branch changed.
(cherry picked from commit 97292da960)
Conflicts:
modules/actions/github.go
routers/web/repo/setting/default_branch.go
routers/web/repo/setting/setting.go
services/repository/branch.go
services/repository/setting.go
tests/integration/actions_trigger_test.go
Repositories displaying an "Add more..." tab on the header is a neat way
to let people discover they can enable more units. However, displaying
it all the time for repository owners, even when they deliberately do
not want to enable more units gets noisy very fast.
As such, this patch introduces a new setting which lets people disable
this hint under the appearance settings.
Fixes #2378.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
This PR do some loading speed optimization for feeds user interface
pages.
- Load action users batchly but not one by one.
- Load action repositories batchly but not one by one.
- Load action's Repo Owners batchly but not one by one.
- Load action's possible issues batchly but not one by one.
- Load action's possible comments batchly but not one by one.
(cherry picked from commit aed3b53abdd02a3ffbf9e8eb90272ff567333073)
Unlike other async processing in the queue, we should sync branches to
the DB immediately when handling git hook calling. If it fails, users
can see the error message in the output of the git command.
It can avoid potential inconsistency issues, and help #29494.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Regression of #2507, which switched the HEAD from `pr.GetGitRefName()`
to `pr.HeadCommitID` but it had to be `prInfo.HeadCommitID`. Resolves #2656
I was able to reproduce this locally with _some_ pull requests, haven't
been able to get a reproducer trough integration testing.
(cherry picked from commit 25b842df261452a29570ba89ffc3a4842d73f68c)
Conflicts:
routers/web/repo/wiki.go
services/repository/branch.go
services/repository/migrate.go
services/wiki/wiki.go
also apply to Forgejo specific usage of the refactored functions
Refactor the webhook logic, to have the type-dependent processing happen
only in one place.
---
1. An event happens
2. It is pre-processed (depending on the webhook type) and its body is
added to a task queue
3. When the task is processed, some more logic (depending on the webhook
type as well) is applied to make an HTTP request
This means that webhook-type dependant logic is needed in step 2 and 3.
This is cumbersome and brittle to maintain.
Updated webhook flow with this PR:
1. An event happens
2. It is stored as-is and added to a task queue
3. When the task is processed, the event is processed (depending on the
webhook type) to make an HTTP request
So the only webhook-type dependent logic happens in one place (step 3)
which should be much more robust.
- the raw event must be stored in the hooktask (until now, the
pre-processed body was stored)
- to ensure that previous hooktasks are correctly sent, a
`payload_version` is added (version 1: the body has already been
pre-process / version 2: the body is the raw event)
So future webhook additions will only have to deal with creating an
http.Request based on the raw event (no need to adjust the code in
multiple places, like currently).
Moreover since this processing happens when fetching from the task
queue, it ensures that the queuing of new events (upon a `git push` for
instance) does not get slowed down by a slow webhook.
As a concrete example, the PR #19307 for custom webhooks, should be
substantially smaller:
- no need to change `services/webhook/deliver.go`
- minimal change in `services/webhook/webhook.go` (add the new webhook
to the map)
- no need to change all the individual webhook files (since with this
refactor the `*webhook_model.Webhook` is provided as argument)
(cherry picked from commit 26653b196bd1d15c532af41f60351596dd4330bd)
Conflicts:
services/webhook/deliver_test.go
trivial context conflict
Fix #29000
Fix #28685
Fix #18568
Related: #27497
And by the way fix #24036, add a Cancel button there (one line)
(cherry picked from commit 5cddab4f74bbb307ddf13e458c7ac22f93b9283a)
The tests on migration tests failed but CI reports successfully
https://github.com/go-gitea/gitea/actions/runs/7364373807/job/20044685969#step:8:141
This PR will fix the bug on migration v283 and also the CI hidden
behaviour.
The reason is on the Makefile
`GITEA_ROOT="$(CURDIR)" GITEA_CONF=tests/mysql.ini $(GO) test
$(GOTESTFLAGS) -tags='$(TEST_TAGS)' $(MIGRATE_TEST_PACKAGES)` will
return the error exit code.
But
`for pkg in $(shell $(GO) list
code.gitea.io/gitea/models/migrations/...); do \
GITEA_ROOT="$(CURDIR)" GITEA_CONF=tests/mysql.ini $(GO) test
$(GOTESTFLAGS) -tags '$(TEST_TAGS)' $$pkg; \
done`
will not work.
This also fix #29602
(cherry picked from commit 45277486c2c6213b7766b1da708a991cdb1f3565)
Conflicts:
.github/workflows/pull-db-tests.yml
Makefile
models/migrations/v1_22/v283.go
models/migrations/v1_22/v286_test.go
models/migrations/v1_22/v287_test.go
already in Forgejo for the Makefile & CI logic but Gitea changes
otherwise rule
Follow #29522
Administrators should be able to set a user's email address even if the
email address is not in `EMAIL_DOMAIN_ALLOWLIST`
(cherry picked from commit 136dd99e86eea9c8bfe61b972a12b395655171e8)
Fix #27457
Administrators should be able to manually create any user even if the
user's email address is not in `EMAIL_DOMAIN_ALLOWLIST`.
(cherry picked from commit 4fd9c56ed09b31e2f6164a5f534a31c6624d0478)
