1
0
Fork 0
mirror of https://codeberg.org/forgejo/forgejo.git synced 2024-12-22 12:54:53 -05:00
forgejo/routers/web
Gusted ac6ece75c9
feat: improve performance of notifications page for MySQL
- For the notifications page the unread and pinned notifications are
gathered for doer those that and are ordered by the updated unix.
MariaDB makes a bad decision (sometimes, for most users it does not make
this decision) with this query, it uses the index for the `updated_unix`
column to speed up this query, however this is not the correct index to
be taking, if the doer does not have more than 20 (the
page size) unread and pinned notifications combined MariaDB will
traverse the whole notifications table before it realizes that there are
no more notifications to be gathered. It instead should use the index
for the `user_id` column (this is what MariaDB already does for most
users), so the list that has to be traversed is limited to the doer's
notifications which is significantly less than the whole notifications
table.
- This is a different approach than what Gitea has taken to solve this
problem, which is to add a index to the (status, userid, updated_unix)
tuple (Ref: https://github.com/go-gitea/gitea/pull/32395). Adding more
and more indexes is not a good way if we can use existing indexes to get
a query to a acceptable performance.
- The code cannot use `db.Find` as it's hard to add a index hint option
specifically for this query and not for the other instances that uses
`activities_model.FindNotificationOptions`.
- Only add a index hint for MySQL as I have not been able to test if
SQLite or PostgreSQL are smart enough to use the better index (as you
need a large enough dataset to test this meaningfully).
- Integration test added to ensure the SQL is run by all databases.

---

Performance numbers (from Codeberg's database - MariaDB
10.11.6-MariaDB-0+deb12u1):

Currently:
```sql
SELECT * FROM `notification` WHERE notification.user_id=26734 AND (notification.status=3 OR notification.status=1) ORDER BY notification.updated_unix DESC LIMIT 20;
(5.731 sec)
+------+-------------+--------------+-------+--------------------------------------------------+-------------------------------+---------+-------+---------+------------+----------+------------+-------------+
| id   | select_type | table        | type  | possible_keys                                    | key                           | key_len | ref   | rows    | r_rows     | filtered | r_filtered | Extra       |
+------+-------------+--------------+-------+--------------------------------------------------+-------------------------------+---------+-------+---------+------------+----------+------------+-------------+
|    1 | SIMPLE      | notification | index | IDX_notification_status,IDX_notification_user_id | IDX_notification_updated_unix | 8       | const | 1376836 | 1474066.00 |    50.03 |       0.00 | Using where |
+------+-------------+--------------+-------+--------------------------------------------------+-------------------------------+---------+-------+---------+------------+----------+------------+-------------+
```

Using the better index:
```sql
SELECT * FROM `notification` USE INDEX (IDX_notification_user_id) WHERE notification.user_id=26734 AND (notification.status=3 OR notification.status=1) ORDER BY notification.updated_unix DESC LIMIT 20;
(0.834 sec)

+------+-------------+--------------+--------+----------------------------------------------------------+--------------------------+---------+----------------------------------+-------+----------+----------+------------+----------------------------------------------+
| id   | select_type | table        | type   | possible_keys                                            | key                      | key_len | ref                              | rows  | r_rows   | filtered | r_filtered | Extra                                        |
+------+-------------+--------------+--------+----------------------------------------------------------+--------------------------+---------+----------------------------------+-------+----------+----------+------------+----------------------------------------------+
|    1 | PRIMARY     | notification | ref    | PRIMARY,IDX_notification_status,IDX_notification_user_id | IDX_notification_user_id | 8       | const                            | 22042 | 10756.00 |    50.03 |       0.02 | Using where; Using temporary; Using filesort |
|    1 | PRIMARY     | notification | eq_ref | PRIMARY                                                  | PRIMARY                  | 8       | gitea_production.notification.id | 1     | 1.00     |   100.00 |     100.00 |                                              |
+------+-------------+--------------+--------+----------------------------------------------------------+--------------------------+---------+----------------------------------+-------+----------+----------+------------+----------------------------------------------+
```
2024-12-04 15:19:59 +01:00
..
admin Merge pull request 'fix: use ValidateEmail as binding across web forms' (#5158) from solomonv/consolidate-email-validation into forgejo 2024-10-21 14:31:32 +00:00
auth feat: Add option to disable builtin authentication. 2024-12-01 15:50:10 +01:00
devtest Move context from modules to services (#29440) 2024-03-06 12:10:43 +08:00
events Move context from modules to services (#29440) 2024-03-06 12:10:43 +08:00
explore Add DISABLE_ORGANIZATIONS_PAGE and DISABLE_CODE_PAGE settings for explore pages and fix an issue related to user search (#32288) 2024-10-27 11:27:40 +01:00
feed fix: don't show truncated comments in RSS/Atom feeds 2024-10-22 15:15:09 +02:00
healthcheck Add health-check test 2024-04-06 00:34:55 +02:00
misc fix: Preview picture not visible on Markdown file (#5781) 2024-11-23 15:00:18 +00:00
org Merge pull request '[gitea] week 2024-47 cherry pick (gitea/main -> forgejo)' (#5997) from earl-warren/wcp/2024-47 into forgejo 2024-11-21 18:15:34 +00:00
repo Merge pull request '[gitea] week 2024-49 cherry pick (gitea/main -> forgejo)' (#6110) from earl-warren/wcp/2024-49 into forgejo 2024-12-03 15:06:55 +00:00
shared feat(UI): add package counter to repo/user/org overview pages 2024-07-31 12:40:24 +02:00
user feat: improve performance of notifications page for MySQL 2024-12-04 15:19:59 +01:00
base.go Fix missing signature key error when pulling Docker images with SERVE_DIRECT enabled (#32365) 2024-11-05 09:33:15 +01:00
githttp.go Move context from modules to services (#29440) 2024-03-06 12:10:43 +08:00
goget.go Move context from modules to services (#29440) 2024-03-06 12:10:43 +08:00
home.go migrate some more "OptionalBool" to "Option[bool]" (#29479) 2024-03-06 12:10:44 +08:00
metrics.go Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
nodeinfo.go Move context from modules to services (#29440) 2024-03-06 12:10:43 +08:00
swagger_json.go Move context from modules to services (#29440) 2024-03-06 12:10:43 +08:00
web.go Add a "summary card" to issues & PRs for consumption by OpenGraph clients (#6053) 2024-11-29 15:02:03 +00:00
webfinger.go Enable more revive linter rules (#30608) 2024-04-28 15:39:00 +02:00