foster/forgejo-docs
0
0
Fork 0
mirror of https://codeberg.org/forgejo/docs.git synced 2024-11-21 17:36:59 -05:00
Code Issues Activity

docs: admin: storage: create a new section

Browse source
This commit is contained in:
Earl Warren 2023-08-08 00:03:00 +02:00 committed by Caesar Schinas
parent bb6b1a39b5
commit d1b6f00b45
No known key found for this signature in database
GPG key ID: AE9108461BEA5ACF
3 changed files with 143 additions and 135 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

148
admin/config-cheat-sheet.md
View file

@ -474,8 +474,8 @@ Configuration at `[queue]` will set defaults for queues with overrides for indiv
- `LENGTH`: **100**: Maximal queue size before channel queues block
- `BATCH_LENGTH`: **20**: Batch data before passing to the handler
- `CONN_STR`: **redis://127.0.0.1:6379/0**: Connection string for the redis queue type. For `redis-cluster` use `redis+cluster://127.0.0.1:6379/0`. Options can be set using query params. Similarly, LevelDB options can also be set using: **leveldb://relative/path?option=value** or **leveldb:///absolute/path?option=value**, and will override `DATADIR`
- `QUEUE_NAME`: **_queue**: The suffix for default redis and disk queue name. Individual queues will default to **`name`**`QUEUE_NAME` but can be overridden in the specific `queue.name` section.
- `SET_NAME`: **_unique**: The suffix that will be added to the default redis and disk queue `set` name for unique queues. Individual queues will default to **`name`**`QUEUE_NAME`_`SET_NAME`_ but can be overridden in the specific `queue.name` section.
- `QUEUE_NAME`: **\_queue**: The suffix for default redis and disk queue name. Individual queues will default to **`name`**`QUEUE_NAME` but can be overridden in the specific `queue.name` section.
- `SET_NAME`: **\_unique**: The suffix that will be added to the default redis and disk queue `set` name for unique queues. Individual queues will default to **`name`**`QUEUE_NAME`_`SET_NAME`_ but can be overridden in the specific `queue.name` section.
- `MAX_WORKERS`: **(dynamic)**: Maximum number of worker go-routines for the queue. Default value is "CpuNum/2" clipped to between 1 and 10.
Forgejo creates the following non-unique queues:
@ -800,18 +800,6 @@ Default templates for project boards:
- `ALLOWED_TYPES`: **.csv,.docx,.fodg,.fodp,.fods,.fodt,.gif,.gz,.jpeg,.jpg,.log,.md,.mov,.mp4,.odf,.odg,.odp,.ods,.odt,.patch,.pdf,.png,.pptx,.svg,.tgz,.txt,.webm,.xls,.xlsx,.zip**: Comma-separated list of allowed file extensions (`.zip`), mime types (`text/plain`) or wildcard type (`image/*`, `audio/*`, `video/*`). Empty value or `*/*` allows all types.
- `MAX_SIZE`: **4**: Maximum size (MB).
- `MAX_FILES`: **5**: Maximum number of attachments that can be uploaded at once.
- `STORAGE_TYPE`: **local**: Storage type for attachments, `local` for local disk or `minio` for s3 compatible object storage service, default is `local` or other name defined with `[storage.xxx]`
- `SERVE_DIRECT`: **false**: Allows the storage driver to redirect to authenticated URLs to serve files directly. Currently, only Minio/S3 is supported via signed URLs, local does nothing.
- `PATH`: **data/attachments**: Path to store attachments only available when STORAGE_TYPE is `local`
- `MINIO_ENDPOINT`: **localhost:9000**: Minio endpoint to connect only available when STORAGE_TYPE is `minio`
- `MINIO_ACCESS_KEY_ID`: Minio accessKeyID to connect only available when STORAGE_TYPE is `minio`
- `MINIO_SECRET_ACCESS_KEY`: Minio secretAccessKey to connect only available when STORAGE_TYPE is `minio`
- `MINIO_BUCKET`: **gitea**: Minio bucket to store the attachments only available when STORAGE_TYPE is `minio`
- `MINIO_LOCATION`: **us-east-1**: Minio location to create bucket only available when STORAGE_TYPE is `minio`
- `MINIO_BASE_PATH`: **attachments/**: Minio base path on the bucket only available when STORAGE_TYPE is `minio`
- `MINIO_USE_SSL`: **false**: Minio enabled ssl only available when STORAGE_TYPE is `minio`
- `MINIO_INSECURE_SKIP_VERIFY`: **false**: Minio skip SSL verification available when STORAGE_TYPE is `minio`
- `MINIO_CHECKSUM_ALGORITHM`: **default**: Minio checksum algorithm: `default` (for MinIO or AWS S3) or `md5` (for Cloudflare or Backblaze)
## Log (`log`)
@ -1218,128 +1206,20 @@ WARNING: Changing the settings below can break federation.
- `DEFAULT_INTERVAL`: **8h**: Default interval between each check
- `MIN_INTERVAL`: **10m**: Minimum interval for checking. (Must be >1m).
## LFS (`lfs`)
Storage configuration for lfs data. It will be derived from default `[storage]` or
`[storage.xxx]` when set `STORAGE_TYPE` to `xxx`. When derived, the default of `PATH`
is `data/lfs` and the default of `MINIO_BASE_PATH` is `lfs/`.
- `STORAGE_TYPE`: **local**: Storage type for lfs, `local` for local disk or `minio` for s3 compatible object storage service or other name defined with `[storage.xxx]`
- `SERVE_DIRECT`: **false**: Allows the storage driver to redirect to authenticated URLs to serve files directly. Currently, only Minio/S3 is supported via signed URLs, local does nothing.
- `PATH`: **./data/lfs**: Where to store LFS files, only available when `STORAGE_TYPE` is `local`. If not set it fall back to deprecated LFS_CONTENT_PATH value in [server] section.
- `MINIO_ENDPOINT`: **localhost:9000**: Minio endpoint to connect only available when `STORAGE_TYPE` is `minio`
- `MINIO_ACCESS_KEY_ID`: Minio accessKeyID to connect only available when `STORAGE_TYPE` is `minio`
- `MINIO_SECRET_ACCESS_KEY`: Minio secretAccessKey to connect only available when `STORAGE_TYPE is` `minio`
- `MINIO_BUCKET`: **gitea**: Minio bucket to store the lfs only available when `STORAGE_TYPE` is `minio`
- `MINIO_LOCATION`: **us-east-1**: Minio location to create bucket only available when `STORAGE_TYPE` is `minio`
- `MINIO_BASE_PATH`: **lfs/**: Minio base path on the bucket only available when `STORAGE_TYPE` is `minio`
- `MINIO_USE_SSL`: **false**: Minio enabled ssl only available when `STORAGE_TYPE` is `minio`
- `MINIO_INSECURE_SKIP_VERIFY`: **false**: Minio skip SSL verification available when STORAGE_TYPE is `minio`
## Storage (`storage`)
Default storage configuration for attachments, lfs, avatars, repo-avatars, repo-archive, packages, actions_log, actions_artifact.
| storage | default base path | app.ini sections |
| ----------------- | ------------------ | -------------------------------------------------- |
| attachments | attachments/ | [attachments] or [storage.attachements] |
| lfs | lfs/ | [lfs] or [storage.lfs] |
| avatars | avatars/ | [avatars] or [storage.avatars] |
| repo-avatars | repo-avatars/ | [repo-avatars] or [storage.repo-avatars] |
| repo-archive | repo-archive/ | [repo-archive] or [storage.repo-archive] |
| packages | packages/ | [packages] or [storage.packages] |
| actions_log | actions_log/ | [actions_log] or [storage.actions_log] |
| actions_artifacts | actions_artifacts/ | [actions_artifacts] or [storage.actions_artifacts] |
- `STORAGE_TYPE`: **local**: Storage type, `local` for local disk or `minio` for s3 compatible object storage service.
- `SERVE_DIRECT`: **false**: Allows the storage driver to redirect to authenticated URLs to serve files directly. Currently, only Minio/S3 is supported via signed URLs, local does nothing.
- `MINIO_ENDPOINT`: **localhost:9000**: Minio endpoint to connect only available when `STORAGE_TYPE` is `minio`
- `MINIO_ACCESS_KEY_ID`: Minio accessKeyID to connect only available when `STORAGE_TYPE` is `minio`
- `MINIO_SECRET_ACCESS_KEY`: Minio secretAccessKey to connect only available when `STORAGE_TYPE is` `minio`
- `MINIO_BUCKET`: **gitea**: Minio bucket to store the data only available when `STORAGE_TYPE` is `minio`
- `MINIO_LOCATION`: **us-east-1**: Minio location to create bucket only available when `STORAGE_TYPE` is `minio`
- `MINIO_USE_SSL`: **false**: Minio enabled ssl only available when `STORAGE_TYPE` is `minio`
- `MINIO_INSECURE_SKIP_VERIFY`: **false**: Minio skip SSL verification available when STORAGE_TYPE is `minio`
The recommanded storage configuration for minio like below:
```ini
[storage]
STORAGE_TYPE = minio
; Minio endpoint to connect only available when STORAGE_TYPE is `minio`
MINIO_ENDPOINT = localhost:9000
; Minio accessKeyID to connect only available when STORAGE_TYPE is `minio`
MINIO_ACCESS_KEY_ID =
; Minio secretAccessKey to connect only available when STORAGE_TYPE is `minio`
MINIO_SECRET_ACCESS_KEY =
; Minio bucket to store the attachments only available when STORAGE_TYPE is `minio`
MINIO_BUCKET = gitea
; Minio location to create bucket only available when STORAGE_TYPE is `minio`
MINIO_LOCATION = us-east-1
; Minio enabled ssl only available when STORAGE_TYPE is `minio`
MINIO_USE_SSL = false
; Minio skip SSL verification available when STORAGE_TYPE is `minio`
MINIO_INSECURE_SKIP_VERIFY = false
SERVE_DIRECT = true
```
Defaultly every storage has their default base path like below
| storage | default base path |
| ----------------- | ------------------ |
| attachments | attachments/ |
| lfs | lfs/ |
| avatars | avatars/ |
| repo-avatars | repo-avatars/ |
| repo-archive | repo-archive/ |
| packages | packages/ |
| actions_log | actions_log/ |
| actions_artifacts | actions_artifacts/ |
And bucket, basepath or `SERVE_DIRECT` could be special or overrided, if you want to use a different you can:
```ini
[storage.actions_log]
MINIO_BUCKET = forgejo_actions_log
SERVE_DIRECT = true
MINIO_BASE_PATH = my_actions_log/ ; default is actions_log/ if blank
```
If you want to customerize a different storage for `lfs` if above default storage defined
```ini
[lfs]
STORAGE_TYPE = my_minio
[storage.my_minio]
STORAGE_TYPE = minio
; Minio endpoint to connect only available when STORAGE_TYPE is `minio`
MINIO_ENDPOINT = localhost:9000
; Minio accessKeyID to connect only available when STORAGE_TYPE is `minio`
MINIO_ACCESS_KEY_ID =
; Minio secretAccessKey to connect only available when STORAGE_TYPE is `minio`
MINIO_SECRET_ACCESS_KEY =
; Minio bucket to store the attachments only available when STORAGE_TYPE is `minio`
MINIO_BUCKET = forgejo
; Minio location to create bucket only available when STORAGE_TYPE is `minio`
MINIO_LOCATION = us-east-1
; Minio enabled ssl only available when STORAGE_TYPE is `minio`
MINIO_USE_SSL = false
; Minio skip SSL verification available when STORAGE_TYPE is `minio`
MINIO_INSECURE_SKIP_VERIFY = false
```
## Repository Archive Storage (`storage.repo-archive`)
Configuration for repository archive storage. It will inherit from default `[storage]` or
`[storage.xxx]` when set `STORAGE_TYPE` to `xxx`. The default of `PATH`
is `data/repo-archive` and the default of `MINIO_BASE_PATH` is `repo-archive/`.
- `STORAGE_TYPE`: **local**: Storage type for repo archive, `local` for local disk or `minio` for s3 compatible object storage service or other name defined with `[storage.xxx]`
- `SERVE_DIRECT`: **false**: Allows the storage driver to redirect to authenticated URLs to serve files directly. Currently, only Minio/S3 is supported via signed URLs, local does nothing.
- `PATH`: **./data/repo-archive**: Where to store archive files, only available when `STORAGE_TYPE` is `local`.
- `MINIO_ENDPOINT`: **localhost:9000**: Minio endpoint to connect only available when `STORAGE_TYPE` is `minio`
- `MINIO_ACCESS_KEY_ID`: Minio accessKeyID to connect only available when `STORAGE_TYPE` is `minio`
- `MINIO_SECRET_ACCESS_KEY`: Minio secretAccessKey to connect only available when `STORAGE_TYPE is` `minio`
- `MINIO_BUCKET`: **gitea**: Minio bucket to store the lfs only available when `STORAGE_TYPE` is `minio`
- `MINIO_LOCATION`: **us-east-1**: Minio location to create bucket only available when `STORAGE_TYPE` is `minio`
- `MINIO_BASE_PATH`: **repo-archive/**: Minio base path on the bucket only available when `STORAGE_TYPE` is `minio`
- `MINIO_USE_SSL`: **false**: Minio enabled ssl only available when `STORAGE_TYPE` is `minio`
- `MINIO_INSECURE_SKIP_VERIFY`: **false**: Minio skip SSL verification available when STORAGE_TYPE is `minio`
## Repository Archives (`repo-archive`)
- `STORAGE_TYPE`: **local**: Storage type for actions logs, `local` for local disk or `minio` for s3 compatible object storage service, default is `local` or other name defined with `[storage.xxx]`
- `MINIO_BASE_PATH`: **repo-archive/**: Minio base path on the bucket only available when STORAGE_TYPE is `minio`
The storage configuration for attachments, lfs, avatars, repo-avatars, repo-archive, packages, actions_log, actions_artifact is [explained in detail in a dedicated section](../storage/).
## Proxy (`proxy`)
@ -1359,8 +1239,6 @@ PROXY_HOSTS = *.github.com
- `ENABLED`: **false**: Enable/Disable actions
- `DEFAULT_ACTIONS_URL`: **https://code.forgejo.org**: Default address to get action plugins, e.g. the default value means downloading from "https://code.forgejo.org/actions/checkout" for "uses: actions/checkout@v3"
- `STORAGE_TYPE`: **local**: Storage type for actions logs, `local` for local disk or `minio` for s3 compatible object storage service, default is `local` or other name defined with `[storage.xxx]`
- `MINIO_BASE_PATH`: **actions_log/**: Minio base path on the bucket only available when STORAGE_TYPE is `minio`
## Other (`other`)

1
admin/index.md
View file

@ -8,6 +8,7 @@ These documents are targeted to people who run Forgejo on their machines.
- [Installation](./installation/)
- [Database Preparation](./database-preparation/)
- [Configuration Cheat Sheet](./config-cheat-sheet/)
- [Storage settings](./storage/)
- [Upgrade guide](./upgrade/)
- [Reverse Proxy](./reverse-proxy/)
- [Email setup](./email-setup/)

129
admin/storage.md Normal file
View file

@ -0,0 +1,129 @@
---
title: 'Storage settings'
license: 'CC-BY-SA-4.0'
---
The storage for each subsystem (`attachments`, `lfs`, `avatars`,
`repo-avatars`, `repo-archive`, `packages`, `actions_log`,
`actions_artifact`) is defined in `app.ini`. It can either be on disk
(`local`) or using a MinIO server (`minio`). The default is `local`
storage, using the following hierarchy under the `WORK_PATH` directory:
| storage | default base path | app.ini sections |
| ----------------- | ------------------ | -------------------------------------------------- |
| attachments | attachments/ | [attachments] or [storage.attachements] |
| lfs | lfs/ | [lfs] or [storage.lfs] |
| avatars | avatars/ | [avatars] or [storage.avatars] |
| repo-avatars | repo-avatars/ | [repo-avatars] or [storage.repo-avatars] |
| repo-archive | repo-archive/ | [repo-archive] or [storage.repo-archive] |
| packages | packages/ | [packages] or [storage.packages] |
| actions_log | actions_log/ | [actions_log] or [storage.actions_log] |
| actions_artifacts | actions_artifacts/ | [actions_artifacts] or [storage.actions_artifacts] |
For instance if `WORK_PATH` was `/appdata`, the default directory to
store attachments would be `/appdata/attachements`.
## Overriding the defaults
These defaults can be modified for all subsystems in the `[storage]`
section. For instance setting:
```
[storage]
PATH = /mystorage
```
would change the default for storing attachements to
`/mystorage/attachments`. It is also possible to change these settings
for each subsystem in a `[storage.XXXX]` section. For instance setting:
```
[storage]
PATH = /mystorage
[storage.attachments]
PATH = /otherstorage/attachements
```
would store attachments in `/otherstorage/attachements` while `lfs`
files would be stored in `/mystorage/lfs`.
Finally, instead of using `[storage.XXXX]` it is also possible to use
`[XXXX]` as a shorthand.
## Storage type
The value of `STORAGE_TYPE` can be `local` (the default) or `minio`. For instance:
```
[storage]
STORAGE_TYPE = minio
```
Will use `minio` for all subsystems (`attachements`, `lfs`, etc.)
instead of storing them on disk. Each storage type has its own
settings, as explained below.
## `local` storage
There is just one setting when the `STORAGE_TYPE` is set to `local`,
`PATH`. For instance:
```
[storage]
STORAGE_TYPE = local
PATH = /mystorage
```
If the value of PATH for the `XXXX` subsystem is relative, it is
constructed as follows:
- The default base path is `WORK_PATH` (for instance `/appdata`)
- If `[storage].PATH` is relative (for instance `storage`), the default base path becomes `WORK_PATH`/`[storage].PATH` (for instance `/appdata/storage`)
- If `[storage.XXXX].PATH` is relative, the path becomes `WORK_PATH`/`[storage].PATH`/`[storage.XXXX].PATH` (for instance`/appdata/storage/lfs`)
It is recommended to always set the `PATH` values to an absolute path
name because it is easier to understand and maintain.
## `minio` storage
When the `STORAGE_TYPE` is set to `minio`, the settings available in
all sections (`[storage]`, `[storage.XXXX]` and `[XXXX]`) are:
- `SERVE_DIRECT`: **false**: Allows the storage driver to redirect to authenticated URLs to serve files directly. Only supported via signed URLs.
- `MINIO_ENDPOINT`: **localhost:9000**: Minio endpoint to connect.
- `MINIO_ACCESS_KEY_ID`: Minio accessKeyID to connect.
- `MINIO_SECRET_ACCESS_KEY`: Minio secretAccessKey to connect.
- `MINIO_BUCKET`: **gitea**: Minio bucket to store the data.
- `MINIO_LOCATION`: **us-east-1**: Minio location to create bucket.
- `MINIO_USE_SSL`: **false**: Minio enabled ssl.
- `MINIO_INSECURE_SKIP_VERIFY`: **false**: Minio skip SSL verification.
One setting is only available in the `[storage.XXXX]` or `[XXXX]` sections:
- `MINIO_BASE_PATH`: defaults to the `default base path` of the `XXXX`
subsystem (see the table above) and is a relative path within the
MinIO bucket defined by `MINIO_BUCKET`.
## Sections precedence
The sections in which a setting is found have the following priority:
- [XXXX] is first
- [storage.XXXX] is second
- [storage] is last
For instance:
```
[storage]
PATH = /last
[storage.attachments]
PATH = /second
[attachments]
PATH = /first
```
Will set the value of `PATH` for attachements to `/first`.