0
0
Fork 0
mirror of https://codeberg.org/forgejo/docs.git synced 2024-11-24 18:09:26 -05:00

developer: workflow: manual testing expectations

Refs: https://codeberg.org/forgejo/discussions/issues/100#
Refs: https://codeberg.org/forgejo/forgejo/pulls/2265
This commit is contained in:
Earl Warren 2024-01-30 18:51:41 +01:00 committed by Earl Warren
parent 2b0bd8d011
commit b023ac6a9b

View file

@ -146,7 +146,15 @@ _Feature branches_ can contain a number of commits grouped together, for instanc
Observing the granularity of the existing _Feature branches_ is the best way to figure out what works and what does not. It requires adjustments from time to time depending on the number of contributors and the complexity of the Forgejo codebase.
## End to end testing
## Testing
Software is always tested with different methods which are more or less costly. Forgejo has a lot of technical debt and large portions of the codebase are very difficult to test. A [reasonable effort should be made](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md) to improve the situation whenever a change is proposed.
### Automated tests
They do not require manual intervention and can be run as often as necessary from the CI.
Ideally all aspects of a code can be verified with unit tests (e.g. `make unit-test-coverage`). Integration tests may involve third party components such as a database (e.g. `make test-sqlite`).
Tests that require launching a Forgejo instance are found in the [end-to-end repository](https://code.forgejo.org/forgejo/end-to-end).
@ -155,3 +163,9 @@ It is possible to run them on a pull request by setting the `run-end-to-end-test
- builds a Forgejo binary including the pull request and uploads it as [an artifact](https://forgejo.org/docs/next/user/actions/#artifacts) of the PR
- creates a pull request in the [end-to-end repository](https://code.forgejo.org/forgejo/end-to-end) against the [forgejo-pr branch](https://code.forgejo.org/forgejo/end-to-end/src/branch/forgejo-pr)
- run [a workflow](https://code.forgejo.org/forgejo/end-to-end/src/branch/forgejo-pr/.forgejo/workflows/pr.yml) using the Forgejo binary found in the PR artifact
### Manual tests
When the test infrastructure is lacking or the test to verify a change (such as a oneliner) is out of proportion with the effort to create and automated test (as a few days of work may be needed if this is in the web UI), the instructions to manually verify the change is good must be documented in the [forgejo-manual-testing](https://codeberg.org/forgejo/forgejo-manual-testing) repository.
The person who wrote the instructions is expected to run them on a regular basis to ensure there are no regressions. **If this is not done, the regression will always happen over time and will be discovered by end users which by far the [most costly and frustrating method of testing](https://codeberg.org/forgejo/discussions/issues/103).**