foster/forgejo-docs
0
0
Fork 0
mirror of https://codeberg.org/forgejo/docs.git synced 2024-11-28 18:42:51 -05:00
Code Issues Activity

Rework contributor test instructions with new developers in mind

Browse source
This commit is contained in:
Otto Richter 2024-09-08 13:16:56 +02:00 committed by Otto
parent 830595982f
commit f25e842fec
1 changed files with 121 additions and 22 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

143
docs/contributor/testing.md
View file

@ -3,22 +3,126 @@ title: 'Testing'
license: 'CC-BY-SA-4.0' license: 'CC-BY-SA-4.0'
--- ---
A [reasonable effort should be made to test changes](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md) If you want to contribute to Forgejo,
proposed to Forgejo. keep in mind that we ask you to do a
[reasonable effort to test your changes](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md).
## Running the tests from sources Why would we do this?
Your submission to Forgejo could be wasted if you add it,
but no one checks that it still works in a few months, or a few years.
We want your contribution to be meaningful and to last.
To run automated frontend and backend tests: We cannot test all the behaviour of Forgejo.
It is complex, and certain aspects are only important to a handful of users.
To reduce the burden on maintainer,
you should do your best to make testing easy and automated.
## What is a "reasonable" effort?
There is no simple answer to this,
but we want to see that you gave your best.
It will be decided on a case-by-case basis by the maintainers.
Some changes are easy to write a test for.
Other tests can be really hard and challenging.
This guide tries to provide you some starting points,
and if you don't know how to continue,
please submit your contribution and ask for help in writing a test.
We will happily assist you.
## How to write a good test?
Let's list some basic things when writing software tests.
Creating _good tests_ can be a creative challenge,
and you'll benefit from experience
(both from writing tests and working with the Forgejo codebase).
These basic rules should hopefully guide you the way.
### Test the actual change
Some parts of the codebase don't bring extensive testing yet.
Asking you to cover all the behaviour with tests would often be too much.
If in doubt, try to focus on the change you made:
- Which behaviour did you change?
- Can you demonstrate that something works differently?
- Can you write a test that fails before,
and passes after your actual contribution?
Take a look at [this example](https://codeberg.org/forgejo/forgejo/pulls/5110/files#diff-f1c02e9204a32c192cc594c01dfb737e14bc4601).
It tests a very simple change in diff generation.
It does not add tests for all the (edge) cases of diff generation,
but it provides a simple verification that the diff is generated on larger words,
not letter-by-letter,
by adding samples that are affected by the change.
### Test the edge cases
If you test different input, consider the behaviour at edge cases.
For example, what happens with an empty string?
If you want to test a size limitation,
do not only test content that is "way too large",
also test content that is a perfect fit,
and the first content size that should be rejected.
### Get inspiration or help
Please don't hesitate to reach out for help or ask for feedback.
In some cases, good knowledge of the codebase is helpful
(e.g. to think of testing your feature's compatibility with another feature you never heard of).
We recommend that you take inspiration from existing tests.
Take a look at the Git history around your files
(or the kind of tests such as e2e/integration)
to see who made recent changes.
They have high chances of being more idiomatic and modern than the average test code,
and feel free to ping people directly for help.
## Getting Started
There are three kinds of tests in Forgejo.
### Unit tests
These live next to the code in files suffixed
`_test.go` (for backend code)
or `.test.js` (for frontend code).
They test small portions of the code,
mostly functions that process data and have no side-effects.
Use these where possible.
Run these with:
```bash ```bash
make test make test
``` ```
## Frontend tests ### Integration tests
See [the e2e README](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md). Find these in `tests/integration`.
They test more complex behaviour of the app
(such as performing an action and seeing that related activity is recorded).
They are mostly useful when a behaviour is hard to test with isolated unit tests.
## Interactive testing There is more detailed information in the
[integration test README](https://codeberg.org/forgejo/forgejo/src/tests/integration/README.md).
### End-to-end / browser tests
Find these in `tests/e2e`.
They run real browsers and test the frontend of Forgejo.
Use them to verify that things look like they should,
or that pressing a specific button does what it is supposed to do.
You can also run an automated accessibility analysis.
There is more detailed information in the
[e2e test README](https://codeberg.org/forgejo/forgejo/src/tests/e2e/README.md).
## Interactive testing during development
During development, you'll probably want to interact with Forgejo to see if your changes work.
To run and continuously rebuild when the source files change: To run and continuously rebuild when the source files change:
@ -26,18 +130,9 @@ To run and continuously rebuild when the source files change:
TAGS='sqlite sqlite_unlock_notify' make watch TAGS='sqlite sqlite_unlock_notify' make watch
``` ```
> **NOTE:** do not set the `bindata` tag such as in `TAGS="bindata" make watch` or the browser may fail to load pages with an error like `Failed to load asset` > **NOTE:**
> Do not set the `bindata` tag such as in `TAGS="bindata" make watch`.
## Manual run of the binary > The browser may fail to load pages with an error like `Failed to load asset`.
After following the [steps to compile from source](../from-source/), a `forgejo` binary will be available
in the working directory.
It can be tested from this directory or moved to a directory with test data. When Forgejo is
launched manually from command line, it can be killed by pressing `Ctrl + C`.
```bash
./forgejo web
```
## Automated tests ## Automated tests
@ -49,6 +144,9 @@ directory.
### In the end-to-end repository ### In the end-to-end repository
> **Note:**
> Do not confuse this kind of testing with the "e2e" (frontend) tests in the Forgejo repository.
Some tests are best served by running Forgejo from a compiled binary, Some tests are best served by running Forgejo from a compiled binary,
for instance to verify the result of a workflow run by the [Forgejo runner](https://code.forgejo.org/forgejo/runner). They can be for instance to verify the result of a workflow run by the [Forgejo runner](https://code.forgejo.org/forgejo/runner). They can be
run by adding the `run-end-to-end-test` label to the pull request. It will: run by adding the `run-end-to-end-test` label to the pull request. It will:
@ -68,8 +166,9 @@ forgejo-runner exec --workflows .forgejo/workflows/testing.yml
## Manual testing ## Manual testing
When the change to be tested lacks the proper framework, the manual When the change to be tested lacks the proper framework,
test steps must be documented in the description of the pull request. we sometimes add manual testing instructions as a last resort.
The exact test steps must be documented in the description of the pull request.
Changes that are associated with manual tests must be labeled Changes that are associated with manual tests must be labeled
labeled ["test/manual"](https://codeberg.org/forgejo/forgejo/issues?labels=201028). ["test/manual"](https://codeberg.org/forgejo/forgejo/issues?labels=201028).