mirror of
https://codeberg.org/forgejo/docs.git
synced 2024-11-24 18:09:26 -05:00
Rework contributor test instructions with new developers in mind
This commit is contained in:
Otto Richter
Otto
parent
830595982f
commit
f25e842fec
1 changed files with 121 additions and 22 deletions
|
|
@ -3,22 +3,126 @@ title: 'Testing'
|
|||
license: 'CC-BY-SA-4.0'
|
||||
---
|
||||
|
||||
A [reasonable effort should be made to test changes](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md)
|
||||
proposed to Forgejo.
|
||||
If you want to contribute to Forgejo,
|
||||
keep in mind that we ask you to do a
|
||||
[reasonable effort to test your changes](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md).
|
||||
|
||||
## Running the tests from sources
|
||||
Why would we do this?
|
||||
Your submission to Forgejo could be wasted if you add it,
|
||||
but no one checks that it still works in a few months, or a few years.
|
||||
We want your contribution to be meaningful and to last.
|
||||
|
||||
To run automated frontend and backend tests:
|
||||
We cannot test all the behaviour of Forgejo.
|
||||
It is complex, and certain aspects are only important to a handful of users.
|
||||
To reduce the burden on maintainer,
|
||||
you should do your best to make testing easy and automated.
|
||||
|
||||
## What is a "reasonable" effort?
|
||||
|
||||
There is no simple answer to this,
|
||||
but we want to see that you gave your best.
|
||||
It will be decided on a case-by-case basis by the maintainers.
|
||||
|
||||
Some changes are easy to write a test for.
|
||||
Other tests can be really hard and challenging.
|
||||
This guide tries to provide you some starting points,
|
||||
and if you don't know how to continue,
|
||||
please submit your contribution and ask for help in writing a test.
|
||||
We will happily assist you.
|
||||
|
||||
## How to write a good test?
|
||||
|
||||
Let's list some basic things when writing software tests.
|
||||
Creating _good tests_ can be a creative challenge,
|
||||
and you'll benefit from experience
|
||||
(both from writing tests and working with the Forgejo codebase).
|
||||
|
||||
These basic rules should hopefully guide you the way.
|
||||
|
||||
### Test the actual change
|
||||
|
||||
Some parts of the codebase don't bring extensive testing yet.
|
||||
Asking you to cover all the behaviour with tests would often be too much.
|
||||
If in doubt, try to focus on the change you made:
|
||||
|
||||
- Which behaviour did you change?
|
||||
- Can you demonstrate that something works differently?
|
||||
- Can you write a test that fails before,
|
||||
and passes after your actual contribution?
|
||||
|
||||
Take a look at [this example](https://codeberg.org/forgejo/forgejo/pulls/5110/files#diff-f1c02e9204a32c192cc594c01dfb737e14bc4601).
|
||||
It tests a very simple change in diff generation.
|
||||
It does not add tests for all the (edge) cases of diff generation,
|
||||
but it provides a simple verification that the diff is generated on larger words,
|
||||
not letter-by-letter,
|
||||
by adding samples that are affected by the change.
|
||||
|
||||
### Test the edge cases
|
||||
|
||||
If you test different input, consider the behaviour at edge cases.
|
||||
For example, what happens with an empty string?
|
||||
If you want to test a size limitation,
|
||||
do not only test content that is "way too large",
|
||||
also test content that is a perfect fit,
|
||||
and the first content size that should be rejected.
|
||||
|
||||
### Get inspiration or help
|
||||
|
||||
Please don't hesitate to reach out for help or ask for feedback.
|
||||
In some cases, good knowledge of the codebase is helpful
|
||||
(e.g. to think of testing your feature's compatibility with another feature you never heard of).
|
||||
|
||||
We recommend that you take inspiration from existing tests.
|
||||
Take a look at the Git history around your files
|
||||
(or the kind of tests such as e2e/integration)
|
||||
to see who made recent changes.
|
||||
They have high chances of being more idiomatic and modern than the average test code,
|
||||
and feel free to ping people directly for help.
|
||||
|
||||
## Getting Started
|
||||
|
||||
There are three kinds of tests in Forgejo.
|
||||
|
||||
### Unit tests
|
||||
|
||||
These live next to the code in files suffixed
|
||||
`_test.go` (for backend code)
|
||||
or `.test.js` (for frontend code).
|
||||
|
||||
They test small portions of the code,
|
||||
mostly functions that process data and have no side-effects.
|
||||
Use these where possible.
|
||||
|
||||
Run these with:
|
||||
|
||||
```bash
|
||||
make test
|
||||
```
|
||||
|
||||
## Frontend tests
|
||||
### Integration tests
|
||||
|
||||
See [the e2e README](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md).
|
||||
Find these in `tests/integration`.
|
||||
They test more complex behaviour of the app
|
||||
(such as performing an action and seeing that related activity is recorded).
|
||||
They are mostly useful when a behaviour is hard to test with isolated unit tests.
|
||||
|
||||
## Interactive testing
|
||||
There is more detailed information in the
|
||||
[integration test README](https://codeberg.org/forgejo/forgejo/src/tests/integration/README.md).
|
||||
|
||||
### End-to-end / browser tests
|
||||
|
||||
Find these in `tests/e2e`.
|
||||
They run real browsers and test the frontend of Forgejo.
|
||||
Use them to verify that things look like they should,
|
||||
or that pressing a specific button does what it is supposed to do.
|
||||
You can also run an automated accessibility analysis.
|
||||
|
||||
There is more detailed information in the
|
||||
[e2e test README](https://codeberg.org/forgejo/forgejo/src/tests/e2e/README.md).
|
||||
|
||||
## Interactive testing during development
|
||||
|
||||
During development, you'll probably want to interact with Forgejo to see if your changes work.
|
||||
|
||||
To run and continuously rebuild when the source files change:
|
||||
|
||||
|
|
@ -26,18 +130,9 @@ To run and continuously rebuild when the source files change:
|
|||
TAGS='sqlite sqlite_unlock_notify' make watch
|
||||
```
|
||||
|
||||
> **NOTE:** do not set the `bindata` tag such as in `TAGS="bindata" make watch` or the browser may fail to load pages with an error like `Failed to load asset`
|
||||
|
||||
## Manual run of the binary
|
||||
|
||||
After following the [steps to compile from source](../from-source/), a `forgejo` binary will be available
|
||||
in the working directory.
|
||||
It can be tested from this directory or moved to a directory with test data. When Forgejo is
|
||||
launched manually from command line, it can be killed by pressing `Ctrl + C`.
|
||||
|
||||
```bash
|
||||
./forgejo web
|
||||
```
|
||||
> **NOTE:**
|
||||
> Do not set the `bindata` tag such as in `TAGS="bindata" make watch`.
|
||||
> The browser may fail to load pages with an error like `Failed to load asset`.
|
||||
|
||||
## Automated tests
|
||||
|
||||
|
|
@ -49,6 +144,9 @@ directory.
|
|||
|
||||
### In the end-to-end repository
|
||||
|
||||
> **Note:**
|
||||
> Do not confuse this kind of testing with the "e2e" (frontend) tests in the Forgejo repository.
|
||||
|
||||
Some tests are best served by running Forgejo from a compiled binary,
|
||||
for instance to verify the result of a workflow run by the [Forgejo runner](https://code.forgejo.org/forgejo/runner). They can be
|
||||
run by adding the `run-end-to-end-test` label to the pull request. It will:
|
||||
|
|
@ -68,8 +166,9 @@ forgejo-runner exec --workflows .forgejo/workflows/testing.yml
|
|||
|
||||
## Manual testing
|
||||
|
||||
When the change to be tested lacks the proper framework, the manual
|
||||
test steps must be documented in the description of the pull request.
|
||||
When the change to be tested lacks the proper framework,
|
||||
we sometimes add manual testing instructions as a last resort.
|
||||
The exact test steps must be documented in the description of the pull request.
|
||||
|
||||
Changes that are associated with manual tests must be labeled
|
||||
labeled ["test/manual"](https://codeberg.org/forgejo/forgejo/issues?labels=201028).
|
||||
["test/manual"](https://codeberg.org/forgejo/forgejo/issues?labels=201028).
|
||||
|
|
|
|||
Loading…
Reference in a new issue