The basic idea here is to separate a few intertwined notions:
1. Not all "run bash tests" are "install tests"
2. Not all "run bash tests" use `tests/functional/init.sh`, or any
pre-test initialization at all.
This will used in the next commit when we have a test that check unit
test golden master data.
Also, move our custom `PS4` from the test to the test runner, as it is
part of how we want to display the tests, not the test themselves.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
I think it is bad for these reasons when `tests/` contains a mix of
functional and integration tests
- Concepts is harder to understand, the documentation makes a good
unit vs functional vs integration distinction, but when the
integration tests are just two subdirs within `tests/` this is not
clear.
- Source filtering in the `flake.nix` is more complex. We need to
filter out some of the dirs from `tests/`, rather than simply pick
the dirs we want and take all of them. This is a good sign the
structure of what we are trying to do is not matching the structure
of the files.
With this change we have a clean:
```shell-session
$ git show 'HEAD:tests'
tree HEAD:tests
functional/
installer/
nixos/
```
Continue with the characterization testing idioms begun in
c70484454f, but this time for unit tests.
Co-authored-by: Andreas Rammhold <andreas@rammhold.de>
Grouping our tests should make it easier to understand the intent than
one long poorly-arranged list. It also is convenient for running just
the tests for a specific component when working on that component.
We need at least one test group so this isn't dead code; I decided to
collect the tests for the `ca-derivations` and `dynamic-derivations`
experimental features in groups. Do
```bash
make ca.test-group -jN
```
and
```bash
make dyn-drv.test-group -jN
```
to try running just them.
I originally did this as part of #8397 for being able to just the local
overlay store alone. I am PRing it separately now so we can separate
general infra from new features.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
* Lang now verifies errors and parse output
* Some new miscellaneous tests
* Easy way to update the tests
* Document workflow in manual
* Use `!` not `~` as separater char for sed
It is confusing to use `~` when we are talking about paths and home
directories!
* Test test suite itself (`test/lang-test/infra.sh`)
Additionally, run shellcheck on `tests/lang.sh` to help ensure it is
correct, now that is is more complex.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
