Using tags to indicate context in which tests are able to run - CI - Fedora mailing-lists

24 Jul 2017


      Many of the tests that are being written for dist-git packages or
modules have special assumptions about where they can execute. For
example, tests may interact with packages not present in an Atomic Host,
or run things that don't make sense in a container.
While some of this info can be inferred from the "test subject" in the
spec [0], there was a need for a rigorous way for packagers to what a
test was designed for.
To that end, Miroslav, Merlin and I worked on a change to the spec,
which would allow for tests to indicate in which context they are
designed to operate.
We'd use Ansible Tags for this. So far there are three tags "atomic",
"container" and "classic". Here's the updated page:
https://fedoraproject.org/wiki/Changes/InvokingTestsThree
And here's the specific changes:
https://fedoraproject.org/w/index.php?title=Changes%2FInvokingTestsThree&...
And here's an example of it in use:
https://upstreamfirst.fedorainfracloud.org/bash/pull-request/2
Broadly speaking the changes are backwards compatible, although tests
will want to start to tag themselves to at least operate in the
"classic" context.
Does this make sense? If so, next steps are to:
1. Merge those changes back into the main spec:
https://fedoraproject.org/wiki/Changes/InvokingTests
2. Update the tutorial style documentation:
https://fedoraproject.org/wiki/CI/Tests
3. Work through the current set of staged tests and put tags in.
Cheers,
Stef
[0] https://fedoraproject.org/wiki/Changes/InvokingTests