On Wed, 22 May 2019 at 14:55, Bruno Wolff III bruno@wolff.to wrote:
On Wed, May 22, 2019 at 13:25:17 +0200, Petr Šplíchal psplicha@redhat.com wrote:
Hi Bruno,
for your use case (I see you have a single shell test script in your pull request [1]) I would recommend to use the basic role:
https://docs.fedoraproject.org/en-US/ci/standard-test-roles/#_basic
It would take care of correctly copying artifacts and also handle test script return code to report appropriate result. Also adding a list of required packages is easy there. You can also have a look at a few more examples here:
The script doesn't return a failure unless it has a problem with something outside what is being tested. You need to run grep to see if any of the tests, run in loops, fails. That messes with the output. I could probably string something together not much different than I did for the wrapper case though.
You could do something like this:
- hosts: localhost roles: - role: standard-test-basic tags: - classic tests: - compression: dir: . run: "./Squashfs-compression-test.sh 2>&1 | tee /tmp/test.log && ! grep -iq failed /tmp/test.log" required_packages: - kernel-modules-extra - rpmdevtools
Which looks much more concise.
I'm close enough with the wrapper set up now, that I'll probably see if I can finish with it. However, the documentation being off could cause a lot of confusion for others trying to create tests for their packages and something should be done to fix it or warn people off.
What exactly in the docs should cause the confusion? Should any of these pages be updated?
https://docs.fedoraproject.org/en-US/ci/quick-start-guide/ https://docs.fedoraproject.org/en-US/ci/standard-test-roles/ https://docs.fedoraproject.org/en-US/ci/examples/
Or are you missing some essential content there?
I've also been seeing random failures with set up of the environment. Not always the same thing. I'm worried there might be issues with gating because of this. I'm not sure how to rerun a test that has been used for gating.
Including the "[citest]" string in a pull request comment does rerun all enabled tests.
psss...