I'm trying to have shared tests between all 400+ rust packages.
I have few problems though:
* There is no package NEVRA in test environment so the shared test doesn't know what to test - it is necessary to know which test to run * No dist-git sources of a package - while trying to workaround previous problem I have found that dist-git doesn't get into the environment so I can't do something like dnf builddep
Please help me ;)
P.S> I'm using standard-test-basic role.
Thanks in advance!
On Thu, Jul 19, 2018 at 9:08 PM Igor Gnatenko < ignatenkobrain@fedoraproject.org> wrote:
I'm trying to have shared tests between all 400+ rust packages.
I have few problems though:
- There is no package NEVRA in test environment so the shared test doesn't
know what to test - it is necessary to know which test to run.
We trigger tests for dist-git. I think about dist-git as a SRPM. SRPM can produce many RPMs. CI pipeline: install all RPMS from Koji build and run tests from /tests for all installed RPMs. The same approach is for manual testing. Conclusion: test-environment-system has installed RPMS. Tests from /tests dir check test-environment-system as a whole.
https://fedoraproject.org/wiki/CI/Standard_Test_Interface doesn't define NEVRA. Reason: STI tests are for all test-environment-system (all installed packages). /tests directory from dist-git knows that packages goes from dist-git/pkg /tests/tests.yml is written for each SRPM package. /tests/tests.yml knows that it will run against: SRPM + distro version. /tests/tests.yml for Fedora 27 and for Fedora 28 can are different files. If you have common tests, you have different /tests/tests.yml where you can specify exactly that this tests are for this SRPM and for Branch 28 (for example).
- No dist-git sources of a package - while trying to workaround previous
problem I have found that dist-git doesn't get into the environment so I can't do something like dnf builddep
Please help me ;)
We should decide: if tests can exist independently without dist-git. For now, we presume they can. This is current state. If yes. There is nothing to do.
If no. We should tight together dist-git branch + /tests as one entity. And copy /tests to test-environment system together with dist-git.
If we decide to change this, than we need to make correspondent changes to STI. https://fedoraproject.org/wiki/CI/Standard_Test_Interface
P.S> I'm using standard-test-basic role.
Thanks in advance!
-Igor Gnatenko _______________________________________________ CI mailing list -- ci@lists.fedoraproject.org To unsubscribe send an email to ci-leave@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org/mes...
On Fri, Jul 20, 2018 at 1:44 PM Andrei Stepanov astepano@redhat.com wrote:
On Thu, Jul 19, 2018 at 9:08 PM Igor Gnatenko < ignatenkobrain@fedoraproject.org> wrote:
I'm trying to have shared tests between all 400+ rust packages.
I have few problems though:
- There is no package NEVRA in test environment so the shared test
doesn't know what to test - it is necessary to know which test to run.
We trigger tests for dist-git. I think about dist-git as a SRPM. SRPM can produce many RPMs. CI pipeline: install all RPMS from Koji build and run tests from /tests for all installed RPMs. The same approach is for manual testing. Conclusion: test-environment-system has installed RPMS.
Just want to mention that this won't work if your subpackages conflict (e.g. curl and curl-minimal.
Tests from /tests dir check test-environment-system as a whole.
https://fedoraproject.org/wiki/CI/Standard_Test_Interface doesn't define NEVRA. Reason: STI tests are for all test-environment-system (all installed packages). /tests directory from dist-git knows that packages goes from dist-git/pkg /tests/tests.yml is written for each SRPM package. /tests/tests.yml knows that it will run against: SRPM + distro version. /tests/tests.yml for Fedora 27 and for Fedora 28 can are different files. If you have common tests, you have different /tests/tests.yml where you can specify exactly that this tests are for this SRPM and for Branch 28 (for example).
This is wwhat I have done so far, however test runner has all this information, why not to simply pass it to the tests rather than asking every maintainer to do that manually.
- No dist-git sources of a package - while trying to workaround previous
problem I have found that dist-git doesn't get into the environment so I can't do something like dnf builddep
Please help me ;)
We should decide: if tests can exist independently without dist-git. For now, we presume they can. This is current state. If yes. There is nothing to do.
Why would they do?
If no. We should tight together dist-git branch + /tests as one entity. And copy /tests to test-environment system together with dist-git.
If we decide to change this, than we need to make correspondent changes to STI. https://fedoraproject.org/wiki/CI/Standard_Test_Interface
P.S> I'm using standard-test-basic role.
Thanks in advance!
-Igor Gnatenko
CI mailing list -- ci@lists.fedoraproject.org To unsubscribe send an email to ci-leave@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org/mes...
CI mailing list -- ci@lists.fedoraproject.org To unsubscribe send an email to ci-leave@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org/mes...
On Fri, Jul 20, 2018 at 1:54 PM Igor Gnatenko < ignatenkobrain@fedoraproject.org> wrote:
On Fri, Jul 20, 2018 at 1:44 PM Andrei Stepanov astepano@redhat.com wrote:
On Thu, Jul 19, 2018 at 9:08 PM Igor Gnatenko < ignatenkobrain@fedoraproject.org> wrote:
I'm trying to have shared tests between all 400+ rust packages.
I have few problems though:
- There is no package NEVRA in test environment so the shared test
doesn't know what to test - it is necessary to know which test to run.
We trigger tests for dist-git. I think about dist-git as a SRPM. SRPM can produce many RPMs. CI pipeline: install all RPMS from Koji build and run tests from /tests for all installed RPMs. The same approach is for manual testing. Conclusion: test-environment-system has installed RPMS.
Just want to mention that this won't work if your subpackages conflict (e.g. curl and curl-minimal.
Right, and I would ask contra@ team or other guys for clarification, how CI pipeline does care about above moment?
Tests from /tests dir check test-environment-system as a whole.
https://fedoraproject.org/wiki/CI/Standard_Test_Interface doesn't define NEVRA. Reason: STI tests are for all test-environment-system (all installed packages). /tests directory from dist-git knows that packages goes from dist-git/pkg /tests/tests.yml is written for each SRPM package. /tests/tests.yml knows that it will run against: SRPM + distro version. /tests/tests.yml for Fedora 27 and for Fedora 28 can are different files. If you have common tests, you have different /tests/tests.yml where you can specify exactly that this tests are for this SRPM and for Branch 28 (for example).
This is wwhat I have done so far, however test runner has all this information, why not to simply pass it to the tests rather than asking every maintainer to do that manually.
- No dist-git sources of a package - while trying to workaround previous
problem I have found that dist-git doesn't get into the environment so I can't do something like dnf builddep
Please help me ;)
We should decide: if tests can exist independently without dist-git. For now, we presume they can. This is current state. If yes. There is nothing to do.
Why would they do?
If you think that:
"parent dir for /tests must be contents from dist-git branch" , then: please file a issue at https://pagure.io/standard-test-roles/issues That I can announce discussion about this changes to STI to our team. Please remember, this change should be consistent for all dist-git repos.
If no. We should tight together dist-git branch + /tests as one entity.
And copy /tests to test-environment system together with dist-git.
If we decide to change this, than we need to make correspondent changes to STI. https://fedoraproject.org/wiki/CI/Standard_Test_Interface
P.S> I'm using standard-test-basic role.
Thanks in advance!
-Igor Gnatenko
CI mailing list -- ci@lists.fedoraproject.org To unsubscribe send an email to ci-leave@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org/mes...
CI mailing list -- ci@lists.fedoraproject.org To unsubscribe send an email to ci-leave@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org/mes...
--
-Igor Gnatenko _______________________________________________ CI mailing list -- ci@lists.fedoraproject.org To unsubscribe send an email to ci-leave@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org/mes...
On Fri, Jul 20, 2018 at 8:05 AM, Andrei Stepanov astepano@redhat.com wrote:
On Fri, Jul 20, 2018 at 1:54 PM Igor Gnatenko < ignatenkobrain@fedoraproject.org> wrote:
On Fri, Jul 20, 2018 at 1:44 PM Andrei Stepanov astepano@redhat.com wrote:
On Thu, Jul 19, 2018 at 9:08 PM Igor Gnatenko < ignatenkobrain@fedoraproject.org> wrote:
I'm trying to have shared tests between all 400+ rust packages.
I have few problems though:
- There is no package NEVRA in test environment so the shared test
doesn't know what to test - it is necessary to know which test to run.
We trigger tests for dist-git. I think about dist-git as a SRPM. SRPM can produce many RPMs. CI pipeline: install all RPMS from Koji build and run tests from /tests for all installed RPMs. The same approach is for manual testing. Conclusion: test-environment-system has installed RPMS.
Just want to mention that this won't work if your subpackages conflict (e.g. curl and curl-minimal.
Right, and I would ask contra@ team or other guys for clarification, how CI pipeline does care about above moment?
I don't think there is a solution for this currently. We do not have a static blacklist going and I, at least, am not aware of some other way to dynamically choose which rpms from the koji task to install (right now, it just repoqueries the repo created by all the artifact rpms and tries installing them).
Tests from /tests dir check test-environment-system as a whole.
https://fedoraproject.org/wiki/CI/Standard_Test_Interface doesn't define NEVRA. Reason: STI tests are for all test-environment-system (all installed packages). /tests directory from dist-git knows that packages goes from dist-git/pkg /tests/tests.yml is written for each SRPM package. /tests/tests.yml knows that it will run against: SRPM + distro version. /tests/tests.yml for Fedora 27 and for Fedora 28 can are different files. If you have common tests, you have different /tests/tests.yml where you can specify exactly that this tests are for this SRPM and for Branch 28 (for example).
This is wwhat I have done so far, however test runner has all this information, why not to simply pass it to the tests rather than asking every maintainer to do that manually.
- No dist-git sources of a package - while trying to workaround previous
problem I have found that dist-git doesn't get into the environment so I can't do something like dnf builddep
Please help me ;)
We should decide: if tests can exist independently without dist-git. For now, we presume they can. This is current state. If yes. There is nothing to do.
Why would they do?
If you think that:
"parent dir for /tests must be contents from dist-git branch" , then: please file a issue at https://pagure.io/standard-test-roles/issues That I can announce discussion about this changes to STI to our team. Please remember, this change should be consistent for all dist-git repos.
If no. We should tight together dist-git branch + /tests as one entity.
And copy /tests to test-environment system together with dist-git.
If we decide to change this, than we need to make correspondent changes to STI. https://fedoraproject.org/wiki/CI/Standard_Test_Interface
P.S> I'm using standard-test-basic role.
Thanks in advance!
-Igor Gnatenko
CI mailing list -- ci@lists.fedoraproject.org To unsubscribe send an email to ci-leave@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ci@lists. fedoraproject.org/message/ESOHHMZ37AZIVVOPAWQNTRBXDEYKWAF6/
CI mailing list -- ci@lists.fedoraproject.org To unsubscribe send an email to ci-leave@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ci@lists. fedoraproject.org/message/45YUVGGKW3BSFRAFPRGAZ2PKPTFHXLTP/
--
-Igor Gnatenko _______________________________________________ CI mailing list -- ci@lists.fedoraproject.org To unsubscribe send an email to ci-leave@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ci@lists. fedoraproject.org/message/QCO7WV2N5LHUBPN62PZMEGBJL2SKUI33/
CI mailing list -- ci@lists.fedoraproject.org To unsubscribe send an email to ci-leave@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ci@lists. fedoraproject.org/message/EWUIYSJKUZ6DPX3YGJ2ZEXUZFOEFC3PU/
-
Andrei Stepanov -
Igor Gnatenko -
Johnny Bieren