CI July 2020

ci@lists.fedoraproject.org

12 participants
5 discussions

Can we use emulation of other architectures to run integration tests?
by Aleksandra Fedorova 06 Aug '20

06 Aug '20

Hi, all, I'd like to get some understanding on the current state of emulation of other architectures. In the current CI infra we have infinite(*) access to x86_64 compute resources, but we haven't yet got our hands on any non x86_64 hardware. As COPR has recently got support for s390 builds, the question is: if emulation is good enough for building packages, can we use it for testing? What are the limitations there? Is it worth it? If yes, can you suggest some links, tips on the topic? (*) Not really, and some of our infra services are not yet good at scaling up, but we are getting better at it. -- Aleksandra Fedorova bookwar

10 18

Re: We have to talk about annobin... again
by Neal Gompa 25 Jul '20

25 Jul '20

On Sat, Jul 25, 2020 at 11:25 AM Jeff Law <law(a)redhat.com> wrote: > > On Sat, 2020-07-25 at 10:46 +0100, Peter Robinson wrote: > > On Sat, Jul 25, 2020 at 10:35 AM Neal Gompa <ngompa13(a)gmail.com> wrote: > > > Hey all, > > > > > > So I was trying to update libseccomp last night, and I was able to > > > build it for everything except aarch64 on Rawhide because it says the > > > compiler can't build executables[1]. > > > > > > Looking a bit closer, it looks like the compiler stack is out of sync > > > again with annobin. > > > > > > Is there anything that can be done to keep the compiler teams from > > > submitting gcc into rawhide without doing the required rebuild cycle > > > to make it so annobin works? > > > > > > And we're going to have the same problem with clang now that annobin > > > grew a clang plugin, so I would want neither LLVM nor GCC to land in > > > Rawhide unless those teams are literally ensuring that annobin isn't > > > breaking the compiler afterward. > > > > > > I'm personally very tired of having the compiler break so frequently > > > because of that plugin. Either some kind of mechanism to hold back GCC > > > builds until annobin works is implemented, or I'd much rather see the > > > whole thing go away. Obviously, you could just *bundle* annobin into > > > the GCC package and build it together to ensure it never broke, but > > > that option was discarded already[2]. > > > > > > Somebody fix it. ASAP. > > > > I've kicked off an annobin build for the short term issue. > > > > I agree here, we used to have the same issue with gcc-plugins back in > > the day, sadly this is nothing new :-/ > :( I'll raise it again with Nick and Jakub, it's a sore point for everyone I > think and it causes far more friction than just what we see here in Fedora. > There is now a FESCo ticket proposing all GCC and Clang updates to be blocked on some kind of gating mechanism: https://pagure.io/fesco/issue/2454 -- 真実はいつも一つ！/ Always, there's only one truth!

1 0

We have to talk about annobin... again
by Neal Gompa 25 Jul '20

25 Jul '20

Hey all, So I was trying to update libseccomp last night, and I was able to build it for everything except aarch64 on Rawhide because it says the compiler can't build executables[1]. Looking a bit closer, it looks like the compiler stack is out of sync again with annobin. Is there anything that can be done to keep the compiler teams from submitting gcc into rawhide without doing the required rebuild cycle to make it so annobin works? And we're going to have the same problem with clang now that annobin grew a clang plugin, so I would want neither LLVM nor GCC to land in Rawhide unless those teams are literally ensuring that annobin isn't breaking the compiler afterward. I'm personally very tired of having the compiler break so frequently because of that plugin. Either some kind of mechanism to hold back GCC builds until annobin works is implemented, or I'd much rather see the whole thing go away. Obviously, you could just *bundle* annobin into the GCC package and build it together to ensure it never broke, but that option was discarded already[2]. Somebody fix it. ASAP. [1]: https://koji.fedoraproject.org/koji/taskinfo?taskID=47796366 [2]: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org… -- 真実はいつも一つ！/ Always, there's only one truth!

1 0

Re: As a developer I want to easily execute/enable tests
by Petr Šplíchal 23 Jul '20

23 Jul '20

Hi Dominik, Thank you for your questions. See my comments inline. On Thu, 23 Jul 2020 at 14:39, Dominik 'Rathann' Mierzejewski <dominik(a)greysector.net> wrote: > > On Wednesday, 22 July 2020 at 14:49, Petr Šplíchal wrote: > > Hi! > > > > I am writing today to share a proposal for how we can make it > > easier to enable tests in Fedora CI. > > > > We've heard the user story mentioned in the subject many times. > > Fedora developers have frequently mentioned the difficulties which > > they face with enabling tests in Fedora CI. Some people don't > > understand the complex yaml syntax. Other people find that they > > can't easily reproduce issues revealed by Fedora CI on their > > laptops. People have asked us to provide a tool for local testing: > > > > https://pagure.io/fedora-ci/general/issue/4 > > > > A few of us have been working on a tool which we hope will improve > > the situation considerably. The proof of concept below shows how > > everyday developer tasks (related to testing) might look in the > > near future: > > > > Enable basic smoke test for my component: > > > > dnf install -y tmt > > tmt init --template mini > > vim plans/example.fmf > > git add . && git commit -m tests && git push > > > > Create a new test based on a shell/beakerlib template: > > > > tmt test create --template shell tests/basic > > tmt test create --template beakerlib tests/advanced > > > > Run all/selected tests safely in a virtual machine: > > > > tmt run > > tmt run test --name smoke > > I'd say it's good to have such option, but for quick testing a full VM > is often too heavy and slow to boot. Even a container takes time to set > up and run. Yes, there are different needs and preferences related to the environment where the tests should be executed. We want to provide enough flexibility here so the 'provision' step allows you to choose your preferred way. > My primary environment would be local. So for your case that would be 'provision --how local'. If you have a test box prepared for testing you could also connect to it using 'provision --how connect'. > > Execute tests in my preferred environment: > > > > tmt run --all provision --how container > > tmt run --all provision --how virtual > > tmt run --all provision --how local > > tmt run --all provision --how ... > > I find it difficult to guess what the above commands actually do. > Do they run the tests or do they provision environments to run the > tests? Or both? The 'tmt run' command is used to run tests. By default it runs all steps (discover, provision, prepare, execute, report, finish) so the examples above would run tests in the requested environment. It is also possible to run only selected steps: For example, one of the use cases shared with us was 'just tell me what tests would be run, do not run them', this can be achieved with the following: tmt run discover For detailed description of individual steps have a look here: https://tmt.readthedocs.io/en/latest/spec/steps.html > > I would like to ask for your feedback about this. What do you > > think? Is this something that would be useful for you? What do > > you think of the syntax? We would love to hear back from you > > concerning what essential features you might expect from such a > > tool. It would also be helpful to know if there is anything that > > should be changed early in the tool design. > > > > For more examples and frequently asked questions have a look here: > > > > https://tmt.readthedocs.io/en/latest/examples.html > > https://docs.fedoraproject.org/en-US/ci/tmt/ > > > > Looking forward to your feedback and ideas! :) > > tmt examples are overcomplicated and not explained in enough detail. > What is "metadata" and why do you need 4 "levels" of it? Metadata levels allow to store all needed test execution data at appropriate location, the levels are described here: https://tmt.readthedocs.io/en/latest/spec.html We want the config for simple/common use cases (such as running a shell script as a test) as simple as possible so that users do not have to study all the details but just create something like this: execute: script: ./run-my-test.sh If one needs to maintain larger set of tests the L1 metadata can come handy: https://tmt.readthedocs.io/en/latest/spec/tests.html > What are "tiers" for? They are used to organize tests into groups by priority. https://tmt.readthedocs.io/en/latest/spec/tests.html#tier > Which items are mandatory and which are optional? In order to enable tests you just have to define the 'execute' step. Use 'tmt init --template mini' to get a quick start. > What's "relevancy"? It's a test (L1) attribute allowing to filter relevant tests. https://tmt.readthedocs.io/en/latest/spec/tests.html#relevancy Currently we're working on open-sourcing the implementation. > Why is there "extra-summary" in addition to "summary"? > What's "extra-task"? These are because of backward compatibility, for more see: https://tmt.readthedocs.io/en/latest/stories/cli.html?highlight=extra-task#… > What are "plans" for? Plans define which tests should be executed, how the environment should be provisioned and prepared, how the results should be reported. See the following page for more details about plans. https://tmt.readthedocs.io/en/latest/spec/plans.html > What are "stories" for? Stories are an advanced, optional feature which can help you to track which features/requirements have been already implemented, tested and documented. https://tmt.readthedocs.io/en/latest/spec/stories.html https://github.com/psss/tmt/tree/master/stories > You're using a lot of terms as if everyone knows what they mean > in your context. Sorry for that. We're trying to document the concept as best as we can. Hopefully, the links above have helped. Perhaps we could add some more references to the examples page. Thank you for all your feedback. Please, let us know what's still missing and how we can further improve the docs. Thanks. psss... > Regards, > Dominik > -- > Fedora https://getfedora.org | RPM Fusion http://rpmfusion.org > There should be a science of discontent. People need hard times and > oppression to develop psychic muscles. > -- from "Collected Sayings of Muad'Dib" by the Princess Irulan > _______________________________________________ > devel mailing list -- devel(a)lists.fedoraproject.org > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

1 0

As a developer I want to easily execute/enable tests
by Petr Šplíchal 22 Jul '20

22 Jul '20

Hi! I am writing today to share a proposal for how we can make it easier to enable tests in Fedora CI. We've heard the user story mentioned in the subject many times. Fedora developers have frequently mentioned the difficulties which they face with enabling tests in Fedora CI. Some people don't understand the complex yaml syntax. Other people find that they can't easily reproduce issues revealed by Fedora CI on their laptops. People have asked us to provide a tool for local testing: https://pagure.io/fedora-ci/general/issue/4 A few of us have been working on a tool which we hope will improve the situation considerably. The proof of concept below shows how everyday developer tasks (related to testing) might look in the near future: Enable basic smoke test for my component: dnf install -y tmt tmt init --template mini vim plans/example.fmf git add . && git commit -m tests && git push Create a new test based on a shell/beakerlib template: tmt test create --template shell tests/basic tmt test create --template beakerlib tests/advanced Run all/selected tests safely in a virtual machine: tmt run tmt run test --name smoke Execute tests in my preferred environment: tmt run --all provision --how container tmt run --all provision --how virtual tmt run --all provision --how local tmt run --all provision --how ... I would like to ask for your feedback about this. What do you think? Is this something that would be useful for you? What do you think of the syntax? We would love to hear back from you concerning what essential features you might expect from such a tool. It would also be helpful to know if there is anything that should be changed early in the tool design. For more examples and frequently asked questions have a look here: https://tmt.readthedocs.io/en/latest/examples.html https://docs.fedoraproject.org/en-US/ci/tmt/ Looking forward to your feedback and ideas! :) psss...

1 0

2024

2023

2022

2021

2020

2019

2018

2017

CI July 2020