CI December 2018

ci@lists.fedoraproject.org

9 participants
4 discussions

Clarify test result reporting in Standard Test Interface
by Petr Šplíchal 15 May '19

15 May '19

Hi! Currently the Standard Test Interface does not define how to distinguish test failure from an infrastructure error. When ansible playbook is run it usually does some environment preparation and only then executes tests. Upon a problem encountered during provisioning or configuration of the test environment ansible returns non-zero exit code but there is not an easy way how to detect such error from a successful test execution of test which failed (returns non-zero exit code as well). Let's make this part of the specification more clear so that tools can detect infrastructure errors and report them as such to the end user. This will help to direct problems (errors versus failures) to the right audience (infrastructure staff versus developer/tester). See the following ticket for proposed solution and share your thoughts there: https://pagure.io/fedora-ci/general/issue/18 Thanks. psss...

1 4

Is it really necessary for the pipeline to submit a result for every package it ignores?
by Adam Williamson 18 Dec '18

18 Dec '18

I had reason today to go poking through the ResultsDB fedmsg history: https://apps.fedoraproject.org/datagrepper/raw?category=resultsdb doing that makes it apparent that the pipeline is kinda flooding ResultsDB with "test results" that seem to be nothing more than "the pipeline decided not to test this package". Something like 99% of pipeline results appear to be of this type - either "ci.pipeline.allpackages-build.package.ignored" or "ci.pipeline.package.ignore". There've been 20 ci.pipeline.allpackages- build.package.ignored results filed in just the last 7 minutes, for instance, and it seems like there's a more or less constant stream of them. Are these results really necessary or useful? ResultsDB is built to handle the load, I think, and my use case of poking through fedmsg history looking for messages from the pipeline *actually doing something* is not a super common one, but still, if this isn't really necessary, it seems like it might be a good idea to just not do it. (Also, on the topic of why there are two topics - is one from the new pipeline and one from the old? Do we still need the old pipeline running?) Thanks folks! -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net

8 13

Rework CI documentation into docs site
by Aleksandra Fedorova 10 Dec '18

10 Dec '18

Hi, all, tl;dr ------------- We created a repo for Fedora CI documentation. It is easy to contribute. Please join :) ------------- Currently we have a lot of CI related information in Fedora wiki. While wiki provides easy way to edit things, there are certain downsides of storing data there. There is no clear structure, so it is hard to navigate and differentiate docs targeting users from docs describing certain implementation details. It is not flexible in terms of media formats. There is no way to propose and review changes. No issue tracker and so on. Fortunately, Fedora Docs team has solution for that - Fedora Docs site. Fedora Docs site has modular structure, so we can add CI documentation as a component to it. It uses Asciidoc format which is relatively easy and widely adopted. So me and Petr (psss) stepped forward and created a repository on pagure called fedora-ci/docs [1] I've configured initial template and placeholder pages based on template provided by Fedora Docs team, so it is ready to go: we don't need to dig into any Antora internals, only to write articles in the asciidoc format. Also I've created the issue [2] with the list of wiki pages related to CI topics. How you can help (choose one or more): 0) join the conversation about high level structure [3], 1) check that I haven't missed certain pieces of information worth migrating, see [2], 2) review current list of wiki pages in [2], if there is smth outdated, which shouldn't be migrated at all, 3) take one of the pages in [2], transform to asciidoc (rework if needed) and send a pull request to fedora-ci/docs repo, 3) add new article, 4) proof-read articles, added by others. [1] https://pagure.io/fedora-ci/docs [2] https://pagure.io/fedora-ci/docs/issue/1 [3] https://pagure.io/fedora-ci/docs/issue/2 -- Aleksandra Fedorova bookwar

2 1

Re: Fedora Rawhide-20181206.n.0 compose check report
by Adam Williamson 06 Dec '18

06 Dec '18

On Thu, 2018-12-06 at 15:25 +0000, Fedora compose checker wrote: > No missing expected images. > > Compose FAILS proposed Rawhide gating check! > 5 of 47 required tests failed > openQA tests matching unsatisfied gating requirements shown with **GATING** below So - this is a new feature of check-compose I wrote yesterday :) As part of the 'NoMoreAlpha' change from a few releases back: https://fedoraproject.org/wiki/Changes/NoMoreAlpha we planned to start gating Rawhide composes - only syncing them out to become the current public state of the 'Rawhide' tree if they pass some subset of the automated compose tests. However, this has not yet been implemented. An initial gating policy was actually committed to the Fedora Greenwave config some months back by Ralph Bean, and since then it's been possible to ask Greenwave whether a given compose passes or fails the policy - but nothing was hooked up beyond that; no-one's been doing anything with that information. Ultimately the idea is that the Greenwave decision should be hooked into the compose sync process, but I thought before we go that far, it'd be useful to know just how often composes actually *do* meet the requirements. So I enhanced check-compose to request the decision from Greenwave and include it in the output, like this. This should help us get an idea just how often we'd be refusing to sync composes if we turned gating on. The failures that fail the gating check today are down to https://bugzilla.redhat.com/show_bug.cgi?id=1654537 and the Everything netinst tests for both x86_64 and UEFI failing because of the network not being up in the guest for some reason. The FreeIPA bug I'm hopeful we can get fixed soon, I'm not sure what's going on with the netinst failure, I'll have to look into that. (It's a bit tricky because, when the network in the guest doesn't work, we don't get any logs from the test...) Eagle-eyed readers may not a slight bug in the new feature: Greenwave reports *5* failed requirements, but the details below show only *4* tests with **GATING**. This is because the other failed test - the FreeIPA server test - doesn't show up in the check-compose output for some reason. If you re-run check-compose now, it *does* show up, so this is likely some kind of ordering issue to do with exactly what state that job is in when check-compose runs (my working theory is that it's in 'uploading' state - that could cause it to fall between some code cracks). I'll see if I can fix that. Thanks folks! -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net

1 0

2024

2023

2022

2021

2020

2019

2018

2017

CI December 2018