qa-devel June 2016

qa-devel@lists.fedoraproject.org

6 participants
7 discussions

rpmgrill in taskotron
by Kamil Paral 04 Jul '16

04 Jul '16

Hey Ralph (ccing qa-devel), just to let you know, I added some code to Koji directive to download build logs. The patch is posted here: https://phab.qadevel.cloud.fedoraproject.org/D916 (you can use 'arch patch D916' to apply the diff in your libtaskotron checkout) The corresponding changes for task-rpmgrill are in feature/buildlog branch: https://bitbucket.org/fedoraqa/task-rpmgrill/branch/feature/buildlog I could have implemented shuffling the build.log files around in run_rpmgrill.py, but it seemed easier to have it in a bash script. I moved all the rpmgrill* commands in there as well, and used our new bash directive to execute it (it still has rough edges, we're working on that). But having it in a bash script is not necessary, if you prefer to do it all in python, we definitely don't have to merge the code as it is :) I also removed your failsafe decorator, but again, it's just a suggestion. The first reason was that since I moved most of the setup code and rpmgrill execution into the bash directive, the decorator didn't have any affect on it anyway. The second reason is that we seem to be of an opinion that ResultsDB should only contain real results (passed, failed), and crashes and similar execution statuses should be only present in ExecDB [1] (which, however, cannot be searched yet). This is all up to discussion, and our current solutions have a lot of rough edges, so if you want to continue sending CRASHED to ResultsDB in case of errors, we have no problem with it, and we can convert task-rpmgrill once we're 100% sure how we want things to look like and have it implemented properly. Please note, though, that we currently receive emails for all tests that crash hard, so we will be able to spot any task-rpmgrill issues even if not wrapped in the failsafe decorator. If you have any questions or concerns, please ask :) Cheers, Kamil [1] https://taskotron.fedoraproject.org/execdb/

1 1

Automatic ABI checks for new package updates in bodhi using taskotron
by Sinny Kumari 29 Jun '16

29 Jun '16

Hi Kamil, Thank you for the reply. On Wed, Apr 13, 2016 at 7:34 PM, Kamil Paral <kparal(a)redhat.com> wrote: > > Hello Sinny, > thanks for your work and sorry for late response. I'll review your > taskotron task and let you know if there's something that should be changed > or not. Afterwards, we will start mirroring your git repo on our taskotron > servers, and patch our taskotron-trigger to know about task-libabigail. We > can then execute it on every new Koji build (or Bodhi update, your choice). > Your task will probably be the first one that we will execute regularly > while not being written and maintained directly by us, so if there are any > rough edges in the process, I apologize in advance :-) > Okay, sounds good to me. > > You'll need to have two branches in your git: > master - this will be used on our production server > https://taskotron.fedoraproject.org/ > develop - this will be used on our dev and staging server > http://taskotron-dev.fedoraproject.org/ and > https://taskotron.stg.fedoraproject.org/ Okay, works for me. You need to decide whether it is better to run libabigain against every new > Koji build, or just against every new Bodhi update. From a quick look, I > think it makes more sense to run libabigail on every new Koji build, so > that people can see the results even before creating the update (that > requires looking into ResultsDB manually at the moment). If we run it on > every Koji build, the results will still show up in Bodhi - Bodhi should > query ResultsDB and show the results for those particular builds present in > the update. (We might need to teach Bodhi about libabigail existence, I'm > not sure). Ultimately it's your choice, what makes more sense for your > check. > I believe that when we say every new Koji build, we are talking about non-scratch build which doesn't include scratch build done by anyone. If my assumption is right, then yes running libabigail task on each koji build will be good. It is possible to do that with current implementation since libabigail task look for a koji build-id to download corresponding rpms. > Please also create a wiki page at > https://fedoraproject.org/wiki/Taskotron/Tasks/libabigail similar to these > https://fedoraproject.org/wiki/Taskotron/Tasks/depcheck > https://fedoraproject.org/wiki/Taskotron/Tasks/upgradepath > linked from https://fedoraproject.org/wiki/Taskotron/Tasks . > Sure, I will create one. > We try to have at least some basic documentation and FAQ for our checks in > there. Currently it's not very discoverable (we should link to it at least > from ResultsDB, which we currently don't) and the location can change, but > at least it's a link we can give to people when asking basic questions > about one of our tasks. Also, since you're going to maintain the task and > not us, please include some "Contact" section where to post feedback or > report bugs (e.g. github issues page). If people ask us about the task and > we don't know the answer, we will point them to that wiki page. > Will add contact section in wiki page. I wonder if it is better to have this included with other fedora qa tasks? > Can we please continue this discussion in qa-devel [1] mailing list? We > can discuss more implementation details in there, and I'll post my review > findings in there as well. > done! -- http://sinny.io/

4 13

openQA upgrade plans
by Adam Williamson 29 Jun '16

29 Jun '16

Hi, folks! Just wanted to fill everyone in on openQA upgrade plans. I have upgraded the staging server and worker boxes to Fedora 24. prod is still on Fedora 23 for now. I have built current git os-autoinst and openqa, plus latest upstream perl-Mojolicious-Plugin-AssetPack (which is needed for new openqa), for Rawhide. More-or-less identical builds for F24 are available in this COPR: https://copr.fedorainfracloud.org/coprs/adamwill/openQA-stg/ My pet openqa - https://openqa.happyassassin.net - is currently running those packages, as a first step in testing if they work OK. So far it seems to be alright. This week I'm planning - if Emmanuel Seyman from the perl SIG is OK with it - to send out the same three builds to updates-testing for F24, and update staging to the new openqa and os-autoinst. If that all appears to go OK, I'll move on to updating prod to F24 and the new openqa/os-autoinst. The changes to AssetPack are a bit unfortunate; basically, openQA is now pretty tied to a design where the web assets (CSS and JavaScript) that it uses have to be pulled from remote servers and cached at some point, which is awkward for packaging. On the plus side, the cached assets no longer change every time openQA or AssetPack changes, but only when openQA does. After thinking it through I've gone for an approach similar to SUSE's, by adding a script to the package git repo which generates the asset cache and stuffs it into a tarball, which is then included as a Source1 for the package, so the final packages just include a pre-generated copy of the asset cache. Whoever's doing the packaging (so, uh, me...) has to run the script to re-generate the asset cache any time we bump to a new upstream openQA checkout or release. Please do check out the new packages and the deployment on happyassassin and yell if you see any problems. Note that some tests are failing on happyassassin because I haven't built all the hard disk images (that box is limited in disk space) and for some reason tests that try to upload the disk image after they finish tend to fail on that deployment (I've never managed to work out why). The most obvious change is that the web UI has been rather revamped (and now uses the distro-neutral icon that the Fedora design team came up with, yay!) but there's other stuff as well, you can dig through the openqa and os-autoinst git changelogs to see what... -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net

1 2

2016-06-13 @ 14:00 UTC - Fedora QA Devel Meeting
by Tim Flink 13 Jun '16

13 Jun '16

# Fedora QA Devel Meeting # Date: 2016-06-13 # Time: 14:00 UTC (https://fedoraproject.org/wiki/Infrastructure/UTCHowto) # Location: #fedora-meeting-1 on irc.freenode.net I suspect this will be a shorter meeting again, mostly covering tasking and clarifying plans for F25 deliverables. https://phab.qadevel.cloud.fedoraproject.org/w/meetings/20160613-fedoraqade… Tim Proposed Agenda =============== Announcements and Information ----------------------------- - Please list announcements or significant information items below so the meeting goes faster Tasking ------- - Does anyone need tasks to do? Potential Other Topics ---------------------- - Docker testing - packaging - What to do when some base images can't be built (typically rawhide) at all Open Floor ---------- - TBD

1 1

2016-06-06 Fedora QA Devel Meeting Minutes
by Tim Flink 06 Jun '16

06 Jun '16

Despite the lack of a meeting announcement, we were productive and seem to have mostly figured out the remaining namespace issues. That thread will be updated separately with the proposal that we came up with. ================================= #fedora-meeting-1: fedora-qadevel ================================= Minutes: https://meetbot.fedoraproject.org/fedora-meeting-1/2016-06-06/fedora-qadeve… Minutes (text): https://meetbot.fedoraproject.org/fedora-meeting-1/2016-06-06/fedora-qadeve… Log: https://meetbot.fedoraproject.org/fedora-meeting-1/2016-06-06/fedora-qadeve… Meeting summary --------------- * Roll Call (tflink, 14:01:35) * Announcements and Information (tflink, 14:03:55) * taskotron-trigger sent for package review (mkrizek, 14:04:18) * taskotron-trigger sent for package review (mkrizek, 14:05:44) * Task Namespaces (tflink, 14:10:10) * ACTION: tflink to make qa-devel@ archives public (tflink, 14:14:57) * LINK: https://lists.fedoraproject.org/archives/list/qa-devel@lists.fedoraproject.… (tflink, 14:15:28) * AGREED: the items 1-6 above seem reasonable at this time and is good enough to start working on but a more detailed document should be sent out to qa-devel@ for final approval (tflink, 15:17:49) * Open Floor (tflink, 15:19:13) Meeting ended at 15:21:55 UTC. Action Items ------------ * tflink to make qa-devel@ archives public Action Items, by person ----------------------- * tflink * tflink to make qa-devel@ archives public * **UNASSIGNED** * (none) People Present (lines said) --------------------------- * tflink (148) * kparal (79) * Dodji (43) * mkrizek (8) * zodbot (5) * garretraziel (4) * omiday (2) * coremodule (1) * lbrabec (1) Generated by `MeetBot`_ 0.1.4 .. _`MeetBot`: http://wiki.debian.org/MeetBot

1 0

Task Result Namespaces
by Martin Krizek 06 Jun '16

06 Jun '16

Hi, we have deployed task result namespaces support a while ago and put our checks (depcheck, upgradepath, rpmlint) into the 'qa' namespace. With newly added tasks like task-abicheck and task-dockerautotest, we weren't sure where to put them and so we used the scratch namespace which is supposed to be used for testing purposes. Now with those "3rd party" tasks deployed to production, we need to decide what namespaces should they and other future tasks belong in. The starting namespaces, and maintainers of tasks belonging to that namespace, follows: qa.* - Fedora QA pkgs.<pkgname>.* - <pkgname> maintainers fas.<username>.* - <username> fasgroup.<groupname>.* - fas members belonging to <groupname> scratch.* - anyone Now, since we maintain task-dockerautotest, it should go into qa.*, I am not sure though where does task-abicheck belong. I see three options here: 1. we can create fas group and put it there, 2. create additional dist.* namespace where tasks like abicheck and/or rpmgrill would be, or 3. change semantics of qa.* to 'anything Fedora QA maintains or important, not package-specific, tasks". I don't really have strong feelings about either to be honest. Thoughts? Other proposals? Thanks, Martin

4 4

PoC of "configurable trigger"
by Josef Skladanka 01 Jun '16

01 Jun '16

Source: https://bitbucket.org/fedoraqa/taskotron-trigger/branch/pony Diff: https://phab.qadevel.cloud.fedoraproject.org/D872 This started as simple bike-shedding to make more sense in naming (so everything is not named "Trigger"), but it went further :D The main change here is, what I call "configurable trigger" - at the moment, every time we want to add support for even the most basic new task (like the package-specific task for docker), changes are needed in the trigger's source code. These changes add a concept of a "rules engine", that decides what tasks to schedule based on data extracted from the received FedMessage, and a set of rules. The rules-engine is YAML, in a format like this:: ``` - do: - {tasks: [depcheck, upgradepath]} when: {message_type: KojiTagChanged} - do: - {tasks: [dockerautotest]} when: {message_type: KojiBuildCompleted, name: docker} - do: - {tasks: [abicheck]} when: message_type: KojiBuildCompleted name: $in: ${critpath_pkgs} $nin: ['docker'] # critpath excludes ``` The rules are split in two parts `when` and `do`, the `when` clause is a mongo query that will get evaluated against the dataset provided by the FedMsg consumer. For example, the KojiBuildCompletedJobTrigger now publishes this (values are fake, to make it more descriptive:: message_data = { "_msg": {...snipped...}, "message_type": "KojiBuildCompleted", "item": "docker-1.9.1-6.git6ec29ef.fc23", "item_type": "koji_build", "name": "docker", "version": "1.9.1-6.git6ec29ef", "release": "fc23", "critpath_pkgs": [..., "docker", ...] "distgit_branch": "f23", } So taking the rules, and the data, going from the top: # First rule's `when` is `False` as `message_type` is not `KojiTagChanged` # Second rule is `True` because both the `message_type` and name in the `when` clause match the data # Third rule does _not_ schedule anything, because even though `docker` is in `critpath_pkgs`, it also is part of the critpath excludes list, and so the rule is ignored The `when` clauses are in fact mongo queries <https://docs.mongodb.com/manual/tutorial/query-documents/>, evaluated using a Python library that implements it for querying Python objects. The rules engine then takes the `do` clauses of the 'passed' rules, and produces arguments for the `trigger_tasks()` calls. By default, `item`, and `item_type` are taken from the `message_data`, `arches` is set to `config.valid_arches`, and then all the key/values from the `do`'s body are added on top. This means, that we can have a task, that for example forces an architecture different than default:: ``` - do: - {tasks: [awesome_arm_check], arches: [armhfp]} when: {message_type: KojiBuildCompleted} ``` The `do` clause can have multiple items in it, so something like this is possible:: ``` - do: - {tasks: [rpmlint]} - {tasks: [awesome_arm_check], arches: [armhfp]} when: {message_type: KojiBuildCompleted} ``` Triggering `rpmlint` on the default architectures, and `awesome_arm_check` on `armhfp` for each package built in Koji. This means, that when we want to trigger new (somewhat specific) tasks, no changes are needed in the trigger's code, but just in the configuration, to alter the rules. If we come to the point where more functionality is needed, than it obviously calls for changes in the underlying code, in order to add more key/values to the data provided by the Fedmsg consumer, or adding more general functionality overall. A good example of this is the dist-git style tasks problem. To solve it I have added a new command (`$discover`)to the `do` section, that crawls the provided git repo/branch, and schedules jobs for all `runtask.yml`'s found:: ``` - do: - {$discover: {repo: 'http://pkgs.fedoraproject.org/git/rpms-checks/${name}.git', branch: '${distgit_branch}'}} when: {message_type: KojiBuildCompleted} ``` In the bigger picture, this 'rules engine' functionality can be used to make (for example) a web interface, that allows creating/altering the rules, instead of changing the config file (the rules can as easily be taken from a database, as from the config file), or even to provide a per-user triggering capability - we could add a piece of code, that checks (selected) users' Fedorapeople profile for a file, that contains rules in this format, and then could simply run the engine on those rules+data from Fedmsg to decide whether the user-defined tasks should be run. It also somewhat reduces the tight bond between the trigger and FedMessage, as the rules engine does not really care where did the data (used to evaluate the rules) come from. This is by no means final, but it IMO shows quite an interesting PoW/idea, that was not that complicated to implement, and made the trigger lot better at what it can do. Please, share thoughts/comments/rants!

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

qa-devel June 2016