qa-devel March 2014

qa-devel@lists.fedoraproject.org

8 participants
14 discussions

[Fedora QA] #436: SSH access to systems in Beaker lab
by fedora-badges 20 Apr '16

20 Apr '16

#436: SSH access to systems in Beaker lab --------------------------------------+--------------------- Reporter: atodorov | Owner: tflink Type: defect | Status: new Priority: major | Milestone: Component: Blocker bug tracker page | Version: Keywords: | Blocked By: Blocking: | --------------------------------------+--------------------- = bug description = Currently systems in Beaker lab can be accessed only through bastion.fp.o which is not as convenient as direct SSH into the system. There's also the question whether or not to open the systems directly to the Internet. This needs to be discussed with infra. Filing here so it doesn't get lost. -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/436> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 4

[Fedora QA] #443: Better format for test compose (TC) and release candidate (RC) requests
by fedora-badges 29 Jan '15

29 Jan '15

#443: Better format for test compose (TC) and release candidate (RC) requests ----------------------------------------+------------------------ Reporter: jreznik | Owner: Type: enhancement | Status: new Priority: major | Milestone: Fedora 21 Component: Blocker/NTH review process | Version: Keywords: | Blocked By: Blocking: | ----------------------------------------+------------------------ = problem = With Dennis (in CC), we discussed how to make release process, with Fedora.next in mind, more transparent and bullet proof. One issue is that releng request can become pretty messy, with full text included and sometimes it leads to errors (omission of packages in compose etc). = enhancement recommendation = One possibility is to visibly separate full text description (with bug numbers, reasons - it's good to have history) and the list of exact nvrs (maybe in code block?), try to avoid "qt bundle" etc. so it's easier to pick up the right list (for blockers, FEs + exceptional tools requests). Another thing is better coordination between requester/releng - to mark when/which list was picked up etc, in similar way how Go/No-Go decision is stated in the ticket. Now I'll let more space to Dennis, maybe example of how the request should look like to make it easy to parse would help. Long term (and preferred) solution would be to have automation in place, Blocker app talking to releng interface, compose database, web dashboard etc... -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/443> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 3

[Fedora QA] #437: Need to import daily Fedora snapshots into Beaker
by fedora-badges 29 Jan '15

29 Jan '15

#437: Need to import daily Fedora snapshots into Beaker --------------------------------------+--------------------- Reporter: atodorov | Owner: tflink Type: task | Status: new Priority: major | Milestone: Component: Blocker bug tracker page | Version: Keywords: | Blocked By: Blocking: | --------------------------------------+--------------------- = bug description = In order to perform any meaningfull testing Beaker needs to import more recent Fedora trees. It could be daily(nightly) snapshots or less often depending on available resources. The tree directory structure needs to be a copy/snapshot of the current state at the time of import. The reason is b/c devel trees utilize one URL but the contents under this URL are updated in a rolling fashion. We need tree URLs where content is not changing in order to produce consistent test results. -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/437> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 1

[Fedora QA] #433: blocker proposal form forgets everything after login timeout
by fedora-badges 20 Jan '15

20 Jan '15

#433: blocker proposal form forgets everything after login timeout --------------------------------------+--------------------- Reporter: kvolny | Owner: tflink Type: defect | Status: new Priority: major | Milestone: Component: Blocker bug tracker page | Version: Keywords: | Blocked By: Blocking: | --------------------------------------+--------------------- = bug description = I was trying to propose a F20 blocker. I needed to gather information from multiple bugs, so it took me longer to write the justification. After finishing and submitting that, I was presented with a login screen. After logging in again, I was redirected to the proposal form again, but it was completely empty, all the text that took me so long to write was gone. (Okay, I'm a smart guy and I had it in clipboard for such case, but if I had forgotten ... booh.) = bug analysis = Seems the login code doesn't care about other variables in the http request ... = fix recommendation = 1) If there is such a short login timeout, the user should be warned about it (e.g. countdown timer on the page) and the page should allow refresh without submitting the data. 2) Once the login expires, the submitted data should be caried over all the redirects back to the submission form. -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/433> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 2

Taskotron Repos
by Tim Flink 01 Apr '14

01 Apr '14

As I'm working more on the ansible playbooks to deploy Taskotron, I'd like to get a repo setup for our builds until we have stuff in Fedora proper. I think we have 3 options: copr, fedorapeople and qadevel. copr would probably be the easiest of the options - that takes care of building, repo creation and hosting. The only disadvantage that I see to copr is that it limits the number of builds available [1] to the latest successful build and anything else < 14 days old. [1]https://fedorahosted.org/copr/wiki/UserDocs#Howlongdoyoukeepthebuilds fedorapeople is what we've used in the past for both blockerbugs and autoqa. We still need to manage repo content and do the building but it is a system for hosting the repo which we don't have to maintain. When I setup the qadevel machine for phabricator, I planned to also use the machine for at least CI and possibly some docs and repo hosting. While I think this might be a good long-term solution for integration with CI and possible continuous deployment in dev/stg, I'm not sure this would be the best choice right now due to the amount of work which would be required to get everything running. Yet another option would be a hybrid approach (which is what I've been doing for phabricator) would be to build the rpms with copr and mirror to fedorapeople to keep more builds around. The process is a bit of a pain but it can be less painful than building everything locally. I'm of the opinion that either a buildscript or copr and fedorapeople repos is going to be the best option for the moment. Any other thoughts? Tim

3 4

Taskotron Data Interfaces
by Tim Flink 27 Mar '14

27 Mar '14

One of the design goals that I've talked about before is the idea of using restful json at the interfaces between components of taskotron. I'd like to simplify this a bit more to say that any interfaces between components should be a variation on a dictionary. When I say dictionary, I'm referring to a generic implementation that could take one of several forms (json, yaml, python dictionaries are the first that come to mind) Some liberties will likely need to be taken with this concept when interfacing with external systems, the important part is that any single component from the trigger to the runner and reporting systems should be isolate-able by using a simple mock object/system fed by dictionaries. There have been a couple of conversations that have been at least partially related to data interfaces as of late and I thought that it would be helpful to at least make sure we're all on the same page. I've written up how I understand the interfaces below - please comment, ask questions etc. Tim Trigger ======= The trigger is a piece of code which is responsible for starting the process of task execution. In our initial implementation, it will listen for fedmsgs and based on the content of those messages, schedule jobs with buildbot. The interface in this case is: { 'item': <name of item - envr, update id, koji tag, etc.>, 'item_type': <koji_build, koji_tag, bodhi_update>, 'taskname': <task to be scheduled>, 'arch': <arch needed for execution, x86_64, i386 etc.> } Buildbot ======== When jobs are scheduled by the trigger using the above data, buildbot will queue the task and delegate execution to an appropriate client machine based on the input data. The git repo corresponding to the taskname will be cloned on the client but buildbot will add some input data. The resultant interface will be: { 'item': <name of item>, 'item_type': <type of item>, 'build_id': <uuid of buildbot build, used for logs and results>, 'yamlfile': <taskname>.yml } Runner ====== The runner is responsible for taking the specified yaml file and executing the directives contained within. In the case of CLI operation where there is no build_id, a sensible default will be used instead to indicate a filler value (-1000 for example). When running a task, the runner will copy most of its own input as the task's input: { 'item': <name of item>, 'item_type': <type of item>, 'build_id': <uuid of buildbot build, used for logs and results> } While the runner is responsible for running a task, each task is responsible for its own reporting mechanism. The output from the runner itself is in the form of execution information only and does not necessarily parallel the result of the executed task. { 'execution_status': <OK or NOT_OK>, 'status': <status message, possibly including any exeptions> } Task ==== I won't go too far into detail for this as it is being discussed in phabricator: - https://phab.qadevel.cloud.fedoraproject.org/T84 I think that I've captured all that we've been talking about but I'm sure that Josef or Kamil will correct me if I've missed something :) The primary idea is that tasks will generate a list of TAP results which can be farther processed into ResultsDB submissions or Bodhi comments (for now). While TAP isn't scrictly a dictionary, it is a reasonably well defined output format and when TAP13 is used with embedded YAML, we end up with something that doesn't diverge too much from a dictionary. Example TAP output (stolen from D26): ok - $CHECKNAME for Koji build xchat-0.5-1.fc20 --- details: output: |- xchat.x86_64: W: file-not-utf8 /usr/share/doc/xchat/ChangeLog xchat.x86_64: W: non-conffile-in-etc /etc/gconf/schem/apps_xchat_url_handler.schemas xchat.x86_64: W: no-manual-page-for-binary xchat item: xchat-0.5-1.fc20 outcome: PASSED summary: 5 ERRORS, 10 WARNINGS type: koji_build ... not ok - $CHECKNAME for Bodhi update foobar-1.2-3.fc20 # FAIL --- item: foobar-1.2-3.fc20 outcome: FAILED summary: dependency error type: bodhi_update ... ok - $CHECKNAME for Yum repository f20-updates --- item: f20-updates outcome: INFO summary: 2 stale updates type: yum_repository ... Example ResultsDB post (stolen from T84): { "outcome": 'NEEDS_INSPECTION', "testcase_name": 'upgradepath', "summary": 'not ok upgradepath for build foo-1.2-3.fc20' "result_data": { 'item': 'foo-1.2-3.fc20', 'type': 'koji_build', }, "job_id": <resultsdb id>, "log_url": <path to log files> }

2 7

Example Bodhi Task
by Tim Flink 20 Mar '14

20 Mar '14

I wrote an example task that works on bodhi updates today. The idea was to have something simplistic to use for development, figuring out some of the data issues (TAP and resultsdb reporting, CheckDetail etc.) and testing while we get the depcheck and upgradepath tasks complete. The task just reads in the update and assigns a PASS/FAIL status in a pseudo-random fashion. https://bitbucket.org/fedoraqa/task-examplebodhi Feel free to use, update and improve it. Tim

2 1

Beaker 0.16 released
by Dan Callaghan 20 Mar '14

20 Mar '14

Folks, I just wanted to let you know that we released Beaker 0.16.0 this week. This includes the fix for password hashing, which was one of the outstanding security issues with having a public Beaker deployment. https://beaker-project.org/docs/whats-new/release-0.16.html#password-hashes… I can upgrade the Beaker instance at dev.fedoraproject.org (Tim sponsored me into the sysadmin-qa group so that I can help with this stuff). I'll do that early next week unless there are any objections. There will be a brief outage for database upgrades but I don't think that matters given that the Beaker instance is currently not usable :-) (Tim, when you can spare a few minutes please ping me and we can sort out the remaining issues with those Beaker VMs.) -- Dan Callaghan <dcallagh(a)redhat.com> Software Engineer, Hosted & Shared Services Red Hat, Inc.

1 0

QA Infrastructure and Ansible
by Tim Flink 19 Mar '14

19 Mar '14

I don't know if we've talked about it much on list but most of the QA stuff (autoqa, blockerbugs-dev, taskotron, qadevel) is managed with a set of ansible playbooks: https://bitbucket.org/fedoraqa/ansible-playbooks As we've been adding stuff, these playbooks have been getting more and more unwieldy and they aren't quite written in the same style as the Fedora infra playbooks are. I'm going to start some major refactoring of the playbooks this week to farther a couple of goals: 1. Move to a central execution point for the production, staging and non-local dev deployments 2. Migrate to a style more similar to how infra does things so that the eventual migration isn't quite so painful 3. Make the playbooks, as a whole, more manageable and easier to understand so that more folks can learn them, help with sysadmin tasks and decrease the bus factor (which is currently 1 in a couple of areas) I don't think that anyone else is working on the playbooks, but I wanted to send out a heads up in case there is work that I don't know about. If you're using the playbooks for a local taskotron deployment, that'll continue to work for the most part. I'll be pushing changes to a non-master branch at first and you'll probably need to adapt some of your configuration but I'm hoping that the changes won't be huge. Tim

1 1

qadevel seems to be down
by Tim Flink 18 Mar '14

18 Mar '14

As a heads up, qadevel.cloud.fedoraproject.org (and our phabricator instance, which is hosted on that machine) seems to be down right now. I'm not sure what exactly is going on, but most of the fedora cloud seems to be affected. I can still access the machines through an administrative interface, so everything is still there but network into and out of those cloud instances seems to be non-functional. Hopefully, network access will be restored soon. We have daily backups of phabricator, so there won't be any data loss if it turns out to be something bigger. Tim

2 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

qa-devel March 2014