qa-devel May 2013

qa-devel@lists.fedoraproject.org

6 participants
18 discussions

[Fedora QA] #381: Bug and Update syncs shouldn't be halted by an isolated problem with one update or bug
by fedora-badges 13 Jan '14

13 Jan '14

#381: Bug and Update syncs shouldn't be halted by an isolated problem with one update or bug --------------------------------------+---------------------------------- Reporter: tflink | Owner: tflink Type: enhancement | Status: new Priority: major | Milestone: Undetermined Future Component: Blocker bug tracker page | Version: Keywords: | Blocked By: Blocking: | --------------------------------------+---------------------------------- = problem = The current bug and update sync algorithms are very fragile in that one error will stop the entire sync algorithm, leaving stuff out of sync with bodhi or bugzilla. This has caused problems in the past = analysis = This could be solved by better error handling in the sync process such that an error is logged but doesn't bubble up to the main sync code and halt the entire process. -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/381> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 1

[Fedora QA] #384: Improve documentation
by fedora-badges 13 Jan '14

13 Jan '14

#384: Improve documentation --------------------------------------+------------------------ Reporter: tflink | Owner: tflink Type: enhancement | Status: new Priority: major | Milestone: Fedora 20 Component: Blocker bug tracker page | Version: Keywords: | Blocked By: Blocking: | --------------------------------------+------------------------ = problem = At the moment, our documentation is out of date and somewhat incomplete. = analysis = Update the docs and find an appropriate place for them to live. -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/384> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 1

[Fedora QA] #387: Updates pending stable show as stable
by fedora-badges 02 Dec '13

02 Dec '13

#387: Updates pending stable show as stable --------------------------------------+--------------------- Reporter: kparal | Owner: tflink Type: defect | Status: new Priority: minor | Milestone: Component: Blocker bug tracker page | Version: Keywords: | Blocked By: Blocking: | --------------------------------------+--------------------- A package that is in updates-testing and is pending stable is shown as [stable] in the app. That can cause some confusion (i.e. one of us can close a related bug by assuming the package is already stable, and the package push can be cancelled afterwards). I think the app should display the repository the package is currently in, not the repository the package in pending to. What do you think? -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/387> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 4

[Fedora QA] #386: Update URLs should comtain the update ID when possible
by fedora-badges 27 Sep '13

27 Sep '13

#386: Update URLs should comtain the update ID when possible --------------------------------------+---------------------------------- Reporter: tflink | Owner: tflink Type: enhancement | Status: new Priority: minor | Milestone: Undetermined Future Component: Blocker bug tracker page | Version: Keywords: | Blocked By: Blocking: | --------------------------------------+---------------------------------- = problem = The update URLs fetched from bodhi are most reliable when they are based off of the update ID which is generated when the update is pushed to testing. However, the blocker tracking app generally grabs the URL before this happens and thus, end up with the preliminary url based on update contents. = analysis = The update sync process should look to replace any update URLs which are not based on the update ID when the sync runs against bodhi. -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/386> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 4

[Fedora QA] #383: Migrate from FAS to FAS-OpenID
by fedora-badges 29 Jul '13

29 Jul '13

#383: Migrate from FAS to FAS-OpenID --------------------------------------+------------------------ Reporter: tflink | Owner: tflink Type: enhancement | Status: new Priority: major | Milestone: Fedora 20 Component: Blocker bug tracker page | Version: Keywords: | Blocked By: Blocking: | --------------------------------------+------------------------ = problem = With the introduction of FAS-OpenID, the current method of FAS authentication will be going away somewhere around F20. We will need to migrate to the newer FAS-OpenID before that time. = analysis = The OpenID code in python-fedora-flask should make the transition relatively painless but it will still require quite a bit of testing to make sure that everything still works. -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/383> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 1

[Fedora QA] #367: Display flash messages in appropriate place
by fedora-badges 03 Jul '13

03 Jul '13

#367: Display flash messages in appropriate place --------------------------------------+---------------------------------- Reporter: mkrizek | Owner: tflink Type: enhancement | Status: new Priority: minor | Milestone: Undetermined Future Component: Blocker bug tracker page | Version: Keywords: | Blocked By: Blocking: | --------------------------------------+---------------------------------- = problem = Currently flash messages containing information for user are not displayed. = analysis = There should probably be a box right under the header on all pages that would display flash messages. = enhancement recommendation = -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/367> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 2

Review Request 23: Bugzilla Faults During Proposal are not logged, lead to http 500 errors
by Martin Krizek 25 Jun '13

25 Jun '13

----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://reviewboard-tflink.rhcloud.com/r/23/ ----------------------------------------------------------- Review request for blockerbugs. Bugs: 372 https://fedorahosted.org/fedora-qa/ticket/372 Repository: blockerbugs Description ------- This patch wraps bugzilla faults in a BugzillaError and prevents them from not being logged and generating http 500 server error. Diffs ----- blockerbugs/util/bz_interface.py 0579b2455b57a03e3bee570b8cd97aac0eeaf1a4 blockerbugs/controllers/main.py 6d32e5de91900cda33f5e0342c7854adb24fc8b6 Diff: http://reviewboard-tflink.rhcloud.com/r/23/diff/ Testing ------- Thanks, Martin Krizek

1 7

[Fedora QA] #372: Bugzilla Faults During Proposal are not logged, lead to http 500 errors
by fedora-badges 25 Jun '13

25 Jun '13

#372: Bugzilla Faults During Proposal are not logged, lead to http 500 errors --------------------------------------+------------------------ Reporter: tflink | Owner: tflink Type: defect | Status: new Priority: critical | Milestone: Fedora 19 Component: Blocker bug tracker page | Version: Keywords: | Blocked By: Blocking: | --------------------------------------+------------------------ = bug description = When bugzilla faults during proposal don't have fault code 51, the internal !BugzillaError class isn't used and the fault is lost when attempting to extract information from it {{{ [Wed Mar 27 13:26:38 2013] [error] [20130327-13:26PM] [blockerbugs] [ERROR] Exception on /propose_bug [POST] [Wed Mar 27 13:26:38 2013] [error] Traceback (most recent call last): [Wed Mar 27 13:26:38 2013] [error] File "/usr/lib/python2.6/site- packages/flask/app.py", line 1687, in wsgi_app [Wed Mar 27 13:26:38 2013] [error] response = self.full_dispatch_request() [Wed Mar 27 13:26:38 2013] [error] File "/usr/lib/python2.6/site- packages/flask/app.py", line 1360, in full_dispatch_request [Wed Mar 27 13:26:38 2013] [error] rv = self.handle_user_exception(e) [Wed Mar 27 13:26:38 2013] [error] File "/usr/lib/python2.6/site- packages/flask/app.py", line 1358, in full_dispatch_request [Wed Mar 27 13:26:38 2013] [error] rv = self.dispatch_request() [Wed Mar 27 13:26:38 2013] [error] File "/usr/lib/python2.6/site- packages/flask/app.py", line 1344, in dispatch_request [Wed Mar 27 13:26:38 2013] [error] return self.view_functions[rule.endpoint](**req.view_args) [Wed Mar 27 13:26:38 2013] [error] File "/usr/lib/python2.6/site- packages/blockerbugs/controllers/users.py", line 84, in decorated_function [Wed Mar 27 13:26:38 2013] [error] return f(*args, **kwargs) [Wed Mar 27 13:26:38 2013] [error] File "/usr/lib/python2.6/site- packages/blockerbugs/controllers/main.py", line 375, in propose_bug [Wed Mar 27 13:26:38 2013] [error] bugform.bugid.errors = [e.msg] [Wed Mar 27 13:26:38 2013] [error] AttributeError: 'Fault' object has no attribute 'msg' }}} = fix recommendation = Either all of the bugzilla faults need to be wrapped in a !BugzillaError or the error handling needs to be capable of handling multiple error types. -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/372> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 1

Making Beaker a better fit for Fedora QA
by Nick Coghlan 03 Jun '13

03 Jun '13

Hi all, Tim saw my Beaker talk proposal for Flock and asked me to get involved earlier than that, since he's been experimenting with Taskbot and doesn't want to wait until August to discuss things. That sounds perfectly reasonable to me, so here I am :) The short version is that I think Beaker can slot fairly cleanly into Tim's Taskbot vision as the task execution engine, as well as providing a results repository. But wait, you say, doesn't Beaker always provision systems from scratch? Doesn't it only support the arcane task definition syntax we inherited from RHTS? Good questions, and we do have answers for them :) = Defining tasks = The interface to the native test harness (beah) is one we inherited from RHTS, and it has historically been quite poorly documented. The upcoming Beaker 0.13 release includes much improved documentation for anyone that wants to write a native Beaker task: http://beaker-project.org/docs-develop/user-guide/writing-tasks.html However, above and beyond that, we're working with the autotest developers to start supporting autotest as a first class environment for execution of tasks in Beaker, by providing a stable API on the lab controllers for harnesses to talk to (see http://beaker-project.org/dev/tech-roadmap.html#autotest-support) That alternate harness API is also our avenue for bypassing the task library in the future - we're working with the autotest developers to ensure that the details of the tests to be executed can be retrieved directly from git rather than having to be registered as RPMs in the Beaker task library. Even once we get the autotest support on par with the existing beah support, the task library will likely still be useful for solving problems that can otherwise be painful (like Kerberos and AMQP testing - we have some Beaker provided tasks in development for spinning up a KDC or a qpid message broker to test against as part of a multi-host test) = Provisioning systems = Beaker *does* currently always provision systems from scratch - it's the only way to support full installer testing as well as kernel integration testing on a wide range of hardware. However, we're also aware that this *doesn't make sense* for a whole lot of testing that could just as easily be run in a VM. Our first step down the road to fixing this has been to support dynamic provisioning of virtual machines for task execution. The initial attempt relied on oVirt, and this turned out to be a really bad fit - oVirt isn't designed for fast provisioning of ephemeral instances, it's built for stable provisioning of long running core services. We also explored ovirt's support for dynamic image based provisioning and the short answer is "not supported". However, the rest of the dynamic provisioning support is still in place, so our current plans involve tweaking that system to use OpenStack instead (although, if we can, we'll probably use the EC2 compatible APIs for broader compatibility). OpenStack already includes a *lot* of the stuff we want (fast image based provisioning, a cross platform post-install configuration system, etc) so it makes sense to us to try to re-use it rather than writing our own (the development resources being poured into OpenStack by prospective vendors don't hurt, either). = What's in it for Fedora QA? = You don't have to reinvent solutions to problems that Beaker already solved. You also get a task execution engine with several full time engineers assigned to it (in addition to whatever resources others can spare), that was specifically built for the task of testing an integrated Linux distribution rather closely related to Fedora ;) = What's in it for Beaker? = We get Fedora QA's assistance in solving the problems that we haven't solved yet either (like fast image based provisioning). We also get a *public* instance we can reference from our docs rather than having to be somewhat vague and hand-wavey about how all this works because all the other current instances are behind various corporate firewalls :P Regards, Nick. -- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane Test Automation Team Lead Beaker Development Lead (http://beaker-project.org/) PulpDist Development Lead (http://pulpdist.readthedocs.org)

2 1

Automation Tool Evaluation/Comparison
by Tim Flink 31 May '13

31 May '13

There have been a couple tooling suggestions for automation here as of late and I don't want people to think that we're ignoring them because we aren't. I'm working on taskbot as time permits because it's a relatively low priority for us until F19 final is released. I'm planning to do a more in depth evaluation of the suggested tools (beaker and igor, in particular. I'm already pretty familiar with both buildbot and autotest for the purposes of what we're looking for) and have started a wiki page to describe the criteria that we're interested in. https://fedoraproject.org/wiki/User:Tflink/AutomationFrameworkEvaluation In case you're wondering, I'm still planning to go forward with taskbot from a conceptual standpoint. While these frameworks would all occupy slightly different places in the concept, I don't see anything at the moment that would be a reason to reject any of them as part of the system as a whole. I'll update this thread as I get around to those evaluations. Tim

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

qa-devel May 2013