test August 2010

test@lists.fedoraproject.org

119 participants
341 discussions

[Fedora QA] #62: Description page for every test case in Package Update Acceptance Test Plan
by fedora-badges 08 Dec '11

08 Dec '11

#62: Description page for every test case in Package Update Acceptance Test Plan --------------------+------------------------------------------------------- Reporter: kparal | Owner: kparal Type: task | Status: new Priority: major | Milestone: Component: Wiki | Version: Keywords: | --------------------+------------------------------------------------------- For every test case in Package Update Acceptance Test Plan there should be a separate wiki page with detailed description what gets tested, etc. Some tools and autoqa front-ends may then reference this pages. Also it is needed for package maintainers that will surely want to read something about the test case their package has just failed. The test plan proposal referencing all the test cases is currently here: https://fedoraproject.org/wiki/User:Kparal/Proposal:Package_update_acceptan… -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/62> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 3

[Fedora QA] #80: Add dual-boot release criteria
by fedora-badges 07 Dec '11

07 Dec '11

#80: Add dual-boot release criteria ---------------------------+------------------------------------------------ Reporter: jlaska | Owner: Type: enhancement | Status: new Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Keywords: retrospective | ---------------------------+------------------------------------------------ = problem = See [https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=590661 RHBZ#590661 - GRUB bootloader should have a few seconds delay on a multi- boot setup] = analysis = Due to RHBZ #590661, the dual-boot experience was not well understood and tested for the final release. It was discovered late and unclear whether this behavior was critical to Fedora success. = enhancement recommendation = The user experience of dual-boot scenarios was not well understood, as a result RHBZ #590661 did not clearly impact the release criteria. Recommend reviewing and making adjustments to the release criteria for dual-boot expectations. -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/80> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 3

[Fedora QA] #93: Review options to reward key QA contributors
by fedora-badges 07 Dec '11

07 Dec '11

#93: Review options to reward key QA contributors ---------------------------+------------------------------------------------ Reporter: jlaska | Owner: jlaska Type: enhancement | Status: new Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Keywords: retrospective | ---------------------------+------------------------------------------------ = problem = During Fedora 13, a small amount was available to reward several key QA contributors for their efforts. = analysis = If possible, I'd like to repeat this in Fedora 14. = enhancement recommendation = What's the best way to say thanks? 1. Request and secure QA budget to reward key contributors 2. Researching reward options (maxamillion has discussed t-shirt ideas with the design team) -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/93> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 3

[Fedora QA] #89: Improve tracking blocker review status
by fedora-badges 07 Dec '11

07 Dec '11

#89: Improve tracking blocker review status ---------------------------+------------------------------------------------ Reporter: jlaska | Owner: poelstra Type: enhancement | Status: new Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Keywords: retrospective | ---------------------------+------------------------------------------------ = problem = Several bugs were changed so they fell off the tracking lists for Fedora 13. For example... * [http://bugzilla.redhat.com/505189 505189 - Going back to repo UI screen and and modifying Installation Repo causes traceback] * [http://bugzilla.redhat.com/577803 577803 - Adding repository requiring network fails if network is down] = analysis = In F-13, a bugzilla keyword was used to denote blocker status. However, aside from reviewing bugzilla comments, there was no query-able method to determine whether a blocker request is open, approved or denied. This led to several Fedora 13 bugs that were 1) fixed in Rawhide, 2) Moved to MODIFIED or CLOSED and not included, but not included in Fedora 13 (e.g. RHBZ #505189 and RHBZ #577803). Not knowing which bugs were already reviewed, also introduced time wasted reviewing previously reviewed bugs during blocker review meetings. = enhancement recommendation = Recommend reviewing process changes to avoid the scenarios leading up to RHBZ #505189 and RHBZ #577803. Some options discussed so far include 1. Document usage of bugzilla flags (suggested by jkeating) to track blocker requests. This likely involves creating a ''bot'' to police the tags? 2. Hardening the current keyword-based mechanism. Additional discussion points include: 1. Document generating exception report and document action plan for CLOSED Blocker bugs that did not go through VERIFIED. 2. More frequent nag mails leading up to release milestones. Notification of NEW or ASSIGNED bugs goes to the maintainer, and MODIFIED bugs goes to QA. -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/89> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 21

[Fedora QA] #98: Request for a Mentor
by fedora-badges 07 Dec '11

07 Dec '11

#98: Request for a Mentor -----------------------------------------+---------------------------------- Reporter: jdulaney | Owner: Type: proventester request | Status: new Priority: major | Milestone: Component: Proventester Mentor Request | Version: Keywords: | -----------------------------------------+---------------------------------- I would like to now formally request a Proven Tester Mentor. I have been running Fedora since FC1, generally updating every other release (I skipped 9 and went straight to 10). I have several boxes so as to get different hardware involved. Thanks, John H. Dulaney https://fedoraproject.org/wiki/User:Jdulaney -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/98> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 2

[Fedora QA] #60: Devise a better location for critpath.txt
by fedora-badges 07 Dec '11

07 Dec '11

#60: Devise a better location for critpath.txt -------------------------+-------------------------------------------------- Reporter: kparal | Owner: Type: enhancement | Status: new Priority: major | Milestone: Component: Wiki | Version: Keywords: | -------------------------+-------------------------------------------------- critpath.txt lists critical path packages [1]. Currently it is referenced as {{{ http://kojipkgs.fedoraproject.org/mash/rawhide-<<CURRENT DATE>>/logs/critpath.txt }}} or {{{ http://kojipkgs.fedoraproject.org/mash/rawhide-<<YESTERDAY DATE>>/logs/critpath.txt }}} depending on which page. Similarly for branched releases. This is not optimal. Very often the url just doesn't work. For example at the time of writing critpath.txt on F13 Alpha Release Criteria page [2] references rawhide-20100407/ directory, where critpath.txt is not available. Nor it is available in rawhide-20100406/ directory. Only in rawhide-20100405/ it is finally available. The very page Critical Path Packages [1] references critpath.txt for Branched release, which is available ATM, but also references critpath.txt for Rawhide, which is again not available. Let's imagine other use case - some automated tool must decide whether package in the critical path. It can't rely on path which are constantly changing and sometimes are not available. critpath.txt is created dynamically, so we probably must have the contents created every day or so. But we should devise such mechanism that the location is stable and the file is always available (if new contents fails to create, the old contents is still available). That will also allow third-party tools to rely on it. [1] https://fedoraproject.org/wiki/Critical_Path_Packages [[BR]] [2] https://fedoraproject.org/wiki/Fedora_13_Alpha_Release_Criteria -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/60> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 2

[Fedora QA] #79: Add firstboot release criteria
by fedora-badges 09 Aug '11

09 Aug '11

#79: Add firstboot release criteria ---------------------------+------------------------------------------------ Reporter: jlaska | Owner: Type: enhancement | Status: new Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Keywords: retrospective | ---------------------------+------------------------------------------------ = problem = See [https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=574596 RHBZ#574596 - Smolt does not run at firstboot] = analysis = We don't have a way to know what firstboot modules are expected, we need to document this as release criteria, or write a test to report the issue (for details, see RHBZ #574596). = enhancement recommendation = The release criteria do not include what image:Package-x-generic-16.pngfirstboot modules are intended for Fedora. Often, firstboot some modules are missing or disabled, and we don't notice or know whether this is a blocker. Recommend clarifying the use cases of firstboot in the release criteria. -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/79> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 2

[Fedora QA] #116: Clarify https://fedoraproject.org/wiki/QA:Testcase_Mediakit_FileConflicts to say that explicit Conflicts: are acceptable
by fedora-badges 23 Jun '11

23 Jun '11

#116: Clarify https://fedoraproject.org/wiki/QA:Testcase_Mediakit_FileConflicts to say that explicit Conflicts: are acceptable ----------------------+----------------------------------------------------- Reporter: adamwill | Owner: rhe Type: defect | Status: new Priority: major | Milestone: Component: Wiki | Version: Keywords: | ----------------------+----------------------------------------------------- = bug description = Please update the https://fedoraproject.org/wiki/QA:Testcase_Mediakit_FileConflicts test case to clarify that explicit Conflicts: tags are okay. = bug analysis = The relevant release criterion is "No file conflicts or unresolved package dependencies during a media-based (CD/DVD) install". The term "file conflicts" here is important. I believe it's intended to mean that only cases where the files in two packages conflict, but there is no explicit Conflicts: tag in the packages. Where the conflict is properly marked in the packages with Conflicts: tags, this should not be considered an infringement of the criterion. = fix recommendation = Change expected results: 2. No file conflicts were detected for packages included in the media kit, unless the conflicting packages also have explicit Conflicts: tags -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/116> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 19

[Fedora QA] #84: Define install matrix baseline
by fedora-badges 23 Jun '11

23 Jun '11

#84: Define install matrix baseline ---------------------------+------------------------------------------------ Reporter: jlaska | Owner: Type: enhancement | Status: new Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Keywords: retrospective | ---------------------------+------------------------------------------------ = problem = * We almost didn't find [http://bugzilla.redhat.com/578633 RHBZ#578633 - Unable to enter passphrase to unlock encrypted disk partitions] * When re-using test results from a previous test run, it's possible to include results for tests that should be re-tested. This would be bad. = analysis = * F-13 Beta candidate#3 didn't include the correct version of ''plymouth'' (see RHBZ #578633). * Since it was Beta#3 and not much was supposed to change from Beta#2, some test results from the previous Beta#2 candidate were carried forward. * Thankfully, QA found the problem before release while running the QA:Testcase_Anaconda_autopart_(encrypted)_install test. = enhancement recommendation = * Establish better guidelines about which test results can be carried forward from one candidate to the next (this isn't easy). Or perhaps, establish a subset of tests that all respins must undergo. * Perhaps link to, or document at http://fedoraproject.org/wiki/QA:Installation_validation_testing -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/84> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 6

[Fedora QA] #86: Add test coverage for text-mode upgrade
by fedora-badges 15 Jun '11

15 Jun '11

#86: Add test coverage for text-mode upgrade ---------------------------+------------------------------------------------ Reporter: jlaska | Owner: Type: enhancement | Status: new Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Keywords: retrospective | ---------------------------+------------------------------------------------ = problem = See [http://bugzilla.redhat.com/590823 RHBZ#590823 - Text-mode upgrade fails when installing new bootloader -- KeyError: 'bootloader'] = analysis = Some tests in the [https://fedoraproject.org/wiki/Test_Results:Current_Installation_Test install test matrix] involve user interface components in both text-mode and graphical-mode (native or VNC). One such bug (RHBZ #590823) involved a problem found only during text-mode upgrades to Fedora 13. The text-mode upgrade screen shows different options thanenhancement recommendation = = enhancement recommendation = I'm hesitant to recommend specific text-mode tests for each user interface element that differs in text-mode and graphical-mode (it would be exhaustive but unachievable). As a catch-all, testing a text-mode install is included in the current matrix. Is there something small that can be done to cover the gap? 1. Perhaps adding a text-mode upgrade test case to the install matrix, or ... 2. Adding variations to the existing upgrade tests [https://fedoraproject.org/wiki/QA:Testcase_Anaconda_Upgrade_Update_Bootload… QA:Testcase_Anaconda_Upgrade_Update_Bootloader] and [https://fedoraproject.org/wiki/QA:Testcase_Anaconda_Upgrade_Skip_Bootloader QA:Testcase_Anaconda_Upgrade_Skip_Bootloader] to also test text-mode, or ... 3. <insert better idea here> -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/86> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 11

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

test August 2010