#151: Tests consistent with Criterion ---------------------------+------------------------------------------------ Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Keywords: retrospective | ---------------------------+------------------------------------------------ = problem =
Criteria was kept changing during F-14 test cycle, but some installation tests didn't update accordingly. On the other hand, the criterion should be modified to include preupgrade_from_older_release test and new tests created for F-15.
= enhancement recommendation =
Installation tests need be reviewed and updated to fit for the criterion. Criterion for preupgrade_from_older_release test is required.
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Changes (by adamwill):
* cc: adamwill, jlaska (added)
Comment:
This is still valid, we have several criteria for which we do not have matching tests, on both installation and 'desktop' side (we may want a third matrix for things that aren't really either). Let's work on this for F16.
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 16 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Changes (by jlaska):
* milestone: Fedora 15 => Fedora 16
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 16 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by adamwill):
I have done a survey of the alpha criteria, checking for associated tests. Here are the results. 'partial' indicates that relevant tests are present but do not entirely cover the criterion's requirements. 'not-in-matrix' indicates a test case is written but not included in the current test matrices. 'NO TEST' indicates that, as far as I could find, there is no test case written for the criterion. Following these keywords, I have listed the relevant test cases for each criterion (where applicable).
We can open subsidiary tickets to this bug for each specific issue identified here.
There must be no file conflicts (cases where the files in some packages conflict but the packages have explicit Conflicts: tags are acceptable) or unresolved package dependencies during a media-based (CD/DVD) install
https://fedoraproject.org/wiki/QA:Testcase_Mediakit_FileConflicts
All dedicated installer images must boot to the graphical boot menu and allow the user to select install options. If no option is selected, the installer should load after a reasonable timeout
partial https://fedoraproject.org/wiki/QA/TestCases/BootMethodsBootIso https://fedoraproject.org/wiki/QA/TestCases/BootMethodsDvd https://fedoraproject.org/wiki/QA:Testcase_Boot_Methods_efidisk.img
The installer must boot (if appropriate) and run on all primary architectures from default live image, DVD, and boot.iso install media
not-in-matrix https://fedoraproject.org/wiki/QA:Testcase_Live_Image_Boot https://fedoraproject.org/wiki/QA/TestCases/BootMethodsBootIso https://fedoraproject.org/wiki/QA/TestCases/BootMethodsDvd
The graphical boot menu for all installation images should include an entry which causes both installation and the installed system to use a generic, highly compatible video driver (such as 'vesa'). This mechanism should work correctly, launching the installer and attempting to use the generic driver
NO TEST
The installer must be able to use at least one of the HTTP or FTP remote package source options
https://fedoraproject.org/wiki/QA:Testcase_Http_Repository (redundant with ftp_repository?) https://fedoraproject.org/wiki/QA:Testcase_Additional_Http_Repository https://fedoraproject.org/wiki/QA:Testcase_Additional_Ftp_Repository https://fedoraproject.org/wiki/QA:Testcase_Ftp_Repository
The installer must be able to use the DVD local package source options
https://fedoraproject.org/wiki/QA/TestCases/InstallSourceDvd
The installer must be able to complete an installation using the text, graphical and VNC installation interfaces
https://fedoraproject.org/wiki/QA/TestCases/InstallSourceDvd https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_Text https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_VNC
The installer must be able to complete package installation with the default package set for each supported installation method
https://fedoraproject.org/wiki/QA/TestCases/InstallSourceDvd https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_Text https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_VNC
The installer must be able to complete an installation using IDE, SATA and SCSI storage devices, with the default file system and LVM
https://fedoraproject.org/wiki/QA/TestCases/PartitioningExt4OnNativeDevice https://fedoraproject.org/wiki/QA:Testcase_install_to_SATA_device https://fedoraproject.org/wiki/QA:Testcase_install_to_SCSI_device https://fedoraproject.org/wiki/QA:Testcase_install_to_PATA_device
The installer must be able to complete an installation using the entire disk, existing free space, or existing Linux partitions methods, with or without encryption enabled
partial https://fedoraproject.org/wiki/QA:Testcase_Anaconda_autopart_install https://fedoraproject.org/wiki/QA:Testcase_Anaconda_autopart_%28use_free_spa... https://fedoraproject.org/wiki/QA:Testcase_Anaconda_autopart_%28encrypted%29... (existing partitions method does not seem to be tested)
The rescue mode of the installer must start successfully and be able to detect and mount an existing default installation
https://fedoraproject.org/wiki/QA:Testcase_Anaconda_rescue_mode
The installer must be able to report failures to Bugzilla, with appropriate information included
https://fedoraproject.org/wiki/QA:Testcase_Anaconda_save_traceback_to_bugzil...
In most cases (see Blocker_Bug_FAQ), a system installed according to any of the above criteria (or the appropriate Beta or Final criteria, when applying this criterion to those releases) must boot to the 'firstboot' utility on the first boot after installation, without unintended user intervention. This includes correctly accessing any encrypted partitions when the correct passphrase is supplied. The firstboot utility must be able to create a working user account
NO TEST
Following on from the previous criterion, after firstboot is completed and on subsequent boots, a system installed according to any of the above criteria (or the appropriate Beta or Final criteria, when applying this criterion to those releases) must boot to a working graphical environment without unintended user intervention. This includes correctly accessing any encrypted partitions when the correct passphrase is supplied
NO TEST (partly implied by more sophisticated desktop tests)
When booting a system installed without a graphical environment, or when using a correct configuration setting to cause an installed system to boot in non-graphical mode, the system should boot to a state where it is possible to log in through at least one of the default virtual consoles
partial https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_Text
It must be possible to run the default web browser and a terminal application from all release-blocking desktop environments. The web browser must be able to download files, load extensions, and log into FAS
https://fedoraproject.org/wiki/QA:Testcase_desktop_browser
The installed system must be able to download and install updates with yum and the default graphical package manager in all release-blocking desktops
https://fedoraproject.org/wiki/QA:Testcase_desktop_updates
The default Fedora artwork must either refer to the current Fedora release under development (Fedora 16), or reference an interim release milestone (e.g. Alpha or Beta). If a release version number is used, it must match the current Fedora release under development. This includes artwork used in the installer, firstboot, graphical boot, graphical login and desktop background.
NO TEST
A system logging infrastructure must be available and enabled by default. It must provide at least basic local file-based logging of kernel messages, and allow other components to write log messages. This must be done in accordance with relevant standards accepted by the Project
NO TEST
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 16 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by adamwill):
For "All dedicated installer images must boot to the graphical boot menu and allow the user to select install options. If no option is selected, the installer should load after a reasonable timeout": I have updated the relevant test cases to specify that navigating the boot menu must work. that resolves the 'partial' issue for this criterion.
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 16 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by adamwill):
For "The graphical boot menu for all installation images should include an entry which causes both installation and the installed system to use a generic, highly compatible video driver (such as 'vesa'). This mechanism should work correctly, launching the installer and attempting to use the generic driver" - filed ticket #200: https://fedorahosted.org/fedora- qa/ticket/200
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 16 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by adamwill):
For "The installer must be able to complete an installation using the entire disk, existing free space, or existing Linux partitions methods, with or without encryption enabled", filed ticket #201 to add an 'existing Linux partitions' test.
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 16 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by adamwill):
For "In most cases (see Blocker_Bug_FAQ), a system installed according to any of the above criteria (or the appropriate Beta or Final criteria, when applying this criterion to those releases) must boot to the 'firstboot' utility on the first boot after installation, without unintended user intervention. This includes correctly accessing any encrypted partitions when the correct passphrase is supplied. The firstboot utility must be able to create a working user account": filed #202 to request a firstboot test case.
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 16 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by adamwill):
For "Following on from the previous criterion, after firstboot is completed and on subsequent boots, a system installed according to any of the above criteria (or the appropriate Beta or Final criteria, when applying this criterion to those releases) must boot to a working graphical environment without unintended user intervention. This includes correctly accessing any encrypted partitions when the correct passphrase is supplied": filed #203 to request a test case.
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 16 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by adamwill):
For "When booting a system installed without a graphical environment, or when using a correct configuration setting to cause an installed system to boot in non-graphical mode, the system should boot to a state where it is possible to log in through at least one of the default virtual consoles": I updated https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_Text to better reflect the relevant release criteria.
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 16 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by adamwill):
For "The default Fedora artwork must either refer to the current Fedora release under development (Fedora 16), or reference an interim release milestone (e.g. Alpha or Beta). If a release version number is used, it must match the current Fedora release under development. This includes artwork used in the installer, firstboot, graphical boot, graphical login and desktop background. " (and corresponding Final criterion): filed #204 to request test cases.
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 16 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by adamwill):
For "A system logging infrastructure must be available and enabled by default. It must provide at least basic local file-based logging of kernel messages, and allow other components to write log messages. This must be done in accordance with relevant standards accepted by the Project": filed #205 to request a test case.
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 16 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by rhe):
Many thanks for your survey, Adam! I've read it carefully and updated sub- tickets accordingly. Please note that the install source and repository tests in the [https://fedoraproject.org/wiki/QA:Fedora_15_Install_Results_Template F-15 install template] have been recategorized and updated to [https://fedoraproject.org/wiki/Category:Installation_Repository install repository category] in [https://fedoraproject.org/wiki/QA:Fedora_16_Install_Results_Template F-16 install template] to cover the repo=, askmethod, and graphically repo adding methods(see ticket#173 ). It wouldn't reduce the test coverage expect that the HTTP and FTP were included in one test now.
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 16 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by adamwill):
update for criterion "The graphical boot menu for all installation images should include an entry which causes both installation and the installed system to use a generic, highly compatible video driver (such as 'vesa'). This mechanism should work correctly, launching the installer and attempting to use the generic driver": rhe points out that there is in fact a test in the install matrix, https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_Basic_Vid... .
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: adamwill Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 16 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Changes (by adamwill):
* owner: rhe => adamwill * status: new => assigned
Comment:
Updates: created test cases for 'firstboot' and 'initial startup' criteria (see tickets #202 and #203). I also re-considered merging the text install, firstboot and startup tests; they should really be separate, so the new test cases cover both text and graphic cases, and the installer tests have been restricted to considering installer functions only.
I've put the new test cases in a new 'base' category (per their page names, anyway) to reflect that I think we're going to need a new matrix for tests that enforce criteria that aren't really installer or desktop related exactly: firstboot, system logging, startup and so on.
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: adamwill Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 16 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by adamwill):
Here are the results of the Beta survey. Will follow up with tickets for the action areas soon.
The network installation image, DVD image, and live images for release- blocking desktops must meet current size requirements
https://fedoraproject.org/wiki/QA:Testcase_Mediakit_ISO_Size
The installer must boot and run on systems using EFI other than Apple Macs
https://fedoraproject.org/wiki/QA:Testcase_Boot_Methods_efidisk.img (and all boot_methods tests)
The installer must be able to use the HTTP, FTP and NFS remote package source options
https://fedoraproject.org/wiki/QA:Testcase_install_repository_NFS_graphical (others are Alpha)
At the package installation stage, the install should show no uncategorized package groups
NO TEST
The installer must be able to use all kickstart delivery methods
https://fedoraproject.org/wiki/QA:Testcase_Kickstart_Http_Server_Ks_Cfg https://fedoraproject.org/wiki/QA:Testcase_Kickstart_File_Path_Ks_Cfg https://fedoraproject.org/wiki/QA:Testcase_Kickstart_Hd_Device_Path_Ks_Cfg https://fedoraproject.org/wiki/QA:Testcase_Kickstart_Nfs_Server_Path_Ks_Cfg
The installer must be able to create and install to software, hardware or BIOS RAID-0, RAID-1 or RAID-5 partitions for anything except /boot
https://fedoraproject.org/wiki/QA:Testcase_Partitioning_On_Software_RAID https://fedoraproject.org/wiki/QA:Testcase_Install_to_BIOS_RAID https://fedoraproject.org/wiki/QA:Testcase_Install_to_Hardware_RAID
The installer must be able to successfully complete an upgrade installation from a clean, fully updated default installation (from any official install medium) of the previous stable Fedora release, either via preupgrade or by booting to the installer manually. The upgraded system must meet all release criteria
https://fedoraproject.org/wiki/QA:Testcase_Anaconda_Upgrade_New_Bootloader https://fedoraproject.org/wiki/QA:Testcase_Anaconda_Upgrade_Skip_Bootloader https://fedoraproject.org/wiki/QA:Testcase_Anaconda_Upgrade_Update_Bootloade... https://fedoraproject.org/wiki/QA:Testcase_Anaconda_Upgrade_Encrypted_Root https://fedoraproject.org/wiki/QA:Testcase_Preupgrade
Any installation method or process designed to run unattended must do so (there should be no prompts requiring user intervention)
NO TEST
The rescue mode of the installer must be able to detect and mount (read- write and read-only) LVM, encrypted, and RAID (BIOS, hardware, and software) installations
partial https://fedoraproject.org/wiki/QA:Testcase_Anaconda_rescue_mode
When booting a system installed without a graphical environment, or when using a correct configuration setting to cause an installed system to boot in non-graphical mode, the system should provide a working login prompt without any unintended user intervention when boot is complete, and all virtual consoles intended to provide a working login prompt should do so
not-in-matrix https://fedoraproject.org/wiki/QA:Testcase_base_startup
The release must boot successfully as a virtual guest in a situation where the virtual host is running the same release (using Fedora's current preferred virtualization technology)
NO TEST (explicit; in practice, much testing is done in virt environment)
The release must boot successfully as a virtual guest in a situation where the virtual host is running the previous stable Fedora release (using Fedora's current preferred virtualization technology)
NO TEST
The release must boot successfully as a virtual guest in a situation where the virtual host is running a supported Xen implementation
NO TEST
In most cases, the installed system must be able to play back sound with gstreamer-based applications (see Blocker_Bug_FAQ)
http://fedoraproject.org/wiki/QA:Testcase_audio_basic
No part of any release-blocking desktop's panel (or equivalent) configuration should crash or be entirely non-functional on boot of the installed system using default installation choices
http://fedoraproject.org/wiki/QA:Testcase_desktop_panel_basic
Automatic mounting on insertion of removable media must work in release- blocking desktops
http://fedoraproject.org/wiki/QA:Testcase_desktop_automount
The default update manager in release-blocking desktops must not periodically check for updates when the system is booted live, but must periodically check for updates when running on an installed system
http://fedoraproject.org/wiki/QA:Testcase_desktop_updates
All release-blocking desktops' offered mechanisms (if any) for shutting down, logging out and rebooting must work
http://fedoraproject.org/wiki/QA:Testcase_desktop_login
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: adamwill Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 16 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by adamwill):
Created tickets for all the deficiencies listed above (except the not-in- matrix Testcase_base_startup - I'm handling the proposed Base matrix through the list):
https://fedorahosted.org/fedora-qa/ticket/216 https://fedorahosted.org/fedora-qa/ticket/217 https://fedorahosted.org/fedora-qa/ticket/218 https://fedorahosted.org/fedora-qa/ticket/219
all are currently assigned to rhe, but if anyone wants to pitch in and help out, please do!
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: adamwill Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 17 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Changes (by adamwill):
* milestone: Fedora 16 => Fedora 17
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: pschindl Type: enhancement | Status: new Priority: major | Milestone: Fedora 17 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Changes (by adamwill):
* owner: adamwill => pschindl * status: assigned => new
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: pschindl Type: enhancement | Status: new Priority: major | Milestone: Fedora 17 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by adamwill):
Petr, the work remaining here is:
* Ensure there are test cases in the matrices for all Final criteria * Reverse concordance: go through the set of all test cases and make sure all the ones that are listed as 'Alpha', 'Beta' or 'Final' are actually associated with a criterion from the appropriate level. If there's no criterion that matches the test case, then decide whether it makes more sense to create a criterion, or to adjust the 'importance level' of the test case (or drop it altogether).
Please let me know if you have any problems or questions!
#151: Tests consistent with Criterion --------------------------+------------------------------------------------- Reporter: rhe | Owner: pschindl Type: enhancement | Status: new Priority: major | Milestone: Fedora 17 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by adamwill):
er, by 'set of all test cases', I mean all *release validation* test cases - all the test cases that show up on the Install, Desktop and Base validation matrices.
#151: Tests consistent with Criterion --------------------------+--------------------------- Reporter: rhe | Owner: pschindl Type: enhancement | Status: new Priority: major | Milestone: Fedora 17 Component: Wiki | Version: Resolution: | Keywords: retrospective Blocked By: | Blocking: --------------------------+---------------------------
Comment (by adamwill):
Petr, I think we got a long way along with this. Would you say we completed it? I don't recall if we've finished the reverse concordance.
#151: Tests consistent with Criterion --------------------------+--------------------------- Reporter: rhe | Owner: pschindl Type: enhancement | Status: new Priority: major | Milestone: Fedora 17 Component: Wiki | Version: Resolution: | Keywords: retrospective Blocked By: | Blocking: --------------------------+---------------------------
Comment (by pschindl):
I don't think we can complete this any time. Criteria are changing quite quickly. Changes I did before Fedora 17 pre-release testing:
I added new test case for uncovered criterion: "All services in a default install must start properly" -> https://fedoraproject.org/wiki/QA:Testcase_Services_start (there is a problem with this, which we should discuss: http://lists.fedoraproject.org/pipermail/test/2012-February/105685.html)
I removed http://lists.fedoraproject.org/pipermail/test/2012-February/105685.html test case from installation matrix.
I changed release level of https://fedoraproject.org/wiki/QA:Testcase_Kickstart_Http_Server_Ks_Cfg and https://fedoraproject.org/wiki/QA:Testcase_Mediakit_ISO_Size to beta
I changed release level of https://fedoraproject.org/wiki/QA:Testcase_Anaconda_traceback_debug_mode to non-blocking, because debug mode is not supposed to be used by users.
I changed release level of https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_serial_co... to Beta and level of https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_Cmdline to Final.
I added new alpha criterion: "A correct checksum must be published for each official release image." and final criterion "If there is embedded checksum on ISO media, it must be correct." and I set release level of https://fedoraproject.org/wiki/QA:Testcase_Mediakit_ISO_Checksums to Alpha / Final
I added new alpha criteria: * "The installer must be able to download and use an [[Anaconda/Updates| installer update image]] from an HTTP server"
I added new beta criteria: * "The installer must be able to use an [[Anaconda/Updates|installer update image]] retrieved from removable media, remote installation source and HTTP url" - this one was lately moved to the final" * "The installer must be able to complete an installation using the serial console interface"
I added new final criteria: * "All release media must include a standalone memory test utility. A boot menu option to launch this utility must be present and must work correctly." * "The installer must be able to complete an installation using all supported interfaces"
I changed criterion "The installer must be able to report failures to Bugzilla, with appropriate information included" to "The installer must be able to report failures to Bugzilla and local disk, with appropriate information included" and I changed release level of https://fedoraproject.org/wiki/QA:Testcase_Anaconda_save_traceback_to_remote... to non-blocking
All those changes were made after discussion on test mailing list.
#151: Tests consistent with Criterion --------------------------+--------------------------- Reporter: rhe | Owner: pschindl Type: enhancement | Status: new Priority: major | Milestone: Fedora 17 Component: Wiki | Version: Resolution: | Keywords: retrospective Blocked By: | Blocking: --------------------------+---------------------------
Comment (by pschindl):
Because there have been added new criteria since I went through this I propose to revise it again before Fedora 18 pre-release testing. I know about some unclosed topics (http://lists.fedoraproject.org/pipermail/test/2012-February/105468.html http://lists.fedoraproject.org/pipermail/test/2012-January/105260.html). I'm not sure if I should open new ticket or only change milestone to this one, what do you think Adam?
Also, when we will be accepting a new criterion we should probably demand test case for testing feature from this criterion and reverse approach - if we create new test case and add release level to it, we should make sure that it's covered by some criterion.
#151: Tests consistent with Criterion --------------------------+--------------------------- Reporter: rhe | Owner: pschindl Type: enhancement | Status: new Priority: major | Milestone: Fedora 17 Component: Wiki | Version: Resolution: | Keywords: retrospective Blocked By: | Blocking: --------------------------+---------------------------
Comment (by adamwill):
pschindl: I think it's probably best to open a new ticket, I don't think everlasting tickets are great, it gets messy and doesn't allow us to track what we've achieved in the past. I agree it would probably be a good idea to require newly-proposed validation tests and criteria to match as you suggest.
You have an entry on the retrospective that's broadly about this area, I know - I was already planning to file a new ticket as a follow-up for the retrospective.
#151: Tests consistent with Criterion --------------------------+--------------------------- Reporter: rhe | Owner: pschindl Type: enhancement | Status: new Priority: major | Milestone: Fedora 17 Component: Test cases | Version: Resolution: | Keywords: retrospective Blocked By: | Blocking: --------------------------+--------------------------- Changes (by adamwill):
* component: Wiki => Test cases
#151: Tests consistent with Criterion --------------------------+--------------------------- Reporter: rhe | Owner: pschindl Type: enhancement | Status: closed Priority: major | Milestone: Fedora 17 Component: Test cases | Version: Resolution: fixed | Keywords: retrospective Blocked By: | Blocking: --------------------------+--------------------------- Changes (by pschindl):
* resolution: => fixed * status: new => closed
Comment:
There are separate tickets for newer Fedora.
-
fedora-badges