#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Changes (by adamwill):

* cc: jlaska (added)

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Changes (by fenris02):

* cc: fenris02 (added)

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Changes (by fcami):

* cc: fcami (added)

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Changes (by jlaska):

* milestone: => Fedora 15

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by adamwill):

here's a draft of a sample test case, for mdadm:

https://fedoraproject.org/wiki/User:Adamwill/Draft_QA_testcase_mdadm_package...

the production name would be QA:Testcase_mdadm_package_software , with QA:Testcase being our standard prefix for test cases (it's not up for debate within the context of this ticket, if we change it we must change it for all test cases), then 'mdadm' being the source package name, 'package' indicating this (that it's named based on a src.rpm name - perhaps it should come before the name rather than after?), and 'software' being the name of this particular test (referring to software RAID).

what does everyone think, particularly of the naming and the categorization? I don't quite feel that this one's nailed down yet, if anyone has smart ideas that'd be great. thanks.

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by jlaska):

Replying to [comment:9 adamwill]:

here's a draft of a sample test case, for mdadm:

https://fedoraproject.org/wiki/User:Adamwill/Draft_QA_testcase_mdadm_package...

the production name would be QA:Testcase_mdadm_package_software , with

QA:Testcase being our standard prefix for test cases (it's not up for debate within the context of this ticket, if we change it we must change it for all test cases), then 'mdadm' being the source package name, 'package' indicating this (that it's named based on a src.rpm name - perhaps it should come before the name rather than after?), and 'software' being the name of this particular test (referring to software RAID).

Seems sensible from a general best practices point of view (QA:Testcase_%{sourcerpm}_%{userdefined}). The "_package" seems extraneous, but I might be missing something. With regards to finding applicable tests for a certain update, the most important thing is the category structure, right? That's how other tools will locate applicable tests.

what does everyone think, particularly of the naming and the

categorization? I don't quite feel that this one's nailed down yet, if anyone has smart ideas that'd be great.

In the page you linked, that test is in three categories ... 1. Category:critpath_test_cases 2. Category:mdadm_test_cases 3. Category:mdadm_critpath_test_cases I don't yet see the big picture for how the Categories will be grouped/organized, can you provide a hierarchical example?

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by jlaska):

Awesome, I like how you've focused on writing the test case *once*, and distinguishing the content from how it's grouped/organized. That distinction will help us with a possible future TCMS migration.

We discussed on IRC, but following up here for a larger audience.... I wasn't sure if the 3 groups you listed (critpath_test_cases, mdadm_test_cases, mdadm_critpath_test_cases) were all needed. Certainly they help for convenience, but it sounds like the consumers of this hierarchy would only need to find 1. all mdadm test cases (''Category:mdadm_test_cases'') ... OR ... 2. All critical path mdadm test cases (intersection between ''Category:mdadm_test_cases'' and ''Category:Critpath_test_cases'').

I didn't initially think the intersection would be challenging to do. But thinking ahead to some remote API that consumers would use to find applicable tests, we may want to use the three categories as you suggest to avoid having each consumer defining logic for how it finds applicable tests. So in your example, we would treat ''Category:Critpath_test_cases'' as a way to denote test priority (where high == critpath, media, low). ''Category:mdadm_test_cases'' would be used to indicate membership to a test plan (the mdadm test plan). And finally, ''Category:Critpath_mdadm_test_cases'' would be our way to provide all *high* priority tests in the mdadm test plan. Does that make sense?

I know I'm getting too far ahead, but I'm trying to articulate our wiki organization in TCMS terms, and to reduce the requests consumers would have down to a simple API.

Using the 3 category suggestion, my ASCII-art attempt at a top-down view from ''Category:Test_cases'' would be ... {{{ Category:Test_Cases | +-> Category:mdadm_test_cases | | | +-> Category:Critpath_mdadm_test_cases (e.g. high priority tests) | | | | | +-> QA:Testcase_mdadm_important_use_case_1 | | +-> QA:Testcase_mdadm_important_use_case_2 | | +-> QA:Testcase_mdadm_important_use_case_3 | | | +-> QA:Testcase_mdadm_obscure_use_case_4 | +-> QA:Testcase_mdadm_obscure_use_case_5 | +-> QA:Testcase_mdadm_obscure_use_case_6 | +-> Category:kernel_test_cases | | | +-> Category:Critpath_kernel_test_cases (e.g. high priority tests) | | | +-> ... . . . }}}

And then the same view, but for just ''Category:Critpath_test_cases''. This is probably the easiest thing for consumers of this data to manage.

{{{ Category:Critpath_Test_Cases | +-> Category:Critpath_mdadm_test_cases (e.g. high priority tests) | | | +-> QA:Testcase_mdadm_important_use_case_1 | +-> QA:Testcase_mdadm_important_use_case_2 | +-> QA:Testcase_mdadm_important_use_case_3 +-> Category:Critpath_kernel_test_cases (e.g. high priority tests) | | | +-> QA:Testcase_kernel_important_use_case_1 | +-> QA:Testcase_kernel_important_use_case_2 | +-> ... . . . }}}

While it's an extra category for each src.rpm, this seems like a really sane approach.

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by jlaska):

Replying to [comment:13 adamwill]:

Honestly, not really, no. =) The thinking behind creating the

technically-not-needed combined category was 90% 'half-assing it as I was going along' and 10% for *non*-tool based cases: people just browsing the tests. It's easy via the API for tools to determine which test cases are members of both categories, but it's not easy for a user just browsing the Wiki to do it, AFAIK - there's no easy way to browse pages that are members of both one category and another category in the mediawiki user interface, that I know of.

Correct, not without http://www.mediawiki.org/wiki/Extension:Multi- Category_Search

But really, I'm not married to the idea. (That's why I like it, ba-dum

tish. Tip your server.) I don't really see any problem with having tools derive the combination themselves - it's not like they'll all be implementing it in wildly different ways, I can't see how that would happen.

I can and have, I'd like to stay away from that. We're talking about a *small* set of tools at this point so the fallout would be minimal and manageable, agreed. My main point is, let's do our best to keep test cases and related test case metadata within the realm of TCMS (whether it's wiki or nitrate).

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by adamwill):

sigh, I keep getting ready to send this out to lists and then hitting one more snag. =)

so, in discussion with jlaska, we both think it makes most sense to only create two new types of category: one for associating test cases with specific packages, one for marking them as critical path test cases.

Critical path test cases should be in the category Critical_path_Test_Cases . Note that this means *any test case which is primarily concerned with verifying critical path test cases* should be in this category. It doesn't mean any test case associated with a critical path package should be in the category, because critical path packages can have non-critical path uses and hence non-critical path test cases: a test case to test, say, tape drive functionality in the kernel is associated with a critical path package (the kernel) but is not a critical path test.

The category name for specific source packages should be:

Package_(srcrpmname)_Test_Cases

same reasoning as my initial proposal for the test case name: we need to specify that (srcrpmname), in this context, is specifically a source RPM name. I can imagine, for instance, we might want to create a generic category for Python test packages, called Category:Python_Test_Cases, and put test cases into that category which are not actually test cases for the Python .src.rpm. So we'd also have a Category:Package_python_Test_Cases . That could probably be a member of Category:Python_Test_Cases , but not vice versa.

I think the page name of each test case can be a bit more flexible and we don't really need to worry about defining it, the categories are the crucial thing.

Do we also want to suggest a category for each .src.rpm along the lines of:

Category:Package_python_core_Test_Cases

? Rationale: I'm thinking about how we want the bodhi or fedora-easy-karma listing for each package to look. I'm thinking of something like:

-------------------------------------------------------

Update: python-2.7.9-blahblah

This update does blah blah foo foo whee whee poop.

The following test cases can be used to check the functionality of this package.

'''Critical path tests'''

These tests verify critical path functionality (link to critical path process page) and should take highest priority:

* links to all tests in Category:Package_python_Test_Cases *and* Category:Critical_path_Test_Cases

'''Core functionality tests'''

These tests verify core functionality of the package and should take highest priority after critical path tests:

* links to all tests in Category:Package_python_core_Test_Cases (but not Category:Critical_path_Test_Cases)

'''Remaining tests'''

These tests verify non-critical path, non-core functionality:

* links to all other tests (in Category:Package_python_Test_Cases but not Category:Package_python_core_Test_Cases or Category:Critical_path_Test_Cases)

------------------------------------------------------ ------------------------------------------------------

The thinking being that some packages will grow fairly large sets of tests (abrt already has, for instance) and it would help to isolate core functionality tests for rapid sanity checking.

random side observation: our naming of test case pages and categories currently stinks.

next big concern: where do we effectively document this? All we currently have on test cases is:

https://fedoraproject.org/wiki/Category:Test_Cases

I can add some overview and stuff to that page but it's not a terribly visible page, really. I guess the best approach is to edit that page and add some links to it from strategic other places, especially developer instruction pages. Thoughts?

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by jlaska):

Replying to [comment:16 adamwill]:

The category name for specific source packages should be:

Package_(srcrpmname)_Test_Cases

That works fine. I think the "Package_" prefix is not needed, but it certainly doesn't hurt anything to have it (just longer wiki page names "Category:Package_mobile-broadband-provider-info_Test_Cases" vs "Category :Mobile-broadband-provider-info_Test_Cases", but that's certainly not horrible).

same reasoning as my initial proposal for the test case name: we need to

specify that (srcrpmname), in this context, is specifically a source RPM name. I can imagine, for instance, we might want to create a generic category for Python test packages, called Category:Python_Test_Cases, and put test cases into that category which are not actually test cases for the Python .src.rpm. So we'd also have a Category:Package_python_Test_Cases . That could probably be a member of Category:Python_Test_Cases , but not vice versa.

I think the page name of each test case can be a bit more flexible and

we don't really need to worry about defining it, the categories are the crucial thing.

Definitely!

Do we also want to suggest a category for each .src.rpm along the lines

of:

Category:Package_python_core_Test_Cases

? Rationale: I'm thinking about how we want the bodhi or fedora-easy-

karma listing for each package to look. I'm thinking of something like:

---

Update: python-2.7.9-blahblah

This update does blah blah foo foo whee whee poop.

The following test cases can be used to check the functionality of this

package.

'''Critical path tests'''

These tests verify critical path functionality (link to critical path

process page) and should take highest priority:

• links to all tests in Category:Package_python_Test_Cases *and*

Category:Critical_path_Test_Cases

'''Core functionality tests'''

These tests verify core functionality of the package and should take

highest priority after critical path tests:

• links to all tests in Category:Package_python_core_Test_Cases (but not

Category:Critical_path_Test_Cases)

'''Remaining tests'''

These tests verify non-critical path, non-core functionality:

• links to all other tests (in Category:Package_python_Test_Cases but

not Category:Package_python_core_Test_Cases or Category:Critical_path_Test_Cases)

---

---

The thinking being that some packages will grow fairly large sets of

tests (abrt already has, for instance) and it would help to isolate core functionality tests for rapid sanity checking.

The way I read your organization above is that the front-ends will present the tests by priority (critical path, core, *). I really like this visualization, but wonder if it is outside the scope for the first pass. Meaning, I think I'd want to see more focus on just getting critpath tests defined for as many critpath packages, rather than spending time on non- critpath or non-core stuff. With the setup you've designed, it doesn't exclude people from creating non-critpath tests.

An aside, I think the previously discussed Category: organization would lend better to allowing maintainers to prioritize tests themselves. For example ...

bodhi and f-e-k will look for wiki pages under Category:Package_%{sourcerpm}_Test_cases. Any tests in that category would be listed, any tests in sub-categories of that page will be presented as you demonstrate above. So to achieve your desired output, you might organize tests like ...

{{{ Category:Package_python_Test_Cases | +--> Category:Critpath_python_Test_Cases | | | +-> QA:Testcase_python_do_something_1 [1] | +-> QA:Testcase_python_do_something_2 [1] | +-> QA:Testcase_python_do_something_3 [1] | +--> Category:Core_python_Test_Cases | | | +-> QA:Testcase_python_do_something_4 | +-> QA:Testcase_python_do_something_5 | +--> Category:Low_python_Test_Cases | | | +-> QA:Testcase_python_do_something_6 | +-> QA:Testcase_python_do_something_7 | |--> QA:Testcase_python_do_something_8 |--> QA:Testcase_python_do_something_9

[1] Could also exist in Category:Critpath_test_cases, but not required }}}

This would allow maintainers to group the tests as they see fit, but holds that everything must be anchored out of Category:Package_python_Test_Cases. We would still focus on prioritizing development of critpath tests, but the support exists for maintainers/packagers to organize as they feel is appropriate. Also, the mediawiki API provides the needed methods to find and present the information listed above.

Another something to keep in mind, any heavy/complex wiki organization we do may require extra work when migrating data to another system (which is a possible mid-term future scenario).

random side observation: our naming of test case pages and categories

currently stinks.

Sure, page naming wasn't as important when we weren't attempting to integrate it with other tools so I gather some wiki renaming may be required. Or did it stink in another regard?

next big concern: where do we effectively document this? All we

currently have on test cases is:

https://fedoraproject.org/wiki/Category:Test_Cases

I can add some overview and stuff to that page but it's not a terribly

visible page, really. I guess the best approach is to edit that page and add some links to it from strategic other places, especially developer instruction pages. Thoughts?

Agreed. Do we need an SOP/HOWTO for each of the use cases where proventesters, or maintainers, will interact with this setup? Such as ... 1. How to create a test case 2. How to organize test cases 3. How to post results for test cases 4. ... ?

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by adamwill):

here's a draft of a test case creation SOP:

https://fedoraproject.org/wiki/User:Adamwill/Draft_QA_SOP_test_case_creation

it's not really anything specific to this ticket, but it is a prerequisite. I plan to write a (short) SOP on package-specific test cases, linking to this more generic SOP, and then edit the test case category page to link to both (and possibly the test case template page), and then edit strategic pages in the packaging instructions section to point out to these pages. sound good?

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by adamwill):

I tweaked https://fedoraproject.org/wiki/User:Adamwill/Draft_QA_SOP_package_test_plan_... somewhat and https://fedoraproject.org/wiki/User:Adamwill/Draft_QA_SOP_test_case_creation a small amount. I think they're ready to go out for review now. I'll mail the lists for feedback.

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by fcami):

Thank you Adam and James! This is excellent work. Two comments: The extended test cases (at least, the 3D ones) should be run on Intel also, and maybe on Nouveau (do we support 3D there?), so we might need a way to share these across drivers. We also probably need to define a list of hardware we want the tests executed on, as a lot of the bugs we encounter are hardware-dependent, hitting different paths in the radeon/mesa stack. Any idea how to handle this?

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by fcami):

Sorry, I should have used more generic terms in my explanation. Let me try again :)

1. we should share test cases between all supported drivers and not duplicate them - because otherwise, editing a test case relevant to multiple hardware drivers means editing multiple pages.

2. how do we define success/failure (for a go/nogo meeting for instance) when some people manage to run the test cases and others don't, depending on hardware?

Note that 2. is probably more a "see what it's bugzilla" than a "see the test case results" question, so it's moot.

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Changes (by bruno):

* cc: bruno (added)

Comment:

I have tried to create a test for squashfs-tools at: https://fedoraproject.org/wiki/QA:Testcase_squashfs-tools_compression

Does it look reasonable? Most of it is running a test script and making sure it doesn't print any "failed" messages.

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by bruno):

There is no upstream. I needed to make one so that I would have a reasonable test before putting a development version of squashfs-tools in rawhide. (Kyle asked me not to break things :-) I'll take a look at doing the upload and if I think it is as simple as it looks at first glance I'll do it shortly.

This is mostly crit path. squashfs-tools is used in live images. If uncompressing doesn't work then the images won't work. Arguably only gzip needs to work now, but hopefully we'll also need xz for F15. Also the unsquashfs path isn't used by live images, so in theory that could be skipped as well. But it seemed convenient to test these together. If it runs too long, the test data size could be cut back and the test would probably be as good. In my tests it runs reasonably quick on an older machine.

In theory there could be other core tests to make sure extended attributes and special files work correctly. But those features aren't really used by live images currently.

A more complete test, would be to test live images, but there could be other things blocking a live image test and they would take longer to do.

I can add a good results example. The only bad results I saw were when I had bugs in my test script.

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by jlaska):

Replying to [comment:31 bruno]:

I fixed up the test case page for squashfs-tools.

Nice, looks good to me!

One other note about this is that in the past I think testing of

squashfs-tools by proventesters was slow because there wasn't a test case and people didn't know what to test. Eventually nirik ended up doing when I asked him about it. Hopefully this won't be as big of an issue in the future.

That's one of the big reasons Adam is proposing the wiki structure discussed in this ticket, so that we now have a place for people to document repeatable test procedures, and a consistent way to find them. Yay, thank you Adam!

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by bruno):

I think when I first read them I wasn't sure it was required. And since I just had the one test case, I wasn't sure if it was worthwhile. I might have also seen an early version of the instructions as they were being worked on at least somewhat after I did the test case. I'll take care of the category change.

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by adamwill):

thanks for the feedback, I'll update the page soon. yes, you should create a category even for just one test case, because it is still useful to help both people and programs locate it. If there isn't a category, tools like f-e-k and bodhi will not have a way to figure out that the test case should be associated with the package.

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by bruno):

That looks good. I found the real problem in my case, is that I did not realize that there were two documents and I was going mostly off the test case document, rather than the test plan document. I think I ran across both at various times, but when actually writing up the test case I was looking at the test case document. There is a link back to the test plan document, but I didn't check at that time.

I still like the changes to the test plan document. The sort trick I only knew about from the conversation about the radeon test cases. I'd be less likely to have made the mistake of not having a package category if I had been referring to both pages at the time I was actually making up the test case, but with the current wording I would have been sure to have made the package category.

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by jlaska):

I think we can put a fork in the initial proof-of-concept of this ticket. Is there anything else you wanted to track here before closing?

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Changes (by athmane):

* cc: athmane (added)

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by jlaska):

Assuming I'm using my own script correctly ([http://fedorapeople.org/gitweb?p=jlaska/public_git/scripts.git;a=blob;f =get-mediawiki-data get-mediawiki-data]), we have the following packages with wiki test cases.

{{{ Category:Package abrt test cases Category:Package anaconda test cases Category:Package biosdevname test cases Category:Package clamav test cases Category:Package Django test cases Category:Package evince test cases Category:Package firefox test cases Category:Package gnome-color-manager test cases Category:Package gnome-disk-utility test cases Category:Package gammu test cases Category:Package gnome-shell test cases Category:Package httping test cases Category:Package imsettings test cases Category:Package ibus test cases Category:Package mysql test cases Category:Package mysql-libs test cases Category:Package mysql-server test cases Category:Package NetworkManager test cases Category:Package nmap test cases Category:Package nikto test cases Category:Package openvas-scanner test cases Category:Package openvas-libraries test cases Category:Package openvas-client test cases Category:Package preupgrade test cases Category:Package report test cases Category:Package rkhunter test cases Category:Package squashfs-tools test cases Category:Package smolt test cases Category:Package wireshark test cases Category:Package xorg-x11-drv-ati test cases Category:Package xorg-x11-drv-nouveau test cases Category:Package xorg-x11-drv-intel test cases }}}

Combining the list above, with a [http://kojipkgs.fedoraproject.org/mash/rawhide-20110610/logs/critpath.txt recent critpath log], the following critpath packages have wiki test cases:

{{{ anaconda NetworkManager report squashfs-tools xorg-x11-drv-ati xorg-x11-drv-nouveau xorg-x11-drv-intel }}}

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by athmane):

I've categorized and tested two test-cases for cronie and perl (they are on critpath category) so they'll show up in Bodhi updates.

https://fedoraproject.org/wiki/QA:Testcase_Perl_sanity

http://fedoraproject.org/wiki/QA:Testcase_crontabs

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by athmane):

Fixed <pre> tag issue on:

https://fedoraproject.org/wiki/QA:Testcase_Dracut_root%3Dpartition

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by jlaska):

Athmane pointed out that the linked critpath.log shows binary packages, not src.rpms. I've attached a src.rpm list to this ticket for review. The list is taken form the 0610 rawhide critpath.log.

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by athmane):

I've just created a test-case for yum (=> critpath):

https://fedoraproject.org/wiki/QA:Testcase_Yum_basics

#154: Tracker: critical path test case creation --------------------------+------------------------------------------------- Reporter: adamwill | Owner: adamwill Type: enhancement | Status: closed Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: fixed | Keywords: --------------------------+------------------------------------------------- Changes (by adamwill):

* status: new => closed * resolution: => fixed

Comment:

Looking at this again, I've had a change of heart and I agree with Past James and not Past Adam: a single ticket for 'create test cases for 400+ critpath components' isn't much use, especially given trac doesn't let you have tickets depend on each other. So let's close it. That doesn't mean we should stop working on critpath test cases, though, this is just bureaucracy. :) Thanks athmane for your work so far on writing and categorizing test cases!