Hi,
I'd like to see some more detailed test plans for the Fedora desktop. These need not be complicated or long, just a set of basic things we want to try out in each of the major apps and in the OS itself.
Here are some notes I have on things we might include. I know that some of these tests would not pass at the moment, no need to point that out ;-)
- basic run through a login session; log in, log out, launch some apps; does anything embarassing happen - install previous version of the OS; log in to a user account, play with some things, change settings; tar up and save the user home directory; install current version of the OS, and unpack same user home directory; repeat the above basic run through a login session - develop list of major apps that must be tested, check basic functionality of each. e.g. browser, office, mail, IM, etc. - input methods work in these apps - printing works properly in all 10 languages from these major apps - Windows printers are detected and shown automatically in print dialog - CUPS printers are detected and shown automatically in print dialog - in Nautilus, verify that Windows file shares appear automatically and the files there can be opened on double click, specifically check MS Office files - verify that plugging in several representative USB storage devices results in them appearing on the desktop and working - verify that if a local USB printer is plugged in, it is automatically configured and made available - verify that on plugging a network cable the link is automatically brought up and dhcp run - verify that wireless networking can be used without resorting to the command line at all - login to FC1/2, login to FC3 same homedir, run through major apps and check for confusion with shared NFS homedir - login on two different machines, same homedir, with same FC version, check for badness - login twice on same machine same user and check for badness - test lockdown of settings, ensure desktop behaves properly and user can't avoid the lockdown - test X on some list of video hardware; automated smoke test - accessibility, e.g. test screen reader and onscreen keyboard - correct fonts used in 10 languages for set of common font names - suite of Microsoft documents to be imported correctly (screenshots of correct appearance?) - suite of web pages to render correctly - cut-and-paste to/from a big matrix of sources/targets - out of disk space, behavior on login and when saving from the major apps - set date/time backward, see if desktop or login breaks - run any test suites we have for specific components - verify that all browser plugins work (including the proprietary ones available from third parties) - verify important MIME associations, e.g. .doc opens in OpenOffice.org, etc. - test use of an Active Directory account for Linux login
As you can see from the list, the plans need to be more specific, and point to the files (tarred up homedirs, web pages, office documents, etc.) to be used.
Havoc
Havoc Pennington hp@redhat.com writes:
Hi,
I'd like to see some more detailed test plans for the Fedora desktop. These need not be complicated or long, just a set of basic things we want to try out in each of the major apps and in the OS itself.
<snip>
- accessibility, e.g. test screen reader and onscreen keyboard
The accessibility project has helpfully put together a test plan at:
http://developer.gnome.org/projects/gap/testing/a11y_sanity_suite.html
-Jonathan
On Thu, 29 Jul 2004 15:15:14 -0400, Havoc Pennington hp@redhat.com wrote:
Hi,
I'd like to see some more detailed test plans for the Fedora desktop. These need not be complicated or long, just a set of basic things we want to try out in each of the major apps and in the OS itself.
Is this aimed at other developers, or is this list aimed for testers to run? I'd very much like to see a set of tasks laid out for community testers to work through in an organized way... homework assignments for lack of a better phrase. I've been waiting on the prophetic CMS system for fedora's website instead going off on my own and starting a crappy attempt to write a testing guide. But if this a serious attempt to codify a set of specific tasks for people to test and report back failures on, I'm more than willing to help generate things like office documents if I still don't have access to contributing webpage text for the guide.
-jef
Havoc Pennington wrote:
- develop list of major apps that must be tested, check basic functionality of each. e.g. browser, office, mail, IM, etc.
- input methods work in these apps
- printing works properly in all 10 languages from these major apps
...
- verify that all browser plugins work (including the proprietary ones available from third parties)
Mozilla's basic functionality test suite (smoketest) which includes plugins testing is at http://www.mozilla.org/quality/smoketests/index.html
On Thu, 2004-07-29 at 16:06, Jeff Spaleta wrote:
Is this aimed at other developers, or is this list aimed for testers to run?
I'd like the developers to help define the tests, then I figure Red Hat QA guys will run as much of it as they can, and Red Hat developers and perhaps Fedora contributors (if there's interest) can pitch in and help run the rest of it.
I'd very much like to see a set of tasks laid out for community testers to work through in an organized way... homework assignments for lack of a better phrase. I've been waiting on the prophetic CMS system for fedora's website instead going off on my own and starting a crappy attempt to write a testing guide. But if this a serious attempt to codify a set of specific tasks for people to test and report back failures on, I'm more than willing to help generate things like office documents if I still don't have access to contributing webpage text for the guide.
That sounds great. I wouldn't wait for us to have a wiki, you can always write content elsewhere, including fedora.us wiki, and then we can move stuff to the wiki when we have it. Or whatever.
I don't know what the best approach is. Maybe we should put the files to be used in the tests in an RPM package - then the kickstart files for the test machines can just include that package ;-)
The Mozilla smoke test caillon posted looks like exactly the right kind of thing that we should have covering all the major desktop bits.
Havoc
On Thu, 29 Jul 2004 15:15:14 -0400, Havoc Pennington wrote:
- Windows printers are detected and shown automatically in print dialog
- in Nautilus, verify that Windows file shares appear automatically and the files there can be opened on double click, specifically check MS Office files
Last time I checked these two were broken by the default firewall configuration. There is a bug filed on this, assigned to Bill Nottingham I think
- suite of Microsoft documents to be imported correctly (screenshots of correct appearance?)
I suspect OpenOffice has a test suite of these, why does it need to be duplicated in Fedora?
- cut-and-paste to/from a big matrix of sources/targets
- out of disk space, behavior on login and when saving from the major apps
- Session management works correctly in major apps?
- verify that all browser plugins work (including the proprietary ones available from third parties)
Heh, does that include CrossOver Plugin? :)
thanks -mike
On Fri, 2004-07-30 at 09:16, Mike Hearn wrote:
Last time I checked these two were broken by the default firewall configuration. There is a bug filed on this, assigned to Bill Nottingham I think
Aha, Colin was trying to figure that out. ;-)
- suite of Microsoft documents to be imported correctly (screenshots of correct appearance?)
I suspect OpenOffice has a test suite of these, why does it need to be duplicated in Fedora?
What we have to do is *run* the test suite on the package in Fedora. That means having a pointer to said suite and instructions on how to run it in the Fedora test plan.
- Session management works correctly in major apps?
I think we've more or less given up on session management these days, though it would be nice.
- verify that all browser plugins work (including the proprietary ones available from third parties)
Heh, does that include CrossOver Plugin? :)
Yep, ideally.
Havoc
Is there currently any way to watch quicktime movies inside Mozilla/Mozilla Firefox?
Thanks Chris
On Sat, 2004-07-31 at 16:18, Chris Farber wrote:
Is there currently any way to watch quicktime movies inside Mozilla/Mozilla Firefox?
fedora-list@redhat.com is the right place to ask this sort of questions
http://www.redhat.com/archives/fedora-list/2004-January/msg03811.html (might be of some help)
On Fri, 2004-07-30 at 23:19 -0400, Havoc Pennington wrote:
On Fri, 2004-07-30 at 09:16, Mike Hearn wrote:
Last time I checked these two were broken by the default firewall configuration. There is a bug filed on this, assigned to Bill Nottingham I think
Aha, Colin was trying to figure that out. ;-)
I dropped the firewall long ago, so I'm pretty sure that wasn't the problem.
We have one solution to this over at the unofficial Fedora F.A.Q.
www.fedorafaq.org/#browsermovies
lør, 31.07.2004 kl. 08.18 skrev Chris Farber:
Is there currently any way to watch quicktime movies inside Mozilla/Mozilla Firefox?
Thanks Chris
On Thu, Jul 29, 2004 at 04:35:01PM -0400, Christopher Aillon wrote:
Mozilla's basic functionality test suite (smoketest) which includes plugins testing is at http://www.mozilla.org/quality/smoketests/index.html
Similiarly OpenOffice has a qadev cvs module which has lots of qa tests, and there is a quicker smoketest 10erTest_680.sxw document which runs through some basic sanity tests. With OOo UNO interface its relative trivial to write tests in e.g. python to automate OOo specific tests like document load/convert comparisons.
C.
desktop@lists.stg.fedoraproject.org
-
Caolán McNamara -
Chris Farber -
Christopher Aillon -
Colin Charles -
Colin Walters -
Havoc Pennington -
Jef Spaleta -
Jonathan Blandford -
Mike Hearn -
Sindre Pedersen Bjordal