Hi,
I'd like to see some more detailed test plans for the Fedora desktop. These need not be complicated or long, just a set of basic things we want to try out in each of the major apps and in the OS itself.
Here are some notes I have on things we might include. I know that some of these tests would not pass at the moment, no need to point that out ;-)
- basic run through a login session; log in, log out, launch some apps; does anything embarassing happen - install previous version of the OS; log in to a user account, play with some things, change settings; tar up and save the user home directory; install current version of the OS, and unpack same user home directory; repeat the above basic run through a login session - develop list of major apps that must be tested, check basic functionality of each. e.g. browser, office, mail, IM, etc. - input methods work in these apps - printing works properly in all 10 languages from these major apps - Windows printers are detected and shown automatically in print dialog - CUPS printers are detected and shown automatically in print dialog - in Nautilus, verify that Windows file shares appear automatically and the files there can be opened on double click, specifically check MS Office files - verify that plugging in several representative USB storage devices results in them appearing on the desktop and working - verify that if a local USB printer is plugged in, it is automatically configured and made available - verify that on plugging a network cable the link is automatically brought up and dhcp run - verify that wireless networking can be used without resorting to the command line at all - login to FC1/2, login to FC3 same homedir, run through major apps and check for confusion with shared NFS homedir - login on two different machines, same homedir, with same FC version, check for badness - login twice on same machine same user and check for badness - test lockdown of settings, ensure desktop behaves properly and user can't avoid the lockdown - test X on some list of video hardware; automated smoke test - accessibility, e.g. test screen reader and onscreen keyboard - correct fonts used in 10 languages for set of common font names - suite of Microsoft documents to be imported correctly (screenshots of correct appearance?) - suite of web pages to render correctly - cut-and-paste to/from a big matrix of sources/targets - out of disk space, behavior on login and when saving from the major apps - set date/time backward, see if desktop or login breaks - run any test suites we have for specific components - verify that all browser plugins work (including the proprietary ones available from third parties) - verify important MIME associations, e.g. .doc opens in OpenOffice.org, etc. - test use of an Active Directory account for Linux login
As you can see from the list, the plans need to be more specific, and point to the files (tarred up homedirs, web pages, office documents, etc.) to be used.
Havoc