#261: Revise upgrade test case set -----------------------------------------+---------------------------------- Reporter: adamwill | Owner: Type: task | Status: new Priority: major | Milestone: Fedora 17 Component: Proventester Mentor Request | Version: Keywords: | -----------------------------------------+---------------------------------- Our present upgrade test cases are probably a sub-optimal set. We exercise various rarely-used choices - bootloader action, and text upgrade path - but don't exercise more common variations, like installed package sets, install media, and disk layouts. While we can't commit to supporting absolutely any upgrade, we should cover at least some _more_ cases in order to catch major fails.
Based on the experience in F16 and the recommendations in the retrospective, I'd suggest we could consider dropping or downgrading the importance of some tests, perhaps:
* QA:Testcase_Anaconda_Upgrade_Skip_Bootloader * QA:Testcase_Anaconda_Upgrade_Skip_Bootloader_Text_Mode * QA:Testcase_Anaconda_Upgrade_Update_Bootloader_Text_Mode
and add test cases for all or some of the following scenarios (please add any other suggestions as comments):
* upgrade from image written to USB stick (livecd-iso-to-disk) * upgrade with KDE (and possibly Xfce and LXDE as 'optional' test cases) * upgrade an EFI install (see also https://fedorahosted.org/fedora- qa/ticket/256 for organizational issues here) * upgrade with bootloader on first partition (not MBR) * upgrade with separate /usr , /home and /var partitions * upgrade a RAID install
This should be completed ideally by Fedora 17 Alpha phase, and definitely by Beta phase (when upgrade issues become blockers).
#261: Revise upgrade test case set ------------------------------------------+--------------------------------- Reporter: adamwill | Owner: hongqing Type: task | Status: assigned Priority: major | Milestone: Fedora 17 Component: Proventester Mentor Request | Version: Resolution: | Keywords: ------------------------------------------+--------------------------------- Changes (by hongqing):
* owner: => hongqing * status: new => assigned
#261: Revise upgrade test case set --------------------------+------------------------------------------------- Reporter: adamwill | Owner: hongqing Type: task | Status: assigned Priority: major | Milestone: Fedora 17 Component: Test Review | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Changes (by hongqing):
* component: Proventester Mentor Request => Test Review
#261: Revise upgrade test case set --------------------------+------------------------------------------------- Reporter: adamwill | Owner: hongqing Type: task | Status: assigned Priority: major | Milestone: Fedora 17 Component: Test Review | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by kparal):
We should discuss the issue of upgrading over more than 1 release (e.g. F14 -> F16 directly). We don't have it in the release criteria, but we have a test case for it. That's not optimal.
The installation guide says a direct upgrade over multiple releases is possible:
http://docs.fedoraproject.org/en-US/Fedora/16/html/Installation_Guide/ch- upgrade-x86.html
But it does not say over how many releases. The best way would probably be to ask FESCo how many releases do we want *officialy* support to upgrade over. And then QA should reflect this approach in its test cases. Anaconda should also warn when user tries to upgrade over more than the officially supported number of releases.
I would love to see only one officially supported release to upgrade from (only F15->F16), but that's probably not gonna happen. Anyway, some clear decision would be highly appreciated.
#261: Revise upgrade test case set --------------------------+------------------------------------------------- Reporter: adamwill | Owner: hongqing Type: task | Status: assigned Priority: major | Milestone: Fedora 17 Component: Test Review | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by adamwill):
The current situation is how it's meant to be - though obviously, since it's possible to interpret it as a problem like you did, we need to clarify things.
It's actually okay to have tests in the matrix which are associated with no release phase. I call these 'advisory tests'. The idea is that there are things we want to test so we catch and file any bugs in them and so we know if they're broken or buggy and can document the problems, but bugs in them are _not_ release blockers. That's fine.
That's what the 'upgrade two releases' test is: it's not a release blocking issue if it fails, but we do want to know whether there are major problems we should document and at least _try_ to fix.
Obviously, we need to make this more clear somehow, so people don't have the same questions about it that you did!
The current 'official' support is exactly what's in the criteria: upgrade from one release ago is release-blocking, upgrade from more than one release ago is not.
#261: Revise upgrade test case set --------------------------+----------------------- Reporter: adamwill | Owner: hongqing Type: task | Status: assigned Priority: major | Milestone: Fedora 17 Component: Test Review | Version: Resolution: | Keywords: Blocked By: | Blocking: --------------------------+-----------------------
Comment (by athmane):
Xfce upgrade test case: https://fedoraproject.org/wiki/QA:Testcase_Xfce_Upgrade
#261: Revise upgrade test case set --------------------------+----------------------- Reporter: adamwill | Owner: hongqing Type: task | Status: assigned Priority: major | Milestone: Fedora 17 Component: Test Review | Version: Resolution: | Keywords: Blocked By: | Blocking: --------------------------+-----------------------
Comment (by jdulaney):
I would remove the bit about GDM being the display manager in case someone has KDM.
#261: Revise upgrade test case set --------------------------+----------------------- Reporter: adamwill | Owner: hongqing Type: task | Status: assigned Priority: major | Milestone: Fedora 17 Component: Test Review | Version: Resolution: | Keywords: Blocked By: | Blocking: --------------------------+-----------------------
Comment (by athmane):
Replying to [comment:6 jdulaney]:
I would remove the bit about GDM being the display manager in case
someone has KDM.
Fixed, however it seems that GDM is pulled when XFCE group is selected. see [1]
[1] http://git.fedorahosted.org/git/?p=comps.git;a=blob_plain;f=comps-f16.xml.in...
#261: Revise upgrade test case set --------------------------+----------------------- Reporter: adamwill | Owner: hongqing Type: task | Status: assigned Priority: major | Milestone: Fedora 17 Component: Test Review | Version: Resolution: | Keywords: Blocked By: | Blocking: --------------------------+-----------------------
Comment (by adamwill):
athmane: that's correct. Fedora's Xfce uses GDM as its default login manager, at least presently.
For the record - by default, in Fedora, GNOME uses GDM, KDE uses KDM, Xfce uses GDM, and LXDE uses LXDM.
#261: Revise upgrade test case set --------------------------+----------------------- Reporter: adamwill | Owner: hongqing Type: task | Status: assigned Priority: major | Milestone: Fedora 17 Component: Test Review | Version: Resolution: | Keywords: Blocked By: | Blocking: --------------------------+-----------------------
Comment (by athmane):
KDE upgrade test case: https://fedoraproject.org/wiki/QA:Testcase_KDE_Upgrade
#261: Revise upgrade test case set --------------------------+----------------------- Reporter: adamwill | Owner: hongqing Type: task | Status: assigned Priority: major | Milestone: Fedora 17 Component: Test Review | Version: Resolution: | Keywords: Blocked By: | Blocking: --------------------------+-----------------------
Comment (by jdulaney):
Adamw, apologies. I was thinking of LXDE by mistake.
Need to get these pesky brain farts under control.
#261: Revise upgrade test case set --------------------------+----------------------- Reporter: adamwill | Owner: hongqing Type: task | Status: assigned Priority: major | Milestone: Fedora 18 Component: Test Review | Version: Resolution: | Keywords: Blocked By: | Blocking: --------------------------+----------------------- Changes (by adamwill):
* milestone: Fedora 17 => Fedora 18
Comment:
We haven't completed this yet (though thanks to Athmane for the added test cases). We still should. Bumping out to F18.
#261: Revise upgrade test case set -------------------------+----------------------- Reporter: adamwill | Owner: hongqing Type: task | Status: assigned Priority: major | Milestone: Fedora 18 Component: Test cases | Version: Resolution: | Keywords: Blocked By: | Blocking: -------------------------+----------------------- Changes (by adamwill):
* component: Test Review => Test cases
#261: Revise upgrade test case set -------------------------+----------------------- Reporter: adamwill | Owner: hongqing Type: task | Status: closed Priority: major | Milestone: Fedora 18 Component: Test cases | Version: Resolution: fixed | Keywords: Blocked By: | Blocking: -------------------------+----------------------- Changes (by adamwill):
* cc: test@… (added) * resolution: => fixed * status: assigned => closed
Comment:
This more or less got done - we have a rather different set of test cases now. We probably should consider re-*re*-doing it for the Fedora.next flavor split, so we have tests for upgrading each flavor, but we can have a new ticket for that.
-
fedora-badges