Hey folks! Just thought I'd keep people up to date on this: I'm gonna temporarily disable the validation event creation bot because we happen to know that F24 composes are completely borked right now, and I don't really want to get an event for one of them. We need a new anaconda build to fix the C.UTF-8 default locale bug, without that, all installs will simply crash.
sbueno is aiming to do a new anaconda build tomorrow, so once we do get one we'll figure a way to get it karma'ed and pushed stable - either just by people building their own images to test or updating anaconda in live images, or I'll ask releng for a Beta candidate compose that includes it.
For now Alpha is still the thing to test if you want to do some release validation testing, it's still useful to fill in missing Beta and Final tests in the Alpha matrices even though Alpha has been released, because the results will be tracked in testcase_stats and of course if you find any bugs you can file them and nominate them as blockers if appropriate.
Thanks folks!
Il giorno mer, 30/03/2016 alle 14.40 -0700, Adam Williamson ha scritto:
Hey folks! Just thought I'd keep people up to date on this: I'm gonna temporarily disable the validation event creation bot because we happen to know that F24 composes are completely borked right now, and I don't really want to get an event for one of them. We need a new anaconda build to fix the C.UTF-8 default locale bug, without that, all installs will simply crash.
sbueno is aiming to do a new anaconda build tomorrow, so once we do get one we'll figure a way to get it karma'ed and pushed stable - either just by people building their own images to test or updating anaconda in live images, or I'll ask releng for a Beta candidate compose that includes it.
For now Alpha is still the thing to test if you want to do some release validation testing, it's still useful to fill in missing Beta and Final tests in the Alpha matrices even though Alpha has been released, because the results will be tracked in testcase_stats and of course if you find any bugs you can file them and nominate them as blockers if appropriate.
Thanks folks!
Hi Adam.
I'd like to ask a question about this temporary part of release validation testing.
Which one of the following scenario do you recommend? Scenario 1) Sticking with Alpha 1.7, without performing updates. Fill in the matrices with results coming from it. Scenario 2) Running the test cases with an up to date F24 Alpha system. Fill in the matrices with it.
Both of these have drawbacks: Scenario 1) The wiki ends up with a mix of Alpha 1.7 and "updated Alpha" results. So do the stats. Scenario 2) If I remember right, Gnome 3.20 has been pushed stable after the release of F24 Alpha. So, testing the version shipped with Alpha 1.7, might not be so useful, when approaching Beta and Final test cases.
In my opinion, the first scenario is not a big problem. Stats are interesting, but a proper testing looks more important and nobody will complain if we mess up some matrices. But, if we report an already fixed problem, we waste time.
So... I'd rather test an up to date system. What do you think?
Thank you.
--- Giulio (juliuxpigface)
On Thu, 2016-03-31 at 19:38 +0200, Giulio E. wrote:
Hi Adam.
I'd like to ask a question about this temporary part of release validation testing.
Which one of the following scenario do you recommend? Scenario 1) Sticking with Alpha 1.7, without performing updates. Fill in the matrices with results coming from it. Scenario 2) Running the test cases with an up to date F24 Alpha system. Fill in the matrices with it.
Both of these have drawbacks: Scenario 1) The wiki ends up with a mix of Alpha 1.7 and "updated Alpha" results. So do the stats. Scenario 2) If I remember right, Gnome 3.20 has been pushed stable after the release of F24 Alpha. So, testing the version shipped with Alpha 1.7, might not be so useful, when approaching Beta and Final test cases.
In my opinion, the first scenario is not a big problem. Stats are interesting, but a proper testing looks more important and nobody will complain if we mess up some matrices. But, if we report an already fixed problem, we waste time.
So... I'd rather test an up to date system. What do you think?
So far as the matrices go, I personally prefer reporting results really *for the compose in question* - it gets confusing if different people report results for different updates to the same matrix pages, you have 'bob' saying GNOME works fine and 'anna' saying it's completely broken and it becomes a bit of a mess. But I'd say for non-install-related tests it would be more valuable to test the up-to-date bits and just don't report results to the wiki - just file bugs if you find any, and nominate them as blockers if appropriate. We'll get a newer event soon enough.
Always keep in mind that *ultimately* the goal of the testing is to find bugs and fix them, not check boxes in wiki pages =) And you get 'Heroes of Fedora' credit for filing bugs as well as filling in wiki boxes!
This is another reason it would be nice to have a more flexible test result system than the wiki, of course. We'll get that off the todo list one day!