test August 2022

test@lists.fedoraproject.org

25 participants
44 discussions

F37 Beta blocker status email
by Ben Cotton 02 Sep '22

02 Sep '22

With F37 now branched from Rawhide, it's time to pick up our weekly blocker status emails! Action summary ==================== Accepted blockers ----------------- 1. kde-settings — KDE needs to pick up F37 backgrounds — NEW ACTION: Maintainers to make new kde-settings build Proposed blockers ----------------- 1. anaconda — Anaconda crashed on signal 11 - keyboard layout encryption password prompt — ON_QA ACTION: QA, reporter to verify anaconda-37.12-1 NEEDINFO: doug.hs 2. anaconda — Anaconda will not start F37 Raw — NEW ACTION: Reporter to provide requested log files NEEDINFO: aalmeleh.whatever.0101 3. dnf — "dnf update" runs out of memory on swapless 0.5 GiB RAM machine — NEW ACTION: Manitainers to investigate short-term dnf improvements NEEDINFO: jmracek 4. dracut — Media check fails with checkisomd5 "service failed because the control process exited with error code" — POST ACTION: Maintainers to build with upstream PR 5. polkit — Switching users in Gnome results in polkitd errors that prevent powering off the computer. — ASSIGNED ACTION: Maintainers to diagnose issue 6. sddm — Logout from KDE session on Rawhide goes to console, not SDDM — NEW ACTION: Maintainers to diagnose issue Bug-by-bug detail ============= Accepted blockers ----------------- 1. kde-settings — https://bugzilla.redhat.com/show_bug.cgi?id=2117706 — NEW KDE needs to pick up F37 backgrounds New desktop backgrounds require a new KDE settings update. Proposed blockers ----------------- 1. anaconda — https://bugzilla.redhat.com/show_bug.cgi?id=2070823 — ON_QA Anaconda crashed on signal 11 - keyboard layout encryption password prompt Clicking the keyboard layout button in the LUKS decryption popup crashes anaconda. anaconda-37.12-1 contains a candidate fix. 2. anaconda — https://bugzilla.redhat.com/show_bug.cgi?id=2101229 — NEW Anaconda will not start F37 Raw Neither clicking the desktop icon nor the taskbar icon will launch anaconda. openQA is not experiencing this. Using basic graphics mode results in the expected behavior. 3. dnf — https://bugzilla.redhat.com/show_bug.cgi?id=1907030 — NEW "dnf update" runs out of memory on swapless 0.5 GiB RAM machine dnf is killed by oom-kill on machines with 512 MB of RAM. Adding a swap disk or disabling the updates repo works around this issue. Recently, FCOS has seen this behavior with 1 GB VMs. The issue appears to be the size of the metadata for the repository. The short term fixes look like they may need to be made on the repo side, not in dnf. 4. dracut — https://bugzilla.redhat.com/show_bug.cgi?id=2107858 — POST Media check fails with checkisomd5 "service failed because the control process exited with error code" Media check fails on certain hardware. Skipping the media check results in a successful boot. Upstream PR #1882 contains a candidate fix: https://github.com/dracutdevs/dracut/pull/1882 5. polkit — https://bugzilla.redhat.com/show_bug.cgi?id=2109145 — ASSIGNED Switching users in Gnome results in polkitd errors that prevent powering off the computer. Under certain conditions (including but not limited to several user switches), the power off button disappears from the GNOME menu due to a polkitd error. `systemctl poweroff -i` as an unprivileged user does not work. It may be an issue with using the duktape JS engine instead of mozjs91. polkit-121-3 reverts to mozjs91, but is not a sufficient fix. 6. sddm — https://bugzilla.redhat.com/show_bug.cgi?id=2110801 — NEW Logout from KDE session on Rawhide goes to console, not SDDM After logging out of KDE Plasma, you end up at a console on tty2. SDDM had been running on tty1 but just shows a flashing cursor. -- Ben Cotton He / Him / His Fedora Program Manager Red Hat TZ=America/Indiana/Indianapolis

2 4

Request for testing: Fedora 37 pre-Beta validation tests
by Adam Williamson 01 Sep '22

01 Sep '22

Hey folks! So we're in freeze for Fedora 37 Beta now, and the first go/no-go meeting should be on September 8. It would be really great if we can get the validation tests run now so we can find any remaining blocker bugs in good time to get them fixed. Right now the blocker list looks short, but there are definitely some tests that have not been run. You can use the testcase_stats view to find tests that need running: https://openqa.fedoraproject.org/testcase_stats/37/ For each validation test set (Base, Desktop etc.) it shows when each test was last performed. So you can easily look for Basic and Beta tests that have not yet been run. We need to run all of these. You can enter results using `relval report-results`, or edit the summary results page at https://fedoraproject.org/wiki/Test_Results:Current_Summary . That's a redirect link which will always point to the validation results page for the currently-nominated compose, which right now is 20220826.n.0. Sumantro will be running a validation 'test week' starting on Wednesday, so you can drop by the Fedora Test Day room on chat.fedoraproject.org to hang out with other testers and get any help you need in testing. See https://lists.fedoraproject.org/archives/list/test-announce@lists.fedorapro… for that announcement. Thanks folks! -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net

4 4

Re: Thoughts welcome: interface between automated test gating and the "critical path"
by Adam Williamson 31 Aug '22

31 Aug '22

On Wed, 2022-08-31 at 12:43 +0000, Zbigniew Jędrzejewski-Szmek wrote: > On Tue, Aug 30, 2022 at 12:10:00PM -0700, Adam Williamson wrote: > > On Tue, 2022-08-30 at 09:14 -0400, Ben Cotton wrote: > > > From my perspective, anything that blocks the release is on the > > > critical path. So any time there's a violation of the release criteria > > > and the package is not on the critical path definition, that's a bug > > > in the definition. > > > > > > I recognize that this is a somewhat naïve view. For one, it may > > > broaden the definition beyond the current capacity of our test > > > infrastructure. It also may broaden the definition beyond what > > > maintainers are willing to put up with. These are both legitimate > > > problems. But the closer we can get to this ideal state, the better. > > > > > > For anyone who is curious, I just searched for all accepted blockers > > > in the "Fedora" product in Bugzilla. 327 components have been a > > > blocker at least once. Some of those may no longer be blocking and > > > others will be added over time as our criteria change. The full list > > > with counts is at > > > https://bcotton.fedorapeople.org/release-blocking-components.csv if > > > you're interested. > > > > Honestly, something along these lines would be my preference too, I > > just don't know if others would agree/support changing the critical > > path definition to "all release-blocking functionality" rather than > > "functionality needed to boot a basically-functional system". > > I'd support increasing the scope to cover more packages in this fashion. > > Being on critpath list is somewhat annoying because the bodhi update > minimum times are twice as long. For many packages this is a not a problem, > popular packages get enough karma within a day or two, > but if we expanded the list a lot, it could prove annoying to those > packagers. I think we could discuss lowering the minium update time > if this turns out to be the case. So, that's an interesting question to consider as well, of course: what's the actual impact of critpath? It'd be an interesting outcome if we broadened the critpath definition but diluted the Bodhi requirements, because the original purpose of critpath was more or less entirely the stricter Bodhi requirements. Until openQA came along, that was all it really meant: updates to these packages have stricter Bodhi requirements. Now, because I glued openQA to the critpath because it was handy, there are two sets of consequences to a package being in critical path: 1. Tighter Bodhi requirements 2. openQA tests are run, and results gate the update (except Rawhide) So, one of the implicit questions here is, is it OK to keep twinning these two sets of consequences, or should we split them up? Splitting them up kinda implies answer 2) from my original mail: "Keep the current "critical path" concept but define a broader group of "gated packages" somewhere". Because we would then need some new concept that isn't "critical path". As I said, that's more *work* - it'd require us to write new code in several places[0]. Even if we decide it'd be nice to do this, is it nice *enough* to be worth doing that work? If we don't think it's worth doing that work, then we're kinda stuck with openQA glomming onto the critpath definition to decide which updates to test and gate, because I don't have any other current viable choices for that, really. And we'd have to figure out a critpath definition that's as viable as possible for both purposes. BTW, one other thought I've had in relation to all this is that we could enhance the current critpath definition somewhat. Right now, it's built out of package groups in comps which are kinda topic-separated: there's a critpath-kde, a critpath-gnome, a critpath-server, and so on. But the generated critical path package list is a monolith: it doesn't distinguish between a package that's on the GNOME critpath and a package that's on the KDE critpath, you just get a big list of all critpath packages. It might be nice if we actually did distinguish between those - the critpath definition could keep track of which critpath topic(s) a package is included in, and Bodhi could display that information in the web UI and provide it via the API. That way manual testers could get a bit more info on why a package is critpath and what areas to test, and openQA could potentially target its test runs to conserve resources a bit, though this might require a bit more coding work on the gating stuff now I think about it. [0]: at least: releng critpath generation scripts, bodhi, the openQA scheduler, greenwave policies, and possibly greenwave -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net

1 0

Fedora 37 Pre Beta #2 testing
by Luna Jernberg 31 Aug '22

31 Aug '22

1 0

Thoughts welcome: interface between automated test gating and the "critical path"
by Adam Williamson 30 Aug '22

30 Aug '22

Hi folks! I have one of those definitional quandaries and I figured I'd throw it at the lists for some input. Right now, we kind of take advantage of the "critical path" concept for automated update testing and gating via openQA. openQA does not test all updates, only critical path updates plus updates containing any package on a short allowlist. Bodhi and Greenwave do not gate all updates on the openQA tests since not all updates are tested; again we use the critpath definition. If an update is critical path, it gets gated on the openQA tests. If it isn't, it doesn't. If you've been paying attention, that means there's a bit of a hole: the packages on the 'allowlist'. These are tested, but not gated. What's on the allowlist? Basically, FreeIPA-related packages. We have a good set of FreeIPA tests in openQA, and both I and the FreeIPA team wanted relevant updates to be tested with them. But those packages are not on the critical path. So I set up this 'allowlist' mechanism. However, the lack of gating kinda sucks. I would much prefer if we could gate these updates as well as just testing them. The obvious thing to do is add the FreeIPA-related packages to the critical path, but that really is not supported by the critical path definition: https://fedoraproject.org/wiki/Critical_path_package which defines it as packages required to "perform the most fundamental actions on a system", with a list of specific areas, none of which is "run a domain server". So...what to do? I can think of I guess four options: 1. Broaden the definition of the "critical path" somehow. We could just write in that it includes FreeIPA functionality, I guess, though that seems special purpose. We could broaden it to include any functionality covered by the release criteria, which would be quite a big increase but seems like a reasonable and concise definition. Or we could write in that it includes any functionality that is exercised by the gating openQA tests, though that seems a bit arbitrary - there's no particularly great organizing principle to the openQA tests we choose to run on updates, if I'm honest, it's a sort of grab bag I came up with. 2. Keep the current "critical path" concept but define a broader group of "gated packages" somewhere (could be comps or somewhere else). This would require engineering work - we'd have to touch probably the openQA scheduler, Bodhi, and greenwave configs. It's also another maintenance burden. 3. Add gating config to each allowlisted package repo's gating.yml one by one. I don't really like this option. It'd be a chunk of work to do initially, and multiples the maintenance required when the list of tests to gate on changes for some reason. 4. Do nothing, just keep only gating things that are "critical path" on the current definition. In a utopian future, we'd manage to deploy openQA in the cloud or get a giant pile of super fast servers so we could test and gate *every* update, and this wouldn't be an issue any more. What do folks think? Any preferences? Thanks! -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net

3 3

Re: Thoughts welcome: interface between automated test gating and the "critical path"
by Adam Williamson 30 Aug '22

30 Aug '22

On Tue, 2022-08-30 at 09:39 +0300, Alexander Bokovoy wrote: > On ma, 29 elo 2022, Adam Williamson wrote: > > On Tue, 2022-08-30 at 00:32 -0400, DJ Delorie wrote: > > > It sounds to me like the problem is "how do we best use the available > > > automated test resources?" so I'll answer accordingly. Ignore me if I > > > misunderstood ;-) > > > > No, not really, sorry if I didn't explain clearly enough :D It's more > > just a 'what's the best way to work what I want to do around the > > existing definitions' question. What I want to do is have updates of > > FreeIPA-related packages gated. The awkwardness is that right now the > > definition of what gets gated is "critical path packages", and those > > clearly don't meet the current definition of "critical path". > > Not all of them meet the current definition of 'critical path' but many > do. In fact, almost all crypto-implementing packages should be on that > list, if not already. Sure, but freeipa itself, for instance, can't really be as things stand. Nor could bind, I don't think. And several others. > > Ideally, use of gating tests should be defined in the component itself. > Do we have a mechanism to trigger OpenQA runs from the gating.yaml in > the specific component? If we do, then we really should move it there. No. I don't think this is in-scope for gating.yaml. I *could* extend the openQA scheduler to check some kind of per-repo config for every package in the update, but there is nothing like this at present. In any case, triggering the *tests* isn't the problem here; we already do that, with the allowlist. It's doing the *gating* that's the problem. As I said, we could add *that* to gating.yaml for each repo, but I don't really love that idea, because you have to provide a full list of what tests to gate on, and doing that for each repo and keeping it updated seems like kind of a drag. I guess in a sense it's appropriate for these packages, though, as you could say we'd "really" want to gate on the FreeIPA-specific tests, or at least just the server tests. But honestly, if an update to a freeipa-related package causes GNOME to start crashing or something, that's something I want to look into before the update goes stable, even if it seems very weird. > > Another part of the problem is that most of these runs should probably > be consulting, not gating in the sense that failing them should give you > a clue that things might be wrong but there needs to be a human overview > of the failures. It is what we have right now with OpenQA-reported > failures for Bodhi updates in most cases, the test results aren't really > preventing the submissions from the going forward unless a maintainer > defined the update configuration to be so. The problem cases here are what I was thinking about. They are: Branched before Beta freeze, and Rawhide. For those cases, updates are automatically created on `fedpkg build` and immediately submitted for stable. If there is no applicable gating policy they are immediately pushed stable. There is no time for anyone to consult. If you use a side tag there's no automatic submission, I don't think, but people do not use side tags as a matter of course. Right now we don't gate Rawhide updates, but I would like to and I am actively working on it (this would prevent me having the kind of nightmare week I had last week where I spent dozens of hours tracking down things that had broken in Rawhide, getting them fixed, and re- running tests). We *do* currently gate Branched updates before Beta freeze, and I want FreeIPA-related packages included in that for the next cycle. I would have liked it if, for the F37 cycle, FreeIPA-related updates that were submitted between Branch point and Beta freeze were gated. But they weren't. Practically speaking, the way we do things in Fedora, there is very little difference between "gating" and "consulting"; because of the waiver mechanism they're effectively identical. You can always issue waivers to override the gating. So really, gating is consulting. You can look at the failures, decide they're bogus, and hit the waiver button, which will immediately make the update 'pass' gating. I do encourage people not to abuse the waiver function - if the failure seems like a flake please use the 're-run tests' button, if you really think it's bogus please poke me or Lukas or someone as I'm probably working right then on dealing with it - but it is right there. > > Ideally, we should have reverse dependency updates to trigger FreeIPA > tests and give their results a visibility but don't block on failures. > This would give an opportunnity to notice a failurer earlier but don't > discourage maintainers from participating in a joint development. We would like reverse dependency testing for all sorts of reasons, but it's also all sorts of difficult (and, potentially, resource- intensive). Of course, in openQA testing we do sort of have this already. We run *all* tests on all critpath updates. So if an update to a critpath component breaks FreeIPA, we find out about it immediately. We don't have to wait for a FreeIPA-related update to show up. If the dep isn't critpath, though, we don't test it. > Perhaps, The Fedora Packager Dashboard could be extended to pick up > results of tests relevant to your packages and display them together? > This way FreeIPA maintainers can see an overview of all tests related to > FreeIPA in one place and see if any help is needed to resolve failures. This would be lovely, sure, but at this point we're quite far away from the fairly concrete initial scope of my mail. I don't want to get sidetracked into sky castle building where we file tickets and wait two years for people to build stuff. I would like to gate updates to FreeIPA-related packages *now*. I was all set to submit a pull request to comps to add them to the server critpath definition, but then I checked the official definition of "critical path" and noticed this problem. That's where I'm coming from here: I want to make a concrete change now, not design awesome speculative improvements on paper. -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net

1 0

Thoughts welcome: interface between automated test gating and the "critical path"
by pmkellly＠frontier.com 30 Aug '22

30 Aug '22

On 8/29/22 6:50 PM, Adam Williamson wrote: > Hi folks! > > I have one of those definitional quandaries and I figured I'd throw it > at the lists for some input. I spent some years as a project manager and I have lots of experience with schedules. The term Crital Path was formalized in the business world when project managers learned to use Gantt Charts for their scheduling. The Critical Path was fluid and changed as various task dependancies were actually or potentially impacting the project completion date. The tasks and resources on the critical path got special attention to mitigate the delay. This usually led to changes in the critical path or new a critical path would come up. Redefining the term is is not a clean solution ot this problem. First, I think the terminology needs to change. I propose having a critical package list. The critical package list would be defined as something along the lines: "These packages should be working in Fedora as released as Beta or Final. Failing packages must have blocker bugs and/or freeze exceptions." Notice the should gives room for making exceptions as packages fail near release time. After that all critical packages should be tested and evaluated with openQA and Gating. To adopt this approach their two tasks: First decide what packages are on the Critical Package List. The reasons will be many and I don't see a need to document the reasons. A small group of people can make up the list and put it before the comunity for agreement. Second change the testing and gatiing so they work from the list Have Great Day! Pat tablepc

1 0

Re: Thoughts welcome: interface between automated test gating and the "critical path"
by Adam Williamson 30 Aug '22

30 Aug '22

On Tue, 2022-08-30 at 00:32 -0400, DJ Delorie wrote: > It sounds to me like the problem is "how do we best use the available > automated test resources?" so I'll answer accordingly. Ignore me if I > misunderstood ;-) No, not really, sorry if I didn't explain clearly enough :D It's more just a 'what's the best way to work what I want to do around the existing definitions' question. What I want to do is have updates of FreeIPA-related packages gated. The awkwardness is that right now the definition of what gets gated is "critical path packages", and those clearly don't meet the current definition of "critical path". > > We currently have a small list of packages that are gated behind openQA, > and insufficient openQA resources to expand this list to all packages. > > We also have a gating system based on karma, where users provide the QA > manually. At least, one hopes. The karma system has some > configurability for how much karma is needed, and for how long, etc. Ah, no, again, I guess I should've explained that in more detail. When I talk about "gating" I'm not talking about the manual karma mechanism. I'm talking about the mechanism that prevents updates being pushed stable if they fail the automated tests. That's a separate, parallel thing which does not involve karma or manual feedback at all, it's entirely automated. > > What about a merger of those systems, where the openQA results count as > a type of user, contributing to the status? Give each package a "QA > Priority" that ranges from "full gating" to "five second rule", where > the openQA resources and the user work together to prioritize and test > packages according to need: > > * full gating requires all openQA tests to pass AND meet karma > requirements, openQA does these first. > > * partial gating is like above, but either one (openQA or karma) can be > sufficient if enough time passes. > > * reject only allows an openQA failure to reject an update, but the lack > of openQA success need not stop it if it has enough karma. (glibc uses > this paradigm - a ci/cd failure is an automatic reject, but a ci/cd > pass adds nothing to the consensus needed) > > ... and so on down the list. Each user, including openQA, can "vote" a > pass or fail, and the rules say how all those passes and fails result in > the update's overall pass or fail. > > This would allow the openQA resources to prioritize updates that it > knows *needs* openQA results, which allocating spare cycles to packages > that could benefit from testing but don't require it. It also means > that additional openQA resources can be put to use immediately, while > still allowing for growth in critical path size, without being wasted on > unseen results. This is an interesting idea, but rather complex and doesn't solve any problem I have right now. :P Sorry for the confusion! -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net

1 0

Self-introduction: Tommy Nguyen
by Tommy Nguyen 29 Aug '22

29 Aug '22

Reposting this as I sent this to the wrong list by accident. Hello all, I've been using Fedora for many years and recently became interested in QA testing. My interest developed when I wanted to test programs before they reached stable so that I could ensure that I wouldn't be surprised by an update. But I find the QA process to be kind of fun and useful. My previous experience includes maintaining a website (see footer) and I have a few COPR packages as well. I'm not really interested in joining currently as I probably will not be able to commit, but I'd like to keep testing, filing bug reports and be useful in some way. Thanks. ---------------------- Site: https://linuxguideandhints.com/ COPR: https://copr.fedorainfracloud.org/coprs/remyabel/ IRC: remyabel2

1 0

[Call for Action] Fedora 37 Pre-Beta Validation Event kicks off 2022-08-31 through 2022-08-07
by Sumantro Mukherjee 29 Aug '22

29 Aug '22

Hey All, Some of you may know from my presentation at Nest '22 about the Validation events that are getting added! Historically, Fedora QA has relied on a lot of contributor feedback. Mostly, we have set up multiple ways how to capture the results and there has been no better time than to start capturing feedback about the composes as a part of the Release Validation Event! As the name suggests, we will be validating composes throughout a week and we will do it twice per cycle, once for pre-beta and other for pre-final. Validation Events such as this are done to invite more community folks to test the latest compose(s) for each day and help fill the Release Validation Matrix leading up to the Final Go/No-Go meeting. We will start this Validation Event with the compose for 2022-08-31[0] and the respective "current" validation page can be found [1]. As the days change, so will the latest composes and thus will be the current validation pages. Feel free to jump in anytime, and submit validation results on the "current" validation page. You can use "relval" [2] to submit reports for composes as well apart from the regular editing wiki pages. I will be posting about the new images and the QA candidates on this list every day and will be up for taking any question there is! [0] https://kojipkgs.fedoraproject.org/compose/branched/ [1] https://fedoraproject.org/wiki/Test_Results:Current_Summary [2] https://pagure.io/fedora-qa/relval -- //sumantro Fedora QE TRIED AND PERSONALLY TESTED, ERGO TRUSTED

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

test August 2022