On Mon, 2019-05-13 at 16:29 -0600, Lars Kurth wrote:
Hi all,
I am going to step in here with my hat as Xen Project community manager. We had a discussion about this issue as part of last week's community call. I CC'ed a number of stake-holders, which probably should have been on the thread such as ITL (which builds QubesOS on top of Fedora) and Michael A Young (the Xen package manager).
First of all apologies that this issue has lingered so long. Key members of the community were not aware of the issues raised in this thread, otherwise we would have acted earlier. With this in mind, please in future raise issues with me, on xen-devel@, committers@ or the Xen-Fedora package manager. The Xen Community would like to see Fedora running as guest: in fact it would be somewhat odd if there was a Xen-Dom0 package and guest support didn't work. And there are some downstreams such as QubesOS, which depend on this support.
Thanks for stepping in. Of course we always want as much stuff as possible to work, but that does not mean we block the release on it. We certainly want Fedora to work as a guest on VMWare, VirtualBox and Parallels too; we don't block the release on any of those either...
Well, I mean, every few *days* a compose gets nominated for validation testing, and a mail is sent to test-announce. Just check your test- announce archives for mails with "nominated for testing" in their subject lines, and you'll see dozens. Is this not sufficient notification?
We discussed this at the community call and came to the conclusion that we can run regular tests of Fedora RC's as part of our OSSTEST infrastructure. Ian Jackson volunteered to implement this, but there are some questions on a) The installer (which we can handle ourselves) b) When we would trigger a test - aka is there some trigger other than the c) How would results best be reported back to Fedora
Apologies, I am not very familiar with how the Fedora Test group works. Is there some documentation which would help integrate what you to with the test system of another open source project?
b) you can use fedmsg / fedora-messaging: https://github.com/fedora-infra/fedmsg https://github.com/fedora-infra/fedora-messaging A message is emitted every time a compose attempt finishes (on the fedmsg topic 'org.fedoraproject.prod.pungi.compose.status.change': see https://apps.fedoraproject.org/datagrepper/raw?topic=org.fedoraproject.prod.... for a log of past messages). What you will want to do is listen for completed Branched and Rawhide composes and run tests whenever one completes successfully. This is already exactly what we do to schedule openQA tests; you can crib from the openQA test scheduler: https://pagure.io/fedora-qa/fedora_openqa particularly the fedmsg consumer: https://pagure.io/fedora-qa/fedora_openqa/blob/master/f/fedora_openqa/consum...
c) ideally it would be good to report to both resultsdb and to the wiki. Again, we already do this for openQA, and you can crib from the code there: https://pagure.io/fedora-qa/fedora_openqa/blob/master/f/fedora_openqa/report... reporting to ResultsDB might be tricky due to authentication issues, I'm not sure if we ever put the openID auth stuff into production. For wiki reporting you will either have to auth manually every so often or ask Fedora infra for a special token that doesn't expire (this is what we do for the openQA results). Reporting to ResultsDB you do through resultsdb_api - https://pagure.io/taskotron/resultsdb_api - and optionally you can use my resultsdb_conventions - https://pagure.io/taskotron/resultsdb_conventions - which makes it somewhat easier (IMO anyway) and will make your results consistent with those from openQA and Autocloud. Reporting to the wiki you can do through my crazy python-wikitcms library - https://pagure.io/fedora-qa/python-wikitcms . Again, fedora_openqa does all this for openQA results, so you can crib from that. Let me know if you have trouble with this.
from Oracle. On that basis, I'm proposing we remove this Final criterion:
s/Oracle/Xen Project/ I believe?
Perhaps, it's just that it always seemed to be you doing the testing, so they got a bit conflated :)
Can we come to some arrangement, by which such issues get communicated to the Xen Project earlier? Aka me, xen-devel@ or committers@
It would be nice if you could ensure someone from Xen is actually watching the Fedora lists, if working in Fedora is a target for Xen. We *could* try and CC stuff all the time, but imagine if we tried to do that for everybody. But yes, for future conversations of this nature I'll try and remember to include those lists.
"The release must boot successfully as Xen DomU with releases providing a functional, supported Xen Dom0 and widely used cloud providers utilizing Xen."
and change the 'milestone' for the test case - https://fedoraproject.org/wiki/QA:Testcase_Boot_Methods_Xen_Para_Virt - from Final to Optional.
Thoughts? Comments? Thanks!
I would prefer for it to remain as it is.
This is only practical if it's going to be tested, and tested regularly
- not *only* on the final release candidate, right before we sign off
on the release. It needs to be tested regularly throughout the release cycle, on the composes that are "nominated for testing".
Would the proposal above work for you? I think it satisfies what you are looking for. We would also have someone who monitors these test results pro-actively.
In theory, yeah, but given the history here I'm somewhat sceptical. I'd also say we still haven't really got a convincing case for why we should continue to block the release (at least in theory) on Fedora working in Xen when we don't block on any other virt stack apart from our 'official' one, and we don't block on all sorts of other stuff we'd "like to have working" either. Regardless of the testing issues, I'd like to see that too if we're going to keep blocking on Xen...