To all Rawhide users, be careful with update of mutter. It seems that mutter-3.23.2-2.fc26 is broken [1], which prevents you from log in to your system. Downgrade to mutter-3.23.1-2.fc26.x86_64 workarounded the issues for me.
Vít
To all Rawhide users, be careful with update of mutter. It seems that mutter-3.23.2-2.fc26 is broken [1], which prevents you from log in to your system. Downgrade to mutter-3.23.1-2.fc26.x86_64 workarounded the issues for me.
Vít
This serves as a nice example why we need to tweak how Rawhide works if we want people actually running on it. I talked to rtcm on #fedora-desktop and the reason is that gnome-shell 3.23.2 needs to be updated in sync with mutter. Mutter has been built just fine [1], but gnome-shell build failed [2]. And, snap, GNOME users are immediately (at the next repo push) screwed because of a failed build.
Of course developers will sooner or later fix the broken build, but often there are issues can't be fixed immediately. And the successful mutter build can't be taken back either. No good solution there.
So, we either need: a) updates-testing for Rawhide - e.g. with auto-push even with 0 karma after 3 days, doesn't matter, it would still allow us to prevent many of these issues or b) decouple building and submitting to Rawhide - because the idea that everything and anything built is immediately part of the release seems very broken to me. At least if we want people to use it. If we want just a huge repo of frequently broken bleeding edge packages, yes, that's exactly what we have.
With b) solution the QA still can't stop bad updates, but at least the maintainer can decide to not push it to Rawhide (seeing that only one of two mandatory packages has been successfully built). With a) solution the maintainers can do the same thing, and other people can detect and stop broken updates as well.
[1] http://koji.fedoraproject.org/koji/buildinfo?buildID=820286 [2] http://koji.fedoraproject.org/koji/buildinfo?buildID=822182
Kamil Paral wrote:
This serves as a nice example why we need to tweak how Rawhide works if we want people actually running on it.
And why do we need that? Rawhide is a place to do development, not a rolling release distro.
So, we either need: a) updates-testing for Rawhide - e.g. with auto-push even with 0 karma after 3 days, doesn't matter, it would still allow us to prevent many of these issues or b) decouple building and submitting to Rawhide - because the idea that everything and anything built is immediately part of the release seems very broken to me.
Those proposals will both slow down development to a crawl and prevent us from doing our packaging work.
At least if we want people to use it. If we want just a huge repo of frequently broken bleeding edge packages, yes, that's exactly what we have.
The latter is exactly what you get when you do actual development. The former is a totally impractical illusion.
Kevin Kofler
On Wed, 2016-12-07 at 06:21 +0100, Kevin Kofler wrote:
Kamil Paral wrote:
This serves as a nice example why we need to tweak how Rawhide works if we want people actually running on it.
And why do we need that? Rawhide is a place to do development, not a rolling release distro.
So, we either need: a) updates-testing for Rawhide - e.g. with auto-push even with 0 karma after 3 days, doesn't matter, it would still allow us to prevent many of these issues or b) decouple building and submitting to Rawhide - because the idea that everything and anything built is immediately part of the release seems very broken to me.
Those proposals will both slow down development to a crawl and prevent us from doing our packaging work.
At least if we want people to use it. If we want just a huge repo of frequently broken bleeding edge packages, yes, that's exactly what we have.
The latter is exactly what you get when you do actual development. The former is a totally impractical illusion.
I quote, from https://fedoraproject.org/wiki/Releases/Rawhide :
"Rawhide has the following Goals:
To allow package maintainers to integrate the newest *usable* versions of their packages into Fedora. To allow advanced users access to the newest *usable* packages in a rolling manner."
Emphasis is original, not mine.
More to the point: it is entirely awful for the quality of Fedora as a whole if Rawhide is allowed to be completely broken for substantial periods of time - and this *did* make Rawhide completely broken. Just about any package set besides minimal could not be installed or updated. Several release-blocking deliverables entirely failed to compose.
This isn't 2005 any more. It's not OK for us to go for weeks without Rawhide working, then finally clean it all up a week before the Alpha release and try to figure out where we stand. We have mechanisms now which make it a reasonable goal to keep Rawhide working. As long as we actually pay attention, they make it very easy to pinpoint breakage and fix it much more easily than before.
But if we just let things be completely broken, all that goes away: we've no idea if half the rest of the distro is broken if we can't get to the point of testing it because of bugs like this. Keeping a baseline level of fundamental working-ness lets us test in much more depth and identify breakages *as they happen* and fix them rapidly. Refusing to revert changes like this and instead insisting that their consequences be fixed live, while the distro fails to work or build properly for the entire time until that's done, is just a crappy way to do things. We have the capability to do better, thus we should.
Adam Williamson wrote:
More to the point: it is entirely awful for the quality of Fedora as a whole if Rawhide is allowed to be completely broken for substantial periods of time - and this *did* make Rawhide completely broken.
You may consider it awful, but it is necessary to allow development to be done.
Just about any package set besides minimal could not be installed or updated.
Not true. Plasma does not depend on mutter.
Several release-blocking deliverables entirely failed to compose.
If the KDE/Plasma Spin also failed to compose, that would be due to some unrelated issue. If not, "Just about any package set besides minimal could not be installed or updated." is untrue.
This isn't 2005 any more.
Just because there is a different integer written on our calendars now does not magically make it possible to do development and deliver a usable rolling release in the same tree.
Kevin Kofler
On Thu, 2016-12-08 at 02:13 +0100, Kevin Kofler wrote:
Adam Williamson wrote:
More to the point: it is entirely awful for the quality of Fedora as a whole if Rawhide is allowed to be completely broken for substantial periods of time - and this *did* make Rawhide completely broken.
You may consider it awful, but it is necessary to allow development to be done.
Just about any package set besides minimal could not be installed or updated.
Not true. Plasma does not depend on mutter.
But it does depend on several other things that went through the libldb / libtdb complex. KDE images failed to compose and KDE package sets failed to install just like Server and Workstation. (KDE is in fact still broken even now ldb and tdb have been fixed, but that's your problem, not mine...)
Several release-blocking deliverables entirely failed to compose.
If the KDE/Plasma Spin also failed to compose, that would be due to some unrelated issue. If not, "Just about any package set besides minimal could not be installed or updated." is untrue.
No, the thing you're missing is that the CFLAGS change affected far more things than just GNOME.
Adam Williamson wrote:
But it does depend on several other things that went through the libldb / libtdb complex. KDE images failed to compose and KDE package sets failed to install just like Server and Workstation. (KDE is in fact still broken even now ldb and tdb have been fixed, but that's your problem, not mine...)
KDE packages are broken in Rawhide mainly due to 2 issues: 1. breakage due to OpenSSL API and ABI changes (OpenSSL 1.1), 2. breakage due to a glibc header change confusing Qt's moc tool. (The Qt 4 version and the Qt 5 version get confused in different ways.)
For 1., we are now building affected packages (where there is no working patch porting them to OpenSSL 1.1) against compat-openssl10-devel. There was a problem with that: mariadb-devel had a Requires on openssl-devel which conflicts with compat-openssl10-devel. Rex Dieter fixed that in mariadb- devel now.
For 2., we have workarounds now that we have identified the issue. See: https://bugzilla.redhat.com/show_bug.cgi?id=1396755
Neither is related in any way to the CFLAGS fiasco.
Kevin Kofler
On 12/07/2016 05:13 PM, Kevin Kofler wrote:
Adam Williamson wrote:
More to the point: it is entirely awful for the quality of Fedora as a whole if Rawhide is allowed to be completely broken for substantial periods of time - and this *did* make Rawhide completely broken.
You may consider it awful, but it is necessary to allow development to be done.
You keep making this point, but I haven't seen any explanation for it. Why is it necessary to have Rawhide broken in order to do development? Don't you develop locally and then push the result when it's functional?
Samuel Sieb wrote:
You keep making this point, but I haven't seen any explanation for it. Why is it necessary to have Rawhide broken in order to do development? Don't you develop locally and then push the result when it's functional?
No (at least not for the typical packaging "development" where there is just a bunch of packages to bump to a new upstream version), and besides, even doing so does not prevent issues like in the case that started this thread, where one package out of a bunch of interdependent packages unexpectedly fails to build due to an unrelated change in another component.
Also note that I run only stable releases, so even when I actually do development and testing locally, it will NOT catch random unrelated Rawhide breakage because I will be doing the builds for the released Fedora (locally or in a Copr). Rawhide is just too broken to use on a machine that I use also for non-Fedora work, and no amount of your misguided developer- unfriendly attempts to make it not so will change that. If you try to force me to use Rawhide to do Fedora packaging, I will be unable to continue contributing to Fedora.
Kevin Kofler
On Wed, Dec 7, 2016 at 2:13 AM, Adam Williamson adamwill@fedoraproject.org wrote:
Emphasis is original, not mine.
More to the point: it is entirely awful for the quality of Fedora as a whole if Rawhide is allowed to be completely broken for substantial periods of time - and this *did* make Rawhide completely broken. Just about any package set besides minimal could not be installed or updated. Several release-blocking deliverables entirely failed to compose.
Given that the breakage was not for rawhide as a whole, but only for Gnome, the problem does not seem to be as large as you imply. The complexity and fragility of *Gnome*, and its vulnerability to even very small changes of internal and external components, is a strong reason to keep other window managers accessible for developers to encounter, and be able to debug, the mess.
But if we just let things be completely broken, all that goes away: we've no idea if half the rest of the distro is broken if we can't get to the point of testing it because of bugs like this. Keeping a
Gnome is not "half the rest of the distro".
baseline level of fundamental working-ness lets us test in much more depth and identify breakages *as they happen* and fix them rapidly. Refusing to revert changes like this and instead insisting that their consequences be fixed live, while the distro fails to work or build properly for the entire time until that's done, is just a crappy way to do things. We have the capability to do better, thus we should.
Adam, it's clear it was a problematic breakage. But there are levels of debugging that are feasible for such large and complex structures, whether they be all of Fedora or merely the Gnome toolkit. And there are *going* to be interactions that break other components.
Kamil Paral wrote:
And why do we need that? Rawhide is a place to do development, not a rolling release distro.
What is the purpose of Rawhide if it could be broken anytime? I can't understand that, the development could be done and Rawhide could be functional at the same time. It's not exclusive.
If Rawhide is not usable, nobody would use it. And if nobody use Rawhide, it couldn't be tested in many configurations and your development couldn't be tested correctly.
So, you can push in Rawhide when the package doesn't brake the system or the software entirely. It's not interesting to push packages with these issues into Rawhide.
In my work, you can't push a commit or package in devel branch if it is not functional. If you want to do development, you can do that locally before push it, can't you?
And, as mentioned by Adam, Rawhide definition is a rolling functional release. Of course, not recommended for end users, but with functional GUI or software...
Regards, Charles-Antoine Couret
On Wednesday, December 7, 2016 6:21:02 AM CET Kevin Kofler wrote:
Kamil Paral wrote:
This serves as a nice example why we need to tweak how Rawhide works if we want people actually running on it.
And why do we need that?
This is clear to me: Early testing -> early fixes -> faster development.
Having no Rawhide users would degrade releaes of stable fedoras into the actual Rawhide quality.
Pavel
Dne 6.12.2016 v 15:19 Kamil Paral napsal(a):
To all Rawhide users, be careful with update of mutter. It seems that mutter-3.23.2-2.fc26 is broken [1], which prevents you from log in to your system. Downgrade to mutter-3.23.1-2.fc26.x86_64 workarounded the issues for me.
Vít
This serves as a nice example why we need to tweak how Rawhide works if we want people actually running on it. I talked to rtcm on #fedora-desktop and the reason is that gnome-shell 3.23.2 needs to be updated in sync with mutter. Mutter has been built just fine [1], but gnome-shell build failed [2]. And, snap, GNOME users are immediately (at the next repo push) screwed because of a failed build.
Of course developers will sooner or later fix the broken build, but often there are issues can't be fixed immediately. And the successful mutter build can't be taken back either. No good solution there.
So, we either need: a) updates-testing for Rawhide - e.g. with auto-push even with 0 karma after 3 days, doesn't matter, it would still allow us to prevent many of these issues or b) decouple building and submitting to Rawhide - because the idea that everything and anything built is immediately part of the release seems very broken to me. At least if we want people to use it. If we want just a huge repo of frequently broken bleeding edge packages, yes, that's exactly what we have.
With b) solution the QA still can't stop bad updates, but at least the maintainer can decide to not push it to Rawhide (seeing that only one of two mandatory packages has been successfully built). With a) solution the maintainers can do the same thing, and other people can detect and stop broken updates as well.
[1] http://koji.fedoraproject.org/koji/buildinfo?buildID=820286 [2] http://koji.fedoraproject.org/koji/buildinfo?buildID=822182 _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org
In this context, I have to mention my "Bodhi For Rawhide?" [1] proposal again ....
Vít
[1] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
-
Adam Williamson -
Charles-Antoine Couret -
Kamil Paral -
Kevin Kofler -
Nico Kadel-Garcia -
Pavel Raiskup -
Samuel Sieb -
Vít Ondruch