= System Wide Change: GCC8 = https://fedoraproject.org/wiki/Changes/GCC8
Change owner(s): * Jakub Jelínek <jakub AT redhat DOT com>
Switch GCC in Fedora 28 to 8.x.y, rebuild all packages with it, or optionally rebuild just some packages with it and rebuild all packages only in Fedora 29.
== Detailed Description == GCC 8 is currently in stage3, will move to stage4 on January 14th, in prerelease state with only regression bugfixes and documentation fixes allowed. The release will happen probably in the middle of April. We are working on scratch gcc rpms and will perform a test mass rebuild.
== Scope == All packages should be rebuilt with the new gcc once it hits f28, or, if there is not enough time for that, just all packages built after the new gcc hits the buildroots.
* Proposal owners: Build gcc in f28, rebuild packages that have direct dependencies on exact gcc version (libtool, llvm, gcc-python-plugin, odb).
* Other developers: First few days/weeks just voluntary rebuilds using the new system gcc, if things fail, look at http://gcc.gnu.org/gcc-8/porting_to.html and fix bugs in packages or, if there is a gcc bug or suspected gcc bug, analyze and report.
* Release engineering: PR#7249: https://pagure.io/releng/issue/7249 Mass rebuild requested for F28.
* Policies and guidelines: No policies need to be changed
On Tue, 2018-01-09 at 12:11 +0100, Jan Kurik wrote:
= System Wide Change: GCC8 = https://fedoraproject.org/wiki/Changes/GCC8
Change owner(s):
- Jakub Jelínek <jakub AT redhat DOT com>
Switch GCC in Fedora 28 to 8.x.y, rebuild all packages with it, or optionally rebuild just some packages with it and rebuild all packages only in Fedora 29.
== Detailed Description == GCC 8 is currently in stage3, will move to stage4 on January 14th, in prerelease state with only regression bugfixes and documentation fixes allowed. The release will happen probably in the middle of April. We are working on scratch gcc rpms and will perform a test mass rebuild.
The timing on this looks a bit awkward when compared with the current schedule, which has the Beta going out in March and Final early in May:
https://fedoraproject.org/wiki/Releases/28/Schedule
That would appear to mean we'd have to do a mass rebuild with a pre- release GCC, or do a mass rebuild between Beta and Final, or suddenly introduce a new compiler post-Beta, potentially meaning that we need to rebuild something to fix a blocker bug quite late in the release and discover that the new GCC which has suddenly appeared causes problems with building it. None of those sound like great options to me.
Is there significant enough benefit from the new GCC to outweigh all these potential negative impacts to it going stable between our Beta and Final releases?
On Tue, Jan 09, 2018 at 09:39:41AM -0800, Adam Williamson wrote:
The timing on this looks a bit awkward when compared with the current schedule, which has the Beta going out in March and Final early in May:
https://fedoraproject.org/wiki/Releases/28/Schedule
That would appear to mean we'd have to do a mass rebuild with a pre- release GCC, or do a mass rebuild between Beta and Final, or suddenly introduce a new compiler post-Beta, potentially meaning that we need to rebuild something to fix a blocker bug quite late in the release and discover that the new GCC which has suddenly appeared causes problems with building it. None of those sound like great options to me.
Mass rebuild with a prerelease version of the compiler, like every year at least in the past 10 years in Fedora.
Is there significant enough benefit from the new GCC to outweigh all these potential negative impacts to it going stable between our Beta and Final releases?
Yes.
Jakub
On Tue, 2018-01-09 at 18:45 +0100, Jakub Jelinek wrote:
On Tue, Jan 09, 2018 at 09:39:41AM -0800, Adam Williamson wrote:
The timing on this looks a bit awkward when compared with the current schedule, which has the Beta going out in March and Final early in May:
https://fedoraproject.org/wiki/Releases/28/Schedule
That would appear to mean we'd have to do a mass rebuild with a pre- release GCC, or do a mass rebuild between Beta and Final, or suddenly introduce a new compiler post-Beta, potentially meaning that we need to rebuild something to fix a blocker bug quite late in the release and discover that the new GCC which has suddenly appeared causes problems with building it. None of those sound like great options to me.
Mass rebuild with a prerelease version of the compiler, like every year at least in the past 10 years in Fedora.
Well, true, but then just like every year, we'll wind up doing a lot of the spadework of fixing things to build with the new GCC. And probably at first some critical things will fail to build and that'll mess up the stability of the distro for a couple of weeks. I guess if everyone else is still loving that grind, hey.
On Tue, Jan 9, 2018 at 12:58 PM Adam Williamson adamwill@fedoraproject.org wrote:
On Tue, 2018-01-09 at 18:45 +0100, Jakub Jelinek wrote:
On Tue, Jan 09, 2018 at 09:39:41AM -0800, Adam Williamson wrote:
The timing on this looks a bit awkward when compared with the current schedule, which has the Beta going out in March and Final early in May:
https://fedoraproject.org/wiki/Releases/28/Schedule
That would appear to mean we'd have to do a mass rebuild with a pre- release GCC, or do a mass rebuild between Beta and Final, or suddenly introduce a new compiler post-Beta, potentially meaning that we need to rebuild something to fix a blocker bug quite late in the release and discover that the new GCC which has suddenly appeared causes problems with building it. None of those sound like great options to me.
Mass rebuild with a prerelease version of the compiler, like every year
at
least in the past 10 years in Fedora.
Well, true, but then just like every year, we'll wind up doing a lot of the spadework of fixing things to build with the new GCC. And probably at first some critical things will fail to build and that'll mess up the stability of the distro for a couple of weeks. I guess if everyone else is still loving that grind, hey.
This is the cost of being "First". Fedora has long enjoyed a tight coupling with the GCC upstream. It's a symbiosis: they use our mass-rebuild to help identify any issues before GCC goes stable and in turn Fedora gets to have the newest compiler features before anyone else.
Realistically, since Fedora is the first real-world exercise of new GCC, if we waited for the upstream stable release, it would be exactly as it is now. Fedora would hit all the same issues and GCC would have to release updates to fix them for us.
On Tue, Jan 09, 2018 at 07:50:10PM +0000, Stephen Gallagher wrote:
Well, true, but then just like every year, we'll wind up doing a lot of the spadework of fixing things to build with the new GCC. And probably at first some critical things will fail to build and that'll mess up the stability of the distro for a couple of weeks. I guess if everyone else is still loving that grind, hey.
This is the cost of being "First". Fedora has long enjoyed a tight coupling with the GCC upstream. It's a symbiosis: they use our mass-rebuild to help identify any issues before GCC goes stable and in turn Fedora gets to have the newest compiler features before anyone else.
To be fair, Ubuntu (or Debian or both, dunno) has already performed test mass rebuilds with GCC 8 prerelease some time ago and OpenSUSE usually performs them roughly at the same time as we do. We are likely the first one or one of the first ones to deploy it as a stable compiler in the distro and it is mutually beneficial both for the distro and for GCC.
Jakub
On Tue, 2018-01-09 at 21:51 +0100, Jakub Jelinek wrote:
On Tue, Jan 09, 2018 at 07:50:10PM +0000, Stephen Gallagher wrote:
Well, true, but then just like every year, we'll wind up doing a lot of the spadework of fixing things to build with the new GCC. And probably at first some critical things will fail to build and that'll mess up the stability of the distro for a couple of weeks. I guess if everyone else is still loving that grind, hey.
This is the cost of being "First". Fedora has long enjoyed a tight coupling with the GCC upstream. It's a symbiosis: they use our mass-rebuild to help identify any issues before GCC goes stable and in turn Fedora gets to have the newest compiler features before anyone else.
To be fair, Ubuntu (or Debian or both, dunno) has already performed test mass rebuilds with GCC 8 prerelease some time ago and OpenSUSE usually performs them roughly at the same time as we do. We are likely the first one or one of the first ones to deploy it as a stable compiler in the distro and it is mutually beneficial both for the distro and for GCC.
Just for the record, it is now 11pm the day before we are supposed to branch Fedora 28, and I have spent the whole evening fixing OpenColorIO's Python bindings to build with GCC 8:
https://github.com/imageworks/OpenColorIO/pull/518
only to find that it fails to build on i686 because since pdftex got rebuilt with GCC 8 (OK, haven't confirmed that yet, but it's the most obvious suspect), it's segfaulting:
https://koji.fedoraproject.org/koji/taskinfo?taskID=25176488
Also noted by QuLogic trying to build R-htmltools:
https://koji.fedoraproject.org/koji/taskinfo?taskID=25173683
So now I am running the build in an i686 mock so I can shell into the mock and hopefully get a traceback of the pdftex crash and try to do *something* about fixing it.
This is the kind of 'spadework' I was talking about. (And I haven't even mentioned any of the *other* cases we've hit).
On Mon, 2018-02-19 at 22:57 -0800, Adam Williamson wrote:
On Tue, 2018-01-09 at 21:51 +0100, Jakub Jelinek wrote:
On Tue, Jan 09, 2018 at 07:50:10PM +0000, Stephen Gallagher wrote:
Well, true, but then just like every year, we'll wind up doing a lot of the spadework of fixing things to build with the new GCC. And probably at first some critical things will fail to build and that'll mess up the stability of the distro for a couple of weeks. I guess if everyone else is still loving that grind, hey.
This is the cost of being "First". Fedora has long enjoyed a tight coupling with the GCC upstream. It's a symbiosis: they use our mass-rebuild to help identify any issues before GCC goes stable and in turn Fedora gets to have the newest compiler features before anyone else.
To be fair, Ubuntu (or Debian or both, dunno) has already performed test mass rebuilds with GCC 8 prerelease some time ago and OpenSUSE usually performs them roughly at the same time as we do. We are likely the first one or one of the first ones to deploy it as a stable compiler in the distro and it is mutually beneficial both for the distro and for GCC.
Just for the record, it is now 11pm the day before we are supposed to branch Fedora 28, and I have spent the whole evening fixing OpenColorIO's Python bindings to build with GCC 8:
https://github.com/imageworks/OpenColorIO/pull/518
only to find that it fails to build on i686 because since pdftex got rebuilt with GCC 8 (OK, haven't confirmed that yet, but it's the most obvious suspect), it's segfaulting:
https://koji.fedoraproject.org/koji/taskinfo?taskID=25176488
Also noted by QuLogic trying to build R-htmltools:
https://koji.fedoraproject.org/koji/taskinfo?taskID=25173683
So now I am running the build in an i686 mock so I can shell into the mock and hopefully get a traceback of the pdftex crash and try to do *something* about fixing it.
So, here lies a hilarious tale of dynamic generation of entirely undocumented C code using single-character variable names by a ridiculously arcane build system tracked in Subversion:
https://bugzilla.redhat.com/show_bug.cgi?id=1546964
Excuse me while I go break some stuff.
On Feb 20, 2018 08:54, "Adam Williamson" adamwill@fedoraproject.org wrote:
On Mon, 2018-02-19 at 22:57 -0800, Adam Williamson wrote:
On Tue, 2018-01-09 at 21:51 +0100, Jakub Jelinek wrote:
On Tue, Jan 09, 2018 at 07:50:10PM +0000, Stephen Gallagher wrote:
Well, true, but then just like every year, we'll wind up doing a
lot of
the spadework of fixing things to build with the new GCC. And
probably
at first some critical things will fail to build and that'll mess up the stability of the distro for a couple of weeks. I guess if
everyone
else is still loving that grind, hey.
This is the cost of being "First". Fedora has long enjoyed a tight
coupling
with the GCC upstream. It's a symbiosis: they use our mass-rebuild to
help
identify any issues before GCC goes stable and in turn Fedora gets to
have
the newest compiler features before anyone else.
To be fair, Ubuntu (or Debian or both, dunno) has already performed
test mass
rebuilds with GCC 8 prerelease some time ago and OpenSUSE usually
performs them
roughly at the same time as we do. We are likely the first one or one
of
the first ones to deploy it as a stable compiler in the distro and it is mutually beneficial both for the distro and for GCC.
Just for the record, it is now 11pm the day before we are supposed to branch Fedora 28, and I have spent the whole evening fixing OpenColorIO's Python bindings to build with GCC 8:
https://github.com/imageworks/OpenColorIO/pull/518
only to find that it fails to build on i686 because since pdftex got rebuilt with GCC 8 (OK, haven't confirmed that yet, but it's the most obvious suspect), it's segfaulting:
https://koji.fedoraproject.org/koji/taskinfo?taskID=25176488
Also noted by QuLogic trying to build R-htmltools:
https://koji.fedoraproject.org/koji/taskinfo?taskID=25173683
So now I am running the build in an i686 mock so I can shell into the mock and hopefully get a traceback of the pdftex crash and try to do *something* about fixing it.
So, here lies a hilarious tale of dynamic generation of entirely undocumented C code using single-character variable names by a ridiculously arcane build system tracked in Subversion:
Wow, that's exceptionally awful. You're a hero for tracking down and fixing those things. Really. Thank you.
Fabio
https://bugzilla.redhat.com/show_bug.cgi?id=1546964
Excuse me while I go break some stuff. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org
BTW dovecot is broken by all the compiler and crypto changes, both for the two last rebuilds (that failed) and the build before (that succeeded but has broken auth which is sort of needed for an imap server)
"NM" == Nicolas Mailhot nicolas.mailhot@laposte.net writes:
NM> BTW dovecot is broken by all the compiler and crypto changes, both NM> for the two last rebuilds (that failed) and the build before (that NM> succeeded but has broken auth which is sort of needed for an imap NM> server)
Interestingly enough, cyrus-imapd is also broken. Technically it compiles fine and mostly runs fine, but the extensive test suite (which we now run at build time) causes the xapian-based search indexing component to segfault deep down in the bowels of some tests.
In what is surely a plan to make me lose the rest of my hair, the failures _only_ happen when run in mock. In a clean rawhide VM or docker container with enough build deps to run fedpkg local, things build fine. So far the only thing resembling a backtrace I've been able to get out of anything is just this which coredumpctl saves:
#0 0x0000000000000120 n/a (n/a) #1 0x00007ffd8745b720 n/a (n/a) #2 0x00007fb063abac80 _ZTVNSt7__cxx1119basic_ostringstreamIcSt11char_traitsIcESaIcEEE (libstdc++.so.6)
That makes me wonder if I'm hitting a libstdc++ bug. If I do coredumpctl gdb and then to a backtrace, I just get:
(gdb) bt #0 0x0000000000000120 in ?? () #1 0x00007fb06352881e in ?? () #2 0x00007ffd8745b7b0 in ?? () #3 0x00007ffd8745b528 in ?? () #4 0x0000000000000000 in ?? ()
which obviously isn't helpful. And this was obtained by running mock on a rawhide VM where I've made sure that the environment inside and outside of the chroot have the same packages, all debuginfo is installed and the test suite is run in %install instead of %check so that the unstripped binaries are executed. I can't figure out a way to get more useful information out.
- J<
Le 2018-02-20 19:51, Jason L Tibbitts III a écrit :
"NM" == Nicolas Mailhot nicolas.mailhot@laposte.net writes:
NM> BTW dovecot is broken by all the compiler and crypto changes, both NM> for the two last rebuilds (that failed) and the build before (that NM> succeeded but has broken auth which is sort of needed for an imap NM> server)
Interestingly enough, cyrus-imapd is also broken. Technically it compiles fine and mostly runs fine, but the extensive test suite (which we now run at build time) causes the xapian-based search indexing component to segfault deep down in the bowels of some tests.
Yes, the failure for the latest dovecot that built is weird, and for some reason abrt refuses to collect traces on the process. In the end I had to downgrade (I need my mail) and thankfully the previous devel binary build in koji does not bomb. I'll try to at least post the stack bits journalctl captured (investigating more would be nice but I doubt I’ll find the time, I’m kind of over-over-over-booked right now)
Regards,
On Mon, 2018-02-19 at 22:57 -0800, Adam Williamson wrote:
On Tue, 2018-01-09 at 21:51 +0100, Jakub Jelinek wrote:
On Tue, Jan 09, 2018 at 07:50:10PM +0000, Stephen Gallagher wrote:
Well, true, but then just like every year, we'll wind up doing a lot of the spadework of fixing things to build with the new GCC. And probably at first some critical things will fail to build and that'll mess up the stability of the distro for a couple of weeks. I guess if everyone else is still loving that grind, hey.
This is the cost of being "First". Fedora has long enjoyed a tight coupling with the GCC upstream. It's a symbiosis: they use our mass- rebuild to help identify any issues before GCC goes stable and in turn Fedora gets to have the newest compiler features before anyone else.
To be fair, Ubuntu (or Debian or both, dunno) has already performed test mass rebuilds with GCC 8 prerelease some time ago and OpenSUSE usually performs them roughly at the same time as we do. We are likely the first one or one of the first ones to deploy it as a stable compiler in the distro and it is mutually beneficial both for the distro and for GCC.
Just for the record, it is now 11pm the day before we are supposed to branch Fedora 28, and I have spent the whole evening fixing OpenColorIO's Python bindings to build with GCC 8:
pdftex Segmentation fault in arches i686 and amrv7 when build gdal [1]
gdal build fails and have broken deps, which makes openv broken deps , which make mlt broken. So I have a big chain of packages with broken deps and I can't do nothing
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1546913
Best regards,
https://github.com/imageworks/OpenColorIO/pull/518
only to find that it fails to build on i686 because since pdftex got rebuilt with GCC 8 (OK, haven't confirmed that yet, but it's the most obvious suspect), it's segfaulting:
https://koji.fedoraproject.org/koji/taskinfo?taskID=25176488
Also noted by QuLogic trying to build R-htmltools:
https://koji.fedoraproject.org/koji/taskinfo?taskID=25173683
So now I am running the build in an i686 mock so I can shell into the mock and hopefully get a traceback of the pdftex crash and try to do *something* about fixing it.
This is the kind of 'spadework' I was talking about. (And I haven't even mentioned any of the *other* cases we've hit). -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org
-
Adam Williamson -
Fabio Valentini -
Jakub Jelinek -
Jan Kurik -
Jason L Tibbitts III -
Nicolas Mailhot -
Stephen Gallagher -
Sérgio Basto