Recently, I've been participating in some discussion as to how to enable C++11 support for EPEL builds. Specifically, R (and its large universe of addons in CRAN) would benefit significantly from C++11 support.
After much discussion, it seems like the only sane way to do this is to use the Red Hat Developer Toolset (for el6 and el7). It is my understanding that all RHEL customers (and CentOS users) should be able to enable this repository without restrictions.
I'd like to propose that we enable the Developer Toolset repo in EPEL and allow packages to depend on it. Thoughts?
~tom
== Red Hat
On Mon, Aug 15, 2016 at 12:24 PM, Tom Callaway tcallawa@redhat.com wrote:
Recently, I've been participating in some discussion as to how to enable C++11 support for EPEL builds. Specifically, R (and its large universe of addons in CRAN) would benefit significantly from C++11 support.
After much discussion, it seems like the only sane way to do this is to use the Red Hat Developer Toolset (for el6 and el7). It is my understanding that all RHEL customers (and CentOS users) should be able to enable this repository without restrictions.
I'd like to propose that we enable the Developer Toolset repo in EPEL and allow packages to depend on it. Thoughts?
Previously, the big hold up was that it was at least partially behind a paywall, but now that it's open and headed up by a SIG, I don't see any reason why we shouldn't start benefit from using it. It might be nice to have some rules or at least recommendations on which version to use and there will need to be a policy when versions are EOLed, but maybe just an automated set of rebuilds would be sufficient.
On Mon, Aug 15, 2016 at 2:40 PM, Dave Johansen davejohansen@gmail.com wrote:
On Mon, Aug 15, 2016 at 12:24 PM, Tom Callaway tcallawa@redhat.com wrote:
Recently, I've been participating in some discussion as to how to enable C++11 support for EPEL builds. Specifically, R (and its large universe of addons in CRAN) would benefit significantly from C++11 support.
After much discussion, it seems like the only sane way to do this is to use the Red Hat Developer Toolset (for el6 and el7). It is my understanding that all RHEL customers (and CentOS users) should be able to enable this repository without restrictions.
I'd like to propose that we enable the Developer Toolset repo in EPEL and allow packages to depend on it. Thoughts?
Previously, the big hold up was that it was at least partially behind a paywall, but now that it's open and headed up by a SIG, I don't see any reason why we shouldn't start benefit from using it. It might be nice to have some rules or at least recommendations on which version to use and there will need to be a policy when versions are EOLed, but maybe just an automated set of rebuilds would be sufficient.
Probably the best way to do this would be to create a "wrapper" package that would point to the latest version and provide a macro for setting up the environment so that the spec uses the Developer Toolset compiler for building the software instead of the system one. Maybe "devtoolset-toolchain" or something like that?
On Mon, 15 Aug 2016 14:24:21 -0400 Tom Callaway tcallawa@redhat.com wrote:
Recently, I've been participating in some discussion as to how to enable C++11 support for EPEL builds. Specifically, R (and its large universe of addons in CRAN) would benefit significantly from C++11 support.
After much discussion, it seems like the only sane way to do this is to use the Red Hat Developer Toolset (for el6 and el7). It is my understanding that all RHEL customers (and CentOS users) should be able to enable this repository without restrictions.
I'd like to propose that we enable the Developer Toolset repo in EPEL and allow packages to depend on it. Thoughts?
So, this is a SCL of newer tools right?
Does this result in a runtime dependency? Or just a build time one? ie, will everyone using packages built with this have to install it also, or it's just a buildrequires?
kevin
On Tue, Aug 16, 2016 at 11:07 AM, Kevin Fenzi kevin@scrye.com wrote:
On Mon, 15 Aug 2016 14:24:21 -0400 Tom Callaway tcallawa@redhat.com wrote:
Recently, I've been participating in some discussion as to how to enable C++11 support for EPEL builds. Specifically, R (and its large universe of addons in CRAN) would benefit significantly from C++11 support.
After much discussion, it seems like the only sane way to do this is to use the Red Hat Developer Toolset (for el6 and el7). It is my understanding that all RHEL customers (and CentOS users) should be able to enable this repository without restrictions.
I'd like to propose that we enable the Developer Toolset repo in EPEL and allow packages to depend on it. Thoughts?
So, this is a SCL of newer tools right?
Does this result in a runtime dependency? Or just a build time one? ie, will everyone using packages built with this have to install it also, or it's just a buildrequires?
It's a BuildRequires only because any newer features from the compiler are statically linked into the binary. See other notes at the bottom of Section 2.3: https://access.redhat.com/documentation/en-US/Red_Hat_Developer_Toolset/2/ht...
On 08/16/2016 11:07 AM, Kevin Fenzi wrote:
On Mon, 15 Aug 2016 14:24:21 -0400 Tom Callaway tcallawa@redhat.com wrote:
Recently, I've been participating in some discussion as to how to enable C++11 support for EPEL builds. Specifically, R (and its large universe of addons in CRAN) would benefit significantly from C++11 support.
After much discussion, it seems like the only sane way to do this is to use the Red Hat Developer Toolset (for el6 and el7). It is my understanding that all RHEL customers (and CentOS users) should be able to enable this repository without restrictions.
I'd like to propose that we enable the Developer Toolset repo in EPEL and allow packages to depend on it. Thoughts?
So, this is a SCL of newer tools right?
Does this result in a runtime dependency? Or just a build time one? ie, will everyone using packages built with this have to install it also, or it's just a buildrequires?
kevin
Well, one question is the R-core-devel package. Currently it requires gcc-c++ and gcc-gfortran. Presumably if R were compiled with the devtoolset compiler, it would need to require those to build add-ons locally. Not sure though. That's perhaps not many people though.
I'll also note that R currently BRs gcc-objc but I don't see any devtoolset gcc-objc packages.
Can we not just have an updated gcc package in EPEL? It wouldn't be hard to make one. Or do we have to not conflict with whatever Red Hat might put in one of their addons? Even then it would be easy to avoid both file and package name conflicts.
- J<
On 08/18/2016 05:57 PM, Jason L Tibbitts III wrote:
Can we not just have an updated gcc package in EPEL? It wouldn't be hard to make one. Or do we have to not conflict with whatever Red Hat might put in one of their addons? Even then it would be easy to avoid both file and package name conflicts.
I don't see why not (we only promise to not conflict with a couple base channels), though I'm willing to believe that it would be hard, especially for EL6.
On Thu, Aug 18, 2016 at 10:03 PM, Orion Poplawski orion@cora.nwra.com wrote:
On 08/18/2016 05:57 PM, Jason L Tibbitts III wrote:
Can we not just have an updated gcc package in EPEL? It wouldn't be hard to make one. Or do we have to not conflict with whatever Red Hat might put in one of their addons? Even then it would be easy to avoid both file and package name conflicts.
I don't see why not (we only promise to not conflict with a couple base channels), though I'm willing to believe that it would be hard, especially for EL6.
devtoolset is designed to do all of this and is already done, so it seems that the only advantage to putting it in EPEL itself would be to reduce the number of repos during build time. In my opinion that's not a big enough advantage to justify all of the work that would be involved.
"DJ" == Dave Johansen davejohansen@gmail.com writes:
DJ> devtoolset is designed to do all of this and is already done, so it DJ> seems that the only advantage to putting it in EPEL itself would be DJ> to reduce the number of repos during build time.
So is devtoolset something I get access to as a CentOS user? How do I build these packages myself (i.e. in mock)?
- J<
On Mon, Aug 22, 2016 at 11:30 AM, Jason L Tibbitts III tibbs@math.uh.edu wrote:
"DJ" == Dave Johansen davejohansen@gmail.com writes:
DJ> devtoolset is designed to do all of this and is already done, so it DJ> seems that the only advantage to putting it in EPEL itself would be DJ> to reduce the number of repos during build time.
So is devtoolset something I get access to as a CentOS user? How do I build these packages myself (i.e. in mock)?
The official version used to be behind a paywall, but there were builds by CentOS and CERN. It's now been turned over to a SIG and is all open: https://www.softwarecollections.org/en/scls/?search=devtoolset
Building just the compiler part isn't too bad, but building the IDE and other parts of devtoolset takes some working out of the build order that I never worked out because I only cared about the compiler: https://www.redhat.com/archives/sclorg/2016-January/msg00002.html
Here's a blog on building SCLs in mock: http://developers.redhat.com/blog/2015/01/07/using-mock-to-build-python27-so...
For building in COPR, I followed these instructions (but the main magic point is editting each of the supported chroots to have "scl-utils-build <scl_name>-build"): https://www.redhat.com/archives/sclorg/2014-November/msg00015.html
On 22/08/16 18:30, Jason L Tibbitts III wrote:
"DJ" == Dave Johansen davejohansen@gmail.com writes:
DJ> devtoolset is designed to do all of this and is already done, so it DJ> seems that the only advantage to putting it in EPEL itself would be DJ> to reduce the number of repos during build time.
So is devtoolset something I get access to as a CentOS user? How do I build these packages myself (i.e. in mock)?
you should be able to 'yum install centos-release-scl' on a CentOS Linux machine and get access to all the SCLs
On Tue, 23 Aug 2016 14:21:24 +0100 Karanbir Singh mail-lists@karan.org wrote:
On 22/08/16 18:30, Jason L Tibbitts III wrote:
> "DJ" == Dave Johansen davejohansen@gmail.com writes:
DJ> devtoolset is designed to do all of this and is already done, DJ> so it seems that the only advantage to putting it in EPEL DJ> itself would be to reduce the number of repos during build DJ> time.
So is devtoolset something I get access to as a CentOS user? How do I build these packages myself (i.e. in mock)?
you should be able to 'yum install centos-release-scl' on a CentOS Linux machine and get access to all the SCLs
Yeah, but if we enable that for EPEL builds, we are going to get all SCLs right? So, people could start depending on them at runtime instead of just install time.
I'm not opposed to devtoolset, but I don't think we want to allow runtime scls without actual scl guidelines.
kevin
On Wed, Aug 24, 2016 at 4:22 PM, Kevin Fenzi kevin@scrye.com wrote:
On Tue, 23 Aug 2016 14:21:24 +0100 Karanbir Singh mail-lists@karan.org wrote:
On 22/08/16 18:30, Jason L Tibbitts III wrote:
>> "DJ" == Dave Johansen davejohansen@gmail.com writes:
DJ> devtoolset is designed to do all of this and is already done, DJ> so it seems that the only advantage to putting it in EPEL DJ> itself would be to reduce the number of repos during build DJ> time.
So is devtoolset something I get access to as a CentOS user? How do I build these packages myself (i.e. in mock)?
you should be able to 'yum install centos-release-scl' on a CentOS Linux machine and get access to all the SCLs
Yeah, but if we enable that for EPEL builds, we are going to get all SCLs right? So, people could start depending on them at runtime instead of just install time.
I'm not opposed to devtoolset, but I don't think we want to allow runtime scls without actual scl guidelines.
I seem to remember that Fedora didn't allow SCLs because there was some compatibility problem or something of the sort. Do you know the details or what the current state is?
Also, RedHat has been pushing devtoolset pretty hard. The response to a few bugzillas has even been "use devtoolset because the issue is fixed there and we're not going to fix the system gcc/libc/etc". So it seems like allowing SCLs in Fedora/EPEL makes sense and fits with the direction of RHEL in general.
On Wed, 24 Aug 2016 16:43:31 -0600 Dave Johansen davejohansen@gmail.com wrote:
On Wed, Aug 24, 2016 at 4:22 PM, Kevin Fenzi kevin@scrye.com wrote:
On Tue, 23 Aug 2016 14:21:24 +0100 Karanbir Singh mail-lists@karan.org wrote:
On 22/08/16 18:30, Jason L Tibbitts III wrote:
>>> "DJ" == Dave Johansen davejohansen@gmail.com writes:
DJ> devtoolset is designed to do all of this and is already DJ> done, so it seems that the only advantage to putting it in DJ> EPEL itself would be to reduce the number of repos during DJ> build time.
So is devtoolset something I get access to as a CentOS user? How do I build these packages myself (i.e. in mock)?
you should be able to 'yum install centos-release-scl' on a CentOS Linux machine and get access to all the SCLs
Yeah, but if we enable that for EPEL builds, we are going to get all SCLs right? So, people could start depending on them at runtime instead of just install time.
I'm not opposed to devtoolset, but I don't think we want to allow runtime scls without actual scl guidelines.
I seem to remember that Fedora didn't allow SCLs because there was some compatibility problem or something of the sort. Do you know the details or what the current state is?
It's not a compatibility problem. It's lack of guidelines.
Also, RedHat has been pushing devtoolset pretty hard. The response to a few bugzillas has even been "use devtoolset because the issue is fixed there and we're not going to fix the system gcc/libc/etc". So it seems like allowing SCLs in Fedora/EPEL makes sense and fits with the direction of RHEL in general.
As I noted, I'd probibly be for devtoolset because the guidelines would be pretty simple. Just do X Y and Z in your spec, build as normal and users wouldn't see any run time dep on it.
It gets weird where other SCL's come in. Can your package require say postgresql from SCL instead of the rhel one? If so, would our users all be able to install that? What happens if two packages need different SCL versions? Can SCL's depend on each other? Can you make a package thats an SCL? we have no guidelines for that, and just saying "do whatever you want" is likely to cause mass confusion and make everyone miserable.
kevin
On Wed, Aug 24, 2016 at 5:28 PM, Kevin Fenzi kevin@scrye.com wrote:
On Wed, 24 Aug 2016 16:43:31 -0600 Dave Johansen davejohansen@gmail.com wrote:
On Wed, Aug 24, 2016 at 4:22 PM, Kevin Fenzi kevin@scrye.com wrote:
On Tue, 23 Aug 2016 14:21:24 +0100 Karanbir Singh mail-lists@karan.org wrote:
On 22/08/16 18:30, Jason L Tibbitts III wrote:
>>>> "DJ" == Dave Johansen davejohansen@gmail.com writes:
DJ> devtoolset is designed to do all of this and is already DJ> done, so it seems that the only advantage to putting it in DJ> EPEL itself would be to reduce the number of repos during DJ> build time.
So is devtoolset something I get access to as a CentOS user? How do I build these packages myself (i.e. in mock)?
you should be able to 'yum install centos-release-scl' on a CentOS Linux machine and get access to all the SCLs
Yeah, but if we enable that for EPEL builds, we are going to get all SCLs right? So, people could start depending on them at runtime instead of just install time.
I'm not opposed to devtoolset, but I don't think we want to allow runtime scls without actual scl guidelines.
I seem to remember that Fedora didn't allow SCLs because there was some compatibility problem or something of the sort. Do you know the details or what the current state is?
It's not a compatibility problem. It's lack of guidelines.
Also, RedHat has been pushing devtoolset pretty hard. The response to a few bugzillas has even been "use devtoolset because the issue is fixed there and we're not going to fix the system gcc/libc/etc". So it seems like allowing SCLs in Fedora/EPEL makes sense and fits with the direction of RHEL in general.
As I noted, I'd probibly be for devtoolset because the guidelines would be pretty simple. Just do X Y and Z in your spec, build as normal and users wouldn't see any run time dep on it.
It gets weird where other SCL's come in. Can your package require say postgresql from SCL instead of the rhel one? If so, would our users all be able to install that? What happens if two packages need different SCL versions? Can SCL's depend on each other? Can you make a package thats an SCL? we have no guidelines for that, and just saying "do whatever you want" is likely to cause mass confusion and make everyone miserable.
I agree that how to handle SCLs can get really mess really fast, but a lot of projects are jumping on the "modern C++" bandwagon and allows devtoolset is low risk, easy to do and enables a lot of packages to be built with EPEL that otherwise wouldn't be.
Basically, I think that figuring out how to handle SCLs is a long term issue that will take some serious work, but coming up with some simple policies that allow it to be used in EPEL is something that should be well within the realm of the possible.
When I proposed importing gcc-5 to EPEL6 back in 04/2016 ( https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject... ) the response was an unequivocal no, EPEL does not install to /opt/ , so it dies right there.
Now you are proposing the same ( devtoolset/scl installs to /opt except for the wrapper call) but using a scheme that is somewhat less convenient (In scl the binutils and gcc have to be coupled, and as noted the imported gcc suite is incomplete), much less frequent (the most updated version is gcc-5.2, while I maintain both gcc-5.x and gcc-6.1), and much less complete (I import everything but gcc-gnat, while devtoolset4 only has gcc,gcc-c++ and gcc-gfortran. No gcc-objc, no gcc-go, no cpp, and none of the libs (cilk, gccjit, atomic, asan etc...).
I'm still building and maintaining both gcc and bintutils for my own purposes, which are based off of fc24 srpms, and with optionally gcc-c++ specs file hardcoded to use binutils tools at the new prefix so use of env-modules is not required.
I'm just wandering why the different treatment - the automatic knee-jerk reaction of dismissal to a newbie proposal vs. accepting the exact same proposal (although wrapped so it's less convenient to use) when it comes from someone else.
On 25/08//2016 06:59, Dave Johansen wrote:
On Wed, Aug 24, 2016 at 5:28 PM, Kevin Fenzi <kevin@scrye.com mailto:kevin@scrye.com> wrote:
On Wed, 24 Aug 2016 16:43:31 -0600 Dave Johansen <davejohansen@gmail.com <mailto:davejohansen@gmail.com>> wrote: > On Wed, Aug 24, 2016 at 4:22 PM, Kevin Fenzi <kevin@scrye.com <mailto:kevin@scrye.com>> wrote: > > > On Tue, 23 Aug 2016 14:21:24 +0100 > > Karanbir Singh <mail-lists@karan.org <mailto:mail-lists@karan.org>> wrote: > > > > > On 22/08/16 18:30, Jason L Tibbitts III wrote: > > > >>>>>> "DJ" == Dave Johansen <davejohansen@gmail.com <mailto:davejohansen@gmail.com>> writes: > > > > > > > > DJ> devtoolset is designed to do all of this and is already > > > > DJ> done, so it seems that the only advantage to putting it in > > > > DJ> EPEL itself would be to reduce the number of repos during > > > > DJ> build time. > > > > > > > > So is devtoolset something I get access to as a CentOS user? > > > > How do I build these packages myself (i.e. in mock)? > > > > > > you should be able to 'yum install centos-release-scl' on a CentOS > > > Linux machine and get access to all the SCLs > > > > Yeah, but if we enable that for EPEL builds, we are going to get all > > SCLs right? So, people could start depending on them at runtime > > instead of just install time. > > > > I'm not opposed to devtoolset, but I don't think we want to allow > > runtime scls without actual scl guidelines. > > > > I seem to remember that Fedora didn't allow SCLs because there was > some compatibility problem or something of the sort. Do you know the > details or what the current state is? It's not a compatibility problem. It's lack of guidelines. > Also, RedHat has been pushing devtoolset pretty hard. The response to > a few bugzillas has even been "use devtoolset because the issue is > fixed there and we're not going to fix the system gcc/libc/etc". So > it seems like allowing SCLs in Fedora/EPEL makes sense and fits with > the direction of RHEL in general. As I noted, I'd probibly be for devtoolset because the guidelines would be pretty simple. Just do X Y and Z in your spec, build as normal and users wouldn't see any run time dep on it. It gets weird where other SCL's come in. Can your package require say postgresql from SCL instead of the rhel one? If so, would our users all be able to install that? What happens if two packages need different SCL versions? Can SCL's depend on each other? Can you make a package thats an SCL? we have no guidelines for that, and just saying "do whatever you want" is likely to cause mass confusion and make everyone miserable.I agree that how to handle SCLs can get really mess really fast, but a lot of projects are jumping on the "modern C++" bandwagon and allows devtoolset is low risk, easy to do and enables a lot of packages to be built with EPEL that otherwise wouldn't be.
Basically, I think that figuring out how to handle SCLs is a long term issue that will take some serious work, but coming up with some simple policies that allow it to be used in EPEL is something that should be well within the realm of the possible.
epel-devel mailing list epel-devel@lists.fedoraproject.org https://lists.fedoraproject.org/admin/lists/epel-devel@lists.fedoraproject.o...
On 25 August 2016 at 02:14, dani dani@letai.org.il wrote:
When I proposed importing gcc-5 to EPEL6 back in 04/2016 ( https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject... ) the response was an unequivocal no, EPEL does not install to /opt/ , so it dies right there.
Now you are proposing the same ( devtoolset/scl installs to /opt except for the wrapper call) but using a scheme that is somewhat less convenient (In scl the binutils and gcc have to be coupled, and as noted the imported gcc suite is incomplete), much less frequent (the most updated version is gcc-5.2, while I maintain both gcc-5.x and gcc-6.1), and much less complete (I import everything but gcc-gnat, while devtoolset4 only has gcc,gcc-c++ and gcc-gfortran. No gcc-objc, no gcc-go, no cpp, and none of the libs (cilk, gccjit, atomic, asan etc...).
I'm still building and maintaining both gcc and bintutils for my own purposes, which are based off of fc24 srpms, and with optionally gcc-c++ specs file hardcoded to use binutils tools at the new prefix so use of env-modules is not required.
I'm just wandering why the different treatment - the automatic knee-jerk reaction of dismissal to a newbie proposal vs. accepting the exact same proposal (although wrapped so it's less convenient to use) when it comes from someone else.
You are misreading both responses. There is no knee-jerk acceptance and there wasn't a knee jerk dismissal because you were a newbie. Please don't find malice where none was intended.
On Thu, 25 Aug 2016 12:52:46 -0400 Stephen John Smoogen smooge@gmail.com wrote:
On 25 August 2016 at 02:14, dani dani@letai.org.il wrote:
When I proposed importing gcc-5 to EPEL6 back in 04/2016 ( https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject... ) the response was an unequivocal no, EPEL does not install to /opt/ , so it dies right there.
Now you are proposing the same ( devtoolset/scl installs to /opt except for the wrapper call) but using a scheme that is somewhat less convenient (In scl the binutils and gcc have to be coupled, and as noted the imported gcc suite is incomplete), much less frequent (the most updated version is gcc-5.2, while I maintain both gcc-5.x and gcc-6.1), and much less complete (I import everything but gcc-gnat, while devtoolset4 only has gcc,gcc-c++ and gcc-gfortran. No gcc-objc, no gcc-go, no cpp, and none of the libs (cilk, gccjit, atomic, asan etc...).
I'm still building and maintaining both gcc and bintutils for my own purposes, which are based off of fc24 srpms, and with optionally gcc-c++ specs file hardcoded to use binutils tools at the new prefix so use of env-modules is not required.
I'm just wandering why the different treatment - the automatic knee-jerk reaction of dismissal to a newbie proposal vs. accepting the exact same proposal (although wrapped so it's less convenient to use) when it comes from someone else.
You are misreading both responses. There is no knee-jerk acceptance and there wasn't a knee jerk dismissal because you were a newbie. Please don't find malice where none was intended.
What smooge said. ;)
The reason SCL's are under opt is that they got a namespace approved for that purpose:
https://fedoraproject.org/wiki/Packaging:Guidelines#Limited_usage_of_.2Fopt....
"Currently, we have allocated /opt/fedora/scls, /etc/opt/fedora/scls, and /var/opt/fedora/scls for use by Software Collections. "
Perhaps you could explain exactly what you want to propose here again? Just epel6? or 7 as well? Do you have co-maintainers in case you get busy, etc?
I think we are all open to ideas how to do things better, but it's really not clear what is best or even exactly what is proposed. ;)
kevin
On Thu, Aug 25, 2016 at 2:34 PM, Kevin Fenzi kevin@scrye.com wrote:
On Wed, 24 Aug 2016 21:59:55 -0600 Dave Johansen davejohansen@gmail.com wrote:
I agree that how to handle SCLs can get really mess really fast, but a lot of projects are jumping on the "modern C++" bandwagon and allows devtoolset is low risk, easy to do and enables a lot of packages to be built with EPEL that otherwise wouldn't be.
It would be low risk if that was the only SCL in the repo. My understanding is that they are all there in the one repo, so if we enable that, everyone could start using anything thats in there.
They're each in their own repo and I would propose adding devtoolset only in repos used by mock during build time.
On Thu, Aug 25, 2016 at 2:40 PM, Kevin Fenzi kevin@scrye.com wrote:
On Thu, 25 Aug 2016 12:52:46 -0400 Stephen John Smoogen smooge@gmail.com wrote:
On 25 August 2016 at 02:14, dani dani@letai.org.il wrote:
When I proposed importing gcc-5 to EPEL6 back in 04/2016 ( https://lists.fedoraproject.org/archives/list/epel-devel@
lists.fedoraproject.org/message/F5JXEYPKQY77NRBCL4MNUBS3K2YYBBTU/
) the response was an unequivocal no, EPEL does not install to /opt/ , so it dies right there.
Now you are proposing the same ( devtoolset/scl installs to /opt except for the wrapper call) but using a scheme that is somewhat less convenient (In scl the binutils and gcc have to be coupled, and as noted the imported gcc suite is incomplete), much less frequent (the most updated version is gcc-5.2, while I maintain both gcc-5.x and gcc-6.1), and much less complete (I import everything but gcc-gnat, while devtoolset4 only has gcc,gcc-c++ and gcc-gfortran. No gcc-objc, no gcc-go, no cpp, and none of the libs (cilk, gccjit, atomic, asan etc...).
I'm still building and maintaining both gcc and bintutils for my own purposes, which are based off of fc24 srpms, and with optionally gcc-c++ specs file hardcoded to use binutils tools at the new prefix so use of env-modules is not required.
I'm just wandering why the different treatment - the automatic knee-jerk reaction of dismissal to a newbie proposal vs. accepting the exact same proposal (although wrapped so it's less convenient to use) when it comes from someone else.
You are misreading both responses. There is no knee-jerk acceptance and there wasn't a knee jerk dismissal because you were a newbie. Please don't find malice where none was intended.
What smooge said. ;)
The reason SCL's are under opt is that they got a namespace approved for that purpose:
https://fedoraproject.org/wiki/Packaging:Guidelines# Limited_usage_of_.2Fopt.2C_.2Fetc.2Fopt.2C_and_.2Fvar.2Fopt
"Currently, we have allocated /opt/fedora/scls, /etc/opt/fedora/scls, and /var/opt/fedora/scls for use by Software Collections. "
Perhaps you could explain exactly what you want to propose here again? Just epel6? or 7 as well? Do you have co-maintainers in case you get busy, etc?
I think we are all open to ideas how to do things better, but it's really not clear what is best or even exactly what is proposed. ;)
Here's one proposal: 1) Add version X of devtoolset only in repos used by mock 2) N months (maybe 6?) before version X is EOLed, make version X+1 (or whatever the latest is) available in a buildroot override (or something similar) for testing 3) Rebuild all packages that use devtoolset using version X+1 and make available for testing 4) After testing period (maybe 1 month? or 3 months?), upgrade to version X+1 and move all rebuilt packages from testing repos to main repos
Or here's another option: 1) Add all non-EOLed versions of devtoolset only in repos used by mock 2) N months (at least 6) before version X is EOLed require that it be rebuilt with a newer version of devtoolset 3) If it's not rebuilt before the EOL happens, then retire the package
I like the second option better, because it's easier to setup/maintain from a mock/koji perspective, provides flexibility and doesn't force a mass rebuild/test every 12-18 months when a devtoolset is retired (their life cycle is 2 years). However, it also means that the update is decentralized and depends on maintainers rather an an automated system.
On Thu, 25 Aug 2016 15:04:58 -0600 Dave Johansen davejohansen@gmail.com wrote:
On Thu, Aug 25, 2016 at 2:34 PM, Kevin Fenzi kevin@scrye.com wrote:
On Wed, 24 Aug 2016 21:59:55 -0600 Dave Johansen davejohansen@gmail.com wrote:
I agree that how to handle SCLs can get really mess really fast, but a lot of projects are jumping on the "modern C++" bandwagon and allows devtoolset is low risk, easy to do and enables a lot of packages to be built with EPEL that otherwise wouldn't be.
It would be low risk if that was the only SCL in the repo. My understanding is that they are all there in the one repo, so if we enable that, everyone could start using anything thats in there.
They're each in their own repo and I would propose adding devtoolset only in repos used by mock during build time.
Oh? thats good... I was understanding that they were all in a scl repo/channel.
Here's one proposal:
- Add version X of devtoolset only in repos used by mock
- N months (maybe 6?) before version X is EOLed, make version X+1 (or
whatever the latest is) available in a buildroot override (or something similar) for testing 3) Rebuild all packages that use devtoolset using version X+1 and make available for testing 4) After testing period (maybe 1 month? or 3 months?), upgrade to version X+1 and move all rebuilt packages from testing repos to main repos
Or here's another option:
- Add all non-EOLed versions of devtoolset only in repos used by mock
- N months (at least 6) before version X is EOLed require that it be
rebuilt with a newer version of devtoolset 3) If it's not rebuilt before the EOL happens, then retire the package
I like the second option better, because it's easier to setup/maintain from a mock/koji perspective, provides flexibility and doesn't force a mass rebuild/test every 12-18 months when a devtoolset is retired (their life cycle is 2 years). However, it also means that the update is decentralized and depends on maintainers rather an an automated system.
Right. I would prefer the second one too, but we would still need some automated detection that the packages no longer build.
kevin
On 08/25/2016 11:40 PM, Kevin Fenzi wrote:
Perhaps you could explain exactly what you want to propose here again? Just epel6? or 7 as well? Do you have co-maintainers in case you get busy, etc?
I propose adding several gnu packages (namely gcc, binutils and gdb) with versions following those supplied by fedora, specifically for epel6, but possibly for epel7 if requested.
This could hold a pattern such as /opt/gnu/[gcc|binutils|gdb]/<version>/ to allow several version to co-exist. I don't have any co-maintainers, but I mainly get busy in my day job, which happens to be the reason I maintain those packages.
I propose importing fc24 packages as and when they become available, maintaining the exact same packages - so gcc6.1 srpm would create the entire suite* of rpms currently available on fc24.
My current scheme is relying on environment modules to "switch" between versions - each installation of course creates and installs the modules file which also maintains dependencies - binutils >= 2.24 must be loaded prior to loading gcc6 etc.
(*) I have no experience building and packaging gcc-gnat, so not quite the entire suite, but I could probably learn to build that too.
I think we are all open to ideas how to do things better, but it's really not clear what is best or even exactly what is proposed. ;)
kevin
epel-devel mailing list epel-devel@lists.fedoraproject.org https://lists.fedoraproject.org/admin/lists/epel-devel@lists.fedoraproject.o...
On 26 August 2016 at 06:00, Daniel Letai dani@letai.org.il wrote:
On 08/25/2016 11:40 PM, Kevin Fenzi wrote:
Perhaps you could explain exactly what you want to propose here again? Just epel6? or 7 as well? Do you have co-maintainers in case you get busy, etc?
I propose adding several gnu packages (namely gcc, binutils and gdb) with versions following those supplied by fedora, specifically for epel6, but possibly for epel7 if requested.
This could hold a pattern such as /opt/gnu/[gcc|binutils|gdb]/<version>/ to allow several version to co-exist. I don't have any co-maintainers, but I mainly get busy in my day job, which happens to be the reason I maintain those packages.
OK there were multiple reasons there were reservations for this:
1) /opt/gnu (and many other /opt/*) names are already in use by many site admistrators. Putting our packages in there and over-writing locally compiled apps is going to cause problems. [Remember rpm will overwrite /opt/gnu/gcc/5.0/bin/gcc if it wasn't in the rpm db before hand without any report of a conflict.]
2) What you are proposing is a completely new way of packaging software from how Fedora or EPEL has done it before. That means it needs a fuller proposal and reasoning than a single email. The SCL process took years of work with many iterations because various people found many valid problems that needed addressing. [There were also many useless nitpicks but that seems to be part and process of open source]
Neither of the above is a "no". It is a "you need to do a lot of groundwork before it is going to happen in any form."
On 26 August 2016 at 12:58, Stephen John Smoogen smooge@gmail.com wrote:
On 26 August 2016 at 06:00, Daniel Letai dani@letai.org.il wrote:
On 08/25/2016 11:40 PM, Kevin Fenzi wrote:
Perhaps you could explain exactly what you want to propose here again? Just epel6? or 7 as well? Do you have co-maintainers in case you get busy, etc?
I propose adding several gnu packages (namely gcc, binutils and gdb) with versions following those supplied by fedora, specifically for epel6, but possibly for epel7 if requested.
This could hold a pattern such as /opt/gnu/[gcc|binutils|gdb]/<version>/ to allow several version to co-exist. I don't have any co-maintainers, but I mainly get busy in my day job, which happens to be the reason I maintain those packages.
OK there were multiple reasons there were reservations for this:
- /opt/gnu (and many other /opt/*) names are already in use by many
site admistrators. Putting our packages in there and over-writing locally compiled apps is going to cause problems. [Remember rpm will overwrite /opt/gnu/gcc/5.0/bin/gcc if it wasn't in the rpm db before hand without any report of a conflict.]
In reading some of the FESCO tickets, we can't use /opt/gnu because we are not the GNU organization. https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s13.html
We would need to use the /opt/fedora or go through the process of becoming an entity that the LANANA.org people would recognize.
On Fri, Aug 26, 2016 at 4:24 PM, dani dani@letai.org.il wrote:
On 26/08//2016 21:18, Stephen John Smoogen wrote:
On 26 August 2016 at 12:58, Stephen John Smoogen smooge@gmail.com smooge@gmail.com wrote:
On 26 August 2016 at 06:00, Daniel Letai dani@letai.org.il dani@letai.org.il wrote:
On 08/25/2016 11:40 PM, Kevin Fenzi wrote:
Perhaps you could explain exactly what you want to propose here again? Just epel6? or 7 as well? Do you have co-maintainers in case you get busy, etc?
I propose adding several gnu packages (namely gcc, binutils and gdb) with versions following those supplied by fedora, specifically for epel6, but possibly for epel7 if requested.
This could hold a pattern such as /opt/gnu/[gcc|binutils|gdb]/<version>/ to allow several version to co-exist. I don't have any co-maintainers, but I mainly get busy in my day job, which happens to be the reason I maintain those packages.
OK there were multiple reasons there were reservations for this:
- /opt/gnu (and many other /opt/*) names are already in use by many
site admistrators. Putting our packages in there and over-writing locally compiled apps is going to cause problems. [Remember rpm will overwrite /opt/gnu/gcc/5.0/bin/gcc if it wasn't in the rpm db before hand without any report of a conflict.]
In reading some of the FESCO tickets, we can't use /opt/gnu because we are not the GNU organization.https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s13.html
We would need to use the /opt/fedora or go through the process of becoming an entity that the LANANA.org people would recognize.
I think /opt/fedora would be fine, but it would require an additional level to allow for better flexibility, something like /opt/fedora/epel[6?]/gcc/5.3/... If possible, /opt/epel/gcc/... is more intuitive, but might require registration ( I'm not familiar with any other "epel" out there that might contest the use of "epel" as a new provider). This would also allow for differing python version (/opt/epel/python/[2.7,3.0,3.4...]) or any other multi-version package some might wish to maintain.
I realize this is not the fedora way, to maintain multiple version of the same package, and over the years this had led to some inconsistencies in naming - python might be the most known example, but other packages exists which had to do with all sort of *-compat or name<numer> kludges.
I think it is high time to rethink the single version of a package policy, and come up with some scheme that would allow for any package to maintain multiple versions in a consistent manner. gcc is just a single example where such a need exists. Perl, python and any other tool that breaks api between versions is of course a candidate.
SCL, while apparently solving this issue, seem to break the modular approach to software delivery that is rpm - you have to use fixed versions provided by an scl suite for the entire tooling, rather than upgrading or using tools from different versions as long as they are compatible.
I believe that's just a design decision to make it easier to work with a compiler toolchain. devtoolset could probably be broken up into a few smaller chunks (compiler, IDE, debugging tools, etc), but I don't know if there's any significant benefit to completely separating each component so you can mix different versions (assuming that something like that is even possible).
On Fri, Aug 26, 2016 at 6:24 PM, dani dani@letai.org.il wrote:
I think it is high time to rethink the single version of a package policy, and come up with some scheme that would allow for any package to maintain multiple versions in a consistent manner.
We wouldn't even be the first distro to wind up discarding that policy. Both openSUSE and Mandriva did the same many years ago. That led to the restructuring of the packaging to the current form openSUSE and Mageia have[1][2].
[1]: https://en.opensuse.org/openSUSE:Packaging_Multiple_Version_guidelines [2]: https://wiki.mageia.org/en/Libraries_policy
"d" == dani dani@letai.org.il writes:
d> I think it is high time to rethink the single version of a package d> policy, and come up with some scheme that would allow for any package d> to maintain multiple versions in a consistent manner.
We don't have such a policy. At least Fedora doesn't. In fact, adding another version of an existing package doesn't require a pass through the review process. The packages just need to not conflict and be named appropriately. Dealing with the names of binaries is left up to the packager.
- J<
On 29/08//2016 22:23, Jason L Tibbitts III wrote:
"d" == dani dani@letai.org.il writes:
d> I think it is high time to rethink the single version of a package d> policy, and come up with some scheme that would allow for any package d> to maintain multiple versions in a consistent manner.
We don't have such a policy. At least Fedora doesn't. In fact, adding another version of an existing package doesn't require a pass through the review process. The packages just need to not conflict and be named appropriately. Dealing with the names of binaries is left up to the packager.
- J<
There is no policy for multiversioning, which results in an inconsistent packaging scheme for each multi-version provided, with some relying on alternatives, others add new sub-hierarchy to /usr and almost none consider allowing for other versions other than their own.
Alternatives forces system-wide defaults, with the different versions not easily accessible, or used as a group. SCL solves that issue, but must be applied on a full suite basis only, and only at the version points provided by any scl repo.
As mentioned, it doesn't exists for other architectures, the way native fedora rpms do.
As far as i could tell, the main argument against my scheme was usage of /opt, yet it is approved for scl, despite scl's limitations.
I'm only asking for a clear policy regarding multiversion packages, which would define clear guidelines on any submission requests.
On 31 August 2016 at 04:58, dani dani@letai.org.il wrote:
On 29/08//2016 22:23, Jason L Tibbitts III wrote:
> > "d" == dani dani@letai.org.il writes:
d> I think it is high time to rethink the single version of a package d> policy, and come up with some scheme that would allow for any package d> to maintain multiple versions in a consistent manner.
We don't have such a policy. At least Fedora doesn't. In fact, adding another version of an existing package doesn't require a pass through the review process. The packages just need to not conflict and be named appropriately. Dealing with the names of binaries is left up to the packager.
- J<
There is no policy for multiversioning, which results in an inconsistent packaging scheme for each multi-version provided, with some relying on alternatives, others add new sub-hierarchy to /usr and almost none consider allowing for other versions other than their own.
Alternatives forces system-wide defaults, with the different versions not easily accessible, or used as a group. SCL solves that issue, but must be applied on a full suite basis only, and only at the version points provided by any scl repo.
As mentioned, it doesn't exists for other architectures, the way native fedora rpms do.
As far as i could tell, the main argument against my scheme was usage of /opt, yet it is approved for scl, despite scl's limitations.
No that is not the main argument at all. There isn't an argument here. The issue is that your scheme needs to be fully written up and then reviewed and approved by the Fedora Packaging Committee and FESCO. This is what every other packaging scheme (for everything from SCL to ruby to java) has to do.
No one is going to write up the policy for you. No one is going to champion it for you. Not because we are against what you are doing but because most of us have 20 other jobs that we have to get done at any other time and our 'free' time is very limited.
I'm only asking for a clear policy regarding multiversion packages, which would define clear guidelines on any submission requests.
epel-devel mailing list epel-devel@lists.fedoraproject.org https://lists.fedoraproject.org/admin/lists/epel-devel@lists.fedoraproject.o...
On Wednesday, 31 August 2016 at 10:58, dani wrote: [...]
As far as i could tell, the main argument against my scheme was usage of /opt, yet it is approved for scl, despite scl's limitations.
SCL was never approved for Fedora. It's something that Red Hat provides in their products only.
I'm only asking for a clear policy regarding multiversion packages, which would define clear guidelines on any submission requests.
RPM already supports installation of multiple versions of a package as long as they don't conflict.
Regards, Dominik
On 24 August 2016 at 23:59, Dave Johansen davejohansen@gmail.com wrote:
On Wed, Aug 24, 2016 at 5:28 PM, Kevin Fenzi kevin@scrye.com wrote:
On Wed, 24 Aug 2016 16:43:31 -0600 Dave Johansen davejohansen@gmail.com wrote:
On Wed, Aug 24, 2016 at 4:22 PM, Kevin Fenzi kevin@scrye.com wrote:
On Tue, 23 Aug 2016 14:21:24 +0100 Karanbir Singh mail-lists@karan.org wrote:
On 22/08/16 18:30, Jason L Tibbitts III wrote:
>>>>> "DJ" == Dave Johansen davejohansen@gmail.com writes:
DJ> devtoolset is designed to do all of this and is already DJ> done, so it seems that the only advantage to putting it in DJ> EPEL itself would be to reduce the number of repos during DJ> build time.
So is devtoolset something I get access to as a CentOS user? How do I build these packages myself (i.e. in mock)?
you should be able to 'yum install centos-release-scl' on a CentOS Linux machine and get access to all the SCLs
Yeah, but if we enable that for EPEL builds, we are going to get all SCLs right? So, people could start depending on them at runtime instead of just install time.
I'm not opposed to devtoolset, but I don't think we want to allow runtime scls without actual scl guidelines.
I seem to remember that Fedora didn't allow SCLs because there was some compatibility problem or something of the sort. Do you know the details or what the current state is?
It's not a compatibility problem. It's lack of guidelines.
Also, RedHat has been pushing devtoolset pretty hard. The response to a few bugzillas has even been "use devtoolset because the issue is fixed there and we're not going to fix the system gcc/libc/etc". So it seems like allowing SCLs in Fedora/EPEL makes sense and fits with the direction of RHEL in general.
As I noted, I'd probibly be for devtoolset because the guidelines would be pretty simple. Just do X Y and Z in your spec, build as normal and users wouldn't see any run time dep on it.
It gets weird where other SCL's come in. Can your package require say postgresql from SCL instead of the rhel one? If so, would our users all be able to install that? What happens if two packages need different SCL versions? Can SCL's depend on each other? Can you make a package thats an SCL? we have no guidelines for that, and just saying "do whatever you want" is likely to cause mass confusion and make everyone miserable.
I agree that how to handle SCLs can get really mess really fast, but a lot of projects are jumping on the "modern C++" bandwagon and allows devtoolset is low risk, easy to do and enables a lot of packages to be built with EPEL that otherwise wouldn't be.
Basically, I think that figuring out how to handle SCLs is a long term issue that will take some serious work, but coming up with some simple policies that allow it to be used in EPEL is something that should be well within the realm of the possible.
History and experience has taught us that every time we do that it comes back and bites us big time. Mainly because the simple policies start getting revised and rewritten or violated as soon as the second package gets put in... and in fixing that you break the first one.. and the people who used it. This is ok in Fedora but in EPEL you end up spending a lot of time fixing people who aren't expecting breakage.
On 25 August 2016 at 12:49, Stephen John Smoogen smooge@gmail.com wrote:
On 24 August 2016 at 23:59, Dave Johansen davejohansen@gmail.com wrote:
On Wed, Aug 24, 2016 at 5:28 PM, Kevin Fenzi kevin@scrye.com wrote:
I agree that how to handle SCLs can get really mess really fast, but a lot of projects are jumping on the "modern C++" bandwagon and allows devtoolset is low risk, easy to do and enables a lot of packages to be built with EPEL that otherwise wouldn't be.
Basically, I think that figuring out how to handle SCLs is a long term issue that will take some serious work, but coming up with some simple policies that allow it to be used in EPEL is something that should be well within the realm of the possible.
History and experience has taught us that every time we do that it comes back and bites us big time. Mainly because the simple policies start getting revised and rewritten or violated as soon as the second package gets put in... and in fixing that you break the first one.. and the people who used it. This is ok in Fedora but in EPEL you end up spending a lot of time fixing people who aren't expecting breakage.
I hit send too soon as what is the exact policy proposal you are wanting.
-- Stephen J Smoogen.
On Wed, 24 Aug 2016 21:59:55 -0600 Dave Johansen davejohansen@gmail.com wrote:
I agree that how to handle SCLs can get really mess really fast, but a lot of projects are jumping on the "modern C++" bandwagon and allows devtoolset is low risk, easy to do and enables a lot of packages to be built with EPEL that otherwise wouldn't be.
It would be low risk if that was the only SCL in the repo. My understanding is that they are all there in the one repo, so if we enable that, everyone could start using anything thats in there.
Basically, I think that figuring out how to handle SCLs is a long term issue that will take some serious work, but coming up with some simple policies that allow it to be used in EPEL is something that should be well within the realm of the possible.
Perhaps. :)
kevin
On 16/08/16 06:24, Tom Callaway wrote:
I'd like to propose that we enable the Developer Toolset repo in EPEL and allow packages to depend on it. Thoughts?
There is one issue with this that I don't think others have addressed yet on this list. EPEL 6 supports i686 as a build target, but devtoolset for el6 is available for x86_64 only, so it would cause problems with i686 builds.
Peter
epel-devel@lists.fedoraproject.org
-
dani -
Dave Johansen -
Dominik 'Rathann' Mierzejewski -
Jason L Tibbitts III -
Karanbir Singh -
Kevin Fenzi -
Neal Gompa -
Orion Poplawski -
Peter -
Stephen John Smoogen -
Tom Callaway