Hi All,
So it's been discussed a little about doing a EPEL bootstrap for new architectures. We have a number of arches wanting to do this (ppc64le, aarch64, i686, s390x). So ppc64le will be our first and this is an overview of the process I'll be using to do it. Once it's complete I'll do a better write up as I suspect some things will evolve as we go on down the path.
So the general overview of the process is:
1) add new builders to koji (ppc64le complete) 2) create separate inherited build target and tag (epel7-archbootstrap) with associated architecture 3) run scratch builds in that target using the git commit hashes from the latest builds in the epel7, epel7-testing and epel7-candidate tags 4) For each scratch build completed, run mergeScratch(task_id) 5) when all builds are complete enable the arch on the main epel7 target 6) sign all new packages 7) update bodhi, pkgdb, mash configs 7) mirror out
I have some scripts to do the above which I'll publish for reference once I've cleaned them up a little.
This process isn't perfect. The new arch builds may not have the exact same build dependency NVRs because, at least in the case of ppc64le, the first EL7 release with .el7 distags is 7.2 and obviously most of epel7 is built against < 7.2. The mergeScratch koji API call imports all the build logs which helps mitigate this from a debug PoV. There's not really a good way to deal with this, and obviously once the new arch is enabled they'll be the same moving forward. I don't see it as an issue really, just making note of it here for reference.
Looking at the current stable epel7 builds, as of a couple of days ago, we have around 4763 source packages, of which 2686 are noarch (and hence don't need to be rebuilt) and a touch under 2077 (2077 at time of check) are arch dependent.
Let me know of any queries, questions, concerns etc
Peter
On Wed, 25 Nov 2015 15:25:21 +0000 Peter Robinson pbrobinson@gmail.com wrote:
Hi All,
So it's been discussed a little about doing a EPEL bootstrap for new architectures. We have a number of arches wanting to do this (ppc64le, aarch64, i686, s390x). So ppc64le will be our first and this is an overview of the process I'll be using to do it. Once it's complete I'll do a better write up as I suspect some things will evolve as we go on down the path.
So the general overview of the process is:
- add new builders to koji (ppc64le complete)
- create separate inherited build target and tag
(epel7-archbootstrap) with associated architecture 3) run scratch builds in that target using the git commit hashes from the latest builds in the epel7, epel7-testing and epel7-candidate tags 4) For each scratch build completed, run mergeScratch(task_id) 5) when all builds are complete enable the arch on the main epel7 target 6) sign all new packages 7) update bodhi, pkgdb, mash configs 7) mirror out
I have some scripts to do the above which I'll publish for reference once I've cleaned them up a little.
This process isn't perfect. The new arch builds may not have the exact same build dependency NVRs because, at least in the case of ppc64le, the first EL7 release with .el7 distags is 7.2 and obviously most of epel7 is built against < 7.2. The mergeScratch koji API call imports all the build logs which helps mitigate this from a debug PoV. There's not really a good way to deal with this, and obviously once the new arch is enabled they'll be the same moving forward. I don't see it as an issue really, just making note of it here for reference.
Looking at the current stable epel7 builds, as of a couple of days ago, we have around 4763 source packages, of which 2686 are noarch (and hence don't need to be rebuilt) and a touch under 2077 (2077 at time of check) are arch dependent.
Let me know of any queries, questions, concerns etc
sounds sane :-) But how will be handled packages that require some boostrapping procedure - is a new combo of boostrap and final build required in existing EPEL (dist-git) or or will be the bootstrap build taken from the original EPEL bootstrap (built eg. against RHEL-7 Beta) and the current latest NVR will be then the final build?
Dan
On 11/25/2015 04:48 PM, Dan Horák wrote:
On Wed, 25 Nov 2015 15:25:21 +0000 Peter Robinson pbrobinson@gmail.com wrote:
Hi All,
So it's been discussed a little about doing a EPEL bootstrap for new architectures. We have a number of arches wanting to do this (ppc64le, aarch64, i686, s390x). So ppc64le will be our first and this is an overview of the process I'll be using to do it. Once it's complete I'll do a better write up as I suspect some things will evolve as we go on down the path.
So the general overview of the process is:
- add new builders to koji (ppc64le complete)
- create separate inherited build target and tag
(epel7-archbootstrap) with associated architecture 3) run scratch builds in that target using the git commit hashes from the latest builds in the epel7, epel7-testing and epel7-candidate tags 4) For each scratch build completed, run mergeScratch(task_id) 5) when all builds are complete enable the arch on the main epel7 target 6) sign all new packages 7) update bodhi, pkgdb, mash configs 7) mirror out
I have some scripts to do the above which I'll publish for reference once I've cleaned them up a little.
This process isn't perfect. The new arch builds may not have the exact same build dependency NVRs because, at least in the case of ppc64le, the first EL7 release with .el7 distags is 7.2 and obviously most of epel7 is built against < 7.2. The mergeScratch koji API call imports all the build logs which helps mitigate this from a debug PoV. There's not really a good way to deal with this, and obviously once the new arch is enabled they'll be the same moving forward. I don't see it as an issue really, just making note of it here for reference.
Looking at the current stable epel7 builds, as of a couple of days ago, we have around 4763 source packages, of which 2686 are noarch (and hence don't need to be rebuilt) and a touch under 2077 (2077 at time of check) are arch dependent.
Let me know of any queries, questions, concerns etc
sounds sane :-) But how will be handled packages that require some boostrapping procedure - is a new combo of boostrap and final build required in existing EPEL (dist-git) or or will be the bootstrap build taken from the original EPEL bootstrap (built eg. against RHEL-7 Beta) and the current latest NVR will be then the final build?
Dan
epel-devel mailing list epel-devel@lists.fedoraproject.org http://lists.fedoraproject.org/admin/lists/epel-devel@lists.fedoraproject.or...
Note that I have packages already built in my local ppc64le epel7 environment and some could be used to help in bootstrap.
So it's been discussed a little about doing a EPEL bootstrap for new architectures. We have a number of arches wanting to do this (ppc64le, aarch64, i686, s390x). So ppc64le will be our first and this is an overview of the process I'll be using to do it. Once it's complete I'll do a better write up as I suspect some things will evolve as we go on down the path.
So the general overview of the process is:
- add new builders to koji (ppc64le complete)
- create separate inherited build target and tag
(epel7-archbootstrap) with associated architecture 3) run scratch builds in that target using the git commit hashes from the latest builds in the epel7, epel7-testing and epel7-candidate tags 4) For each scratch build completed, run mergeScratch(task_id) 5) when all builds are complete enable the arch on the main epel7 target 6) sign all new packages 7) update bodhi, pkgdb, mash configs 7) mirror out
I have some scripts to do the above which I'll publish for reference once I've cleaned them up a little.
This process isn't perfect. The new arch builds may not have the exact same build dependency NVRs because, at least in the case of ppc64le, the first EL7 release with .el7 distags is 7.2 and obviously most of epel7 is built against < 7.2. The mergeScratch koji API call imports all the build logs which helps mitigate this from a debug PoV. There's not really a good way to deal with this, and obviously once the new arch is enabled they'll be the same moving forward. I don't see it as an issue really, just making note of it here for reference.
Looking at the current stable epel7 builds, as of a couple of days ago, we have around 4763 source packages, of which 2686 are noarch (and hence don't need to be rebuilt) and a touch under 2077 (2077 at time of check) are arch dependent.
Let me know of any queries, questions, concerns etc
sounds sane :-) But how will be handled packages that require some boostrapping procedure - is a new combo of boostrap and final build required in existing EPEL (dist-git) or or will be the bootstrap build taken from the original EPEL bootstrap (built eg. against RHEL-7 Beta) and the current latest NVR will be then the final build?
Note that I have packages already built in my local ppc64le epel7 environment and some could be used to help in bootstrap.
Sorry, that's not going to happen, but if you've got a list of package sets that need specific bootstrapping that would be useful.
Peter
----- Original Message -----
From: "Peter Robinson" pbrobinson@gmail.com To: "EPEL Development List" epel-devel@lists.fedoraproject.org Sent: Wednesday, November 25, 2015 5:33:17 PM Subject: [EPEL-devel]Re: Boostrapping a new architecture for EPEL
So it's been discussed a little about doing a EPEL bootstrap for new architectures. We have a number of arches wanting to do this (ppc64le, aarch64, i686, s390x). So ppc64le will be our first and this is an overview of the process I'll be using to do it. Once it's complete I'll do a better write up as I suspect some things will evolve as we go on down the path.
So the general overview of the process is:
- add new builders to koji (ppc64le complete)
- create separate inherited build target and tag
(epel7-archbootstrap) with associated architecture 3) run scratch builds in that target using the git commit hashes from the latest builds in the epel7, epel7-testing and epel7-candidate tags 4) For each scratch build completed, run mergeScratch(task_id) 5) when all builds are complete enable the arch on the main epel7 target 6) sign all new packages 7) update bodhi, pkgdb, mash configs 7) mirror out
I have some scripts to do the above which I'll publish for reference once I've cleaned them up a little.
This process isn't perfect. The new arch builds may not have the exact same build dependency NVRs because, at least in the case of ppc64le, the first EL7 release with .el7 distags is 7.2 and obviously most of epel7 is built against < 7.2. The mergeScratch koji API call imports all the build logs which helps mitigate this from a debug PoV. There's not really a good way to deal with this, and obviously once the new arch is enabled they'll be the same moving forward. I don't see it as an issue really, just making note of it here for reference.
Looking at the current stable epel7 builds, as of a couple of days ago, we have around 4763 source packages, of which 2686 are noarch (and hence don't need to be rebuilt) and a touch under 2077 (2077 at time of check) are arch dependent.
Let me know of any queries, questions, concerns etc
sounds sane :-) But how will be handled packages that require some boostrapping procedure - is a new combo of boostrap and final build required in existing EPEL (dist-git) or or will be the bootstrap build taken from the original EPEL bootstrap (built eg. against RHEL-7 Beta) and the current latest NVR will be then the final build?
Note that I have packages already built in my local ppc64le epel7 environment and some could be used to help in bootstrap.
Sorry, that's not going to happen, but if you've got a list of package sets that need specific bootstrapping that would be useful.
I'm wondering if [0] could be of any use for you. It's not quite ready for a release, but it should hopefully make rebuilds (including bootstrapping) a lot simpler (eventually, anyway :) ).
You tell the tool the names of packages you want to rebuild, where it should get those packages from and where it should build them, and it builds them it the right order, using what we call a "recipe" [1] to break a dep cycle whenever it finds one.
If you think it could be helpful, I'll let the author know he should improve the README. :)
Matt
[0] https://github.com/mcyprian/sclbuilder (please disregard the name, it's not SCL specific :) ) [1] https://github.com/mcyprian/sclbuilder/blob/master/input_data/recipe1_py35.y...
So it's been discussed a little about doing a EPEL bootstrap for new architectures. We have a number of arches wanting to do this (ppc64le, aarch64, i686, s390x). So ppc64le will be our first and this is an overview of the process I'll be using to do it. Once it's complete I'll do a better write up as I suspect some things will evolve as we go on down the path.
So the general overview of the process is:
- add new builders to koji (ppc64le complete)
- create separate inherited build target and tag
(epel7-archbootstrap) with associated architecture 3) run scratch builds in that target using the git commit hashes from the latest builds in the epel7, epel7-testing and epel7-candidate tags 4) For each scratch build completed, run mergeScratch(task_id) 5) when all builds are complete enable the arch on the main epel7 target 6) sign all new packages 7) update bodhi, pkgdb, mash configs 7) mirror out
I have some scripts to do the above which I'll publish for reference once I've cleaned them up a little.
This process isn't perfect. The new arch builds may not have the exact same build dependency NVRs because, at least in the case of ppc64le, the first EL7 release with .el7 distags is 7.2 and obviously most of epel7 is built against < 7.2. The mergeScratch koji API call imports all the build logs which helps mitigate this from a debug PoV. There's not really a good way to deal with this, and obviously once the new arch is enabled they'll be the same moving forward. I don't see it as an issue really, just making note of it here for reference.
Looking at the current stable epel7 builds, as of a couple of days ago, we have around 4763 source packages, of which 2686 are noarch (and hence don't need to be rebuilt) and a touch under 2077 (2077 at time of check) are arch dependent.
Let me know of any queries, questions, concerns etc
sounds sane :-) But how will be handled packages that require some boostrapping procedure - is a new combo of boostrap and final build required in existing EPEL (dist-git) or or will be the bootstrap build taken from the original EPEL bootstrap (built eg. against RHEL-7 Beta) and the current latest NVR will be then the final build?
Good question. Shortly after I hit send on the email I realised I missed that and one other point.
On packages that need a bootstrap they'll be handled on a case by case manner as I suspect in each case there's some specific procedure.
The other issue was packages with dependencies in EPEL itself like a toolchain stack or a desktop environment. Similar to the first we might need a couple of passes of builds here.
I'm going to do an initial dry run to see how we stand for both of the above to get a better idea where we stand.
The other issue we'll likely have, as bought up on the list earlier today is EPEL packages that are FTBFS against 7.2. We had a similar issue actually doing the initial arch bringup of ppc64le internally and it was a process of actually fixing the failures on the primary arches at the same time. Again I'll deal with this on a case by case basis.
As mentioned in my original email the above process isn't perfect and it'll be adjusted as necessary as we go, it's not something we're really done before mid rhel cycle.
Peter
epel-devel@lists.fedoraproject.org
-
Dan Horák -
Matej Stuchlik -
Menanteau -
Peter Robinson