Hi list, I'd like to start yet another discussion about some functionality of copr, here it is: One of nice use cases for copr would be building software collections [1]. These however require modified mock chroot (modified config_opts['root'] and config_opts['chroot_setup_cmd']). I'm trying to think of a way how to do this: - We would need to allow users to create their own chroots (or more precisely allow them to do certain modifications). - We would need to be able to send these chroots from frontend to backend. - We would need some mechanism for sane naming, so that we can distinguish between builds of collections for different releases and architectures. E.g. ruby193-x86_64 is not enough, we need to also store the information about os name (epel/fedora) and os version (epel 6, fedora 18, ...).
My proposal: - Use chroot names like ruby193_fedora-18-x86_64. - Allow users to create these chroots by modifying config_opts['chroot_setup_cmd'] (config_opts['root'] would be autogenerated from the chroot name). - Store the modified information in the MockChroot table and create these on backend using communication channel proposed in [2] (parts of it already implemented in frontend). - Allow selecting these on copr creation/modification as more options for mock chroots, by default hidden under some kind of fancy clickable JS dropdown.
Thoughts?
On Fri, 1 Mar 2013 03:54:48 -0500 (EST) Bohuslav Kabrda bkabrda@redhat.com wrote:
Hi list, I'd like to start yet another discussion about some functionality of copr, here it is: One of nice use cases for copr would be building software collections [1]. These however require modified mock chroot (modified config_opts['root'] and config_opts['chroot_setup_cmd']). I'm trying to think of a way how to do this:
What modifications do they need? Also - why can't this just be provided as an additional 'distro' in the form of another mock .cfg file for that distro?
Then the user would just select that in the distro/release in the copr and build?
My proposal:
- Use chroot names like ruby193_fedora-18-x86_64.
- Allow users to create these chroots by modifying
config_opts['chroot_setup_cmd'] (config_opts['root'] would be autogenerated from the chroot name).
Why overload the chroot name for this?
- Store the modified information in the MockChroot table and create
these on backend using communication channel proposed in [2] (parts of it already implemented in frontend).
- Allow selecting these on copr creation/modification as more options
for mock chroots, by default hidden under some kind of fancy clickable JS dropdown.
I need more info about what modification SCLs require to be able to judge this well.
thanks,
-sv
----- Original Message -----
On Fri, 1 Mar 2013 03:54:48 -0500 (EST) Bohuslav Kabrda bkabrda@redhat.com wrote:
Hi list, I'd like to start yet another discussion about some functionality of copr, here it is: One of nice use cases for copr would be building software collections [1]. These however require modified mock chroot (modified config_opts['root'] and config_opts['chroot_setup_cmd']). I'm trying to think of a way how to do this:
What modifications do they need? Also - why can't this just be provided as an additional 'distro' in the form of another mock .cfg file for that distro?
Then the user would just select that in the distro/release in the copr and build?
I meant something like this, only there would need to be a for frontend to let backend know about the new SCL chroots.
My proposal:
- Use chroot names like ruby193_fedora-18-x86_64.
- Allow users to create these chroots by modifying
config_opts['chroot_setup_cmd'] (config_opts['root'] would be autogenerated from the chroot name).
Why overload the chroot name for this?
- Store the modified information in the MockChroot table and create
these on backend using communication channel proposed in [2] (parts of it already implemented in frontend).
- Allow selecting these on copr creation/modification as more
options for mock chroots, by default hidden under some kind of fancy clickable JS dropdown.
I need more info about what modification SCLs require to be able to judge this well.
See above from my original mail. I'm attaching a diff of what I did with epel-6 mock chroot (and if you ignore my added local repo, it's pretty trivial).
thanks,
-sv
On Wed, 13 Mar 2013 05:29:56 -0400 (EDT) Bohuslav Kabrda bkabrda@redhat.com wrote:
----- Original Message -----
On Fri, 1 Mar 2013 03:54:48 -0500 (EST) Bohuslav Kabrda bkabrda@redhat.com wrote:
Hi list, I'd like to start yet another discussion about some functionality of copr, here it is: One of nice use cases for copr would be building software collections [1]. These however require modified mock chroot (modified config_opts['root'] and config_opts['chroot_setup_cmd']). I'm trying to think of a way how to do this:
What modifications do they need? Also - why can't this just be provided as an additional 'distro' in the form of another mock .cfg file for that distro?
Then the user would just select that in the distro/release in the copr and build?
I meant something like this, only there would need to be a for frontend to let backend know about the new SCL chroots.
My proposal:
- Use chroot names like ruby193_fedora-18-x86_64.
- Allow users to create these chroots by modifying
config_opts['chroot_setup_cmd'] (config_opts['root'] would be autogenerated from the chroot name).
Why overload the chroot name for this?
- Store the modified information in the MockChroot table and
create these on backend using communication channel proposed in [2] (parts of it already implemented in frontend).
- Allow selecting these on copr creation/modification as more
options for mock chroots, by default hidden under some kind of fancy clickable JS dropdown.
I need more info about what modification SCLs require to be able to judge this well.
See above from my original mail. I'm attaching a diff of what I did with epel-6 mock chroot (and if you ignore my added local repo, it's pretty trivial).
thanks,
-sv
okay - looking at that mock config you don't even need to do that.
1. you add a repo to the copr itself to use for buildreqs 2. in that repo you setup a groups/comps file for yum - which says
<group> <id>buildsys-build</id> <name>buildsysbuild for scl ruby193</name> <description/> <packagelist> <packagereq type="mandatory">scl-utils-build</packagereq> <packagereq type="mandatory">ruby193-build</packagereq> </packagelist> </group>
then yum will add this buildsys-build to the original buildsys-build and install ALL of the pkgs in the group by that name in ALL repos.
see - just by adding a repo you can change the buildtools.
-sv
----- Original Message -----
On Wed, 13 Mar 2013 05:29:56 -0400 (EDT) Bohuslav Kabrda bkabrda@redhat.com wrote:
----- Original Message -----
On Fri, 1 Mar 2013 03:54:48 -0500 (EST) Bohuslav Kabrda bkabrda@redhat.com wrote:
Hi list, I'd like to start yet another discussion about some functionality of copr, here it is: One of nice use cases for copr would be building software collections [1]. These however require modified mock chroot (modified config_opts['root'] and config_opts['chroot_setup_cmd']). I'm trying to think of a way how to do this:
What modifications do they need? Also - why can't this just be provided as an additional 'distro' in the form of another mock .cfg file for that distro?
Then the user would just select that in the distro/release in the copr and build?
I meant something like this, only there would need to be a for frontend to let backend know about the new SCL chroots.
My proposal:
- Use chroot names like ruby193_fedora-18-x86_64.
- Allow users to create these chroots by modifying
config_opts['chroot_setup_cmd'] (config_opts['root'] would be autogenerated from the chroot name).
Why overload the chroot name for this?
- Store the modified information in the MockChroot table and
create these on backend using communication channel proposed in [2] (parts of it already implemented in frontend).
- Allow selecting these on copr creation/modification as more
options for mock chroots, by default hidden under some kind of fancy clickable JS dropdown.
I need more info about what modification SCLs require to be able to judge this well.
See above from my original mail. I'm attaching a diff of what I did with epel-6 mock chroot (and if you ignore my added local repo, it's pretty trivial).
thanks,
-sv
okay - looking at that mock config you don't even need to do that.
- you add a repo to the copr itself to use for buildreqs
- in that repo you setup a groups/comps file for yum - which says
<group> <id>buildsys-build</id> <name>buildsysbuild for scl ruby193</name> <description/> <packagelist> <packagereq type="mandatory">scl-utils-build</packagereq> <packagereq type="mandatory">ruby193-build</packagereq> </packagelist> </group>
then yum will add this buildsys-build to the original buildsys-build and install ALL of the pkgs in the group by that name in ALL repos.
see - just by adding a repo you can change the buildtools.
Nice :) Question: What would happen if I wanted to start building the SCL from the scratch in copr and therefore ruby193-build package would not exist yet? Would the buildroot initialization fail or just ignore the missing package? Idea: What if we just add "additional buildroot packages" attribute to every build, similarly as we do with repos/mock chroots? Would the backend be able to generate comps file on the fly for every build? This way, we wouldn't even need additional chroots, we would just use the standard ones and backend would only change the buildsys group on demand with certain builds (heck, this way we could even build SCL-macroed packages as non-SCL in the same buildroot; or build multiple SCLs in the buildroot).
Thanks, Slavek.
-sv
On 03/21/2013 12:27 PM, Bohuslav Kabrda wrote:
What if we just add "additional buildroot packages" attribute to every build, similarly as we do with repos/mock chroots? Would the backend be able to generate comps file on the fly for every build? This way, we wouldn't even need additional chroots, we would just use the standard ones and backend would only change the buildsys group on demand with certain builds (heck, this way we could even build SCL-macroed packages as non-SCL in the same buildroot; or build multiple SCLs in the buildroot).
I could not imagine building one hundreds of packages this way. That would be PITA. IMO SCL setting should be related to (chroot, copr).
----- Original Message -----
On 03/21/2013 12:27 PM, Bohuslav Kabrda wrote:
What if we just add "additional buildroot packages" attribute to every build, similarly as we do with repos/mock chroots? Would the backend be able to generate comps file on the fly for every build? This way, we wouldn't even need additional chroots, we would just use the standard ones and backend would only change the buildsys group on demand with certain builds (heck, this way we could even build SCL-macroed packages as non-SCL in the same buildroot; or build multiple SCLs in the buildroot).
I could not imagine building one hundreds of packages this way. That would be PITA. IMO SCL setting should be related to (chroot, copr).
-- Miroslav Suchy Red Hat Systems Management Engineering _______________________________________________ copr-devel mailing list copr-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/copr-devel
While both of the ways seem to have their pros and cons, I tend to agree with Mirek. Before deciding which way to go, there are some questions that will need to be answered for this approach: - How do we map the chroots to the backend repo? What I mean by this is, that if you build two SCLs in one copr, you will need two chroots, say ruby193-fedora-19-x86_64 and python26-fedora-19-x86_64 (will we enforce this naming scheme?). This makes me think that we should place all the builds done in these chroots to fedora-19-x86_64, not in their separate directories (so that we can generate repofile that points just to this dir - I'm not sure whether yum would be able to cope with arbitrary named directories in a copr and installing dependencies from them). CCing @tradej on this, since this would probably need some altering of repofile generation. - As already asked above - do we enforce a specific naming scheme? - Will any user be able to create an alternate chroot? What if two users decide to use same name for a chroot? (This leads me to a thought of making the modified chroots per-user (e.g. bkabrda/ruby193-fedora-19-x86_64), but I may be overengineering here).
Does this make sense? Thanks!
On 06/25/2013 12:39 PM, Bohuslav Kabrda wrote:
While both of the ways seem to have their pros and cons, I tend to agree with Mirek. Before deciding which way to go, there are some questions that will need to be answered for this approach:
- How do we map the chroots to the backend repo? What I mean by this is, that if you build two SCLs in one copr, you will need two chroots, say ruby193-fedora-19-x86_64 and python26-fedora-19-x86_64 (will we enforce this naming scheme?). This makes me think that we should place all the builds done in these chroots to fedora-19-x86_64, not in their separate directories (so that we can generate repofile that points just to this dir - I'm not sure whether yum would be able to cope with arbitrary named directories in a copr and installing dependencies from them). CCing @tradej on this, since this would probably need some altering of repofile generation.
I do not think we should grant user the rights for creating new chroot. I have different suggestion - see below. But when you are speaking about two SCL... this need to be solved on SCL side. Let assume that you have ruby193 and python26 collection and I want to have build libselinux. And I plan to use it under both collection at the same time. But libselinux have subpackages with ruby and python bindings. So I probably need to build it at one shot for both collections. Which is currently impossible. And it is beyond COPR. And even Koji.
- Will any user be able to create an alternate chroot? What if two users decide to use same name for a chroot? (This leads me to a thought of making the modified chroots per-user (e.g. bkabrda/ruby193-fedora-19-x86_64), but I may be overengineering here).
I suggest that users will use chroots, which we will gave them. But that chroots will be editable (see attachment). And when you click on chroot, you will be able to add additional package to chroot.
For example for SCL you will first normally build meta package, then you will click on epel6-x86_64 chroot. And you will say you want to add ruby193-build package to your default buildroot. We will store it in DB in custom_chroot: id, copr_id, mock_chroot_id
Backend will retrieve that info and will create mock config with: config_opts['root'] = 'epel-6-x86_64' config_opts['target_arch'] = 'x86_64' config_opts['legal_host_arches'] = ('x86_64',) config_opts['chroot_setup_cmd'] = 'install @build ruby193-build' config_opts['dist'] = 'el6'
and pass this mock.config to builder. There is no need to give it name. Or more precise the name of such mock config can by ugly as custom-mock-$copr_id-$chroot_id because it will not be seen by user.
----- Original Message -----
On 06/25/2013 12:39 PM, Bohuslav Kabrda wrote:
While both of the ways seem to have their pros and cons, I tend to agree with Mirek. Before deciding which way to go, there are some questions that will need to be answered for this approach:
- How do we map the chroots to the backend repo? What I mean by this is,
that if you build two SCLs in one copr, you will need two chroots, say ruby193-fedora-19-x86_64 and python26-fedora-19-x86_64 (will we enforce this naming scheme?). This makes me think that we should place all the builds done in these chroots to fedora-19-x86_64, not in their separate directories (so that we can generate repofile that points just to this dir
- I'm not sure whether yum would be able to cope with arbitrary named
directories in a copr and installing dependencies from them). CCing @tradej on this, since this would probably need some altering of repofile generation.
I do not think we should grant user the rights for creating new chroot. I have different suggestion - see below. But when you are speaking about two SCL... this need to be solved on SCL side. Let assume that you have ruby193 and python26 collection and I want to have build libselinux. And I plan to use it under both collection at the same time. But libselinux have subpackages with ruby and python bindings. So I probably need to build it at one shot for both collections. Which is currently impossible. And it is beyond COPR. And even Koji.
Not really. You may just want to build multiple separate SCLs in one copr. Sharing libraries between them is issue of SCL concept itself, yes, but it has nothing to do with building more scls in copr.
- Will any user be able to create an alternate chroot? What if two users
decide to use same name for a chroot? (This leads me to a thought of making the modified chroots per-user (e.g. bkabrda/ruby193-fedora-19-x86_64), but I may be overengineering here).
I suggest that users will use chroots, which we will gave them. But that chroots will be editable (see attachment). And when you click on chroot, you will be able to add additional package to chroot.
For example for SCL you will first normally build meta package, then you will click on epel6-x86_64 chroot. And you will say you want to add ruby193-build package to your default buildroot. We will store it in DB in custom_chroot: id, copr_id, mock_chroot_id
Backend will retrieve that info and will create mock config with: config_opts['root'] = 'epel-6-x86_64' config_opts['target_arch'] = 'x86_64' config_opts['legal_host_arches'] = ('x86_64',) config_opts['chroot_setup_cmd'] = 'install @build ruby193-build' config_opts['dist'] = 'el6'
and pass this mock.config to builder. There is no need to give it name. Or more precise the name of such mock config can by ugly as custom-mock-$copr_id-$chroot_id because it will not be seen by user.
So this is basically like my proposal (=personalized chroots), but without actually naming the chroots, right? That has the disadvantage that if you're building multiple SCLs in one copr, you need to alter the chroot for every build in different scl.
-- Miroslav Suchy Red Hat Systems Management Engineering
copr-devel mailing list copr-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/copr-devel
copr-devel@lists.stg.fedorahosted.org
-
Bohuslav Kabrda -
Miroslav Suchý -
Seth Vidal