Using koji what is the meaning of add-pkg ? I understand I use import to add a package and tag-pkg or "call tagBuildBypass" to put a tag on this package. Must I call add-pkg in between?
Thanks, /Otto
On 12/3/14 2:34 PM, Otto Han wrote:
Using koji what is the meaning of add-pkg ? I understand I use import to add a package and tag-pkg or "call tagBuildBypass" to put a tag on this package. Must I call add-pkg in between?
koji add-pkg adds a package to the whitelist for a tag. Before you can tag a build into a tag, you need to add the package to the tag whitelist by calling:
koji add-pkg <tag-name> <package-name> --owner <owner-name>
where <tag-name> is the name of the tag you're trying to tag the build into, or one of the tags that is inherited by that tag. To see the current package whitelist for a tag, run:
koji list-pkgs --tag <tag-name>
This will show the full whitelist, including inherited entries.
Thanks, /Otto -- buildsys mailing list buildsys@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/buildsys
On Wed, Dec 3, 2014 at 3:18 PM, Mike Bonnet mikeb@redhat.com wrote:
On 12/3/14 2:34 PM, Otto Han wrote:
Using koji what is the meaning of add-pkg ? I understand I use import to add a package and tag-pkg or "call tagBuildBypass" to put a tag on this package. Must I call add-pkg in between?
koji add-pkg adds a package to the whitelist for a tag. Before you can tag a build into a tag, you need to add the package to the tag whitelist by calling:
koji add-pkg <tag-name> <package-name> --owner <owner-name>
where <tag-name> is the name of the tag you're trying to tag the build into, or one of the tags that is inherited by that tag. To see the current package whitelist for a tag, run:
koji list-pkgs --tag <tag-name>
This will show the full whitelist, including inherited entries.
OK. But then what "koji list-untagged" shows? Packages not belonging to any white list at all or they may be in a white list but not yet tagged ? If this is true then how can I see packages imported but not yet included in a white-list.
On 12/3/14 4:29 PM, Otto Han wrote:
On Wed, Dec 3, 2014 at 3:18 PM, Mike Bonnet mikeb@redhat.com wrote:
On 12/3/14 2:34 PM, Otto Han wrote:
Using koji what is the meaning of add-pkg ? I understand I use import to add a package and tag-pkg or "call tagBuildBypass" to put a tag on this package. Must I call add-pkg in between?
koji add-pkg adds a package to the whitelist for a tag. Before you can tag a build into a tag, you need to add the package to the tag whitelist by calling:
koji add-pkg <tag-name> <package-name> --owner <owner-name>
where <tag-name> is the name of the tag you're trying to tag the build into, or one of the tags that is inherited by that tag. To see the current package whitelist for a tag, run:
koji list-pkgs --tag <tag-name>
This will show the full whitelist, including inherited entries.
OK. But then what "koji list-untagged" shows? Packages not belonging to any white list at all or they may be in a white list but not yet tagged ? If this is true then how can I see packages imported but not yet included in a white-list.
koji list-untagged shows *builds* that are not associated with any tag. This could be because a build was imported and not tagged, or (more commonly) because a build has been untagged, possibly by the garbage-collection process (koji-gc).
Example: you want to import findutils-4.5.14-1.fc22 and tag it into the my-imports tag
$ koji import findutils-4.5.14-1.fc22.src.rpm \ findutils-4.5.14-1.fc22.x86_64.rpm \ findutils-debuginfo-4.5.14-1.fc22.x86_64.rpm
(this creates a findutils-4.5.14-1.fc22 entry in the database, assuming it doesn't already exist)
$ koji add-pkg my-imports findutils --owner otto
(the argument to --owner would be whatever your koji username is)
$ koji tag-build my-imports findutils-4.5.14-1.fc22
Note that importing rpms isn't common practice. If you're just trying to get to the point where you can build rpms in Koji, you should be setting up external repos as outlined here:
https://fedoraproject.org/wiki/Koji/ExternalRepoServerBootstrap
On Wed, Dec 3, 2014 at 5:21 PM, Mike Bonnet mikeb@redhat.com wrote:
On 12/3/14 4:29 PM, Otto Han wrote:
On Wed, Dec 3, 2014 at 3:18 PM, Mike Bonnet mikeb@redhat.com wrote:
On 12/3/14 2:34 PM, Otto Han wrote:
Using koji what is the meaning of add-pkg ? I understand I use import to add a package and tag-pkg or "call tagBuildBypass" to put a tag on this package. Must I call add-pkg in between?
koji add-pkg adds a package to the whitelist for a tag. Before you can tag a build into a tag, you need to add the package to the tag whitelist by calling:
koji add-pkg <tag-name> <package-name> --owner <owner-name>
where <tag-name> is the name of the tag you're trying to tag the build into, or one of the tags that is inherited by that tag. To see the current package whitelist for a tag, run:
koji list-pkgs --tag <tag-name>
This will show the full whitelist, including inherited entries.
OK. But then what "koji list-untagged" shows? Packages not belonging to any white list at all or they may be in a white list but not yet tagged ? If this is true then how can I see packages imported but not yet included in a white-list.
koji list-untagged shows *builds* that are not associated with any tag. This could be because a build was imported and not tagged, or (more commonly) because a build has been untagged, possibly by the garbage-collection process (koji-gc).
Example: you want to import findutils-4.5.14-1.fc22 and tag it into the my-imports tag
$ koji import findutils-4.5.14-1.fc22.src.rpm \ findutils-4.5.14-1.fc22.x86_64.rpm \ findutils-debuginfo-4.5.14-1.fc22.x86_64.rpm
(this creates a findutils-4.5.14-1.fc22 entry in the database, assuming it doesn't already exist)
$ koji add-pkg my-imports findutils --owner otto
(the argument to --owner would be whatever your koji username is)
$ koji tag-build my-imports findutils-4.5.14-1.fc22
Note that importing rpms isn't common practice. If you're just trying to get to the point where you can build rpms in Koji, you should be setting up external repos as outlined here:
https://fedoraproject.org/wiki/Koji/ExternalRepoServerBootstrap
-- buildsys mailing list buildsys@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/buildsys
This last sentence is very important for me. We are using Koji as sort of Package-MgtSystem (package repository). We are building rpm from srpm in a Jenkins environment. When Jenkins is done and no error was noted we import all built packages in Koji and tag them. The source code is maintained in a perforce CMS. When developers reach a stable phase they generate SRPM from the source code and then run a Jenkins job to generate the RPM. The result is a temporary local repository containing all SRPM and RPM of the build. These packages are then imported in Koji and appropriately tagged: - koji import *.rpm - for all untagged: koji call tagBuildBypass tag package For us importing is a very common practice.
Having said all this what is your advice and what do you think of this way using Koji ?
Rarely we have a brand new package which does not exist yet in Koji. Import *.rpm will insert it in Koji but how can I detect I must run add-pkg for it before tagging?
On 12/04/2014 01:57 PM, Otto Han wrote:
This last sentence is very important for me. We are using Koji as sort of Package-MgtSystem (package repository). We are building rpm from srpm in a Jenkins environment. When Jenkins is done and no error was noted we import all built packages in Koji and tag them. The source code is maintained in a perforce CMS. When developers reach a stable phase they generate SRPM from the source code and then run a Jenkins job to generate the RPM. The result is a temporary local repository containing all SRPM and RPM of the build.
I know that folks have wired Jenkins to trigger builds inside of a Koji instance. I wonder if that would work for you.
These packages are then imported in Koji and appropriately tagged:
- koji import *.rpm
- for all untagged: koji call tagBuildBypass tag package
For us importing is a very common practice.
If I were doing this sort of mass import all the time. I would first generate a list of the srpms these rpms were generated from (which determines the build they will be associated with upon import). E.g.
# rpm --nosignature --nodigest -qp --qf '%{sourcepackage} %{sourcerpm} %{n}-%{v}-%{r}\n' *.rpm |awk '{if ($1=="1") { print $3 } else {print gensub(".src.rpm$", "", "", $2)} }'
(The complication above is to deal with the srpm case (just use that nvr) vs the non-srpm case (use the sourcerpm value).
Anyway, I would save that list and then tag those nvrs after import.
Using list-untagged is not very reliable here, as there are lots of reasons that a build might become untagged. You might end up tagging something that you don't mean to.
Having said all this what is your advice and what do you think of this way using Koji ?
Well you're not really using most of koji, and you're losing many of the advantages. With this approach you have very little build metadata in koji. Garbage collection will be much less intelligent.
Rarely we have a brand new package which does not exist yet in Koji. Import *.rpm will insert it in Koji but how can I detect I must run add-pkg for it before tagging?
You could use the koji list-pkgs command to check for it. Or you could just run koji add-pkg every time. If the package is already present, it is a no-op (though it will print a warning).
On Fri, Dec 12, 2014 at 1:20 PM, Mike McLean mikem@redhat.com wrote:
On 12/04/2014 01:57 PM, Otto Han wrote:
This last sentence is very important for me. We are using Koji as sort of Package-MgtSystem (package repository). We are building rpm from srpm in a Jenkins environment. When Jenkins is done and no error was noted we import all built packages in Koji and tag them. The source code is maintained in a perforce CMS. When developers reach a stable phase they generate SRPM from the source code and then run a Jenkins job to generate the RPM. The result is a temporary local repository containing all SRPM and RPM of the build.
I know that folks have wired Jenkins to trigger builds inside of a Koji instance. I wonder if that would work for you.
These packages are then imported in Koji and appropriately tagged:
- koji import *.rpm
- for all untagged: koji call tagBuildBypass tag package
For us importing is a very common practice.
If I were doing this sort of mass import all the time. I would first generate a list of the srpms these rpms were generated from (which determines the build they will be associated with upon import). E.g.
# rpm --nosignature --nodigest -qp --qf '%{sourcepackage} %{sourcerpm} %{n}-%{v}-%{r}\n' *.rpm |awk '{if ($1=="1") { print $3 } else {print gensub(".src.rpm$", "", "", $2)} }'
(The complication above is to deal with the srpm case (just use that nvr) vs the non-srpm case (use the sourcerpm value).
Anyway, I would save that list and then tag those nvrs after import.
Using list-untagged is not very reliable here, as there are lots of reasons that a build might become untagged. You might end up tagging something that you don't mean to.
Having said all this what is your advice and what do you think of this way using Koji ?
Well you're not really using most of koji, and you're losing many of the advantages. With this approach you have very little build metadata in koji. Garbage collection will be much less intelligent.
Rarely we have a brand new package which does not exist yet in Koji. Import *.rpm will insert it in Koji but how can I detect I must run add-pkg for it before tagging?
You could use the koji list-pkgs command to check for it. Or you could just run koji add-pkg every time. If the package is already present, it is a no-op (though it will print a warning). --
Thanks for the help.
If I can abuse of your time... There is a thing I cannot grasp. If I keep the sources in Koji as SRPM then how do I deal with the project advancement. We are modifying on a regular base the sources (under perforce) then we build the SRPM and RPM and import them in Koji. Is there a Code Management feature in Koji ?
On 12/15/2014 07:11 PM, Otto Han wrote:
On Fri, Dec 12, 2014 at 1:20 PM, Mike McLean mikem@redhat.com wrote:
On 12/04/2014 01:57 PM, Otto Han wrote:
This last sentence is very important for me. We are using Koji as sort of Package-MgtSystem (package repository). We are building rpm from srpm in a Jenkins environment. When Jenkins is done and no error was noted we import all built packages in Koji and tag them. The source code is maintained in a perforce CMS. When developers reach a stable phase they generate SRPM from the source code and then run a Jenkins job to generate the RPM. The result is a temporary local repository containing all SRPM and RPM of the build.
I know that folks have wired Jenkins to trigger builds inside of a Koji instance. I wonder if that would work for you.
These packages are then imported in Koji and appropriately tagged:
- koji import *.rpm
- for all untagged: koji call tagBuildBypass tag package
For us importing is a very common practice.
If I were doing this sort of mass import all the time. I would first generate a list of the srpms these rpms were generated from (which determines the build they will be associated with upon import). E.g.
# rpm --nosignature --nodigest -qp --qf '%{sourcepackage} %{sourcerpm} %{n}-%{v}-%{r}\n' *.rpm |awk '{if ($1=="1") { print $3 } else {print gensub(".src.rpm$", "", "", $2)} }'
(The complication above is to deal with the srpm case (just use that nvr) vs the non-srpm case (use the sourcerpm value).
Anyway, I would save that list and then tag those nvrs after import.
Using list-untagged is not very reliable here, as there are lots of reasons that a build might become untagged. You might end up tagging something that you don't mean to.
Having said all this what is your advice and what do you think of this way using Koji ?
Well you're not really using most of koji, and you're losing many of the advantages. With this approach you have very little build metadata in koji. Garbage collection will be much less intelligent.
Rarely we have a brand new package which does not exist yet in Koji. Import *.rpm will insert it in Koji but how can I detect I must run add-pkg for it before tagging?
You could use the koji list-pkgs command to check for it. Or you could just run koji add-pkg every time. If the package is already present, it is a no-op (though it will print a warning). --
Thanks for the help.
If I can abuse of your time... There is a thing I cannot grasp. If I keep the sources in Koji as SRPM then how do I deal with the project advancement. We are modifying on a regular base the sources (under perforce) then we build the SRPM and RPM and import them in Koji. Is there a Code Management feature in Koji ?
Koji is not a source control solution. We assume folks have their own solution for that. Personally, I like git. Didn't you say you were using perforce?
On Tue, Dec 16, 2014 at 8:23 AM, Mike McLean mikem@redhat.com wrote:
On 12/15/2014 07:11 PM, Otto Han wrote:
On Fri, Dec 12, 2014 at 1:20 PM, Mike McLean mikem@redhat.com wrote:
On 12/04/2014 01:57 PM, Otto Han wrote:
This last sentence is very important for me. We are using Koji as sort of Package-MgtSystem (package repository). We are building rpm from srpm in a Jenkins environment. When Jenkins is done and no error was noted we import all built packages in Koji and tag them. The source code is maintained in a perforce CMS. When developers reach a stable phase they generate SRPM from the source code and then run a Jenkins job to generate the RPM. The result is a temporary local repository containing all SRPM and RPM of the build.
I know that folks have wired Jenkins to trigger builds inside of a Koji instance. I wonder if that would work for you.
These packages are then imported in Koji and appropriately tagged:
- koji import *.rpm
- for all untagged: koji call tagBuildBypass tag package
For us importing is a very common practice.
If I were doing this sort of mass import all the time. I would first generate a list of the srpms these rpms were generated from (which determines the build they will be associated with upon import). E.g.
# rpm --nosignature --nodigest -qp --qf '%{sourcepackage} %{sourcerpm} %{n}-%{v}-%{r}\n' *.rpm |awk '{if ($1=="1") { print $3 } else {print gensub(".src.rpm$", "", "", $2)} }'
(The complication above is to deal with the srpm case (just use that nvr) vs the non-srpm case (use the sourcerpm value).
Anyway, I would save that list and then tag those nvrs after import.
Using list-untagged is not very reliable here, as there are lots of reasons that a build might become untagged. You might end up tagging something that you don't mean to.
Having said all this what is your advice and what do you think of this way using Koji ?
Well you're not really using most of koji, and you're losing many of the advantages. With this approach you have very little build metadata in koji. Garbage collection will be much less intelligent.
Rarely we have a brand new package which does not exist yet in Koji. Import *.rpm will insert it in Koji but how can I detect I must run add-pkg for it before tagging?
You could use the koji list-pkgs command to check for it. Or you could just run koji add-pkg every time. If the package is already present, it is a no-op (though it will print a warning). --
Thanks for the help.
If I can abuse of your time... There is a thing I cannot grasp. If I keep the sources in Koji as SRPM then how do I deal with the project advancement. We are modifying on a regular base the sources (under perforce) then we build the SRPM and RPM and import them in Koji. Is there a Code Management feature in Koji ?
Koji is not a source control solution. We assume folks have their own solution for that. Personally, I like git. Didn't you say you were using perforce?
Yes, we use perforce. I understand importing frequently the SRPM is fine because there is no way to alter the sources through Koji. Doesn't mean this will somehow overload the Koji system in time? Is there a way to "purge" the source RPMs to prevent this situation? I mean once a build is done there is no reason to keep the sources in Koji as they are, already in CMS. On the other hand I understand there is no way to remove / delete a package once in Koji. How can we deal with this case?
-- buildsys mailing list buildsys@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/buildsys
On 12/16/2014 12:07 PM, Otto Han wrote:
Yes, we use perforce. I understand importing frequently the SRPM is fine because there is no way to alter the sources through Koji. Doesn't mean this will somehow overload the Koji system in time? Is there a way to "purge" the source RPMs to prevent this situation? I mean once a build is done there is no reason to keep the sources in Koji as they are, already in CMS. On the other hand I understand there is no way to remove / delete a package once in Koji. How can we deal with this case?
If you are publishing the srpms, then you probably want to keep them. They are probably not that large compared to the rpms themselves.
Koji treats builds (the srpm and all the rpms it creates) as a unit. You can delete old builds as a whole if you need to, or configure the gc to do so. There is currently no way to delete a single rpm (or srpm) from a build.
You described a workflow where you build external to koji and imported. If that is the case, and you don't want the srpms, then perhaps you should just not import them in the first place. A little extra work will be required to get imports to work without the srpms. You could either just add --create-build to the import command to create the build entries without having the srpm, or you could create them directly with an api call before importing the rpms.
buildsys@lists.fedoraproject.org
-
Mike Bonnet -
Mike McLean -
Otto Han