I have latest trunk version of koji, kojid, kojira, etc setup and running. But now what? Does any administration documentation exist? To figure out how to put gas in this engine do I really have to read the source?
Thanks.
On Mon, 2007-09-10 at 10:03 -0700, aberoham@gmail.com wrote:
I have latest trunk version of koji, kojid, kojira, etc setup and running. But now what? Does any administration documentation exist? To figure out how to put gas in this engine do I really have to read the source?
from what i remember, the first step is to import rpms for the distribution you are building against. koji assigns buildids in the order that the rpms are imported or built, so take care to import the oldest rpms first. assign the imported rpms a tag, and then create a build tag.
if you take notes of what is required, it would probably help others to add them here: http://fedoraproject.org/wiki/Koji/ServerHowTo
good luck!
rob.
rob myers rob.myers@gtri.gatech.edu writes:
from what i remember, the first step is to import rpms for the distribution you are building against.
Can be avoided by applying
https://www.redhat.com/archives/fedora-buildsys-list/2007-May/msg00023.html
and using an external repository.
Enrico
On Tuesday 11 September 2007 9:36:31 am Enrico Scholz wrote:
rob myers rob.myers@gtri.gatech.edu writes:
from what i remember, the first step is to import rpms for the distribution you are building against.
Can be avoided by applying
https://www.redhat.com/archives/fedora-buildsys-list/2007-May/msg00023.html
and using an external repository.
Enrico
Which breaks many things in koji. such as reproducability. koji keeps track of and holds all rpms that are used in buildroots.
Dennis
Dennis Gilmore dennis@ausil.us writes:
https://www.redhat.com/archives/fedora-buildsys-list/2007-May/msg00023.html
Which breaks many things in koji.
many? I am only aware of reproducability and did 735 builds/10968 tasks with this patch.
such as reproducability. koji keeps track of and holds all rpms that are used in buildroots.
yes; this requires an enormous amount of space, backup capacity, bandwidth and effort to keep repository up-to-date. This is far too much overhead for my use case...
Enrico
What I have figured out about koji initialization so far from talking to folks is attached. My koji, however, isn't "working."
I'm reproducing my setup from scratch using these notes in the hopes that I simply messed something up with too much parallel mucking along with way. While I'm retracing my steps, I would be much obliged if any of the koji experts could take a gander that these notes and comment.
Thanks, Abe
On 9/10/07, aberoham@gmail.com aberoham@gmail.com wrote:
I have latest trunk version of koji, kojid, kojira, etc setup and running. But now what? Does any administration documentation exist? To figure out how to put gas in this engine do I really have to read the source?
Thanks.
aberoham@gmail.com wrote:
Note: kojira must be running and must have repo perms (perm_id 3)
- import all srpms from the upstream RHEL/Fedora distribution, use --create-build koji import --create-build [SRPM]
In Koji, builds are named after the srpm, i.e they use the e:n-v-r of the source rpm. Because the SOURCERPM field of the rpm does not include an epoch the system cannot be sure it has the correct data for the build entry. So, by default, it only creates the build entry when importing an srpm.
The --create-build option causes import to create a build entry for an rpm even if its srpm is not in the import batch. It will use the n-v-r from the SOURCERPM field, and assume the epoch of the (missing) srpm is the same as the epoch of the rpm.
So --create-build should only be necessary when your import is missing srpms. If you're importing Fedora content, I would hope this is not the case.
- imprt all binary rpms (for each arch) koji import [RPM]
The current import code in the koji cli will sort the srpms on the command line into groups based on their srpm. It will automatically import the srpms first and if there are rpms that do not have a corresponding srpm or existing build entry (and --create-build is not specified) it will warn and exit before taking any action.
So, you should be able to collapse 1 and 2 into one command.
I should also point out: if you're importing a large number of [s]rpms that are already on the same filesystem as your koji store, then the --link option may save you a lot of time. It requires that your (local) user have permission to write to the koji work directory. It bypasses the upload and instead simply hardlinks the file into the work directory (where it would have been uploaded to anyway).
- create a tag, we'll call it mydist koji add-tag mydist koji edit-tag mydist --arches="x86_64 i386"
add-tag supports a --arches argument to specify the initial arch list.
- add all untagged builds to that tag (use koji list-untagged to get the list) koji tag-pkg [package-n-v-r ... package-n-v-r] --force --nowait
heh, that's a clever way to generate the n-v-r list during bootstrap :)
- add each host listed in "koji list-hosts" to createrepo channel # koji list-hosts # koji add-host-to-channel <name of builder> createrepo
We put the repo tasks on a special channel because we found that some hosts in our system were particularly slow at it. By leaving those hosts out of the createrepo channel, we found we could improve overall repo regeneration speed.
It's also conceivable that you might want to keep some hosts out of the createrepo pool simply to reduce load on them.
Mike McLean schrieb:
aberoham@gmail.com wrote:
Note: kojira must be running and must have repo perms (perm_id 3)
- import all srpms from the upstream RHEL/Fedora distribution, use
--create-build koji import --create-build [SRPM]
In Koji, builds are named after the srpm, i.e they use the e:n-v-r of the source rpm. Because the SOURCERPM field of the rpm does not include an epoch the system cannot be sure it has the correct data for the build entry. So, by default, it only creates the build entry when importing an srpm.
The --create-build option causes import to create a build entry for an rpm even if its srpm is not in the import batch. It will use the n-v-r from the SOURCERPM field, and assume the epoch of the (missing) srpm is the same as the epoch of the rpm.
So --create-build should only be necessary when your import is missing srpms. If you're importing Fedora content, I would hope this is not the case.
- imprt all binary rpms (for each arch) koji import [RPM]
The current import code in the koji cli will sort the srpms on the command line into groups based on their srpm. It will automatically import the srpms first and if there are rpms that do not have a corresponding srpm or existing build entry (and --create-build is not specified) it will warn and exit before taking any action.
So, you should be able to collapse 1 and 2 into one command.
I should also point out: if you're importing a large number of [s]rpms that are already on the same filesystem as your koji store, then the --link option may save you a lot of time. It requires that your (local) user have permission to write to the koji work directory. It bypasses the upload and instead simply hardlinks the file into the work directory (where it would have been uploaded to anyway).
- create a tag, we'll call it mydist koji add-tag mydist koji edit-tag mydist --arches="x86_64 i386"
add-tag supports a --arches argument to specify the initial arch list.
- add all untagged builds to that tag (use koji list-untagged to get the list) koji tag-pkg [package-n-v-r ... package-n-v-r] --force --nowait
heh, that's a clever way to generate the n-v-r list during bootstrap :)
- add each host listed in "koji list-hosts" to createrepo channel # koji list-hosts # koji add-host-to-channel <name of builder> createrepo
We put the repo tasks on a special channel because we found that some hosts in our system were particularly slow at it. By leaving those hosts out of the createrepo channel, we found we could improve overall repo regeneration speed.
It's also conceivable that you might want to keep some hosts out of the createrepo pool simply to reduce load on them.
You might also want to take a look at http://wiki.linux-kernel.at/index.php/Koji
:-)
Best, Oliver
buildsys@lists.fedoraproject.org
-
aberoham＠gmail.com -
Dennis Gilmore -
Enrico Scholz -
Mike McLean -
Oliver Falk -
Rob Myers