Hi, I have a question about `checkHost`...
AFAIU:
- kojid's `main` continually loops over daemon.py::TaskManager::getNextTask() - getNextTask() calls daemon.py::TaskManager::takeTask() (when appropriate) - and takeTask() calls back into kojid's build-task-class-specific (i.e. some children of BaseBuildTask) `checkHost()`
The question: Is it the case that a failed `checkHost()` call will simply cause a builder to skip that task and go on to the next, *eventually coming back to check that task again*?
Some context:
OSBS is being redesigned (reimplemented as well, at this point), and the feature that we had in the old OSBS which would "pipeline" multiple tasks created from the exact same repo/branch will no longer work[1].
We're looking at "abusing" `checkHost` by defining an override in OSBS' koji-containerbuild[2] builder to check existing 'OPEN' tasks before taking up a new one.
Our primary concern is making sure that a task which is being skipped by the builders (because it failed the `checkHost` check) will still be picked up once `checkHost` clears.
[1] Primarily to avoid the situation where build 1 starts, then build 2, both from exactly the same repo/branch, but for reasons build *2* finishes first (happens all the time...), and build *1* gets tagged ':latest' in the registry - this is surprising to end-users who expect the last *requested* build to get the ':latest' tag [2] https://github.com/containerbuildsystem/koji-containerbuild
checkHost was added in order to work around issues with "noarch" tasks way back in 2011. It's hardly an idea implementation, but it works reasonably well for the case that prompted it.
You are correct in reading that the host could potentially keep calling checkBuild for the same task. The expectation in the original feature was that another host would take the task before very long.
Querying other open tasks in checkHost is rather suspicious. The intent is to check whether the host is capable of performing the task, but it sounds like you're trying to work around build interdependency issues.
I'm actually working on a significant redesign of task scheduling in Koji. I'd be interested in hearing more about your use case.
On Wed, Feb 16, 2022 at 12:00 PM Ben Alkov ben.alkov@redhat.com wrote:
Hi, I have a question about `checkHost`...
AFAIU:
- kojid's `main` continually loops over
daemon.py::TaskManager::getNextTask()
- getNextTask() calls daemon.py::TaskManager::takeTask() (when appropriate)
- and takeTask() calls back into kojid's build-task-class-specific (i.e.
some children of BaseBuildTask) `checkHost()`
The question: Is it the case that a failed `checkHost()` call will simply cause a builder to skip that task and go on to the next, *eventually coming back to check that task again*?
Some context:
OSBS is being redesigned (reimplemented as well, at this point), and the feature that we had in the old OSBS which would "pipeline" multiple tasks created from the exact same repo/branch will no longer work[1].
We're looking at "abusing" `checkHost` by defining an override in OSBS' koji-containerbuild[2] builder to check existing 'OPEN' tasks before taking up a new one.
Our primary concern is making sure that a task which is being skipped by the builders (because it failed the `checkHost` check) will still be picked up once `checkHost` clears.
[1] Primarily to avoid the situation where build 1 starts, then build 2, both from exactly the same repo/branch, but for reasons build *2* finishes first (happens all the time...), and build *1* gets tagged ':latest' in the registry - this is surprising to end-users who expect the last *requested* build to get the ':latest' tag [2] https://github.com/containerbuildsystem/koji-containerbuild _______________________________________________ koji-devel mailing list -- koji-devel@lists.fedorahosted.org To unsubscribe send an email to koji-devel-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/koji-devel@lists.fedorahosted.o... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
koji-devel@lists.stg.fedorahosted.org