Here's the post-mortem for an exception in koji-hub (EPEL6 1.8.0-2 RPM).
Earlier, I did something weird, and built a kernel RPM with some pieces disabled, including the 'kernel-headers' subpackage. Builds following the repo regeneration failed, since the incomplete kernel package build overrode other kernel packages, including subpackages; 'kernel-headers' is a dependency of build root packages. The incomplete kernel package was removed to the trashcan and repos regenerated.
The next build of another kernel package resulted in the exception pasted below.
*** My best guess at the problem:
Kernel packages have both 'i686' and 'noarch' builds that run simultaneously.
The i686 build inserted the 'kernel-headers' row into the 'rpminfo' table, along with a related row in 'buildroot_listing'.
At the same time, the noarch build attempted to do the same thing, but the i686 build won the race. The new row broke uniqueness constraints, (NVRAs must be unique within an external repo), and hence the exception.
If these two builds' add_external_rpm() calls hadn't overlapped, the second would have created its 'buildroot_listing' row referencing the 'rpminfo' row that the first created.
*** Estimate of problem gravity:
This overlap is probably rarely seen because (1) most packages have either 'noarch' builds or arch-specific builds, but not both; (2) this happened to be the first build after the repos were regenerated after addition of a new package; and (3) even if the first two conditions are met, it is unlikely that the two builds happened to run add_external_rpm() at exactly the same time.
The exception stops the build, but no other harm is done. Rebuilding will not repeat this exception, since the row is created and the race condition does not exist.
John
2013-12-20 13:43:56,876 [WARNING] m=host.setBuildRootList u=infra0.zultron.com p=25442 \ r=192.168.7.29:47734 koji.xmlrpc: Traceback (most recent call last): File "/usr/share/koji-hub/kojixmlrpc.py", line 212, in _wrap_handler response = handler(environ) File "/usr/share/koji-hub/kojixmlrpc.py", line 255, in handle_rpc return self._dispatch(method, params) File "/usr/share/koji-hub/kojixmlrpc.py", line 292, in _dispatch ret = koji.util.call_with_argcheck(func, params, opts) File "/usr/lib/python2.6/site-packages/koji/util.py", line 139, in call_with_argcheck return func(*args, **kwargs) File "/usr/share/koji-hub/kojihub.py", line 10237, in setBuildRootList return br.setList(rpmlist) File "/usr/share/koji-hub/kojihub.py", line 9371, in setList self._setList(rpmlist,update=False) File "/usr/share/koji-hub/kojihub.py", line 9353, in _setList data = add_external_rpm(an_rpm, location, strict=False) File "/usr/share/koji-hub/kojihub.py", line 4590, in add_external_rpm _dml(q, rpminfo) File "/usr/share/koji-hub/kojihub.py", line 3985, in _dml c.execute(operation, values) File "/usr/lib/python2.6/site-packages/koji/db.py", line 105, in execute ret = self.cursor.execute(operation, parameters) File "/usr/lib64/python2.6/site-packages/pgdb.py", line 174, in execute self.executemany(operation, (params,)) File "/usr/lib64/python2.6/site-packages/pgdb.py", line 195, in executemany raise DatabaseError, "error '%s' in '%s'" % ( msg, sql ) DatabaseError: error 'ERROR: duplicate key value violates unique constraint "rpminfo_unique_nvra" ' in 'INSERT INTO rpminfo (id, build_id, buildroot_id, name, version, release, epoch, arch, external_repo_id, payloadhash, size, buildtime) VALUES (10579, NULL, NULL, 'kernel-headers', '3.11.10', '200.fc19', NULL, 'i686', 21, '1e6b65e55f166cb40dff21842941be2b', 3165695, 1386020554) '
buildsys@lists.fedoraproject.org