So, building on the spirited debate going on with respect to a suggested review template, is it time for a review-oh-matic type service?
Many of the things we do for reviews are the same. Check it builds in mock (koji scratch), check the srpm sources against upstream, look @ rpmlint, look at the file lists, etc, etc... Wouldn't it make all our lives to have a simple little tool that a packager could submit a srpm for review to. It could then, say (and off the top of my head):
1) extract the spec, and push it + the srpm somewhere 2) file a review bug 2) kick off a koji scratch build 3) post a link to the build, success/failure, build logs (a la the FTBFS overlord) ==> success, pull the built rpms, and post to the review bug: * rpmlint, requires/provides * md5/sha1 of srpm provided source against upstream * a partial template, good bad, etc.
Provisions could be made for dependencies -- e.g. pass blocking review bug; won't try to do any of the koji bits until the blocking bug(s) is closed. As the review process goes on, new srpms could be submitted against the same review and have the same automatic bits run.
This certainly isn't intended to automatically review and approve packages, and wouldn't be an excuse for reviewers to just rubber stamp. But it certainly could make the process a little less painful and more reliable by consistently automating those bits that it's feasible to.
-Chris
On Fri, Oct 03, 2008 at 12:21:04PM -0700, Chris Weyl wrote:
So, building on the spirited debate going on with respect to a suggested review template, is it time for a review-oh-matic type service?
Many of the things we do for reviews are the same. Check it builds in mock (koji scratch), check the srpm sources against upstream, look @ rpmlint, look at the file lists, etc, etc... Wouldn't it make all our lives to have a simple little tool that a packager could submit a srpm for review to. It could then, say (and off the top of my head):
- extract the spec, and push it + the srpm somewhere
- file a review bug
- kick off a koji scratch build
- post a link to the build, success/failure, build logs (a la the
FTBFS overlord) ==> success, pull the built rpms, and post to the review bug:
- rpmlint, requires/provides
- md5/sha1 of srpm provided source against upstream
- a partial template, good bad, etc.
If you want to get really clever make a database to back this whole thing. Automate as many of the review points as possible and put the results into the database. Leave humans to review the points which need intelligent thought, and have some way to record their decisions for each point, if clarification was needed. eg for an unusual license the DB might storage a comment providing justification for the license being valid in Fedora. Or for each rpmlint check record the pass/warn/error state, and for ones which don't pass again allow for the review to addd a comment justifying why the warn/error is ok to be ignored, or why it must be fixed.
Now fast forward a short while, and have it automatically re-run all checks for the automatable review points and report on changes. We focus alot on initial review, but have very little, if any, oversight' for ongoing package changes. Plenty of specfiles bit-rot and drift out of compliance with review guidelines.
Of course this is all a huge job and I'm not volunteering to write it, but I think more formal automation in the package review process would be very beneficial, particularly if it could help us track or identify changes post-review.
Daniel
On Fri, Oct 3, 2008 at 12:55 PM, Daniel P. Berrange berrange@redhat.com wrote:
On Fri, Oct 03, 2008 at 12:21:04PM -0700, Chris Weyl wrote:
So, building on the spirited debate going on with respect to a suggested review template, is it time for a review-oh-matic type service?
[...snip...]
Of course this is all a huge job and I'm not volunteering to write it, but I think more formal automation in the package review process would be very beneficial, particularly if it could help us track or identify changes post-review.
Also a good project -- but one, I think, whose scope and goals are different than a "just automate the brainless, repetitive tasks" sorta deal :-)
-Chris
packaging@lists.fedoraproject.org
-
Chris Weyl -
Daniel P. Berrange -
Patrice Dumas