#89: Improve tracking blocker review status ---------------------------+------------------------------------------------ Reporter: jlaska | Owner: poelstra Type: enhancement | Status: new Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Keywords: retrospective | ---------------------------+------------------------------------------------ = problem =
Several bugs were changed so they fell off the tracking lists for Fedora 13. For example... * [http://bugzilla.redhat.com/505189 505189 - Going back to repo UI screen and and modifying Installation Repo causes traceback] * [http://bugzilla.redhat.com/577803 577803 - Adding repository requiring network fails if network is down]
= analysis =
In F-13, a bugzilla keyword was used to denote blocker status. However, aside from reviewing bugzilla comments, there was no query-able method to determine whether a blocker request is open, approved or denied. This led to several Fedora 13 bugs that were 1) fixed in Rawhide, 2) Moved to MODIFIED or CLOSED and not included, but not included in Fedora 13 (e.g. RHBZ #505189 and RHBZ #577803).
Not knowing which bugs were already reviewed, also introduced time wasted reviewing previously reviewed bugs during blocker review meetings.
= enhancement recommendation =
Recommend reviewing process changes to avoid the scenarios leading up to RHBZ #505189 and RHBZ #577803. Some options discussed so far include 1. Document usage of bugzilla flags (suggested by jkeating) to track blocker requests. This likely involves creating a ''bot'' to police the tags? 2. Hardening the current keyword-based mechanism.
Additional discussion points include: 1. Document generating exception report and document action plan for CLOSED Blocker bugs that did not go through VERIFIED. 2. More frequent nag mails leading up to release milestones. Notification of NEW or ASSIGNED bugs goes to the maintainer, and MODIFIED bugs goes to QA.
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: new Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by poelstra):
1) We ran out of time and manpower to propose or implement the bugzilla flags that had been discussed on the list. Given what we've seen so far in the Fedora 14 release cycle I'm not convinced that flags would speed our process up very much. The greatest amount of time lag seems to be related to getting clear communication from the package maintainers as to where they are with fixing the bug. Flags will not fix this problem.
2) Blocker meetings appear to be running smoothly and we have started using the whiteboard keyword "AcceptedBlocker" when the people at the meeting determine that a bug has met the specified release criteria. This process happens fairly quickly at the meetings. I'm not convinced it would happen any faster if we waited for the responsible parties to set the flags outside of the meeting.
3) I have an inquiry into the bugzilla team to see if there is a way to get an exception report of blocker bugs that have not transitioned through the VERIFIED state.
4) We added a one week blocker bug nagging period before the Alpha, Beta, and Final change deadlines to the schedule. It seemed to work okay for the Fedora 14 Alpha, but getting rapid maintainer response was difficult.
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: new Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by jlaska):
Replying to [comment:1 poelstra]:
- I have an inquiry into the bugzilla team to see if there is a way to
get an exception report of blocker bugs that have not transitioned through the VERIFIED state.
Any revelations here? I think we're still exposed in this area since bodhi CLOSES bugs when the update ships (regardless of the bug state).
- We added a one week blocker bug nagging period before the Alpha,
Beta, and Final change deadlines to the schedule. It seemed to work okay for the Fedora 14 Alpha, but getting rapid maintainer response was difficult.
We've had a bit more distance from the release, any new ideas/thoughts on the blocker process that we'll want to track for F-15?
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: new Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by poelstra):
Replying to [comment:1 poelstra]:
- We ran out of time and manpower to propose or implement the bugzilla
flags that had been discussed on the list. Given what we've seen so far in the Fedora 14 release cycle I'm not convinced that flags would speed our process up very much. The greatest amount of time lag seems to be related to getting clear communication from the package maintainers as to where they are with fixing the bug. Flags will not fix this problem.
- Blocker meetings appear to be running smoothly and we have started
using the whiteboard keyword "AcceptedBlocker" when the people at the meeting determine that a bug has met the specified release criteria. This process happens fairly quickly at the meetings. I'm not convinced it would happen any faster if we waited for the responsible parties to set the flags outside of the meeting.
Now that Fedora 14 is over I still think the process we ended up with by the end of the release worked well. I still hold the opinion that ROI for the time and energy to implement flags just isn't there.
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Changes (by poelstra):
* status: new => assigned
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by poelstra):
Replying to [comment:2 jlaska]:
Replying to [comment:1 poelstra]:
- I have an inquiry into the bugzilla team to see if there is a way
to get an exception report of blocker bugs that have not transitioned through the VERIFIED state.
Any revelations here? I think we're still exposed in this area since
bodhi CLOSES bugs
when the update ships (regardless of the bug state).
Thanks for the reminder. I didn't get a response to my first query so I'm asking again.
- We added a one week blocker bug nagging period before the Alpha,
Beta, and Final change deadlines to the schedule. It seemed to work okay for the Fedora 14 Alpha, but getting rapid maintainer response was difficult.
We've had a bit more distance from the release, any new ideas/thoughts
on the blocker process that we'll want to track for F-15?
I think we can consider this item along with #1 and #2 closed. Getting timely maintainer response was still an impediment in Beta and Final though we were able to mitigate it successfully enough (to release on time) by reviewing the blocker list and nagging maintainers daily the seven days before the RC compose. So, no, don't really have any other ideas.
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by adamwill):
Mostly I agree with John; we envisaged the AcceptedXXX whiteboard fields as a stop-gap before we introduced flags for F15 cycle, but I think they worked out pretty well and could go long-term. The only thing I'd suggest is making them keywords if we're going to run with them for a while; Whiteboard was fine for a temporary stop-gap system, but for the long term we should set them as keywords to avoid the possibility of typos etc.
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by poelstra):
Replying to [comment:1 poelstra]:
- I have an inquiry into the bugzilla team to see if there is a way to
get an exception report of blocker bugs that have not transitioned through the VERIFIED state.
Here is what I tracked down, thank to information provided by Dave Lawrence. It is possible, but to do so you must provide a list of bug numbers to the advanced query.
To examine bugs blocking the Fedora 14 Beta that did not reach the verified state you'd have to: 1. extract a list of bugs blocking 'f14beta' 2. Feed that list to the advanced query page. It would look something like this: http://bit.ly/dJGRZz
This seems rather cumbersome compared to the value it might provide. Perhaps there are ideas out there on ways this could be automated.
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by bruno):
Does xmlrpc have the same limitation? It seems that one should be able to write a short program to do this.
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by bruno):
I think you can do this query at the advanced query page using boolean charts. Does the output of the following query look like what you want:
https://bugzilla.redhat.com/buglist.cgi?negate0=1&type1-0-0=equals&q...
Note that I couldn't use the F14Beta alias for 611991 in this query.
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by bruno):
I checked the two bug lists from Poelstra's example and my example and found that Poelstra's had one that mine didn't. It doesn't directly block F14Beta, but does block a bug which blocks F14beta. I don't think there is a way to do this with the boolean charts as the block and depend values only check one level and there doesn't seem to be a way build build rules with chains of bugs (or alias expansion for that matter).
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by bruno):
Also worth noting is that Will Woods wrote a program named deptree.py and attached it to https://bugzilla.redhat.com/show_bug.cgi?id=452399 . This also includes the depends on bugs as well as the blocks bugs. The parent bugs could probably be ignored pretty easily. The format of the output is such that it is supposed to be usable in another query. So it could probably be embedded as part of a very simple script. The program also uses what looks like a fixed (though maybe it's just a default, I haven't carefully looked at the script yet) bug of F10Blocker.
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by bruno):
The following query also checks for being currently CLOSED: https://bugzilla.redhat.com/buglist.cgi?negate1=1&type1-0-0=changedto&am...
An important question is for the example bug 608992, which blocks 611991 (F14Beta) but also depends on bug 633827, do we care whether 633827 went through VERIFIED or is only whether or not 608992 did relevant?
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Changes (by bruno):
* cc: bruno (added)
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by jlaska):
Using the script you linked to in bug#452399 .... 1. http://fpaste.org/LF8Q/ -- Generate a list of bugs that depend on a given tracker (I used F14Blocker in my example) 2. http://bit.ly/fEjczx -- Input that list of bugs into the query you provided (btw ... nice, I didn't think about the boolean NOT 'status' 'changed to' filter). I modified the query slightly to restrict the output to Resolution=ERRATA issues
That yields 57 bugs. Quick inspection of a few of those bugs shows that they indeed were not ever in VERIFIED state. I'm completely fine with maintaining a shared bugzilla query that does #2 for us. Periodically throughout the release, as new dependent Tracking bugs are added, we'll need to adjust the query in #2 to include the updated Tacking bugs. That doesn't seem horrible.
So the next question is ... now that we have a list of blocker bugs, fixed by bodhi updates, that weren't tested ... what can we do with that list. I'm inclined to say that the list should serve mainly as another data point during release. We'll periodically monitor the list to make sure any scary issues are VERIFIED? Any other ideas/thoughts?
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by bruno):
I discussed this with James in IRC and I'll be setting up a mockup on my wiki page that shows what I am thinking for this. I'd like to keep the queries clickable (rather than multistep) and the case they miss shouldn't be important. (Normal bug with a dependency on another normal bug.) I will make a link for each tracking bug of interest. I think this will be fairly easy to use. If something supplemental is needed then maybe that can be done in addition.
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by poelstra):
Bruno--thanks so much for your efforts and research. You definitely took it much further than I could.
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by bruno):
I have a mockup for F14 and F15 at:
https://fedoraproject.org/wiki/User:Bruno#Mockup_for_QA_-_Tracking_bug_queri...
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by bruno):
Just to add a bit more to this, I am looking at creating an upstream patch to allow searching for indirect blockers or depends on. Postgres at least, supports with recursive in queries and that allows this to be done without a lot of changes. I am pretty sure doing this is within my skill set. I have a test bugzilla set up at home and will be trying to do it this week. If I do I'll submit it upstream, and if it's not accepted there, then maybe I can talk to our bugzilla people about a local patch for Redhat's server.
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by bruno):
Looking at how to do this, it looks like doing it as a 4.0 extension is probably best. 4.0 is in RC right now, so it may be a bit before Red Hat is using it. But it provides more hooks, so that most if not all of what I need can be handled as an extension instead of changing the core code.
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: assigned Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: | Keywords: retrospective --------------------------+------------------------------------------------- Comment (by bruno):
I am documenting my strategy and status for this project at: https://fedoraproject.org/wiki/User:Bruno#Bugzilla_Extension_development
It looks like the least hackish way to do this is to add some support for views and use an appropriate view in a manner similar to tables. View support should be doable as an alternate database extension. That should be good enough for proof of concept, but may not include all safety checks (as views should be read only). Then the indirect dependency support should be relatively easy to do as a normal extension. I need to need to learn a bit more about doing custom tables. Using aliases in blocks and depends on checks is something else that appears to be relatively easy to do as an extension. It isn't really needed, but would make the queries a bit easier to setup. I am on vacation the 24th through the 2nd and should be able to spend some solid blocks of time during some of that to work on this. I hope to be able to implement this in a way that no core code is touched and that the whole thing is an addon. Then I can submit it upstream as a proof of concept. Also note that I am targeting 4.0 and I am not sure what Red Hat's plans are for updating bugzilla to 4.0. To use it (as a local extension or builtin) we will need to wait on that. 4.0 is currently in release candidate state.
#89: Improve tracking blocker review status --------------------------+------------------------------------------------- Reporter: jlaska | Owner: poelstra Type: enhancement | Status: closed Priority: major | Milestone: Fedora 14 Component: Wiki | Version: Resolution: fixed | Keywords: retrospective --------------------------+------------------------------------------------- Changes (by adamwill):
* status: assigned => closed * resolution: => fixed
Comment:
So for the past few releases we've used the 'ugly quick hack' we set up using whiteboard fields, and notwithstanding how dumb and ugly it is, it seems to work pretty well. We haven't had any actual practical problems caused by 'losing track of' blockers or worrying about non-accepted blockers. We also have the https://fedoraproject.org/wiki/Current_Release_Blockers page which provides info on current blockers in a slightly nicer layout than Bugzilla.
So I'm inclined to consider this fixed. That doesn't mean we can't *improve* the process, but it'd probably work best to consider suggestions for improving the current process as new tickets. Bruno, if you're still interested in a less dumb / more robust way of doing this, by all means, keep working on it, and file a new ticket! Thanks.
-
fedora-badges