There's a rather prominent and obviously serious bug in open-vm-tools right now which causes the daemon to segfault. It seems like each time this happens we get a new bug filed in Bugzilla. So far over 50 bugs have been filed ...
Yes we know, and someone from VMware is working on it, but it probably won't get fixed until the New Year.
Can we temporarily stop new bugs being filed by abrt for this component (open-vm-tools)?
Rich.
On 12/29/2017 09:26 UTC, Richard W.M. Jones wrote:
There's a rather prominent and obviously serious bug in open-vm-tools right now which causes the daemon to segfault. It seems like each time this happens we get a new bug filed in Bugzilla. So far over 50 bugs have been filed ...
0. Exhibit the URL for the bugzilla report against abrt for not recognizing the duplicates, AND for a few of the alleged duplicates. Searching bugzilla for "abrt open-vm-tools", I see only these two recent reports: https://bugzilla.redhat.com/show_bug.cgi?id=1527058 https://bugzilla.redhat.com/show_bug.cgi?id=1526952
1. Roll back open-vm-tools to the most-recent version that does not have the bug.
2. Temporarily remove open-vm-tools from the list of components in bugzilla.
3. Release a version of abrt with a special-case hack that ignores this particular bug.
4. As long as the projected number of bugs to be filed before the VMware fix to open-vm-tools is 3,000 or less, just ignore them.
--
On Fri, 2017-12-29 at 21:26 +0000, Richard W.M. Jones wrote:
There's a rather prominent and obviously serious bug in open-vm-tools right now which causes the daemon to segfault. It seems like each time this happens we get a new bug filed in Bugzilla. So far over 50 bugs have been filed ...
Yes we know, and someone from VMware is working on it, but it probably won't get fixed until the New Year.
Can we temporarily stop new bugs being filed by abrt for this component (open-vm-tools)?
Probably not during the RH holiday shutdown, no, as I would think all the folks in charge of those bits won't be working.
One cause of excessive dupes being filed by abrt (which I suspect is the issue here, but I haven't checked to make sure) is its rather overly-enthusiastic attempts to spot reports which may contain sensitive information. This is a real problem (we've had a few bugs filed which had people's passwords and stuff in the attachments, which is obviously very bad), but in response to that, we would up making abrt crazy cautious, to the point where I think it now believes almost all reports contain sensitive information.
When it thinks this, it defaults to a mode where it wants to make anything submitted to Bugzilla private, and it seems that in this mode, even when it knows a report is a dupe, it won't just add a comment to the parent bug, allegedly because it can't make a comment sufficiently private (I'm not sure on the details of that particular wrinkle). Instead it files a new private bug and immediately closes it as a duplicate of the original bug. You can change this - there's a checkbox during the report workflow which decides the behaviour, when abrt thinks the report contains sensitive data it defaults to being checked, and you can uncheck it if you know it's a false positive - but of course most reporters don't bother with this.
I can think of several things we could do to try and mitigate this (for instance, when just adding a comment to a parent bug abrt doesn't actually *attach* any of the files that might contain sensitive data, so there isn't an awful lot of validity to this behaviour at all), but they'd all sort of require the devs to be around :/
I guess in theory a BZ admin might be able to somehow block the abrt account from filing bugs against a specific component temporarily, but again, the BZ admins are RH staff, and they're probably not working...
On Sat, Dec 30, 2017 at 03:59:13PM -0800, Adam Williamson wrote:
When it thinks this, it defaults to a mode where it wants to make anything submitted to Bugzilla private, and it seems that in this mode, even when it knows a report is a dupe, it won't just add a comment to the parent bug, allegedly because it can't make a comment sufficiently private (I'm not sure on the details of that particular wrinkle).
The new bugs are private to ‘fedora_contrib_private’, but ...
Instead it files a new private bug and immediately closes it as a duplicate of the original bug. You can change this - there's a checkbox during the report workflow which decides the behaviour, when abrt thinks the report contains sensitive data it defaults to being checked, and you can uncheck it if you know it's a false positive - but of course most reporters don't bother with this.
.. I don't believe it's closing these as duplicates, so maybe it's not this?
Here's an example of yet another new bug (which you won't be able to see unless you are in ‘fedora_contrib_private’):
https://bugzilla.redhat.com/show_bug.cgi?id=1529983
I can think of several things we could do to try and mitigate this (for instance, when just adding a comment to a parent bug abrt doesn't actually *attach* any of the files that might contain sensitive data, so there isn't an awful lot of validity to this behaviour at all), but they'd all sort of require the devs to be around :/
I guess in theory a BZ admin might be able to somehow block the abrt account from filing bugs against a specific component temporarily, but again, the BZ admins are RH staff, and they're probably not working...
The good news is that Ravindra from VMware has just pushed an update which ought to fix this mess. Here's hoping :-)
Rich.
On Sun, 2017-12-31 at 16:34 +0000, Richard W.M. Jones wrote:
On Sat, Dec 30, 2017 at 03:59:13PM -0800, Adam Williamson wrote:
When it thinks this, it defaults to a mode where it wants to make anything submitted to Bugzilla private, and it seems that in this mode, even when it knows a report is a dupe, it won't just add a comment to the parent bug, allegedly because it can't make a comment sufficiently private (I'm not sure on the details of that particular wrinkle).
The new bugs are private to ‘fedora_contrib_private’, but ...
Instead it files a new private bug and immediately closes it as a duplicate of the original bug. You can change this - there's a checkbox during the report workflow which decides the behaviour, when abrt thinks the report contains sensitive data it defaults to being checked, and you can uncheck it if you know it's a false positive - but of course most reporters don't bother with this.
.. I don't believe it's closing these as duplicates, so maybe it's not this?
Sounds like it's not, then, indeed - in the cases I'd seen, the new bugs immediately got closed as dupes. So it's probably a case of it not correctly detecting duplicates. I usually file those here:
devel@lists.stg.fedoraproject.org
-
Adam Williamson -
John Reiser -
Richard W.M. Jones