On Tue, Mar 24, 2020 at 12:12:29AM -0000, Michel Alexandre Salim wrote:
Hi,
We run a diverse fleet of Linux laptops and desktops (at Facebook), and sometimes there are regressions that affect some of our fleet but not others.
To pick the latest example:
- pulseaudio 1.3.99.1 (both -1 and -2) breaks Bluetooth support on a Dell XPS 15: https://bugzilla.redhat.com/show_bug.cgi?id=1814556
- but it fixes HDA audio input on ThinkPad T490s and X1 Carbon: https://bodhi.fedoraproject.org/updates/FEDORA-2020-c3e19f5098
We've had similar issues with kernel regressions (e.g. 5.4 kernels had issues with Intel GPUs, and on ThinkPads with Nvidia GPUs).
(Ideally we catch all this before they land -- over the medium term I'm trying to find a way to encourage our users to help test updates)
Would it be possible to keep 2 or 3 versions of the same package in the updates repo, so we can easily keep some of our fleet at a previous version known to work on that particular hardware? And is there a process for proposing this (e.g. file a ticket on Pagure)?
Our workaround right now is to check in the older versions in our internal repo.
So I tilted at this windmill for a while (although from the perspective of rawhide, not updates).
The things that come up:
* It's actually really hard to know what the last 2-3 versions of a package are. koji has no concept of versions, it just has tags. In an ideal world they would be in a nice order in the tag, but there's lots of things that cause this to not be the case. ie, you can get say the last 3 kernels tagged into a tag, but those are always the last 3 version wise. At one point this was very difficult for pungi to do, but might be easier now.
* Keeping 2-3 more packages increases space a great deal. Both updates space and repodata space.
* Keeping 2-3 more packages increases the threat surface about insecure updates. ie, now you could trick someone into downgrading or installing something insecure from the base repo, with this you have 2-3x the chance with all the versions in the updates repo.
Anyhow, I guess this would be something to propose to FESCo (althought they might want discussion on devel list first).
kevin