On 11/9/19 12:51 PM, Jóhann B. Guðmundsson wrote:

Hi

On 11/8/19 4:19 PM, Laura Abbott wrote:

...

Hi,

I'm looking to bring in another piece of downstream work into Fedora sometime next week. The short summary is that Fedora will be bringing in some files to build against a RHEL buildroot but Fedora will still be the same.

If you change a board in a boat is it still the same boat?, If you change 10 boards in the boat is it still the same boat? 100 boards, thousands and in the when you have replaced it all?

This looks like some backward move should not downstream be adopting Fedora not the other way around?

What's the end game here in general and for the other components in the distribution. How long before we be entirely colonized by RHEL?

I can't speak for other components but at least for the kernel this work is attempting to bridge the gap between RHEL and Fedora that's been around for too long and keeps growing. The goal is to help both RHEL and Fedora take fixes and feedback from each other.

In an ideal world, yes downstream would be adopting Fedora and they do to a degree. The bigger issue is that changes happen in downstream and then don't get back into Fedora or only get added back as an afterthought. Some things we don't care about but some are actual bug fixes or fixes we might want. A good example is some downstream work for optimizing module compression in the spec file. This would help Fedora as well but it never got submitted.

Even when RHEL does take what Fedora has done upstream, there have been issues with integration. A good example here is the split into kernel-core and kernel-modules. This worked fine technically but when RHEL picked it up there were some unexpected issues (documentation updates and I think a few scripts)

To try and bridge this gap, we're working to get a src-git tree that works for both Fedora and RHEL. Even without the RHEL component, using a src-git tree for most of our daily work in Fedora would be a good thing. dist-git tends to trip up people who want to build their own kernels, especially if they are trying to build something non-trivial (e.g. an entire graphics branch). src-git is what most work is done in anyway. The RHEL kernel tree has been src-git based for a while so aligning on that makes sense.

I realize my wording about "Fedora will still be the same" was vague. None of this work should meaningfully impact the kernel binaries generated by the Fedora build process. Kernel configuration options are still the same, external modules should still build. Parts of the build process may change and some extra files will be packaged. So yes some things _will_ change but it should be changes that are transparent to users or have minimal impact. If these changes cause regressions or are not acceptable for other reasons, I'm more than willing to fix or revert. One of the changes I brought in (weak-updates) caused a regression due to missing a fix in kmod. I reverted it for now and will be bringing it back when things are built.

Part of "Fedora will still be the same" means that our current policies are still in place. Fedora still has the freedom to make changes how it wants. We can still turn on configuration options and make build changes as we need. We're not getting RHEL process (cheering from the background). What we'd like to change is to be more thoughtful about what will happen for RHEL down the road. This means that when Fedora sets configuration options and build changes, we'll be reviewing what we want to happen for RHEL as well. This doesn't mean RHEL gets to come in an dictate what happens but like other community members, RHEL can say "I see you are making change X. This impacts me in way Y. Can we consider change Z instead to lessen my impact?". Sometimes the answer is no, we're doing X and then RHEL gets to figure out how to workaround this. We already see this happening with things like configuration options (e.g. CONFIG_LIBNVDIMM https://bugzilla.redhat.com/show_bug.cgi?id=1696481) This freedom to make changes makes it easier for downstream as well since we can say "well Fedora has had CONFIG_BLAH on for months now and there have been no complaints".

Fedora has an incredibly valuable community and the goal here is not to subvert it but make sure we can bring the benefits of that community back to downstream in the kernel. I also look forward to bringing more downstream kernel developers into Fedora so they can continue making changes and fixes directly in Fedora.

Thanks, Laura

On Tue, Nov 12, 2019 at 01:55:05AM +0000, Jóhann B. Guðmundsson wrote:

Originally it was the agility of the Fedora distribution, which reflected the industry direction through the community contribution in the project, which gave RH that valuable business insight to build upon and marketing advantage that came thereof but now the fact is, these changes to the project in whole are bubbling up from RH(EL) not tripling down from Fedora's community and all those RH changes that started small as you are proposing/doing have grown into this unmovable monstrosity of chopped fragmented blob resembling an distribution that is only move by "if it builds, let's ship it" chants.

Let's not overgeneralize. If the RHEL spec has some changes that are also beneficial in Fedora (e.g. the module compression work), than we want to merge them "up". If there are differences between the Fedora and RHEL specs that make moving fixes back and forth awkward and make the life of kernel maintainers harder, then we want to synchronize the specs. We're not talking about developing some "technology" here, and while I agree that in general it is better to develop new code upstream, this hardly applies to a spec file.

(Also, in general, when stuff was developed downstream in the past, let's applaud the people who want to move it upstream, not make it sound like they are doing something wrong.)

Zbyszek