On 10/25/2016 10:53 AM, Paul Bolle wrote:

On Tue, 2016-10-25 at 10:46 -0700, Laura Abbott wrote:

...

Anyone have experiences with or opinions about the kernel configuration generation? The goal is to only change the way the configurations are generated and not the options that are enabled.

Naive question: why can't we use one .config per target?

Paul Bolle

This is one of the options being considered. The feedback I got was that it might end up being more work to maintain because there would many large files which all need to be modified. New options would need to be added all of them.

Thanks, Laura

On Tue, Oct 25, 2016 at 02:14:08PM -0400, Don Zickus wrote:

On Tue, Oct 25, 2016 at 10:46:00AM -0700, Laura Abbott wrote:

...

The Fedora kernel has had roughly the same system for generating the kernel configuration for a very long time. There are a series of files listing configuration choices (CONFIG_FOO=y, CONFIG_FOO is not set etc.) that get combined to generate the final config files. This has gotten unsustainable for several reasons:

• When the system was first introduced, the only supported

arches were x86_32 and x86_64. Fedora now supports enough other arches that we have a config-$arch-generic in addition to config-generic

• It's difficult to tell what is actually enabled since

there are several layers of configuration combining (I have to look at config-generic, then config-$arch-generic, then the final config-$specific file to see what the option actually is)

• Keeping the files organized requires manual work and pruning

I've been thinking about alternatives to the existing config generation. One proposal was to take advantage of the upstream kernel now supporting config fragments and keep some part of the fedora configuration upstream. This would have the disadvantage of requiring the configuration to be kept in sync with upstream.

Another option is to switch to a system of generation where each configuration option is kept in a separate file. There is no sorting or organization necessary. This would result in a lot of small files for all the arches Fedora supports though.

Hi Laura,

Just to throw it out there, RHEL has been using the one option per file mechanism for years now with success. Minimizes the maintenance and conflicts. ( I know you already know that, just wanted to publicly state that).

The volume of files is large, but it is hidden away and you only package the resulting kernel.config files into the src.rpm.

I'm quite fond of the was things were done in el7 too, but I'm biased, since I'm the one that implemented it. ;)

Honestly though, part of the reason for doing it was because those various stacked config hunks were *terrible* to deal with in el6, and people constantly touched the wrong one, had no idea how the end configs were actually compiled, etc., and I don't think anyone has ever gotten it wrong with the approach used now in el7. In the end, the srpm uses a merged config for each kernel build/arch, so it's simple for people doing their own custom builds to modify the right config, and the git tree heirarchy still makes it pretty obvious what's enabled where -- the path from single files to kernel-foo.config is pretty straight-forward and obvious. I don't think I have anything bad to say about this approach, other than "there are a lot of files", and the most difficult part was the initial conversion, making sure no end result config values differred from the prior method.

On Tue, Oct 25, 2016 at 04:17:25PM -0400, John W. Linville wrote:

On Tue, 2016-10-25 at 15:59 -0400, Jarod Wilson wrote:

...

On Tue, Oct 25, 2016 at 02:14:08PM -0400, Don Zickus wrote:

...

On Tue, Oct 25, 2016 at 10:46:00AM -0700, Laura Abbott wrote:

...

The Fedora kernel has had roughly the same system for generating the kernel configuration for a very long time. There are a series of files listing configuration choices (CONFIG_FOO=y, CONFIG_FOO is not set etc.) that get combined to generate the final config files. This has gotten unsustainable for several reasons:

• When the system was first introduced, the only supported

arches were x86_32 and x86_64. Fedora now supports enough other arches that we have a config-$arch-generic in addition to config-generic

• It's difficult to tell what is actually enabled since

there are several layers of configuration combining (I have to look at config-generic, then config-$arch-generic, then the final config-$specific file to see what the option actually is)

• Keeping the files organized requires manual work and pruning

I've been thinking about alternatives to the existing config generation. One proposal was to take advantage of the upstream kernel now supporting config fragments and keep some part of the fedora configuration upstream. This would have the disadvantage of requiring the configuration to be kept in sync with upstream.

Another option is to switch to a system of generation where each configuration option is kept in a separate file. There is no sorting or organization necessary. This would result in a lot of small files for all the arches Fedora supports though.

Hi Laura,

Just to throw it out there,  RHEL has been using the one option per file mechanism for years now with success.  Minimizes the maintenance and conflicts.  ( I know you already know that, just wanted to publicly state that).

The volume of files is large, but it is hidden away and you only package the resulting kernel.config files into the src.rpm.

I'm quite fond of the was things were done in el7 too, but I'm biased, since I'm the one that implemented it. ;)

Honestly though, part of the reason for doing it was because those various stacked config hunks were *terrible* to deal with in el6, and people constantly touched the wrong one, had no idea how the end configs were actually compiled, etc., and I don't think anyone has ever gotten it wrong with the approach used now in el7. In the end, the srpm uses a merged config for each kernel build/arch, so it's simple for people doing their own custom builds to modify the right config, and the git tree heirarchy still makes it pretty obvious what's enabled where -- the path from single files to kernel-foo.config is pretty straight-forward and obvious. I don't think I have anything bad to say about this approach, other than "there are a lot of files", and the most difficult part was the initial conversion, making sure no end result config values differred from the prior method.

What happens in RHEL7 if a kernel config option disappears (i.e. is eliminated from all Kconfig files)? I don't think I've ever hit that situation, so I honestly don't know.

The reason I ask, of course, is that such a situation seems much more likely with Fedora kernels than with RHEL kernels, since the latter are ostensibly less tumultuous with features and options coming and going. The answer may be to simply sweep the config to eliminate unused options periodically, but it is worth stating that if so.

Ah, yeah, we rarely have a config disappear in el7, and typically, it's from a rename or some other patch that obsoletes it, so we do have to delete the matching file too. That's a bit problematic with a rebasing kernel, but I can see a relatively easy way to sweep for options that have gone by the wayside: just ls the configs/ sub-dir(s) and git grep for them in the kernel source... Means having to have a separate Linus kernel git tree around too, but meh. Should be a simple enough thing to script even, to get a "stale config options" report, the output of which could be fed to a find command that removes them from the configs/ tree...

On 10/25/2016 01:17 PM, John W. Linville wrote:

On Tue, 2016-10-25 at 15:59 -0400, Jarod Wilson wrote:

...

On Tue, Oct 25, 2016 at 02:14:08PM -0400, Don Zickus wrote:

...

On Tue, Oct 25, 2016 at 10:46:00AM -0700, Laura Abbott wrote:

...

The Fedora kernel has had roughly the same system for generating the kernel configuration for a very long time. There are a series of files listing configuration choices (CONFIG_FOO=y, CONFIG_FOO is not set etc.) that get combined to generate the final config files. This has gotten unsustainable for several reasons:

• When the system was first introduced, the only supported

arches were x86_32 and x86_64. Fedora now supports enough other arches that we have a config-$arch-generic in addition to config-generic

• It's difficult to tell what is actually enabled since

there are several layers of configuration combining (I have to look at config-generic, then config-$arch-generic, then the final config-$specific file to see what the option actually is)

• Keeping the files organized requires manual work and pruning

I've been thinking about alternatives to the existing config generation. One proposal was to take advantage of the upstream kernel now supporting config fragments and keep some part of the fedora configuration upstream. This would have the disadvantage of requiring the configuration to be kept in sync with upstream.

Another option is to switch to a system of generation where each configuration option is kept in a separate file. There is no sorting or organization necessary. This would result in a lot of small files for all the arches Fedora supports though.

Hi Laura,

Just to throw it out there, RHEL has been using the one option per file mechanism for years now with success. Minimizes the maintenance and conflicts. ( I know you already know that, just wanted to publicly state that).

The volume of files is large, but it is hidden away and you only package the resulting kernel.config files into the src.rpm.

I'm quite fond of the was things were done in el7 too, but I'm biased, since I'm the one that implemented it. ;)

Honestly though, part of the reason for doing it was because those various stacked config hunks were *terrible* to deal with in el6, and people constantly touched the wrong one, had no idea how the end configs were actually compiled, etc., and I don't think anyone has ever gotten it wrong with the approach used now in el7. In the end, the srpm uses a merged config for each kernel build/arch, so it's simple for people doing their own custom builds to modify the right config, and the git tree heirarchy still makes it pretty obvious what's enabled where -- the path from single files to kernel-foo.config is pretty straight-forward and obvious. I don't think I have anything bad to say about this approach, other than "there are a lot of files", and the most difficult part was the initial conversion, making sure no end result config values differred from the prior method.

What happens in RHEL7 if a kernel config option disappears (i.e. is eliminated from all Kconfig files)? I don't think I've ever hit that situation, so I honestly don't know.

The reason I ask, of course, is that such a situation seems much more likely with Fedora kernels than with RHEL kernels, since the latter are ostensibly less tumultuous with features and options coming and going. The answer may be to simply sweep the config to eliminate unused options periodically, but it is worth stating that if so.

John

This is a problem we have already. As was pointed out further down, there exists a script that gets run periodically to clean out old configs that are no longer enabled. Similar work would need to happen for the config-per-file model as well. Part of the Outreachy work this past summer was bringing the checking for configs in to the kernel.spec file at build time but that work didn't quite get finished and remains incomplete.

Thanks, Laura