On Tue, 03 Sep 2013 08:38:42 -0400 Stephen Gallagher sgallagh@redhat.com wrote:

On 09/01/2013 03:09 PM, Michael Schwendt wrote:

...

https://bugzilla.redhat.com/1003196 So far, it has always been a packaging mistake to duplicate files (and their Provides as a consequence) in multiple subpackages.

Well, there are a few places where I can see duplicating files making sense (but certainly not to the degree demonstrated in the mate packages).

For example, in the SSSD package, we duplicate the 'sssd_pac' libexec binary in both the 'sssd-provider-ad' and 'sssd-provider-ipa' plugin subpackages, rather than add useless metadata for an extra common subpackage for both to depend on. It seems wasteful to have a whole subpackage for one 150k binary.

You may need to duplicate license files, but I really can't see why you would duplicate binaries. It just makes no sense. That's what packages are for: eliminating redundancies and tracking dependency info.

On Wed, 4 Sep 2013 07:47:55 -0700, Toshio Kuratomi wrote:

https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#...

Embarrassing. :) I was so focused on rpmbuild's "file listed twice" warning that I didn't find this section in the guidelines. And rpmbuild's warning is only per subpackage. It doesn't warn, if a file is included in multiple subpackages.

So that would also be a packaging mistake. It's been many years since this was last touched though. IIRC, mschwendt raised the last issue with it so he may be best able to recall the justifications for this rule and whether the FPC should consider relaxing it.

Relaxing it would be bad.

I agree with the interest in including a small file (other than the license file or small %doc files) in multiple subpackages. That's okay. It doesn't cause any RPM file conflicts.

However, it boils down to: The packager must know what he/she is doing, and particularly must be aware of the consequences. For example, if the duplicate files result in duplicate Provides (e.g. for shared library names), this may affect dependencies in non-obvious ways (e.g. pulling in the wrong package because it happens to include a copy of the needed lib but missing other run-time bits).

To build multiple -devel subpackages, which contain exactly the same set of files, only adds confusion and doesn't add any value at all.

To build multiple subpackages, and in the base package include copies of the files from those subpackages, doesn't make any sense. And it's a pitfall. In bugzilla I've added a comment on the scenario where you want to rename/replace a subpackage.

Originally, avoiding duplicate files in %files sections has been made a checklist item, because of the risks and to have packagers verify painstakingly which file belongs into which (sub)package. Not because duplicated files are always a grave mistake, but because avoiding packaging mistakes (accidents) leads to cleaner packages. Examples: Splitting off a noarch -data subpackage is of no benefit, if the base package includes the data files, too. Easy to fix, though. Splitting off individual plugin subpackages is one of the pitfalls, if the base package contains copies of all plugins (just think about how to recover from that).

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

On 09/04/2013 10:47 AM, Toshio Kuratomi wrote:

On Tue, Sep 03, 2013 at 08:38:42AM -0400, Stephen Gallagher wrote:

...

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

On 09/01/2013 03:09 PM, Michael Schwendt wrote:

...

https://bugzilla.redhat.com/1003196

Based on this suspicious output

mate-dictionary from mate-utils provides libmatedict.so.6()(64bit) mate-utils from mate-utils provides libmatedict.so.6()(64bit) required by: mate-dictionary-devel-1.6.0-7.fc20.x86_64 required by: mate-utils-devel-1.6.0-7.fc20.x86_64

I've only verified in koji that lots of files are included in both sub-packages. Even the descriptions overlap.

And there are even more subpackages, which only contain copies of files included in the base mate-utils package already. Why is that done? Why aren't RPM dependencies used to have the base-package depend on the multiple subpackages?

So far, it has always been a packaging mistake to duplicate files (and their Provides as a consequence) in multiple subpackages.

Well, there are a few places where I can see duplicating files making sense (but certainly not to the degree demonstrated in the mate packages).

For example, in the SSSD package, we duplicate the 'sssd_pac' libexec binary in both the 'sssd-provider-ad' and 'sssd-provider-ipa' plugin subpackages, rather than add useless metadata for an extra common subpackage for both to depend on. It seems wasteful to have a whole subpackage for one 150k binary.

https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#...

So that would also be a packaging mistake. It's been many years since this was last touched though. IIRC, mschwendt raised the last issue with it so he may be best able to recall the justifications for this rule and whether the FPC should consider relaxing it.

For the record, I sent a patch to the SSSD upstream today to add a new common sub-package for just that one file. It'll be cleaned up in the next build.

On Sat, 7 Sep 2013 17:55:46 +0200, Till Maas wrote:

On Wed, Sep 04, 2013 at 07:47:55AM -0700, Toshio Kuratomi wrote:

...

https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#...

So that would also be a packaging mistake. It's been many years since this was last touched though. IIRC, mschwendt raised the last issue with it so he may be best able to recall the justifications for this rule and whether the FPC should consider relaxing it.

The python guidelines contain an example where %doc is used to include the same doc files both in the python2 and python3 package:

https://fedoraproject.org/wiki/Packaging:Python#Example_spec_file

It sounds like it should be mentioned as an exception in the duplicate files guidelines.

Or drop that section from the Packaging and Review Guidelines and replace it with something else. Exceptions soften the guidelines to a degree they become useless. It's a subsection of "File and Directory Ownership":

| In most cases, it should not be necessary for multiple packages to | contain identical copies of the same file. However, if it is necessary,

... that's the backdoor already ... ;)

| multiple packages may contain identical copies of the same file, as long | as the following requirements are met: | | * The packages sharing ownership of the identical files are built from | a single SRPM. | | OR | | * The packages sharing ownership of the identical files are not in a | dependency chain

... sometimes we _create_ package inter-dependencies when we split off -common packages, for example - or a meta-package that depends on multiple packages ...

| (e.g. if package A requires package B, they should | not both contain identical files, either A or B must own the common | files, but not both.)

The ReviewGuidelines refer to it in a second place:

| MUST: Packages must not own files or directories already owned by other | packages. The rule of thumb here is that the first package to be installed | should own the files or directories that other packages may rely | upon. This means, for example, that no package in Fedora should ever share | ownership with any of the files or directories owned by the filesystem or | man package. If you feel that you have a good reason to own a file or | directory that another package owns, then please present that at package | review time.

That links the same "File and Directory Ownership" section and covers subpackages, too, of course.

We need to ask ourselves, do we want to create tiny subpackages for every single file that may be shared by several subpackages? Apparently not according to above section and the exceptions for license/%doc files. And we already have exceptions for directory ownership, too.

A proposal only for the Review Guidelines:

MUST: The spec %files lists must be crafted carefully to ensure that no files or directories get included in the wrong packages.

MUST: Don't duplicate files in more than one subpackage, unless there is a specific requirement to do so. If there is, give the rationale in a comment in the spec file. If it's many files, splitting of a shared package would be preferred.

Known cases here are: the subpackage licensing guidelines, multiple alternative packages that may want to include the same files and/or a few key %doc files.