Hey all, I've just added a draft proposal for limiting file dependencies in packages. It's a "should" recommendation at this point.
http://www.fedoraproject.org/wiki/PackagingDrafts/FileDeps
= File Deps Guideline =
== Problem ==
Resolving file dependencies can be a bottleneck when running yum. With our current repodata format, information on which package owns files in /etc and the bin directories (/bin /sbin /usr/bin /usr/sbin) are kept in primary.xml. All other file deps are kept in filelist.xml. This means that anytime yum has to resolve a file dep in /etc or a bin directory it can just download primary.xml to resolve it. If the file dep is outside of those directories, yum has to download and parse a second xml file, filelist.xml, which is much larger than primary.xml.
Fedora 6 repodata compressed and uncompressed: || ||primary.xml||filelist.xml|| ||Core ||1M/7.6M ||3.1M/39.2M|| ||Extras ||4.7M/11.7M ||4M/50.8M|| ||Updates||0.4M/4.2M ||2.1M/27.5M||
The compressed size has direct bearing on the additional time it will take to download the additional filelist.xml. The uncompressed size is one indicator of how much additional time is needed to parse for the file that satisfies the dependency. If yum does not need to refresh its primary.xml and filelist.xml (because it downloaded a copy previously and the repository has not been updated since), yum won't have to parse a new file and the filelist info will be quicker to find (as the file will have been parsed into an sqlite database), however this is the exception rather than the rule.
If we can avoid having to download and parse filelist.xml, we can greatly speed up yum's dependency resolution times.
== Proposed Guideline ==
'''SHOULD''': If the package has file dependencies outside of /etc, /bin, /sbin, /usr/bin, or /usr/sbin consider requiring the package which provides the file instead of the file itself.
Using file dependencies outside of /etc, /bin, /sbin, /usr/bin, or /usr/sbin requires yum (and other depsolvers using the repomd format) to download and parse a large xml file looking for the dependency. Helping the depsolvers avoid this processing by depending on the package instead of the file saves our end users a lot of time.
-Toshio
On Wed, 20 Dec 2006, Toshio Kuratomi wrote:
'''SHOULD''': If the package has file dependencies outside of /etc, /bin, /sbin, /usr/bin, or /usr/sbin consider requiring the package which provides the file instead of the file itself.
Using file dependencies outside of /etc, /bin, /sbin, /usr/bin, or /usr/sbin requires yum (and other depsolvers using the repomd format) to download and parse a large xml file looking for the dependency. Helping the depsolvers avoid this processing by depending on the package instead of the file saves our end users a lot of time.
Just FWIW, this doesn't actually help smart and apt at all, unless file dependencies outside what's recorded in primary.xml are outright banned in the repodata specification itself by refusing to add dependencies to other paths than what's in primary.xml. Yum is the only one of the three doing dependency resolution "on demand", both apt and smart precalculate the full dependency tree for all operations so they need all the information at all times.
Note that I've absolutely nothing against this proposal, this helps yum (meaning most Fedora users) without hurting the others so nothing wrong with it. File dependencies are a mixed blessing...
- Panu -
On Thu, Dec 21, 2006 at 12:59:37PM +0200, Panu Matilainen wrote:
On Wed, 20 Dec 2006, Toshio Kuratomi wrote:
'''SHOULD''': If the package has file dependencies outside of /etc, /bin, /sbin, /usr/bin, or /usr/sbin consider requiring the package which provides the file instead of the file itself.
Using file dependencies outside of /etc, /bin, /sbin, /usr/bin, or /usr/sbin requires yum (and other depsolvers using the repomd format) to download and parse a large xml file looking for the dependency. Helping the depsolvers avoid this processing by depending on the package instead of the file saves our end users a lot of time.
Just FWIW, this doesn't actually help smart and apt at all,
Why? If unneccessary file dependencies are removed the files sizes get smaller. Of course, yum has the larger benefits, but apt/smart/etc would benefit from say cutting off half of the file dependencies, too.
The only problem I see is to identify when some file dependencies were used "unneccessarily". Using file dependencies was never really preferred over proper package depedencies, so most likely the packager really wanted file dependencies when he explcitely used them.
That means w/o a proper analysis of what we currently have, we shouldn't simply ban them. Instead an analysis would show why packagers used this method and whether we can offer better workarounds for what they wanted to achieve.
On Thu, 21 Dec 2006, Axel Thimm wrote:
On Thu, Dec 21, 2006 at 12:59:37PM +0200, Panu Matilainen wrote:
On Wed, 20 Dec 2006, Toshio Kuratomi wrote:
'''SHOULD''': If the package has file dependencies outside of /etc, /bin, /sbin, /usr/bin, or /usr/sbin consider requiring the package which provides the file instead of the file itself.
Using file dependencies outside of /etc, /bin, /sbin, /usr/bin, or /usr/sbin requires yum (and other depsolvers using the repomd format) to download and parse a large xml file looking for the dependency. Helping the depsolvers avoid this processing by depending on the package instead of the file saves our end users a lot of time.
Just FWIW, this doesn't actually help smart and apt at all,
Why? If unneccessary file dependencies are removed the files sizes get smaller. Of course, yum has the larger benefits, but apt/smart/etc would benefit from say cutting off half of the file dependencies, too.
Looking up a file dependency is just as cheap as looking up a package dependency in apt and smart (AFAIK, haven't studien smart in too much detail), once the initial crunching through the available files is done.
Throwing out *unnecessary* dependencies (file or package) is obviously good, even if it's hardly noticeable in the filesizes in reality. This simply doesn't change the fact that apt and smart need to download the filelists.xml along with primary.xml at all times to guarantee working with any old repomd repository.
- Panu -
On Thu, Dec 21, 2006 at 04:25:10PM +0200, Panu Matilainen wrote:
This simply doesn't change the fact that apt and smart need to download the filelists.xml along with primary.xml at all times to guarantee working with any old repomd repository.
What is an "old" and "new" repomd?
I have two questions:
1) If this is a problem that only affects yum and all the other depsolvers do it efficiently, would it be better to fix yum instead or working around it by spec file changes?
2) If we still use file deps for things in /bin /sbin /usr/bin /usr/sbin, which happens for all GCJ compiled packages, wouldn't it force the reading of the second file anyways?
regards, Fernando
Panu Matilainen wrote:
On Thu, 21 Dec 2006, Axel Thimm wrote:
On Thu, Dec 21, 2006 at 12:59:37PM +0200, Panu Matilainen wrote:
On Wed, 20 Dec 2006, Toshio Kuratomi wrote:
'''SHOULD''': If the package has file dependencies outside of /etc, /bin, /sbin, /usr/bin, or /usr/sbin consider requiring the package which provides the file instead of the file itself.
Using file dependencies outside of /etc, /bin, /sbin, /usr/bin, or /usr/sbin requires yum (and other depsolvers using the repomd format) to download and parse a large xml file looking for the dependency. Helping the depsolvers avoid this processing by depending on the package instead of the file saves our end users a lot of time.
Just FWIW, this doesn't actually help smart and apt at all,
Why? If unneccessary file dependencies are removed the files sizes get smaller. Of course, yum has the larger benefits, but apt/smart/etc would benefit from say cutting off half of the file dependencies, too.
Looking up a file dependency is just as cheap as looking up a package dependency in apt and smart (AFAIK, haven't studien smart in too much detail), once the initial crunching through the available files is done.
Throwing out *unnecessary* dependencies (file or package) is obviously good, even if it's hardly noticeable in the filesizes in reality. This simply doesn't change the fact that apt and smart need to download the filelists.xml along with primary.xml at all times to guarantee working with any old repomd repository.
- Panu --- Fedora-packaging mailing list Fedora-packaging@redhat.com https://www.redhat.com/mailman/listinfo/fedora-packaging
Fernando Nasser wrote:
I have two questions:
- If this is a problem that only affects yum and all the other
depsolvers do it efficiently, would it be better to fix yum instead or working around it by spec file changes?
It's still generally a good idea to minimize file deps.
- If we still use file deps for things in /bin /sbin /usr/bin
/usr/sbin, which happens for all GCJ compiled packages, wouldn't it force the reading of the second file anyways?
No, just for items *outside* of those locations does yum need extra processing. That's the whole point of this exercise.
-- Rex
Rex Dieter wrote:
Fernando Nasser wrote:
I have two questions:
- If this is a problem that only affects yum and all the other
depsolvers do it efficiently, would it be better to fix yum instead or working around it by spec file changes?
It's still generally a good idea to minimize file deps.
Sure, but there are cases where it is desirable. I am afraid a dull guideline will put a straight jacket on developers.
For instance, I know a couple of folks working on a software that in the current release needs two independent packages. Another developer needs one (a single one) file from them, but the file may change places. They thought of adding a virtual provides, but it seems overkill in this case and it is probably only temporary (which would mean we would use a name unnecessarily and regret later). So the best way around it was to require the file, at least until the set of packages stabilize on a future version of the dependency.
Perhaps a "recommendation" guideline? (if there is such a thing)
- If we still use file deps for things in /bin /sbin /usr/bin
/usr/sbin, which happens for all GCJ compiled packages, wouldn't it force the reading of the second file anyways?
No, just for items *outside* of those locations does yum need extra processing. That's the whole point of this exercise.
Ah, thanks for the clarification. The use of file deps prevents an optimization to kick in.
Is this list complete: /bin /sbin /usr/bin /usr/sbin ? Or there are others that do not cause the loading of the second file?
Regards, Fernando
On Thu, 2006-12-21 at 12:13 -0500, Fernando Nasser wrote:
Rex Dieter wrote:
Fernando Nasser wrote:
I have two questions:
- If this is a problem that only affects yum and all the other
depsolvers do it efficiently, would it be better to fix yum instead or working around it by spec file changes?
It's still generally a good idea to minimize file deps.
Actually, what's happening is that all depsolvers have the ability to optimize two time consuming steps if the file deps follow these guidelines. The two steps are 1) downloading filelist.xml.gz and 2) parsing filelist.xml to find the package which provides the file deps. [And this problem is multiplied the more repositories you have defined as the depsolver will have to grab the filelist.xml file from multiple repositories].
Panu points out that only yum currently optimizes this but there's nothing stopping the others doing it as well.
Sure, but there are cases where it is desirable. I am afraid a dull guideline will put a straight jacket on developers.
For instance, I know a couple of folks working on a software that in the current release needs two independent packages. Another developer needs one (a single one) file from them, but the file may change places. They thought of adding a virtual provides, but it seems overkill in this case and it is probably only temporary (which would mean we would use a name unnecessarily and regret later). So the best way around it was to require the file, at least until the set of packages stabilize on a future version of the dependency.
Perhaps a "recommendation" guideline? (if there is such a thing)
Yes. This proposal is for a "Should" rather than a "Must" guideline. In the case you outline, I'd like to see the package depend on the file until it was settled which package would provide it. Then the dependency could be converted to a package dep instead. We don't currently have an ongoing QA requirement, though, so remembering to make the change would largely be the responsibility of the packager.
- If we still use file deps for things in /bin /sbin /usr/bin
/usr/sbin, which happens for all GCJ compiled packages, wouldn't it force the reading of the second file anyways?
No, just for items *outside* of those locations does yum need extra processing. That's the whole point of this exercise.
Ah, thanks for the clarification. The use of file deps prevents an optimization to kick in.
Correct.
Is this list complete: /bin /sbin /usr/bin /usr/sbin ? Or there are others that do not cause the loading of the second file?
/etc is the other one.
That's why the guideline applies to file deps outside of /etc /bin /sbin /usr/bin and /usr/sbin.
-Toshio
Toshio Kuratomi wrote:
On Thu, 2006-12-21 at 12:13 -0500, Fernando Nasser wrote:
Rex Dieter wrote:
Fernando Nasser wrote:
I have two questions:
- If this is a problem that only affects yum and all the other
depsolvers do it efficiently, would it be better to fix yum instead or working around it by spec file changes?
It's still generally a good idea to minimize file deps.
Actually, what's happening is that all depsolvers have the ability to optimize two time consuming steps if the file deps follow these guidelines. The two steps are 1) downloading filelist.xml.gz and 2) parsing filelist.xml to find the package which provides the file deps. [And this problem is multiplied the more repositories you have defined as the depsolver will have to grab the filelist.xml file from multiple repositories].
Panu points out that only yum currently optimizes this but there's nothing stopping the others doing it as well.
OK
Sure, but there are cases where it is desirable. I am afraid a dull guideline will put a straight jacket on developers.
For instance, I know a couple of folks working on a software that in the current release needs two independent packages. Another developer needs one (a single one) file from them, but the file may change places. They thought of adding a virtual provides, but it seems overkill in this case and it is probably only temporary (which would mean we would use a name unnecessarily and regret later). So the best way around it was to require the file, at least until the set of packages stabilize on a future version of the dependency.
Perhaps a "recommendation" guideline? (if there is such a thing)
Yes. This proposal is for a "Should" rather than a "Must" guideline. In the case you outline, I'd like to see the package depend on the file until it was settled which package would provide it. Then the dependency could be converted to a package dep instead. We don't currently have an ongoing QA requirement, though, so remembering to make the change would largely be the responsibility of the packager.
That will work for them. I think it is a sensible approach. Perhaps adding a FIXME to the specfile above that dependency would be a good idea.
- If we still use file deps for things in /bin /sbin /usr/bin
/usr/sbin, which happens for all GCJ compiled packages, wouldn't it force the reading of the second file anyways?
No, just for items *outside* of those locations does yum need extra processing. That's the whole point of this exercise.
Ah, thanks for the clarification. The use of file deps prevents an optimization to kick in.
Correct.
Is this list complete: /bin /sbin /usr/bin /usr/sbin ? Or there are others that do not cause the loading of the second file?
/etc is the other one.
Thanks. I guess the list of exceptions will be in the guidelines.
That's why the guideline applies to file deps outside of /etc /bin /sbin /usr/bin and /usr/sbin.
-Toshio
Thanks for the clarifications
Fernando
On Thu, 21 Dec 2006, Toshio Kuratomi wrote:
On Thu, 2006-12-21 at 12:13 -0500, Fernando Nasser wrote:
Rex Dieter wrote:
Fernando Nasser wrote:
I have two questions:
- If this is a problem that only affects yum and all the other
depsolvers do it efficiently, would it be better to fix yum instead or working around it by spec file changes?
It's still generally a good idea to minimize file deps.
Actually, what's happening is that all depsolvers have the ability to optimize two time consuming steps if the file deps follow these guidelines. The two steps are 1) downloading filelist.xml.gz and 2) parsing filelist.xml to find the package which provides the file deps. [And this problem is multiplied the more repositories you have defined as the depsolver will have to grab the filelist.xml file from multiple repositories].
Panu points out that only yum currently optimizes this but there's nothing stopping the others doing it as well.
Because of their very design, smart and apt CAN NOT optimize it as long as the repodata format permits arbitrary file dependencies. That's what I've been trying to say here.
Assume for a moment that FC + FE are cleaned up from all file dependencies outside of those present in primary.xml. Sure, it'd be possible to make a fedora-specific one-liner patch to avoid downloading the filelists.xml and that would "work." But as long as it's just a Fedora policy and not forced by the tools, things would break as soon as any 3rd party repository not enforcing that particular Fedora policy is used (Axel, this is what I meant with "any old repository", no such thing as "new repodata" at the moment)
What does it mean? Heck, nothing really. Apt and smart have a radically different internal design from that of yum, each has their own benefits. The partial filelist info within primary.xml is really what I'd call a RH/Fedora specific performance hack, based on rough "these directories cover 99% of the filedeps in practice" heuristic. Which is all and well, except it's annoyingly close-but-no-cigar to working for apt and smart as well, they need to download the huge filelists.xml only because of that remaining 1% :)
Again, cleaning up obscure file dependencies is good, nothing against this thing at all. I just wanted to correct the idea that it would help "the other" depsolvers as well - it doesn't, but that doesn't make the idea bad.
- Panu -
On Thu, Dec 21, 2006 at 11:39:38AM -0500, Fernando Nasser wrote:
- If this is a problem that only affects yum and all the other
depsolvers do it efficiently, would it be better to fix yum instead or working around it by spec file changes?
No, it's the other way round: yum is more efficient in downloading because it most often does not need to download everything, while the other depsolvers always download the bist in advance. It's a bit oversimplified, but that servers for a first picture.
The discussion is about whether some packages force yum to go download and drop its bandwidth performance to that of the other depsolvers, but that's just 0.7% of all packages, so not really worth attacking at all IMHO.
Axel Thimm wrote:
On Thu, Dec 21, 2006 at 11:39:38AM -0500, Fernando Nasser wrote:
- If this is a problem that only affects yum and all the other
depsolvers do it efficiently, would it be better to fix yum instead or working around it by spec file changes?
No, it's the other way round: yum is more efficient in downloading because it most often does not need to download everything, while the other depsolvers always download the bist in advance. It's a bit oversimplified, but that servers for a first picture.
Thanks for the clarification. The use of file deps prevents an optimization to kick in.
On Thu, 2006-21-12 at 13:46 +0100, Axel Thimm wrote:
Instead an analysis would show why packagers used this method and whether we can offer better workarounds for what they wanted to achieve.
In the Eclipse SDK package set, we recently went through the multilib-ification process. We had to add a few file-level dependencies to get this to work. This was due to the inability to specify %{arch} in a Requires. How would this have been better accomplished?
Ben can probably explain this better than I can.
Andrew
On Thu, 2006-12-21 at 10:10 -0500, Andrew Overholt wrote:
On Thu, 2006-21-12 at 13:46 +0100, Axel Thimm wrote:
Instead an analysis would show why packagers used this method and whether we can offer better workarounds for what they wanted to achieve.
In the Eclipse SDK package set, we recently went through the multilib-ification process. We had to add a few file-level dependencies to get this to work. This was due to the inability to specify %{arch} in a Requires. How would this have been better accomplished?
Ben can probably explain this better than I can.
Honestly, this is the only valid case for file-level dependencies I can think of, and unfortunately, the only way to work around it is to fix rpm to permit %{arch} specific Requires.
~spot
On Thu, 2006-12-21 at 09:19 -0600, Tom 'spot' Callaway wrote:
On Thu, 2006-12-21 at 10:10 -0500, Andrew Overholt wrote:
In the Eclipse SDK package set, we recently went through the multilib-ification process. We had to add a few file-level dependencies to get this to work. This was due to the inability to specify %{arch} in a Requires. How would this have been better accomplished?
Ben can probably explain this better than I can.
Honestly, this is the only valid case for file-level dependencies I can think of, and unfortunately, the only way to work around it is to fix rpm to permit %{arch} specific Requires.
One thing for which a dir based dependency is the best we have at the moment are plugins for Mozilla compatible browsers (Requires: %{_libdir}/mozilla/plugins). That could be changed if those browser packages had a common Provides indicating that they load plugins from that dir.
By the way, would it be possible/feasible to modify yum so that before downloading filelists metadata, it would check the headers of packages already inserted into a transaction to see if they already satisfy the needed file based dependencies (even if those deps are not available in primary metadata)? One example case where this would be useful is what I currently do in w3c-markup-validator-libs:
Requires: xhtml1-dtds /usr/share/sgml/xhtml1/xhtml1-20020801/DTD
The DTDs are really expected to be found in that exact path. Even though I know it's currently that way in xhtml1-dtds, I'd like to have the path based dependency there for additional safeguard in case the path to the DTDs ever changes (there's a date embedded in the path so it's inherently at least theoretically prone to that).
So when yum sees the above two dependencies without having full filelists available, it'd first insert xhtml1-dtds and try it out if it happened to satisfy /usr/share/sgml/xhtml1/xhtml1-20020801/DTD too, without downloading filelists yet. If yes, no need to download filelists.
On Wed, Dec 20, 2006 at 08:55:25PM -0800, Toshio Kuratomi wrote:
Hey all, I've just added a draft proposal for limiting file dependencies in packages. It's a "should" recommendation at this point.
http://www.fedoraproject.org/wiki/PackagingDrafts/FileDeps
= File Deps Guideline =
== Problem ==
How often does this problem really occur? I just checked core's x86_64 packages and there are exactly 12 files in such dependencies affecting 18 packages out of 2422 (that's 0.7%):
cman-kernel dlm-kernel gamin-python gdm GFS-kernel gnbd-kernel httpd ImageMagick-perl inn libvirt-python libxml2-python libxslt-python openldap-servers openmotif openssh-server python-lcms redhat-lsb vsftpd
That's 0.7%. I thought that the problem was much larger. I either miscalculated the number of file dependencies or there is no problem :)
(At least not one that has to be solved by creating new guidelines, most of these packages like lsb or kernel modules seem to need special handling anyway)
On Thu, Dec 21, 2006 at 02:10:18PM +0100, Axel Thimm wrote:
On Wed, Dec 20, 2006 at 08:55:25PM -0800, Toshio Kuratomi wrote:
Hey all, I've just added a draft proposal for limiting file dependencies in packages. It's a "should" recommendation at this point.
http://www.fedoraproject.org/wiki/PackagingDrafts/FileDeps
= File Deps Guideline =
== Problem ==
How often does this problem really occur? I just checked core's x86_64 packages and there are exactly 12 files in such dependencies affecting 18 packages out of 2422 (that's 0.7%):
cman-kernel dlm-kernel gamin-python gdm GFS-kernel gnbd-kernel httpd ImageMagick-perl inn libvirt-python libxml2-python libxslt-python openldap-servers openmotif openssh-server python-lcms redhat-lsb vsftpd
That's 0.7%. I thought that the problem was much larger. I either miscalculated the number of file dependencies or there is no problem :)
And the same metrics for extras: 26 files required by 27 packages out of 3660 packages:
chess clamav-devel ctapi-cyberjack cvs2cl em8300-devel freefont ftnchek-emacs gnash-plugin libibverbs-devel mimetex mozilla-opensc-signer openbox openct pcsc-perl perl-Image-Info pyxdg SoQt-devel tuxpaint util-vserver util-vserver-legacy util-vserver-sysv uuid-devel uuid-pgsql uuid-php w3c-markup-validator-libs x11-ssh-askpass xmms
That's again 0.7%, so the combined Fedora is at 0.7%, too.
Still hoping my calculations aren't flawed ...
(At least not one that has to be solved by creating new guidelines, most of these packages like lsb or kernel modules seem to need special handling anyway)
On 21.12.2006 14:24, Axel Thimm wrote:
On Thu, Dec 21, 2006 at 02:10:18PM +0100, Axel Thimm wrote:
On Wed, Dec 20, 2006 at 08:55:25PM -0800, Toshio Kuratomi wrote:
Hey all, I've just added a draft proposal for limiting file dependencies in packages. It's a "should" recommendation at this point.
http://www.fedoraproject.org/wiki/PackagingDrafts/FileDeps
= File Deps Guideline =
== Problem ==
How often does this problem really occur? I just checked core's x86_64 packages and there are exactly 12 files in such dependencies affecting 18 packages out of 2422 (that's 0.7%):
cman-kernel dlm-kernel gamin-python gdm GFS-kernel gnbd-kernel httpd ImageMagick-perl inn libvirt-python libxml2-python libxslt-python openldap-servers openmotif openssh-server python-lcms redhat-lsb vsftpd
That's 0.7%. I thought that the problem was much larger. I either miscalculated the number of file dependencies or there is no problem :)
And the same metrics for extras: 26 files required by 27 packages out of 3660 packages:
chess clamav-devel ctapi-cyberjack cvs2cl em8300-devel freefont ftnchek-emacs gnash-plugin libibverbs-devel mimetex mozilla-opensc-signer openbox openct pcsc-perl perl-Image-Info pyxdg SoQt-devel tuxpaint util-vserver util-vserver-legacy util-vserver-sysv uuid-devel uuid-pgsql uuid-php w3c-markup-validator-libs x11-ssh-askpass xmms
That's again 0.7%, so the combined Fedora is at 0.7%, too.
Well, my initial calculations found 32 file based deps outside of "/etc {/usr,}/{s,}bin/" in Extras 6 x64 ;-) Just to note, I mentioned that when I started the whole mess after skvidal poked me about it. See https://www.redhat.com/archives/fedora-packaging/2006-December/msg00077.html
CU thl
On Thu, Dec 21, 2006 at 02:43:31PM +0100, Thorsten Leemhuis wrote:
On 21.12.2006 14:24, Axel Thimm wrote:
On Thu, Dec 21, 2006 at 02:10:18PM +0100, Axel Thimm wrote:
On Wed, Dec 20, 2006 at 08:55:25PM -0800, Toshio Kuratomi wrote:
Hey all, I've just added a draft proposal for limiting file dependencies in packages. It's a "should" recommendation at this point.
http://www.fedoraproject.org/wiki/PackagingDrafts/FileDeps
= File Deps Guideline =
== Problem ==
How often does this problem really occur? I just checked core's x86_64 packages and there are exactly 12 files in such dependencies affecting 18 packages out of 2422 (that's 0.7%):
And the same metrics for extras: 26 files required by 27 packages out of 3660 packages:
That's again 0.7%, so the combined Fedora is at 0.7%, too.
Well, my initial calculations found 32 file based deps outside of "/etc {/usr,}/{s,}bin/" in Extras 6 x64 ;-) Just to note, I mentioned that when I started the whole mess after skvidal poked me about it. See https://www.redhat.com/archives/fedora-packaging/2006-December/msg00077.html
But then it was known that this is negligible, why does it escalate to become even a proposal for guidelines? Do we care that yum fetches the filelists every 140 package updates?
Axel Thimm schrieb:
On Thu, Dec 21, 2006 at 02:43:31PM +0100, Thorsten Leemhuis wrote:
On 21.12.2006 14:24, Axel Thimm wrote:
On Thu, Dec 21, 2006 at 02:10:18PM +0100, Axel Thimm wrote:
On Wed, Dec 20, 2006 at 08:55:25PM -0800, Toshio Kuratomi wrote:
Hey all, I've just added a draft proposal for limiting file dependencies in packages. It's a "should" recommendation at this point. http://www.fedoraproject.org/wiki/PackagingDrafts/FileDeps = File Deps Guideline = == Problem ==
How often does this problem really occur? I just checked core's x86_64 packages and there are exactly 12 files in such dependencies affecting 18 packages out of 2422 (that's 0.7%):
And the same metrics for extras: 26 files required by 27 packages out of 3660 packages: That's again 0.7%, so the combined Fedora is at 0.7%, too.
Well, my initial calculations found 32 file based deps outside of "/etc {/usr,}/{s,}bin/" in Extras 6 x64 ;-) Just to note, I mentioned that when I started the whole mess after skvidal poked me about it. See https://www.redhat.com/archives/fedora-packaging/2006-December/msg00077.html
But then it was known that this is negligible,
"negligible" IMHO is debatable, but yes, it's not a big problem (currently).
And just to clarify: I just started to look into it because skvidal poked me about it. I had no interest in it myself and after the initial research I did te thing rolled on its own.
why does it escalate to become even a proposal for guidelines?
To avoid that there get more of those added in the future?
Do we care that yum fetches the filelists every 140 package updates?
If it can be avoided easiy: yes.
There is also the OLPC hardware that seems to be quite slow and will probably take ages to resolve such a dep. We should keep that in mind, too.
CU thl
Le jeudi 21 décembre 2006 à 19:00 +0100, Thorsten Leemhuis a écrit :
Axel Thimm schrieb:
why does it escalate to become even a proposal for guidelines?
To avoid that there get more of those added in the future?
As the numbers show no one is particularly inclined to add a large number of such deps today. They won't become more common unless they start providing some benefit to users or packagers. (for example proper rpm directory handling at last). I don't see how we can decide today a yum-specific optimisation will take precedence over this yet-to-be-known benefit.
packaging@lists.fedoraproject.org
-
Andrew Overholt -
Axel Thimm -
Fernando Nasser -
Nicolas Mailhot -
Panu Matilainen -
Rex Dieter -
Thorsten Leemhuis -
Tom Callaway -
Toshio Kuratomi -
Ville Skyttä