I know I have probably been hiding under a Rock but can anyone help me understand Fedora's stance on circular dependencies within RPMs?
At least in the past I think circular dependencies have been kept to a minimum as it can cause issues with rpm sorting: i.e. for two rpms A,B with a circular dependency, in different transactions sometimes A goes before B and sometimes B goes before A.
In Fedora 20 just running rpm against the 1500 or so packages I have installed on my Desktop I notice that there are 47 sets of "Strongly Connected Components"; basically 47 cases of a circular dependency.
Was there a point where this became more popular. I don't remember there being more than a few circular deps for EL6 era rpms.
Thanks, Dusty
Am 22.08.2014 um 21:53 schrieb Dusty Mabe:
I know I have probably been hiding under a Rock but can anyone help me understand Fedora's stance on circular dependencies within RPMs?
At least in the past I think circular dependencies have been kept to a minimum as it can cause issues with rpm sorting: i.e. for two rpms A,B with a circular dependency, in different transactions sometimes A goes before B and sometimes B goes before A.
In Fedora 20 just running rpm against the 1500 or so packages I have installed on my Desktop I notice that there are 47 sets of "Strongly Connected Components"; basically 47 cases of a circular dependency.
Was there a point where this became more popular. I don't remember there being more than a few circular deps for EL6 era rpms.
how should it work otherwise?
* each package defines it's depndencies * each of the dependencies have their own * no package knows the whole picture * each package has to pull all deps it needs
if * A requires B * B requires C
it's clear that removal of C would also remove A because it removes B caused by the deps leading to remove A
if you can remove C the B has a package error which is in theory possible because C already could have been installed and nobody noticed the missing dep until you remove it and nothing complais (been there)
On Fri, Aug 22, 2014 at 10:11:25PM +0200, Reindl Harald wrote:
Am 22.08.2014 um 21:53 schrieb Dusty Mabe:
I know I have probably been hiding under a Rock but can anyone help me understand Fedora's stance on circular dependencies within RPMs?
At least in the past I think circular dependencies have been kept to a minimum as it can cause issues with rpm sorting: i.e. for two rpms A,B with a circular dependency, in different transactions sometimes A goes before B and sometimes B goes before A.
In Fedora 20 just running rpm against the 1500 or so packages I have installed on my Desktop I notice that there are 47 sets of "Strongly Connected Components"; basically 47 cases of a circular dependency.
Was there a point where this became more popular. I don't remember there being more than a few circular deps for EL6 era rpms.
how should it work otherwise?
- each package defines it's depndencies
- each of the dependencies have their own
- no package knows the whole picture
- each package has to pull all deps it needs
Sure. Each package defines all deps it needs is how it should be.
if
- A requires B
- B requires C
The scenario I am concerned with here is:
if * A requires B * B requires C * C requires A
This basically yields a case where ordering can't be properly done because rpm doesn't know which dependency is stronger. If all of the rpms in question just deliver files then all is well with the world but if each of them have scriptlets the ordering can definitely matter and this would be a problem.
Dusty
On 08/22/2014 02:23 PM, Dusty Mabe wrote:
The scenario I am concerned with here is:
if
- A requires B
- B requires C
- C requires A
This basically yields a case where ordering can't be properly done because rpm doesn't know which dependency is stronger. If all of the rpms in question just deliver files then all is well with the world but if each of them have scriptlets the ordering can definitely matter and this would be a problem.
Dusty
Scriptlet dependency ordering it specified with for example:
Requires(post): pkg
Are there any circular scriptlet deps?
On Fri, Aug 22, 2014 at 02:29:31PM -0600, Orion Poplawski wrote:
On 08/22/2014 02:23 PM, Dusty Mabe wrote:
The scenario I am concerned with here is:
if
- A requires B
- B requires C
- C requires A
This basically yields a case where ordering can't be properly done because rpm doesn't know which dependency is stronger. If all of the rpms in question just deliver files then all is well with the world but if each of them have scriptlets the ordering can definitely matter and this would be a problem.
Dusty
Scriptlet dependency ordering it specified with for example:
Requires(post): pkg
Are there any circular scriptlet deps?
So that may be the answer. This *new to me* feature solves the case of scriptlets which is really the only place you see errors *during* the install.
Thanks Orion
On Fri, Aug 22, 2014 at 5:15 PM, Dusty Mabe dustymabe@gmail.com wrote:
On Fri, Aug 22, 2014 at 02:29:31PM -0600, Orion Poplawski wrote:
On 08/22/2014 02:23 PM, Dusty Mabe wrote:
The scenario I am concerned with here is:
if
- A requires B
- B requires C
- C requires A
This basically yields a case where ordering can't be properly done because rpm doesn't know which dependency is stronger. If all of the rpms in question just deliver files then all is well with the world but if each of them have scriptlets the ordering can definitely matter and this would be a problem.
Dusty
Scriptlet dependency ordering it specified with for example:
Requires(post): pkg
Are there any circular scriptlet deps?
So that may be the answer. This *new to me* feature solves the case of scriptlets which is really the only place you see errors *during* the install.
Thanks Orion
Installation scripting is not the only source of the problem. Perl modules have been prone to this.
* Perl module A requires perl module B. * Perl module B requires perl module C. * One small script or macro in module C requires one small script from module A. It may not even be a critical component of C, and may be easily segregated, but suddenly there is a circular dependency.
The vortex enters when the author of module B updates to a new dependency or build dependency that is not in the current version of C, and C introduces a new dependency on A but it's based on an older version of A, that has since discarded that macro due to a code cleanup. Hilarity ensues.....
The underlying point is that it's sometimes very helpful to split upstream packages to smaller, individual components, precisely to segregate these dependencies. It's especially useful with Perl modules. I have.... old stories about the mod_perl updates back when HTTPD 2.x was first released and mod_perl compatibility became kind of nutty.
On 08/24/2014 04:37 PM, Nico Kadel-Garcia wrote:
Installation scripting is not the only source of the problem. Perl modules have been prone to this.
- Perl module A requires perl module B.
- Perl module B requires perl module C.
- One small script or macro in module C requires one small script from
module A. It may not even be a critical component of C, and may be easily segregated, but suddenly there is a circular dependency.
The vortex enters when the author of module B updates to a new dependency or build dependency that is not in the current version of C, and C introduces a new dependency on A but it's based on an older version of A, that has since discarded that macro due to a code cleanup. Hilarity ensues.....
The underlying point is that it's sometimes very helpful to split upstream packages to smaller, individual components, precisely to segregate these dependencies. It's especially useful with Perl modules. I have.... old stories about the mod_perl updates back when HTTPD 2.x was first released and mod_perl compatibility became kind of nutty.
Or we can wait for F21, which will have weak dependencies in RPM. And I anticipate that weak dependencies will break a lot of circles.
On 2014-08-25, Miroslav Suchý msuchy@redhat.com wrote:
Or we can wait for F21, which will have weak dependencies in RPM. And I anticipate that weak dependencies will break a lot of circles.
Does Fedora have guidelines what should and what should not be a weak dependency?
My experience with Perl packages is that "declaring dependency because it's recommended but the package works without it" is quite seldom. The majority of build cycles are either build-time dependencies for tests or hard run-time dependencies (the Perl module exits with an exception without the dependency in some code paths.)
In my opinion, it would be much more appreciated if Fedora had a mechanism to express "I want support for PDF" on the installed system and then package manager would use this boolean to install or skip affected dependencies. (This is the case of "some code paths" from previous paragraph.)
-- Petr
On Tue, Aug 26, 2014 at 10:55:56AM +0000, Petr Pisar wrote:
On 2014-08-25, Miroslav Suchý msuchy@redhat.com wrote:
Or we can wait for F21, which will have weak dependencies in RPM. And I anticipate that weak dependencies will break a lot of circles.
Does Fedora have guidelines what should and what should not be a weak dependency?
My experience with Perl packages is that "declaring dependency because it's recommended but the package works without it" is quite seldom. The majority of build cycles are either build-time dependencies for tests or hard run-time dependencies (the Perl module exits with an exception without the dependency in some code paths.)
In my opinion, it would be much more appreciated if Fedora had a mechanism to express "I want support for PDF" on the installed system and then package manager would use this boolean to install or skip affected dependencies. (This is the case of "some code paths" from previous paragraph.)
Would virtual provides give us that?
Packages A, B and C have "Provides: pdf_reader" and Package B has "Requires: pdf_reader". The issue would to determine which provider to install, and do you ask the user to install it first or maybe install a "recommended" package?
On 2014-08-28, Darryl L. Pierce mcpierce@gmail.com wrote:
On Tue, Aug 26, 2014 at 10:55:56AM +0000, Petr Pisar wrote:
In my opinion, it would be much more appreciated if Fedora had a mechanism to express "I want support for PDF" on the installed system and then package manager would use this boolean to install or skip affected dependencies. (This is the case of "some code paths" from previous paragraph.)
Would virtual provides give us that?
Packages A, B and C have "Provides: pdf_reader" and Package B has "Requires: pdf_reader". The issue would to determine which provider to install, and do you ask the user to install it first or maybe install a "recommended" package?
You are still going in circles. The key point is to express the "Requires: pdf_reader" at package B is a weak dependency.
Then an interactive package manager could ask "The pdf_reader B dependency is recommended but not needed. Do you want to fullfill it?", and user had an opportunity to skip it completely.
(Of course, you could create an empty package providing "pdf_reader", but I feel these fake provides just lead to confusion. So I do not recommend this way. See the fakesystemd afair.)
And because users will get tired by explaining their choises with each package manager invocation, the answer should go into local configuration file so the decision could be reused next time.
This per-package decision can even get default value from a global (all-package) configuration. E.g. if an user is not interested in the PDF support at all, then any weak dependency somehow qualified as a PDF could be auto-answered.
And the global configuration could even replace the server roles feature because, at least now as I know, it's only about selecting default package set.
If you know USE flags in Gentoo, then this is it.
-- Petr
On 2014-08-24, Nico Kadel-Garcia nkadel@gmail.com wrote:
Installation scripting is not the only source of the problem. Perl modules have been prone to this.
- Perl module A requires perl module B.
- Perl module B requires perl module C.
- One small script or macro in module C requires one small script from
module A. It may not even be a critical component of C, and may be easily segregated, but suddenly there is a circular dependency.
There is no algorithmic difference between Perl packages and any other packages.
Specifically to Perl, we employ %perl_bootstrap macro which guards the `not even a critical' dependency which allows us to break the dependency cycle.
Perl SIG exhibits weekly rebuilds https://ppisar.fedorapeople.org/perl_rebuild/scratch/index.xhtml which allow us to watch that all Perl packages are buildable.
The vortex enters when the author of module B updates to a new dependency or build dependency that is not in the current version of C, and C introduces a new dependency on A but it's based on an older version of A, that has since discarded that macro due to a code cleanup. Hilarity ensues.....
If you need to do such an upgrade, provided you've already deployed the %perl_bootstrap condition, you just define perl_bootstrap temporarily in one of the upgraded packages, and after upgrading the connected component, you remove the macro definition and rebuild the package again.
The underlying point is that it's sometimes very helpful to split upstream packages to smaller, individual components, precisely to segregate these dependencies. It's especially useful with Perl modules.
Definitely. Once packageres realize that Perl modules can move beetween RPM packages freely, once they realize that specifying all direct dependencies without relying on transitive dependencies is the best practice and a must for healthy distribution, then splitting packages is obvious solution.
Although splitting cannot solve all issues, especially if the splitted modules are really interdependend. Then there is no point in the split. However that's a upstream issue.
-- Petr
devel@lists.stg.fedoraproject.org