Hi everyone,
I've been following the discussion on RPM and GEMS conflict of interest. I found it quite interesting, and I would like to discuss some packaging options already present within ruby and rails.
There are three things I would like to add to Jeroen's page: 1) A valid Rails RPM package *explicit* links to gems required. Strictly speaking, you can lazy load the gems an application needs. This might work, especially if you use gems that are very common (like Rails). Problem is, that a critical dependency is missing. You can add this info in the RPM spec, but I think it is better to make it mandatory more downstream. For RPM packaging, all you need to do is ask for the dependent gems. My question: what are your thoughts on this?
2) A valid Rails RPM package must contain the version when specifying gems and the version must be limited. Example: # require 'aws/s3' config.gem 'aws-s3', :lib => 'aws/s3', :version => '0.4.0', # :source => "http://code.whytheluckystiff.net" (see also http://api.rubyonrails.org/classes/Rails/Configuration.html#M002064) This means that the module will use any version greater than 0.4.0. You can also specify smaller, a range or a specific number. The trick is this: if you do this correctly, you can use multiple gem versions on the same system. In theory, this must work. This is what was causing the issue with Lighthouse. I think you should fix the version or make it a range, never a 'greater than' (as in the example and with Lighthouse). My question: what are your thoughts on this, especially the mandatory 'limit the version number'?
3) Gems can be vendorized. OK, this is the moment where a lot of people are going to be amazed, mouth open and staring at the screen for the next five minutes: you can make a Rails application and include the gems in it, as it where local code. This is called vendorization. A vendorized gem is only available to the application using it: it is part of the source code of that application and located in the /vendor directory of the application root (not in the normal gem directory). In fact, you can vendorize a gem and then even modify the code. On one hand, if you say all gems should be vendorized, you don't have any version conflicts. On the other hand, I heard someone complain about static linking... Personally, I disagree with the vendorization of gems. It obfuscates version management. Question: does Fedore prefer (or mandatorize) a specific approach? Why?
Thanks in advance, Bert
Bert Meerman wrote:
Hi everyone,
I've been following the discussion on RPM and GEMS conflict of interest. I found it quite interesting, and I would like to discuss some packaging options already present within ruby and rails.
There are three things I would like to add to Jeroen's page:
- A valid Rails RPM package *explicit* links to gems required.
Strictly speaking, you can lazy load the gems an application needs. This might work, especially if you use gems that are very common (like Rails). Problem is, that a critical dependency is missing. You can add this info in the RPM spec, but I think it is better to make it mandatory more downstream. For RPM packaging, all you need to do is ask for the dependent gems. My question: what are your thoughts on this?
With a packaged Rails product, for example, one aspect of the RPM package will most likely be:
Requires: rubygem(rails)
One can enforce a specific version of the Ruby gem:
Requires: rubygem(rails) = 2.3.5
in order to prevent breakage on platform or application upgrades, through system capabilities. Pretty much the same goes for Ruby gems, not just packaged applications from downstream consumers or vendors.
Normally, in cases where the upstream of the dependent package is using sane versioning, one could even:
Requires: rubygem(rails) >= 2.3, rubygem(rails) < 2.4
which, logically, should result in the Rails RPM package changing for bug- and security-fixes only. However, such seems to not be common practice in the larger Ruby community. This is a problem mid-streams like Fedora will have to deal with -or let it be a Free For All but then what are we doing here?
Anyway, while the application may have (with emphasis on "may have"):
RAILS_GEM_VERSION = '2.3.2'
this does not correspond with the capabilities provided by the system in any way. It locks the application down to using 2.3.2 and 2.3.2 only, which should thus also not be used lightly in any situation.
Because of it's own little package management, we have two completely separate dependency paths each of them searching to resolve dependencies (but only one of them through global dependencies), and both are as dependent on a closed dependency tree as another.
However, if you install a Ruby gem RPM, and gem update the system, your RPM is still there, and you can depend on it, and you can be sure that, should the system-wide capability be attempted to be removed in any case, the transaction is going to fail because of missing dependencies.
Imagine you gem install rails but then yum remove ruby -just to emphasize the point I'm trying to make here.
In another example, gem install mysql and then yum update mysql from 5.0 to 5.1, or yum update ruby from 1.8.6 to 1.8.7.
There can only be one package manager doing all the work. It's RPM.
Does that limit anyone from using Gem? No, we're shipping it and in the end, it's your system. Knock yourself out.
Do you have the liberty to derive from what we provide, either by using gem or apt or ...? Sure! Have fun.
Does that mean the RPMs provided are worthless? Euh, not so much.
For one, RPM builds are reproducible. If a test fails on one build system, it will fail on another build system in very much the same way (exceptions apply). If compilation fails because it's using a dynamic library instead of a static library to compile against, or it uses the latest and greatest version rather then the compat- version. We'll figure that out and fix it. Documentation for the upstream gem only accounts for so much of so many system aspects.
- A valid Rails RPM package must contain the version when specifying
gems and the version must be limited. Example: # require 'aws/s3' config.gem 'aws-s3', :lib => 'aws/s3', :version => '0.4.0', # :source => "http://code.whytheluckystiff.net"
Actually, this is not the case. in a config.gem statement, one only is required to list the name, and all other parameters are optional.
For example, one my require a mysql gem, but not care about the version - because the platforms the package or application is distributed for includes the correct version and the mysql gem doesn't change that much.
(see also http://api.rubyonrails.org/classes/Rails/Configuration.html#M002064) This means that the module will use any version greater than 0.4.0. You can also specify smaller, a range or a specific number. The trick is this: if you do this correctly, you can use multiple gem versions on the same system. In theory, this must work. This is what was causing the issue with Lighthouse. I think you should fix the version or make it a range, never a 'greater than' (as in the example and with Lighthouse). My question: what are your thoughts on this, especially the mandatory 'limit the version number'?
Multiple versions of a single applications are useful in a situation in which, from the perspective of a midstream distributor, each and every version can be appropriately maintained. Nevermind the bug-fixes and let's go straight to security fixes. We, the mid-stream distributor, have a tendency to solve most of those security issues for you, and give you the appropriate, fixed version, without breaking anything. It's something virtually unheard of in the greater Ruby community, where feature enhancements and API breakage go hand-in-hand in a release marked as a Security related release.
That said, we do have mechanisms in place that allow us to install more then one version of an application through RPM. We currently use that mechanism for kernel packages. It has a setting that is called "installonlyn_limit", so you may choose how many versions (or actually, RPM releases, not application versions) of the package should be preserved.
- Gems can be vendorized.
OK, this is the moment where a lot of people are going to be amazed, mouth open and staring at the screen for the next five minutes: you can make a Rails application and include the gems in it, as it where local code. This is called vendorization.
</sarcasm> ;-)
A vendorized gem is only available to the application using it: it is part of the source code of that application and located in the /vendor directory of the application root (not in the normal gem directory). In fact, you can vendorize a gem and then even modify the code. On one hand, if you say all gems should be vendorized, you don't have any version conflicts. On the other hand, I heard someone complain about static linking... Personally, I disagree with the vendorization of gems. It obfuscates version management. Question: does Fedore prefer (or mandatorize) a specific approach? Why?
We do not allow the shipping of vendored code or libraries. A good explanation of what vendored code entails is at https://bugzilla.redhat.com/show_bug.cgi?id=470696#c37
That does not mean, however, that downstream consumers or Independent Software Vendors are not allowed to vendor parts of their project or product. We discourage it, for all the right reasons might I add, but we can not (and should not) prohibit it.
Vendored code is not allowed for the parts that the Fedora Project distributes, hence if a product is ever to be a part of the distribution, the vendored code needs to go.
-- Jeroen
On 28-6-2010 10:42, Jeroen van Meeuwen wrote:
Bert Meerman wrote:
Hi everyone,
I've been following the discussion on RPM and GEMS conflict of interest. I found it quite interesting, and I would like to discuss some packaging options already present within ruby and rails.
There are three things I would like to add to Jeroen's page:
- A valid Rails RPM package *explicit* links to gems required.
Strictly speaking, you can lazy load the gems an application needs. This might work, especially if you use gems that are very common (like Rails). Problem is, that a critical dependency is missing. You can add this info in the RPM spec, but I think it is better to make it mandatory more downstream. For RPM packaging, all you need to do is ask for the dependent gems. My question: what are your thoughts on this?
With a packaged Rails product, for example, one aspect of the RPM package will most likely be:
Requires: rubygem(rails)
One can enforce a specific version of the Ruby gem:
Requires: rubygem(rails) = 2.3.5
in order to prevent breakage on platform or application upgrades, through system capabilities. Pretty much the same goes for Ruby gems, not just packaged applications from downstream consumers or vendors.
Normally, in cases where the upstream of the dependent package is using sane versioning, one could even:
Requires: rubygem(rails)>= 2.3, rubygem(rails)< 2.4
which, logically, should result in the Rails RPM package changing for bug- and security-fixes only. However, such seems to not be common practice in the larger Ruby community. This is a problem mid-streams like Fedora will have to deal with -or let it be a Free For All but then what are we doing here?
Anyway, while the application may have (with emphasis on "may have"):
RAILS_GEM_VERSION = '2.3.2'
this does not correspond with the capabilities provided by the system in any way. It locks the application down to using 2.3.2 and 2.3.2 only, which should thus also not be used lightly in any situation.
Because of it's own little package management, we have two completely separate dependency paths each of them searching to resolve dependencies (but only one of them through global dependencies), and both are as dependent on a closed dependency tree as another.
However, if you install a Ruby gem RPM, and gem update the system, your RPM is still there, and you can depend on it, and you can be sure that, should the system-wide capability be attempted to be removed in any case, the transaction is going to fail because of missing dependencies.
Imagine you gem install rails but then yum remove ruby -just to emphasize the point I'm trying to make here.
In another example, gem install mysql and then yum update mysql from 5.0 to 5.1, or yum update ruby from 1.8.6 to 1.8.7.
There can only be one package manager doing all the work. It's RPM.
Does that limit anyone from using Gem? No, we're shipping it and in the end, it's your system. Knock yourself out.
Do you have the liberty to derive from what we provide, either by using gem or apt or ...? Sure! Have fun.
Does that mean the RPMs provided are worthless? Euh, not so much.
For one, RPM builds are reproducible. If a test fails on one build system, it will fail on another build system in very much the same way (exceptions apply). If compilation fails because it's using a dynamic library instead of a static library to compile against, or it uses the latest and greatest version rather then the compat- version. We'll figure that out and fix it. Documentation for the upstream gem only accounts for so much of so many system aspects.
- A valid Rails RPM package must contain the version when specifying
gems and the version must be limited. Example: # require 'aws/s3' config.gem 'aws-s3', :lib => 'aws/s3', :version => '0.4.0', # :source => "http://code.whytheluckystiff.net"
Actually, this is not the case. in a config.gem statement, one only is required to list the name, and all other parameters are optional.
For example, one my require a mysql gem, but not care about the version - because the platforms the package or application is distributed for includes the correct version and the mysql gem doesn't change that much.
(see also http://api.rubyonrails.org/classes/Rails/Configuration.html#M002064) This means that the module will use any version greater than 0.4.0. You can also specify smaller, a range or a specific number. The trick is this: if you do this correctly, you can use multiple gem versions on the same system. In theory, this must work. This is what was causing the issue with Lighthouse. I think you should fix the version or make it a range, never a 'greater than' (as in the example and with Lighthouse). My question: what are your thoughts on this, especially the mandatory 'limit the version number'?
Multiple versions of a single applications are useful in a situation in which, from the perspective of a midstream distributor, each and every version can be appropriately maintained. Nevermind the bug-fixes and let's go straight to security fixes. We, the mid-stream distributor, have a tendency to solve most of those security issues for you, and give you the appropriate, fixed version, without breaking anything. It's something virtually unheard of in the greater Ruby community, where feature enhancements and API breakage go hand-in-hand in a release marked as a Security related release.
That said, we do have mechanisms in place that allow us to install more then one version of an application through RPM. We currently use that mechanism for kernel packages. It has a setting that is called "installonlyn_limit", so you may choose how many versions (or actually, RPM releases, not application versions) of the package should be preserved.
3) Gems can be vendorized.
OK, this is the moment where a lot of people are going to be amazed, mouth open and staring at the screen for the next five minutes: you can make a Rails application and include the gems in it, as it where local code. This is called vendorization.
</sarcasm> ;-)
A vendorized gem is only available to the application using it: it is part of the source code of that application and located in the /vendor directory of the application root (not in the normal gem directory). In fact, you can vendorize a gem and then even modify the code. On one hand, if you say all gems should be vendorized, you don't have any version conflicts. On the other hand, I heard someone complain about static linking... Personally, I disagree with the vendorization of gems. It obfuscates version management. Question: does Fedore prefer (or mandatorize) a specific approach? Why?
We do not allow the shipping of vendored code or libraries. A good explanation of what vendored code entails is at https://bugzilla.redhat.com/show_bug.cgi?id=470696#c37
That does not mean, however, that downstream consumers or Independent Software Vendors are not allowed to vendor parts of their project or product. We discourage it, for all the right reasons might I add, but we can not (and should not) prohibit it.
Vendored code is not allowed for the parts that the Fedora Project distributes, hence if a product is ever to be a part of the distribution, the vendored code needs to go.
-- Jeroen
Nice comments! I think we're getting to a nice fundamental problem here: Given the facts: 1) Higher versions of gems (or Ruby itself) can have changed API's. In fact, Ruby 1.9 is very different (just about everything breaks) from Ruby 1.8.x. Also, have a look at the change in http://redmine.ruby-lang.org/issues/show/2713. Yes, it's not a bug but a feature. It also means that just about every gem will break (this is like deleting ncurses from a C development system). 2) Mid stream distributor (Fedora) wants to be able to solve security issues. 3) A principle of mid stream distributor (Fedora) is to be able to patch in a transparent fashion. That is, code may not be obfuscated. This principle means, among other things, that vendorized code will not be packaged. 4) A principle of Ruby is to have your code tested. In fact: 'Writing applications without tests makes you a bad person, incapable of love.' (Wilson - The Rails Way). IMHO this can be used to Fedora advantage: remember, you can confront every Ruby developer with this principle - and they *will* appreciate you for doing so! (This is some of the most valued principles of the Ruby community - in fact, the first point made above is not an issue for ruby developers as they will point out that it's all about the tests, not the code)
This leads me to suggesting the following solution: 1) Given a codebase C, with gems G1..Gn. Let's say one of the gems is the CBAC gem, version 1.0. 2) No gems are vendorized. 2) Run tests and verify application. 3) Package the gems. 4) Package the application. Change the application so the gems are fixed to a specific version. 5) Test and verify. We now have a stable build. Package. Let's say the following happens: 6) A security flaw is detected in the CBAC gem, fixed in version 1.7 7) Change the application to use CBAC gem version 1.1 instead of 1.0. Run tests to verify application. 8) Do this until you reach version 1.7. 9) If everything goes OK, commit the change (eg the version number) to the application, repackage and you're done. 10) If the application breaks (let's say at version 1.4), either do one of the following: - Fix the app so it can work with 1.7 (can be done by the app developer) - Fix the security flaw in version 1.4 (can be done by the gem developer) In both cases, the security flaw is fixed, tested, and packaged. ... Packaging of the applications and gems can be done using a sandbox system: - Basic Fedora version x system - Ruby interpreter - Run test of the application/ gem
Remember: in Ruby land, if the tests pass and the app/ gem fails, the code is bad. You can use Wilson's quote to make your point across. Also, this way Fedora would encourage Ruby principles. Personally, as a Rubyist, I would love this approach.
What do you guys think?
Bert Meerman wrote, at 06/30/2010 02:39 AM +9:00:
On 28-6-2010 10:42, Jeroen van Meeuwen wrote:
Bert Meerman wrote:
Hi everyone,
I've been following the discussion on RPM and GEMS conflict of interest. I found it quite interesting, and I would like to discuss some packaging options already present within ruby and rails.
There are three things I would like to add to Jeroen's page:
- A valid Rails RPM package *explicit* links to gems required.
Strictly speaking, you can lazy load the gems an application needs. This might work, especially if you use gems that are very common (like Rails). Problem is, that a critical dependency is missing. You can add this info in the RPM spec, but I think it is better to make it mandatory more downstream. For RPM packaging, all you need to do is ask for the dependent gems. My question: what are your thoughts on this?
With a packaged Rails product, for example, one aspect of the RPM package will most likely be:
Requires: rubygem(rails)One can enforce a specific version of the Ruby gem:
Requires: rubygem(rails) = 2.3.5in order to prevent breakage on platform or application upgrades, through system capabilities. Pretty much the same goes for Ruby gems, not just packaged applications from downstream consumers or vendors.
Normally, in cases where the upstream of the dependent package is using sane versioning, one could even:
Requires: rubygem(rails)>= 2.3, rubygem(rails)< 2.4which, logically, should result in the Rails RPM package changing for bug- and security-fixes only. However, such seems to not be common practice in the larger Ruby community. This is a problem mid-streams like Fedora will have to deal with -or let it be a Free For All but then what are we doing here?
Anyway, while the application may have (with emphasis on "may have"):
RAILS_GEM_VERSION = '2.3.2'this does not correspond with the capabilities provided by the system in any way. It locks the application down to using 2.3.2 and 2.3.2 only, which should thus also not be used lightly in any situation.
Because of it's own little package management, we have two completely separate dependency paths each of them searching to resolve dependencies (but only one of them through global dependencies), and both are as dependent on a closed dependency tree as another.
However, if you install a Ruby gem RPM, and gem update the system, your RPM is still there, and you can depend on it, and you can be sure that, should the system-wide capability be attempted to be removed in any case, the transaction is going to fail because of missing dependencies.
Imagine you gem install rails but then yum remove ruby -just to emphasize the point I'm trying to make here.
In another example, gem install mysql and then yum update mysql from 5.0 to 5.1, or yum update ruby from 1.8.6 to 1.8.7.
There can only be one package manager doing all the work. It's RPM.
Does that limit anyone from using Gem? No, we're shipping it and in the end, it's your system. Knock yourself out.
Do you have the liberty to derive from what we provide, either by using gem or apt or ...? Sure! Have fun.
Does that mean the RPMs provided are worthless? Euh, not so much.
For one, RPM builds are reproducible. If a test fails on one build system, it will fail on another build system in very much the same way (exceptions apply). If compilation fails because it's using a dynamic library instead of a static library to compile against, or it uses the latest and greatest version rather then the compat- version. We'll figure that out and fix it. Documentation for the upstream gem only accounts for so much of so many system aspects.
- A valid Rails RPM package must contain the version when specifying
gems and the version must be limited. Example: # require 'aws/s3' config.gem 'aws-s3', :lib => 'aws/s3', :version => '0.4.0', # :source => "http://code.whytheluckystiff.net"
Actually, this is not the case. in a config.gem statement, one only is required to list the name, and all other parameters are optional.
For example, one my require a mysql gem, but not care about the version - because the platforms the package or application is distributed for includes the correct version and the mysql gem doesn't change that much.
(see also http://api.rubyonrails.org/classes/Rails/Configuration.html#M002064) This means that the module will use any version greater than 0.4.0. You can also specify smaller, a range or a specific number. The trick is this: if you do this correctly, you can use multiple gem versions on the same system. In theory, this must work. This is what was causing the issue with Lighthouse. I think you should fix the version or make it a range, never a 'greater than' (as in the example and with Lighthouse). My question: what are your thoughts on this, especially the mandatory 'limit the version number'?
Multiple versions of a single applications are useful in a situation in which, from the perspective of a midstream distributor, each and every version can be appropriately maintained. Nevermind the bug-fixes and let's go straight to security fixes. We, the mid-stream distributor, have a tendency to solve most of those security issues for you, and give you the appropriate, fixed version, without breaking anything. It's something virtually unheard of in the greater Ruby community, where feature enhancements and API breakage go hand-in-hand in a release marked as a Security related release.
That said, we do have mechanisms in place that allow us to install more then one version of an application through RPM. We currently use that mechanism for kernel packages. It has a setting that is called "installonlyn_limit", so you may choose how many versions (or actually, RPM releases, not application versions) of the package should be preserved.
- Gems can be vendorized.
OK, this is the moment where a lot of people are going to be amazed, mouth open and staring at the screen for the next five minutes: you can make a Rails application and include the gems in it, as it where local code. This is called vendorization.
</sarcasm> ;-)
A vendorized gem is only available to the application using it: it is part of the source code of that application and located in the /vendor directory of the application root (not in the normal gem directory). In fact, you can vendorize a gem and then even modify the code. On one hand, if you say all gems should be vendorized, you don't have any version conflicts. On the other hand, I heard someone complain about static linking... Personally, I disagree with the vendorization of gems. It obfuscates version management. Question: does Fedore prefer (or mandatorize) a specific approach? Why?
We do not allow the shipping of vendored code or libraries. A good explanation of what vendored code entails is at https://bugzilla.redhat.com/show_bug.cgi?id=470696#c37
That does not mean, however, that downstream consumers or Independent Software Vendors are not allowed to vendor parts of their project or product. We discourage it, for all the right reasons might I add, but we can not (and should not) prohibit it.
Vendored code is not allowed for the parts that the Fedora Project distributes, hence if a product is ever to be a part of the distribution, the vendored code needs to go.
-- Jeroen
Nice comments! I think we're getting to a nice fundamental problem here: Given the facts:
- Higher versions of gems (or Ruby itself) can have changed API's. In
fact, Ruby 1.9 is very different (just about everything breaks) from Ruby 1.8.x. Also, have a look at the change in http://redmine.ruby-lang.org/issues/show/2713. Yes, it's not a bug but a feature. It also means that just about every gem will break (this is like deleting ncurses from a C development system). 2) Mid stream distributor (Fedora) wants to be able to solve security issues. 3) A principle of mid stream distributor (Fedora) is to be able to patch in a transparent fashion. That is, code may not be obfuscated. This principle means, among other things, that vendorized code will not be packaged. 4) A principle of Ruby is to have your code tested. In fact: 'Writing applications without tests makes you a bad person, incapable of love.' (Wilson - The Rails Way). IMHO this can be used to Fedora advantage: remember, you can confront every Ruby developer with this principle - and they *will* appreciate you for doing so! (This is some of the most valued principles of the Ruby community - in fact, the first point made above is not an issue for ruby developers as they will point out that it's all about the tests, not the code)
This leads me to suggesting the following solution:
- Given a codebase C, with gems G1..Gn. Let's say one of the gems is
the CBAC gem, version 1.0. 2) No gems are vendorized. 2) Run tests and verify application. 3) Package the gems. 4) Package the application. Change the application so the gems are fixed to a specific version. 5) Test and verify. We now have a stable build. Package. Let's say the following happens: 6) A security flaw is detected in the CBAC gem, fixed in version 1.7 7) Change the application to use CBAC gem version 1.1 instead of 1.0. Run tests to verify application. 8) Do this until you reach version 1.7. 9) If everything goes OK, commit the change (eg the version number) to the application, repackage and you're done. 10) If the application breaks (let's say at version 1.4), either do one of the following: - Fix the app so it can work with 1.7 (can be done by the app developer) - Fix the security flaw in version 1.4 (can be done by the gem developer) In both cases, the security flaw is fixed, tested, and packaged. ... Packaging of the applications and gems can be done using a sandbox system:
- Basic Fedora version x system
- Ruby interpreter
- Run test of the application/ gem
Remember: in Ruby land, if the tests pass and the app/ gem fails, the code is bad. You can use Wilson's quote to make your point across. Also, this way Fedora would encourage Ruby principles. Personally, as a Rubyist, I would love this approach.
What do you guys think?
Well, if I understand what you want to say here correctly, all you want to discuss here is not limited to ruby or rubygems, and we follow the package updates policy in the distribution (Fedora)
Regards Mamoru
On 30-6-2010 14:19, Mamoru Tasaka wrote:
Bert Meerman wrote, at 06/30/2010 02:39 AM +9:00:
On 28-6-2010 10:42, Jeroen van Meeuwen wrote:
Bert Meerman wrote:
Hi everyone,
I've been following the discussion on RPM and GEMS conflict of interest. I found it quite interesting, and I would like to discuss some packaging options already present within ruby and rails.
There are three things I would like to add to Jeroen's page:
- A valid Rails RPM package *explicit* links to gems required.
Strictly speaking, you can lazy load the gems an application needs. This might work, especially if you use gems that are very common (like Rails). Problem is, that a critical dependency is missing. You can add this info in the RPM spec, but I think it is better to make it mandatory more downstream. For RPM packaging, all you need to do is ask for the dependent gems. My question: what are your thoughts on this?
With a packaged Rails product, for example, one aspect of the RPM package will most likely be:
Requires: rubygem(rails)One can enforce a specific version of the Ruby gem:
Requires: rubygem(rails) = 2.3.5in order to prevent breakage on platform or application upgrades, through system capabilities. Pretty much the same goes for Ruby gems, not just packaged applications from downstream consumers or vendors.
Normally, in cases where the upstream of the dependent package is using sane versioning, one could even:
Requires: rubygem(rails)>= 2.3, rubygem(rails)< 2.4which, logically, should result in the Rails RPM package changing for bug- and security-fixes only. However, such seems to not be common practice in the larger Ruby community. This is a problem mid-streams like Fedora will have to deal with -or let it be a Free For All but then what are we doing here?
Anyway, while the application may have (with emphasis on "may have"):
RAILS_GEM_VERSION = '2.3.2'this does not correspond with the capabilities provided by the system in any way. It locks the application down to using 2.3.2 and 2.3.2 only, which should thus also not be used lightly in any situation.
Because of it's own little package management, we have two completely separate dependency paths each of them searching to resolve dependencies (but only one of them through global dependencies), and both are as dependent on a closed dependency tree as another.
However, if you install a Ruby gem RPM, and gem update the system, your RPM is still there, and you can depend on it, and you can be sure that, should the system-wide capability be attempted to be removed in any case, the transaction is going to fail because of missing dependencies.
Imagine you gem install rails but then yum remove ruby -just to emphasize the point I'm trying to make here.
In another example, gem install mysql and then yum update mysql from 5.0 to 5.1, or yum update ruby from 1.8.6 to 1.8.7.
There can only be one package manager doing all the work. It's RPM.
Does that limit anyone from using Gem? No, we're shipping it and in the end, it's your system. Knock yourself out.
Do you have the liberty to derive from what we provide, either by using gem or apt or ...? Sure! Have fun.
Does that mean the RPMs provided are worthless? Euh, not so much.
For one, RPM builds are reproducible. If a test fails on one build system, it will fail on another build system in very much the same way (exceptions apply). If compilation fails because it's using a dynamic library instead of a static library to compile against, or it uses the latest and greatest version rather then the compat- version. We'll figure that out and fix it. Documentation for the upstream gem only accounts for so much of so many system aspects.
- A valid Rails RPM package must contain the version when specifying
gems and the version must be limited. Example: # require 'aws/s3' config.gem 'aws-s3', :lib => 'aws/s3', :version => '0.4.0', # :source => "http://code.whytheluckystiff.net"
Actually, this is not the case. in a config.gem statement, one only is required to list the name, and all other parameters are optional.
For example, one my require a mysql gem, but not care about the version - because the platforms the package or application is distributed for includes the correct version and the mysql gem doesn't change that much.
(see also http://api.rubyonrails.org/classes/Rails/Configuration.html#M002064) This means that the module will use any version greater than 0.4.0. You can also specify smaller, a range or a specific number. The trick is this: if you do this correctly, you can use multiple gem versions on the same system. In theory, this must work. This is what was causing the issue with Lighthouse. I think you should fix the version or make it a range, never a 'greater than' (as in the example and with Lighthouse). My question: what are your thoughts on this, especially the mandatory 'limit the version number'?
Multiple versions of a single applications are useful in a situation in which, from the perspective of a midstream distributor, each and every version can be appropriately maintained. Nevermind the bug-fixes and let's go straight to security fixes. We, the mid-stream distributor, have a tendency to solve most of those security issues for you, and give you the appropriate, fixed version, without breaking anything. It's something virtually unheard of in the greater Ruby community, where feature enhancements and API breakage go hand-in-hand in a release marked as a Security related release.
That said, we do have mechanisms in place that allow us to install more then one version of an application through RPM. We currently use that mechanism for kernel packages. It has a setting that is called "installonlyn_limit", so you may choose how many versions (or actually, RPM releases, not application versions) of the package should be preserved.
- Gems can be vendorized.
OK, this is the moment where a lot of people are going to be amazed, mouth open and staring at the screen for the next five minutes: you can make a Rails application and include the gems in it, as it where local code. This is called vendorization.
</sarcasm> ;-)
A vendorized gem is only available to the application using it: it is part of the source code of that application and located in the /vendor directory of the application root (not in the normal gem directory). In fact, you can vendorize a gem and then even modify the code. On one hand, if you say all gems should be vendorized, you don't have any version conflicts. On the other hand, I heard someone complain about static linking... Personally, I disagree with the vendorization of gems. It obfuscates version management. Question: does Fedore prefer (or mandatorize) a specific approach? Why?
We do not allow the shipping of vendored code or libraries. A good explanation of what vendored code entails is at https://bugzilla.redhat.com/show_bug.cgi?id=470696#c37
That does not mean, however, that downstream consumers or Independent Software Vendors are not allowed to vendor parts of their project or product. We discourage it, for all the right reasons might I add, but we can not (and should not) prohibit it.
Vendored code is not allowed for the parts that the Fedora Project distributes, hence if a product is ever to be a part of the distribution, the vendored code needs to go.
-- Jeroen
Nice comments! I think we're getting to a nice fundamental problem here: Given the facts:
- Higher versions of gems (or Ruby itself) can have changed API's. In
fact, Ruby 1.9 is very different (just about everything breaks) from Ruby 1.8.x. Also, have a look at the change in http://redmine.ruby-lang.org/issues/show/2713. Yes, it's not a bug but a feature. It also means that just about every gem will break (this is like deleting ncurses from a C development system). 2) Mid stream distributor (Fedora) wants to be able to solve security issues. 3) A principle of mid stream distributor (Fedora) is to be able to patch in a transparent fashion. That is, code may not be obfuscated. This principle means, among other things, that vendorized code will not be packaged. 4) A principle of Ruby is to have your code tested. In fact: 'Writing applications without tests makes you a bad person, incapable of love.' (Wilson - The Rails Way). IMHO this can be used to Fedora advantage: remember, you can confront every Ruby developer with this principle - and they *will* appreciate you for doing so! (This is some of the most valued principles of the Ruby community - in fact, the first point made above is not an issue for ruby developers as they will point out that it's all about the tests, not the code)
This leads me to suggesting the following solution:
- Given a codebase C, with gems G1..Gn. Let's say one of the gems is
the CBAC gem, version 1.0. 2) No gems are vendorized. 2) Run tests and verify application. 3) Package the gems. 4) Package the application. Change the application so the gems are fixed to a specific version. 5) Test and verify. We now have a stable build. Package. Let's say the following happens: 6) A security flaw is detected in the CBAC gem, fixed in version 1.7 7) Change the application to use CBAC gem version 1.1 instead of 1.0. Run tests to verify application. 8) Do this until you reach version 1.7. 9) If everything goes OK, commit the change (eg the version number) to the application, repackage and you're done. 10) If the application breaks (let's say at version 1.4), either do one of the following: - Fix the app so it can work with 1.7 (can be done by the app developer) - Fix the security flaw in version 1.4 (can be done by the gem developer) In both cases, the security flaw is fixed, tested, and packaged. ... Packaging of the applications and gems can be done using a sandbox system:
- Basic Fedora version x system
- Ruby interpreter
- Run test of the application/ gem
Remember: in Ruby land, if the tests pass and the app/ gem fails, the code is bad. You can use Wilson's quote to make your point across. Also, this way Fedora would encourage Ruby principles. Personally, as a Rubyist, I would love this approach.
What do you guys think?
Well, if I understand what you want to say here correctly, all you want to discuss here is not limited to ruby or rubygems, and we follow the package updates policy in the distribution (Fedora)
Regards Mamoru _______________________________________________ ruby-sig mailing list ruby-sig@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/ruby-sig
I can't quite follow you on this one Mamoru, but what I'm trying to say is that there may be a way to use the Ruby development chain to the advantage of Distro packaging. The test philosophy used by Ruby allows a totally different perspective on development and maintenance. Without acknowledgment of this philosophy, I think it will become next to impossible to maintain the packages. Of course, before doing all sorts of stuff, I would like to know what you think of this.
Regards, Bert
ruby-sig@lists.fedoraproject.org
-
Bert Meerman -
Jeroen van Meeuwen -
Mamoru Tasaka