Sending to packaging@lists.fp.o where this belongs.
On Tue, Feb 14, 2012 at 05:27:06PM -0500, Mo Morsi wrote:
Thoughts:
- I am also not a fan of the current approach of always repackaging the gem, seems like extra steps / added complexity which is unneeded, what is the justification for it? In the majority of cases, gem install works out of the box and the gem doesn't need any modifications.
There's a lot of justification in the ticket. There's various pieces.
* The historical ruby/gem guidelines are a huge hack that FPC approved because it seemed that gems wouldn't work with the normal separation of %prep %build %install. The recent draft showed that this is no longer the case. * Proper separation allows rpmbuild -b[pci] --shortcircuit to work properly. * The previous guidelines actually didn't allow patching. The original proposed draft guidelines ended up being overly complex as they had separate cases for gem, gem w/ C extension, patched gem, patched gem w/ C extension, patched gem w/ C extension that fails to build. This strategy has a single way to handle all of this. The cost is three extra lines which are easy to conceptualize and explain. (unpacking the code from an archive, then assembling the code into a form that can be built from). * Proper separation of the steps allows people new to ruby packaging but old hands at rpm packaging (a large portion of reviewers) to come to understand the basics of packaging rubygems.
- what is the rationality behind having a gem package provide ruby (RUBYLIBRARY)? Seems like the non-gem and gem package distinction is more explicit when the gems provide rubygem(gemname) and non-gems provide ruby (libraryname). Since the dependency needs to be represented anyways in any other packages that depend on the gem, an explicit dependency on rubygem (gemname) might not be a bad idea.
In the old guidelines, rubygems were distinct from plain ruby packages. This was because at the time they were written: require 'rubylibrary'
imported a plain ruby library and: gem_require 'rubylibrary'
The had wholly different syntax at the source code level. The piece of vondruch's draft that stated that non-gem subpackages were no longer needed for gem packages lead me to find that the rubygems package now sets up the ruby library path so that "require 'rubylibrary'" works for both gems and nongems. With that change, we don't need to package all ruby libraries in a non-gem version and the need to have separate requires and provides lessens. (The caveat here is that ruby code would need to have require 'rubygems' somewhere early in its code but if that's considered sufficiently compatible for one of these cases, then it's sufficient for both cases)
When might you absolutely need to use Require: rubygem()? I know of the specific case when you are requiring a specific version of code via:: gem 'rubylibrary'
I also imagine that there's ways that ruby code can make use of the metadata that gems provide but I don't know how to detect those. In either of those cases, the rubygem() require would seem to be most appropriate.
When might you want to use the Require ruby() version? When a package's code uses plain "require 'rubylibrary'", it would work whether the rpm package is shipping a rubygem or a plain ruby library. Therefore, if the packager dosn't want to guess whether we're shipping the relevant library as a rubygem or as a non-gem ruby library, they should be able to do a simple "Requires: ruby(library)". If the packager knows that we package that pacticular library as a rubygem, they can use the rubygem() version of the requires instead but they are no longer required to in this case.
So I think answering your question, whether the dependency is satisfied by a gem or non-gem library is no longer of interest in the simple case of a script or piece of code using "require 'rubylibrary'". For places where it does matter, the Guideline should require using Requires: rubygem() (if you have a list of other things to look for, that could be added to flesh out that section).
- Binary Extensions - gem install then recompile doesn't make alot of sense when the other alternative, gem unpack, patch, rebuild, and them install would work in both cases. Why is this split out like this?
I'm not sure I understand this. Are you referring to the draft guideline that vondruch originally proposed with the many separate pages or the changes that I have been making to have only a single case? The single case does the gem unpack, patch, rebuild (via gem install), and finally install via mkdir/cp/mv.
I've forked this to this page: https://fedoraproject.org/wiki/User:Toshio/RubyPackagingDraft
for now so that people can better see what that proposal looks like. I'll have to keep the two pages in sync for a bit until someone does or does not bring up a rubygem where the reduced-complexity guidelines are as direly broken as people are claiming.
- the rspec package will soon finally be updated to rspec2 so the BR: rspec-core in the test guidelines can be changes to a BR: rspec
Nice. Can you specify what versions of Fedora that will be operable on? It's nice to note things like that in the Guidelines.
Answers to specific questions expressed in the draft:
- Interpreter independence - seems reasonable, how we address supporting these
libraries on multiple interpreters needs to be flushed out, abliet not necessarily for this release as time is short
- Move text about interpreter independence to here - seems reasonable to me
So -- vondruch seems to have issues with this. Code doesn't seem to be entirely interpreter independent? Or rather, only certain code is? The FPC really needs to understand this portion of the issue better as it affects things like where the directories belong in the FHS, whether it makes sense to share rubygems across interpreters and whether it makes sense to share vendorlib or vendorarch across interpreters. Example srpms are the best way to make clear what's happening here :-).
Note that when you talk about this being possibly something for later releases -- unfortunately, that seems to be an underlying assumption of the draft guidelines that we were presented with so we need to be able to get correct the parts that make sense.
- Confirm change (remove ruby(rubygems) dep) - also seems reasonable to me
I've finalized this.
- Give examples? and Do we want to tell what the arguments to gem install do? -
both would be appreciated
heh :-) Unfortunately, I don't believe we've had a ruby person on the FPC since lutter (who came up with the original guidelines). So if examples and explanations of arguments are desired, I'll need to recieve them from people who use ruby.
- Replacement instructions - as discussed not a huge fan of this
Unless someone can present examples where this doesn't work that aren't flaws with upstream code, this is likely to become the way to do things.
- Library placement - the bit about 'foo' is somewhat confusing, perhaps
replacing it with something like "[ext|lib]" or similar would work instead?
Yeah -- this was in the portion of the guidelines to be replaced. In my draft, I've simply removed that section (as it's already in the previous section where I renamed it $REQUIRE_PATHS with a comment about where $REQUIRE_PATHS comes from)
-Toshio
On 02/15/2012 11:14 PM, Toshio Kuratomi wrote:
Sending to packaging@lists.fp.o where this belongs.
On Tue, Feb 14, 2012 at 05:27:06PM -0500, Mo Morsi wrote:
Thoughts:
- I am also not a fan of the current approach of always repackaging the gem,
seems like extra steps / added complexity which is unneeded, what is the justification for it? In the majority of cases, gem install works out of the box and the gem doesn't need any modifications.
There's a lot of justification in the ticket. There's various pieces.
- The historical ruby/gem guidelines are a huge hack that FPC approved because it seemed that gems wouldn't work with the normal separation of %prep %build %install. The recent draft showed that this is no longer the case.
- Proper separation allows rpmbuild -b[pci] --shortcircuit to work properly.
- The previous guidelines actually didn't allow patching. The original proposed draft guidelines ended up being overly complex as they had separate cases for gem, gem w/ C extension, patched gem, patched gem w/ C extension, patched gem w/ C extension that fails to build. This strategy has a single way to handle all of this. The cost is three extra lines which are easy to conceptualize and explain. (unpacking the code from an archive, then assembling the code into a form that can be built from).
- Proper separation of the steps allows people new to ruby packaging but old hands at rpm packaging (a large portion of reviewers) to come to understand the basics of packaging rubygems.
OK this makes a lot more sense to me now. The counter-point I would raise is that the majority of gems that I've seen do not require patching. While I agree that this approach brings things more inline w/ the Fedora way of doing things, it does come at the expense of adding additional steps which will not be necessary most of the time.
Perhaps a compromise would be to make the gem unpacking steps optional, and able to be skipped if not necessary?
That being said, I just wanted to verify that "../%{gem_name}-%{version}/" is the correct location of the gem when the user builds it in the rpm spec.
Also it might be desirable to allow the user to specify the gemspec via SOURCE1 (or a subsequent source) so as to be able to pull the original / developer-maintainer gemspec from upstream. I've never used the 'gem spec' command and can't attest to its usage.
Furthermore, this [1] may be of interest. It is a rake task which we wrote and regularly maintain as part of the Aeolus project [2] for building rpms from Rake (the ruby build system). It may be of use to the Fedora community to build rpms from ruby sources (we use it to build our rpms, we also have a yum repo Rake module [3] as well)
- what is the rationality behind having a gem package provide ruby
(RUBYLIBRARY)? Seems like the non-gem and gem package distinction is more explicit when the gems provide rubygem(gemname) and non-gems provide ruby (libraryname). Since the dependency needs to be represented anyways in any other packages that depend on the gem, an explicit dependency on rubygem (gemname) might not be a bad idea.
In the old guidelines, rubygems were distinct from plain ruby packages. This was because at the time they were written: require 'rubylibrary'
imported a plain ruby library and: gem_require 'rubylibrary'
The had wholly different syntax at the source code level. The piece of vondruch's draft that stated that non-gem subpackages were no longer needed for gem packages lead me to find that the rubygems package now sets up the ruby library path so that "require 'rubylibrary'" works for both gems and nongems. With that change, we don't need to package all ruby libraries in a non-gem version and the need to have separate requires and provides lessens. (The caveat here is that ruby code would need to have require 'rubygems' somewhere early in its code but if that's considered sufficiently compatible for one of these cases, then it's sufficient for both cases)
When might you absolutely need to use Require: rubygem()? I know of the specific case when you are requiring a specific version of code via:: gem 'rubylibrary'
I also imagine that there's ways that ruby code can make use of the metadata that gems provide but I don't know how to detect those. In either of those cases, the rubygem() require would seem to be most appropriate.
When might you want to use the Require ruby() version? When a package's code uses plain "require 'rubylibrary'", it would work whether the rpm package is shipping a rubygem or a plain ruby library. Therefore, if the packager dosn't want to guess whether we're shipping the relevant library as a rubygem or as a non-gem ruby library, they should be able to do a simple "Requires: ruby(library)". If the packager knows that we package that pacticular library as a rubygem, they can use the rubygem() version of the requires instead but they are no longer required to in this case.
So I think answering your question, whether the dependency is satisfied by a gem or non-gem library is no longer of interest in the simple case of a script or piece of code using "require 'rubylibrary'". For places where it does matter, the Guideline should require using Requires: rubygem() (if you have a list of other things to look for, that could be added to flesh out that section).
OK again this makes more sense, thanks again for the explanation. If I understand correctly (and perhaps this should be somewhat elaborated in the guidelines), if a ruby package ships a top level module which can simply be 'required' as is that package is to 'Provide: ruby(modulename)'. If a ruby package ships w/ a module that is meant to be pulled in via rubygems it is to 'Provider: rubygem(modulename)'
As far as packages that depend on it, they are to pull in the dependency via whichever means (Requires ruby() or rubygem()) the ruby package which they are shipping pulls it in.
Of course this brings up the issue of naming conflicts, for example if two independent upstream projects separately provide the ruby and rubygem packages (for example the ruby-sqlite3 and rubygem-sqlite3 packages correspond to separate projects), but I imaging this isn't a problem just for ruby.
- Binary Extensions - gem install then recompile doesn't make alot of sense
when the other alternative, gem unpack, patch, rebuild, and them install would work in both cases. Why is this split out like this?
I'm not sure I understand this. Are you referring to the draft guideline that vondruch originally proposed with the many separate pages or the changes that I have been making to have only a single case? The single case does the gem unpack, patch, rebuild (via gem install), and finally install via mkdir/cp/mv.
I've forked this to this page: https://fedoraproject.org/wiki/User:Toshio/RubyPackagingDraft
for now so that people can better see what that proposal looks like. I'll have to keep the two pages in sync for a bit until someone does or does not bring up a rubygem where the reduced-complexity guidelines are as direly broken as people are claiming.
Ah ok yes, was mistaken (thought you added that bit). Can ignore this.
- the rspec package will soon finally be updated to rspec2 so the BR:
rspec-core in the test guidelines can be changes to a BR: rspec
Nice. Can you specify what versions of Fedora that will be operable on? It's nice to note things like that in the Guidelines.
I have the patch to update rspec to rspec2 ready to go, so unless there are objections from ruby-sig (I've heard none up to this point) will push this in for F17.
I just went through and verified the last package that needed to be updated, aeolus-conductor, works as intended against rspec2 so we should be good to go (based on Vit's rspec2 emails, shout out if I missed anything)
Answers to specific questions expressed in the draft:
- Interpreter independence - seems reasonable, how we address supporting these
libraries on multiple interpreters needs to be flushed out, abliet not necessarily for this release as time is short
- Move text about interpreter independence to here - seems reasonable to me
So -- vondruch seems to have issues with this. Code doesn't seem to be entirely interpreter independent? Or rather, only certain code is? The FPC really needs to understand this portion of the issue better as it affects things like where the directories belong in the FHS, whether it makes sense to share rubygems across interpreters and whether it makes sense to share vendorlib or vendorarch across interpreters. Example srpms are the best way to make clear what's happening here :-).
Noarch / pure-ruby code should work across interpreters. eg MRI (c-ruby) interpreter will parse the same pure-ruby-1.9.3 code as JRuby does.
It gets problematic when the Ruby C API is incorporated to build native platform extensions for Ruby. AFAIK (admittedly I have yet to do this myself) bindings need to be written for each interpreter the code needs to run on, so we will need to compile the package again MRI, JRuby, other interpreters specifically (if the upstream project supports those interpreters).
I'm not sure if there is a standard way to build Ruby extensions against multiple interpreters and/or decouple the interpreter implementation from the extension build process, perhaps something worth looking into.
- Give examples? and Do we want to tell what the arguments to gem install do? -
both would be appreciated
heh :-) Unfortunately, I don't believe we've had a ruby person on the FPC since lutter (who came up with the original guidelines). So if examples and explanations of arguments are desired, I'll need to recieve them from people who use ruby.
/me just joined the FPC mailing list :-)
Will see about drafting some examples if I can get some cycles free at some point.
-Mo
[1] https://github.com/aeolusproject/aeolus-configure/blob/master/rake/rpmtask.r... [2] http://aeolusproject.org [3] https://github.com/aeolusproject/aeolus-configure/blob/master/rake/yumtask.r...
Hi, sorry I'm not replying to the original Toshio's post, but I wasn't subscribed to this list at that time. I would like to ask how we are standing with the Ruby guidelines Draft and what are the current blockers, so that we can discuss them here and reach some kind of consensus.
Thanks!
packaging@lists.fedoraproject.org