On 01/03/2012 01:18 PM, Vít Ondruch wrote:

Dne 3.1.2012 18:40, Michael Stahnke napsal(a):

...

On Tue, Jan 3, 2012 at 7:21 AM, Vít Ondruch <vondruch@redhat.com mailto:vondruch@redhat.com> wrote:

Hi everybody,

I am wondering if we should mention Bundler in Ruby's packaging
guidelines and what should be recommendations? Or should we leave
it in gray area of guidelines?

The root issue isn't using bundler per-se, rather the gem dependencies listed in the rpm spec, gem spec, and bundler's Gemfile may become out of sync.

So long as the guidelines has something to address this I think we'll be fine. Something along the lines of it is up to the package maintainer to ensure all the gem dependency subsystems (rpm, gem, and bundler) are kept in sync.

This is the same for end-users, eg it is up to them to make sure they are using an application that works w/ the ruby packages shipped on the given Fedora version.

...

So, I hate bundler, and I don't think we should allow it. We should however, be aware of how that will hinder Fedora as a ruby development and deployment platform.

My intention was not in a sense of "retire Bundler", definitely not. However, I would like to clarify what is our position and suggestions for packagers.

This whole situation is tricky as the ruby-ecosystem up to this point is to allow the flexibility to manage / install many different version of packages, depending on what the developer needs.

Great for the developer, a nightmare for the administrator, and part of the challenge is start pushing some of these practices upstream, eg as gems mature and the APIs become more stable, start relying on the OS's package management system for the common support model.

Perhaps a hybrid solution for the time being would be to encourage end-users to rely on RPMs where they can while providing the ability to install additional dependencies via gem / bundler. Unfortunately this often just leads to people using gem / bundler for everything, a habit that's difficult to break away from.

Conveying the benefits of RPM over gem using specific examples / use cases might go a long way to addressing this.

For example, something along the lines:

1. If test suite is designed to run with Bundler, please run "$ sed -i

-e 's|require "bundler"||' spec/spec_helper.rb" and "$ sed -i -e 's|Bundler.setup||' spec/spec_helper.rb" to remove the dependency and run the test suite without it.

This is fine, but only takes care of this one case. So long as we have the blurb mentioned above (eg make sure dependencies listed in rpm, gem, and bundler are kept in sync), everything should fall into place. After that we can offer suggestions such as this one to help package maintainers move away from bundler usage.

2. If Rails application uses Bundler, remove the dependency by

deleting the Gemfile and Gemfile.lock and the dependency initialization move into some Rails app initializer (not sure if this approach is feasible). Gemfile and Gemfile.lock *must not* be included in package.

IIRC Jay Guiditta wrote something to this effect, a tool that is transparent to the end user that when dropped in place allows bundler usage to be toggled on / off. eg the system administrator and/or end user can configure the underlying package system to use to resolve dependencies without any changes to the code.

cc'ing Jay here for more thoughts. Perhaps we can pull this in and standardize on it in Fedora (and even push it upstream)

I can imagine other variants of the second point like:

2a) Provide specific Gemfile.lock for each Fedora version. However this is fragile, since update of one gem may break the application.

Well some breakage might be the case anyways as an update to a gem dependency may break the app irregardless if specific versions are listed in the Gemfile.

2b) Delete Gemfile.lock from package and run the application for the first time with root privileges.

Not a fan of this, prone to end-users / admins forgetting, and thus more support tickets. ;-)

2c) After first execution, Gemfile.lock is created in some world writable directory, probably somewhere in /var. However this would need patched version of Bundler.

Also not a huge fan of this, if for no other reason world-writable = bad. :-(

As you can see, there is a lot of options, neither one ideal. Question is which one we should pick?

From developer's point of view, I believe that Bundler has its advantages and I support its usage (may be not for 100% but I see some use-cases). The adjustments necessary for Fedora should be done by packager. If they are done sensitively and in collaboration with upstream, then it is ideal.

To summarize, IMO the best solution would be to:

- include a statement in the package guidelines stating it is up to the package-maintainers and application end-users to make sure they keep their stuff in sync with the gems in the Fedora version they are targeting. Maintainers should update the gemspec and Gemfile / Gemfile.lock dependencies for each package of Fedora they are maintaining.

- coordiante and drive a effort in the upstream communities of the advantages of using a single-stack / shared-support model for packages such as rpm/yum or dep/apt-get when possible, while allowing for cases like gem and bundler when not

- use jay's tool and build others allowing the end-users / administrators to switch (as seemlessly as possible) between bundler and yum managed dependencies, using hybrid solutions where possible / necessary.

Hope this all make sense,

-Mo

On 01/04/2012 08:56 AM, Vít Ondruch wrote:

Dne 4.1.2012 14:43, Mo Morsi napsal(a):

...

On 01/03/2012 01:18 PM, Vít Ondruch wrote:

...

Dne 3.1.2012 18:40, Michael Stahnke napsal(a):

...

On Tue, Jan 3, 2012 at 7:21 AM, Vít Ondruch <vondruch@redhat.com mailto:vondruch@redhat.com> wrote:

Hi everybody,

I am wondering if we should mention Bundler in Ruby's packaging
guidelines and what should be recommendations? Or should we
leave it in gray area of guidelines?

The root issue isn't using bundler per-se, rather the gem dependencies listed in the rpm spec, gem spec, and bundler's Gemfile may become out of sync.

So long as the guidelines has something to address this I think we'll be fine. Something along the lines of it is up to the package maintainer to ensure all the gem dependency subsystems (rpm, gem, and bundler) are kept in sync.

I am afraid of scenario such as:

• Having RPM packaged Rails application
• Having Gemfile.lock present
• Update of Rack to 1.4 version.

Now how you will ensure after such update that you did not broke the application? Even though you can find what packages depends on Rack and you check their .gemspec, how you will find the applications with Gemfile.lock? How you will find packages where Gemfile states 'rack', '1.3'?

If the rpm spec, gemspec, and gemfile are required to be kept in sync, this isn't an issue.

The maintainer of the rails application would need to represent all their dependencies in the rpm spec and the Gemfile lock. Afterall they best know what the various versions of their application require in terms of the various versions of underlying dependencies, and can update their package accordingly (to make it is as restrictive or as flexible as desired).

In this case, trying to update Rack would break things at both the rpm level, the gem level, and the bundler level. Thus first and foremost the system wouldn't permit it if doing the update via rpm, and if doing it via gem / bundler, it wouldn't matter as the rpm version would still be present.

This seems to be fragile and huge overload for packagers.

...

This is the same for end-users, eg it is up to them to make sure they are using an application that works w/ the ruby packages shipped on the given Fedora version.

For end user, it's "easy" I would say. I know my application, I know when I update the system, if something breaks, it is possible to localize the issue easily.

'Easy' can take on multiple meanings, it's not out of the question for end-users to use a slightly older version / branch of an application if that satisfies all their requirements and will 'just work' with the dependencies installed on their system.

-Mo

On 01/04/2012 09:24 AM, Vít Ondruch wrote:

Dne 4.1.2012 15:10, Mo Morsi napsal(a):

...

On 01/04/2012 08:56 AM, Vít Ondruch wrote:

...

Dne 4.1.2012 14:43, Mo Morsi napsal(a):

...

On 01/03/2012 01:18 PM, Vít Ondruch wrote:

...

Dne 3.1.2012 18:40, Michael Stahnke napsal(a):

...

On Tue, Jan 3, 2012 at 7:21 AM, Vít Ondruch <vondruch@redhat.com mailto:vondruch@redhat.com> wrote:

Hi everybody,

I am wondering if we should mention Bundler in Ruby's
packaging guidelines and what should be recommendations? Or
should we leave it in gray area of guidelines?

The root issue isn't using bundler per-se, rather the gem dependencies listed in the rpm spec, gem spec, and bundler's Gemfile may become out of sync.

So long as the guidelines has something to address this I think we'll be fine. Something along the lines of it is up to the package maintainer to ensure all the gem dependency subsystems (rpm, gem, and bundler) are kept in sync.

I am afraid of scenario such as:

• Having RPM packaged Rails application
• Having Gemfile.lock present
• Update of Rack to 1.4 version.

Now how you will ensure after such update that you did not broke the application? Even though you can find what packages depends on Rack and you check their .gemspec, how you will find the applications with Gemfile.lock? How you will find packages where Gemfile states 'rack', '1.3'?

If the rpm spec, gemspec, and gemfile are required to be kept in sync, this isn't an issue.

The maintainer of the rails application would need to represent all their dependencies in the rpm spec and the Gemfile lock. Afterall they best know what the various versions of their application require in terms of the various versions of underlying dependencies, and can update their package accordingly (to make it is as restrictive or as flexible as desired).

In this case, trying to update Rack would break things at both the rpm level, the gem level, and the bundler level. Thus first and foremost the system wouldn't permit it if doing the update via rpm, and if doing it via gem / bundler, it wouldn't matter as the rpm version would still be present.

If you update rack, you should ensure that you did not broken the activesupport (possibly fixing its .gemspec and may be some failures).

Why? Nowhere else in Fedora is a package maintainer required to make sure that all the dependencies on that package do not break with an update.

Rather, the policy is to keep versions stable within a Fedora release, and if a major package is being updated in a new release, to give plenty of notification via the mailing lists and what not.

Jeroen is the owner or rubygem-rack, I am the owner of rubygem-activesupport. If he wishes to update rack in the next version of Fedora he should announce on the ruby-sig mailing list this upcoming update, making the rpms available as soon as he can (either hosted by himself or via rawhide), and it is up to me to make sure rubygem-activesupport doesn't break.

If I need to update activesupport to ensure this, I should do the same, eg announce on list and make the rpms available. This way the community is kept in sync and we can discuss updates / resolve issues before hand.

I'll fully admit this isn't always the standard practice but this is more of a community issue and while the the tooling / guidelines can address these practices, it is up to the community to enforce them.

If activesupport still works, everything what depends on it should also work in theory. However, several levels up is sitting Gemfile which needs to be adjusted to new rack. How you would know that? You can't and this is what I am afraid of.

If an other package depends on rack directly, the maintainer should ensure it works when they receive notification rack is being updated on the mailing list.

If the package only depends on activesupport but mistakingly lists the rack dependency, a bug should be files upstream to resolve this.

-Mo

On Tue, Jan 03, 2012 at 09:40:52AM -0800, Michael Stahnke wrote:

I really dislike bundler. However, from the Ruby ecosystem point of view, it's there and it's not going anywhere. It featured on every rubygem page. It certainly conflicts with the bundled library viewpoint Fedora has, but without it, many many applications built upon Ruby won't make it into Fedora. This is especially true since rpm by design cannot have multiple versions of the same package installed; whereas gem can.

Sure it can. Look at kernel as an example. It depends on how the package is installed as to whether the rpm will upgrade an existing package or install in parallel (and also other factors such as locations of files, etc.).

On 01/04/2012 08:04 AM, Vít Ondruch wrote:

Dne 4.1.2012 13:57, Darryl L. Pierce napsal(a):

...

On Tue, Jan 03, 2012 at 09:40:52AM -0800, Michael Stahnke wrote:

...

I really dislike bundler. However, from the Ruby ecosystem point of view, it's there and it's not going anywhere. It featured on every rubygem page. It certainly conflicts with the bundled library viewpoint Fedora has, but without it, many many applications built upon Ruby won't make it into Fedora. This is especially true since rpm by design cannot have multiple versions of the same package installed; whereas gem can.

Sure it can. Look at kernel as an example. It depends on how the package is installed as to whether the rpm will upgrade an existing package or install in parallel (and also other factors such as locations of files, etc.).

---

ruby-sig mailing list ruby-sig@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/ruby-sig

RPM allows to install several versions of package in parallel, however yum cannot handle them and kernel is exceptional case. So if you replace RPM with yum in the paragraph above, then it is correct.

Vit

Yes the kernel is a very exceptional case, special handling for a specific list of packages is hard coded into the yum source.

I don't think we'll be able to do the same for a variable (and ever-growing) amount of rubygems.

-Mo

On Wed, Jan 04, 2012 at 02:10:20PM +0100, Jeroen van Meeuwen (Kolab Systems) wrote:

Actually it's an explicit instruction from YUM to RPM, where YUM uses the 'installonly_pkgs' configuration directive so that it knows to issue this slightly different RPM command to RPM for a certain list of package names.

This is exactly why this mechanism can't (should not) be used for Ruby*; the list of packages that would need to be included in this list of packages to only install (not update, not upgrade) would be enormous, and subject to change quite frequently.

We can create a plugin that does the decision making for us, rather than maintaining a list of packages. Package name accounting shouldn't be a reason to do away with a very powerful feature of gems.