----- Original Message -----

On 06/23/2014 11:20 AM, Joe Rafaniello wrote:

...

Hi all,

Does anyone use gem2rpm to upgrade an existing rpm to new versions of upstream gems?

I'm contemplating working on a pull request to make gem2rpm aware of an existing .spec file and only update specific sections such as: version, requires, buildrequires, and adding a changelog. As it is now, it overwrites the existing rpm spec, removing any changelog entries, etc.

Is this a good idea? What do others do to regenerate the updated version, requires/buildrequires to avoid human error?

Thanks,

You should be able to use polisher from where-ever to do this programmatically.

Just updated the last outstanding PR to incorporate your latest feedback

https://github.com/ManageIQ/polisher/pull/98

As always, glad to review any additional PR's w/ any new polisher features & enhancements.

-Mo

Yeah, Mo. I'm wondering if the complexity should live in polisher or gem2rpm. It seems strange to use gem2rpm for initial rpms and polisher for updates.

If we can solve both initial and updates to a spec in gem2rpm, we can eliminate similar logic in both.

----- Original Message -----

On 06/23/2014 02:01 PM, Joe Rafaniello wrote:

...

----- Original Message -----

...

On 06/23/2014 11:20 AM, Joe Rafaniello wrote:

...

Hi all,

Does anyone use gem2rpm to upgrade an existing rpm to new versions of upstream gems?

I'm contemplating working on a pull request to make gem2rpm aware of an existing .spec file and only update specific sections such as: version, requires, buildrequires, and adding a changelog. As it is now, it overwrites the existing rpm spec, removing any changelog entries, etc.

Is this a good idea? What do others do to regenerate the updated version, requires/buildrequires to avoid human error?

Thanks,

You should be able to use polisher from where-ever to do this programmatically.

Just updated the last outstanding PR to incorporate your latest feedback

https://github.com/ManageIQ/polisher/pull/98

As always, glad to review any additional PR's w/ any new polisher features & enhancements.

-Mo

Yeah, Mo. I'm wondering if the complexity should live in polisher or gem2rpm. It seems strange to use gem2rpm for initial rpms and polisher for updates.

If we can solve both initial and updates to a spec in gem2rpm, we can eliminate similar logic in both.

There are some common aspects to the process and some differences. For creation gem2rpm uses erb-based templates to generate new spec files. For updating, polisher parses the spec & attempts to interpolate gem/rpm contents.

There are tradeoffs either way, generating a new spec gives you a fresh start, but updating it can incorporate existing changes which still apply.

As it stands polisher currently does alot of the parsing / updating legwork. Of course there are edge cases which can be addressed, but those should be able to be taken care of w/ small / targeted PR's (as issues are discovered).

-Mo

If gem2rpm could handle both initial and updates to the spec, we can cleanly separate the responsibilities and just delegate to gem2rpm for all or some of that work. I would be happy to have polisher be the wrapper around gem2rpm for larger "workflows."

I have only glanced at the gem2rpm code and don't know how much effort it is to handle "updates", this is just a curiosity at this point.

For example, I'm curious how are changes in packaging conventions currently handled? Through new gem2rpm templates? Is there a migration path for updating an existing rpm spec from an old template to a new template? Is this all done manually currently? Are there tools to enforce/verify packaging conventions?

Thanks,

----- Original Message -----

On 06/23/2014 09:58 PM, Joe Rafaniello wrote:

...

If gem2rpm could handle both initial and updates to the spec, we can cleanly separate the responsibilities and just delegate to gem2rpm for all or some of that work. I would be happy to have polisher be the wrapper around gem2rpm for larger "workflows."

I have only glanced at the gem2rpm code and don't know how much effort it is to handle "updates", this is just a curiosity at this point.

For example, I'm curious how are changes in packaging conventions currently handled? Through new gem2rpm templates?

Yes. By default is uses this template[0] which was recently updated to reflect new ruby 2.1 changes.

...

Is there a migration path for updating an existing rpm spec from an old template to a new template? Is this all done manually currently? Are there tools to enforce/verify packaging conventions?

Vit, has written a set of scripts which update the specs to current guidelines[1], I don't know if some edge cases are missed though. You can follow a previous discussion[2].

And to chime in the discussion, changing only the sections you mentioned should not be that big a deal.

• version is easy to be fetched from rubygems.org
• Changelog is also easy
• Requires are not needed for fc21 and above (could use a different

template for other branches)

• BR could be tricky, since current implementation [3] (correct me if

I'm wring) checks development deps from gemspec and eg. tools needed for the test suites might not be in that list. So, you can't avoid manual intervention after all.

As Josef wrote, ideally what we need is some sort of a web app that checks/builds every time a new version is out.

[0] https://github.com/lutter/gem2rpm/blob/master/templates/fedora-21-rawhide.sp... [1] https://github.com/voxik/fermig [2] https://lists.fedoraproject.org/pipermail/ruby-sig/2014-April/001537.html [3] https://github.com/lutter/gem2rpm/blob/master/templates/fedora-21-rawhide.sp...

-- FAS : axilleas GPG : 0xABF99BE5 Blog: http://axilleas.me _______________________________________________ ruby-sig mailing list ruby-sig@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/ruby-sig

Thanks. Those are all good references. Does anyone have opinions as to where automation of spec updates belong? I'm wondering why gem2rpm wasn't used for Vit's "fermig" scripts? It would be nice to have one tool that automated some of the rpm spec changes for rubygems packages. I hope to get some cycles to work on adding update to an existing spec in gem2rpm in the next week or so, I'll report back anything I come up with.

Thanks.

----- Original Message -----

Dne 24.6.2014 20:47, Joe Rafaniello napsal(a):

...

----- Original Message -----

...

On 06/23/2014 09:58 PM, Joe Rafaniello wrote:

...

If gem2rpm could handle both initial and updates to the spec, we can cleanly separate the responsibilities and just delegate to gem2rpm for all or some of that work. I would be happy to have polisher be the wrapper around gem2rpm for larger "workflows."

I have only glanced at the gem2rpm code and don't know how much effort it is to handle "updates", this is just a curiosity at this point.

For example, I'm curious how are changes in packaging conventions currently handled? Through new gem2rpm templates?

Yes. By default is uses this template[0] which was recently updated to reflect new ruby 2.1 changes.

...

Is there a migration path for updating an existing rpm spec from an old template to a new template? Is this all done manually currently? Are there tools to enforce/verify packaging conventions?

Vit, has written a set of scripts which update the specs to current guidelines[1], I don't know if some edge cases are missed though. You can follow a previous discussion[2].

And to chime in the discussion, changing only the sections you mentioned should not be that big a deal.

• version is easy to be fetched from rubygems.org
• Changelog is also easy
• Requires are not needed for fc21 and above (could use a different

template for other branches)

• BR could be tricky, since current implementation [3] (correct me if

I'm wring) checks development deps from gemspec and eg. tools needed for the test suites might not be in that list. So, you can't avoid manual intervention after all.

As Josef wrote, ideally what we need is some sort of a web app that checks/builds every time a new version is out.

[0] https://github.com/lutter/gem2rpm/blob/master/templates/fedora-21-rawhide.sp... [1] https://github.com/voxik/fermig [2] https://lists.fedoraproject.org/pipermail/ruby-sig/2014-April/001537.html [3] https://github.com/lutter/gem2rpm/blob/master/templates/fedora-21-rawhide.sp...

-- FAS : axilleas GPG : 0xABF99BE5 Blog: http://axilleas.me _______________________________________________ ruby-sig mailing list ruby-sig@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/ruby-sig

Thanks. Those are all good references. Does anyone have opinions as to where automation of spec updates belong? I'm wondering why gem2rpm wasn't used for Vit's "fermig" scripts?

Well, what is in the upstream fermig repository and what I am actually using during migration periods is not always the same. From my point of view, it is one shot library, which should be ideally used once a year (or how often the guidelines are updated) and that is it.

Actually, I see where are you aiming, i.e. the upgrade of gem is good opportunity to update the .spec, etc. but from tooling POV, it is easier to do it in two steps, e.g. update everything in Fedora when guidelines are updated and upgrade to new version when new version is released.

Vít

Thanks, this is great information. I think it's easier to add "update" functionality to gem2rpm by ensuring the template and spec both get updated to latest conventions. It's really complicated to try to apply changes from a spec generated with new conventions into a spec using old or no conventions.

1) Update or at least highlight changes needed to the template we're using to abide by conventions 2) Manually or script update the existing template to latest conventions in 1) 3) Commit the updated template 4) Call the fermig scripts to update the existing .spec to latest conventions 5) Commit the .spec with latest conventions 6) Use gem2rpm to generate a new .spec from the updated template in 2) for the desired gem version 7) Apply the applicable changes from 4) into 3) (requires, buildrequires, version, etc.) 8) Take a breathe and see if we broke things ;-)

Do you have any links to distgit repos where the template and spec were upgraded for conventions? Vit, the scripts don't appear to be cumulative. So for example, I'd imagine the f19.rb should be run before running update.rb which calls f21.rb. Is this correct?

...

It would be nice to have one tool that automated some of the rpm spec changes for rubygems packages. I hope to get some cycles to work on adding update to an existing spec in gem2rpm in the next week or so, I'll report back anything I come up with.

Thanks.

---

ruby-sig mailing list ruby-sig@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/ruby-sig

On 06/25/2014 10:15 AM, Joe Rafaniello wrote:

...

Well, what is in the upstream fermig repository and what I am actually using during migration periods is not always the same. From my point of view, it is one shot library, which should be ideally used once a year (or how often the guidelines are updated) and that is it.

Actually, I see where are you aiming, i.e. the upgrade of gem is good opportunity to update the .spec, etc. but from tooling POV, it is easier to do it in two steps, e.g. update everything in Fedora when guidelines are updated and upgrade to new version when new version is released.

Vít

Corrected the line 7) below...

Thanks, this is great information. I think it's easier to add "update" functionality to gem2rpm by ensuring the template and spec both get updated to latest conventions.

Would have to agree w/ Vit on this one, it'd be easier to split things into multiple separate use cases:

- generate a new spec from scratch

- updated a single package version to new specifications

- updating a single package between versions

The Unix philosophy is to build and use simple tools that do simple things well. We should try and nail each of these down individually before trying to abstract all three into a single stage / tool.

It's really complicated to try to apply changes from a spec generated with new conventions into a spec using old or no conventions.

Have you attempted this? Perhaps some code would better illustrate what you are going for?

-Mo

On 06/25/2014 11:56 AM, Joe Rafaniello wrote:

----- Original Message -----

...

On 06/25/2014 10:15 AM, Joe Rafaniello wrote:

...

...

Well, what is in the upstream fermig repository and what I am actually using during migration periods is not always the same. From my point of view, it is one shot library, which should be ideally used once a year (or how often the guidelines are updated) and that is it.

Actually, I see where are you aiming, i.e. the upgrade of gem is good opportunity to update the .spec, etc. but from tooling POV, it is easier to do it in two steps, e.g. update everything in Fedora when guidelines are updated and upgrade to new version when new version is released.

Vít

Corrected the line 7) below...

Thanks, this is great information. I think it's easier to add "update" functionality to gem2rpm by ensuring the template and spec both get updated to latest conventions.

Would have to agree w/ Vit on this one, it'd be easier to split things into multiple separate use cases:

• generate a new spec from scratch

• updated a single package version to new specifications

• updating a single package between versions

The Unix philosophy is to build and use simple tools that do simple things well. We should try and nail each of these down individually before trying to abstract all three into a single stage / tool.

...

It's really complicated to try to apply changes from a spec generated with new conventions into a spec using old or no conventions.

Have you attempted this? Perhaps some code would better illustrate what you are going for?

-Mo

---

ruby-sig mailing list ruby-sig@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/ruby-sig

Thanks Mo, that's really my point. In order to update a gem version programmatically, there are a series of steps that should happen, regardless if it's the same or different tools:

That's not what I said at all.

"The Unix philosophy is to build and use simple tools that do simple things well. We should try and nail each of these down individually before trying to abstract all three into a single stage / tool."

If using the gem2rpm mechanism to generate a new spec file for the new gem version and merge the applicable changes, you have a few prerequisites:

1. update the template to latest conventions
2. update the existing spec to latest conventions
3. gem2pm using the updated template for new gem version to create the new version spec file

We're beating on a dead horse here. Obviously I can't stop you from sending patches / following any approach you want. I can only recommend what I believe will / won't work based on my experience with packaging and the automation effort.

Then compare the new version spec to the old version spec and merge any desired changes.

I don't think you can compare two specs unless they're using the same conventions, otherwise there would be lots of noise in the diff.

Again to reiterate my question, have you actually tried developing a solution to this before and have run into problems? Or is this just speculation?

Each version of the guidelines is well documented. We can add some command line and API flags to Polisher where the user can specify the versions there are upgrading from/to (if not inferred).

-Mo

----- Original Message -----

Dne 25.6.2014 16:15, Joe Rafaniello napsal(a):

...

----- Original Message -----

----- Original Message -----

...

Dne 24.6.2014 20:47, Joe Rafaniello napsal(a):

...

----- Original Message -----

...

On 06/23/2014 09:58 PM, Joe Rafaniello wrote:

...

If gem2rpm could handle both initial and updates to the spec, we can cleanly separate the responsibilities and just delegate to gem2rpm for all or some of that work. I would be happy to have polisher be the wrapper around gem2rpm for larger "workflows."

I have only glanced at the gem2rpm code and don't know how much effort it is to handle "updates", this is just a curiosity at this point.

For example, I'm curious how are changes in packaging conventions currently handled? Through new gem2rpm templates?

Yes. By default is uses this template[0] which was recently updated to reflect new ruby 2.1 changes.

...

Is there a migration path for updating an existing rpm spec from an old template to a new template? Is this all done manually currently? Are there tools to enforce/verify packaging conventions?

Vit, has written a set of scripts which update the specs to current guidelines[1], I don't know if some edge cases are missed though. You can follow a previous discussion[2].

And to chime in the discussion, changing only the sections you mentioned should not be that big a deal.

• version is easy to be fetched from rubygems.org
• Changelog is also easy
• Requires are not needed for fc21 and above (could use a different

template for other branches)

• BR could be tricky, since current implementation [3] (correct me if

I'm wring) checks development deps from gemspec and eg. tools needed for the test suites might not be in that list. So, you can't avoid manual intervention after all.

As Josef wrote, ideally what we need is some sort of a web app that checks/builds every time a new version is out.

[0] https://github.com/lutter/gem2rpm/blob/master/templates/fedora-21-rawhide.sp... [1] https://github.com/voxik/fermig [2] https://lists.fedoraproject.org/pipermail/ruby-sig/2014-April/001537.html [3] https://github.com/lutter/gem2rpm/blob/master/templates/fedora-21-rawhide.sp...

-- FAS : axilleas GPG : 0xABF99BE5 Blog: http://axilleas.me _______________________________________________ ruby-sig mailing list ruby-sig@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/ruby-sig

Thanks. Those are all good references. Does anyone have opinions as to where automation of spec updates belong? I'm wondering why gem2rpm wasn't used for Vit's "fermig" scripts?

Well, what is in the upstream fermig repository and what I am actually using during migration periods is not always the same. From my point of view, it is one shot library, which should be ideally used once a year (or how often the guidelines are updated) and that is it.

Actually, I see where are you aiming, i.e. the upgrade of gem is good opportunity to update the .spec, etc. but from tooling POV, it is easier to do it in two steps, e.g. update everything in Fedora when guidelines are updated and upgrade to new version when new version is released.

Vít

Corrected the line 7) below...

Thanks, this is great information. I think it's easier to add "update" functionality to gem2rpm by ensuring the template and spec both get updated to latest conventions. It's really complicated to try to apply changes from a spec generated with new conventions into a spec using old or no conventions.

1. Update or at least highlight changes needed to the template we're using

to abide by conventions 2) Manually or script update the existing template to latest conventions in

2. Commit the updated template
3. Call the fermig scripts to update the existing .spec to latest

conventions 5) Commit the .spec with latest conventions 6) Use gem2rpm to generate a new .spec from the updated template in 2) for the desired gem version 7) Apply the applicable changes from 6) into 5) (requires, buildrequires, version, etc.) 8) Take a breathe and see if we broke things ;-)

Do you have any links to distgit repos where the template and spec were upgraded for conventions? Vit, the scripts don't appear to be cumulative. So for example, I'd imagine the f19.rb should be run before running update.rb which calls f21.rb. Is this correct?

I am afraid I don't follow you. But I'll try to make same remarks which hopefully answers some of your questions.

• No, fermig does not do cumulative updates. It should be the tool,

which should be run after guidelines update and it will update all packages in Rawhide at once and rebuilds them. Since all packages are up2date from that moment, you don't need to care about the history and it does not need to be cumulative. Unfortunately, this is just theory.

• If you maintain several gems, you might create your own template for

gem2rpm. The template would be unique for every gem you maintain. The template would be basically your latest .spec file, with a few macros remaining, Ideally, just the version macro. When new gem version is release, you regenerate the .spec using that template and you are done.

• It should be possible to run fermig on gem2rpm templates and get it

updated. This will get the template up2date with the latest guidelines and you can continue to use the template for updates. The success rate will depend on the template complexity.

• Please note, that the rubygem- package updates are bit easier in F21+,

since you don't need to care about Requires. The requires are autogenerated for you.

Thanks. This does answer many of my questions.

• If I should choose what will be my next target in a war for simplified

gem packaging, then I'd love to focus to simplify %files sections. The first step might be something like %{gem_directories} macro, which would replace %{gem_instdir}, %{gem_spec} and %{gem_cache} set of macros, which are repeated in every gem. The next step of evolution might be something along the way Java (maven) packages do, e.g. you'd do just %file -f filelist and that was it. This would be one huge step forward to remove the need of any upgrade tool.

These are sections that I'm not familiar enough to know if we can automate how to update these values for new gem versions. I was thinking of starting with automatically merging changes from gem2rpm output for the new version gem for only the requires, buildrequires, and version. The remaining spec changes could be manually reviewed and merged as applicable. Of courese, if you know of other fields/sections that should be easy to automatically merge from the gem2rpm output, we can do them too.

Vít _______________________________________________ ruby-sig mailing list ruby-sig@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/ruby-sig