Hello dear list!
I am applying for this year's GSOC [0] and I would like to contribute by bringing GitLab [1][2] in Fedora. For those that don't already know GitLab, it is a Self hosted Git management software written on Ruby on Rails, pretty much like github's interface (you can try a demo here [3]).
It is a ruby app with lots of dependencies and I think this is a great opportunity to extend the list of packaged gems supported by Fedora. Let alone the fact that with the momentum GitLab has gained (it is in the top 50 most popular open source projects on github) it would be an asset if Fedora had this packaged. Also, the core developer was more than willing to help with the process [4].
I could use any help needed on packaging. I have already exchanged a couple of mails with Vit Ondruch and he pointed me to a direction. In fact I am in the middle of reading all the related stuff from the wiki.
So what are your thoughts? Any guidance, ideas or insights are very much appreciated!
Regards,
Axilleas
PS. There is also the thought of deploying GitLab to git repositories on fedorahosted.org, but that's irrelevant to this list.
[0] https://fedoraproject.org/wiki/GSOC_2013 [1] http://gitlab.org [2] https://github.com/gitlabhq/gitlabhq [3] http://demo.gitlabhq.com/ [4] https://groups.google.com/forum/?fromgroups=#!topic/gitlabhq/SQMDi-yyXmU
On Fri, Mar 1, 2013 at 6:49 PM, Axilleas Pipinellis axilleas@archlinux.gr wrote:
So what are your thoughts? Any guidance, ideas or insights are very much appreciated!
Hi Axilleas,
I think this would be great. I'm working on Gitorious myself, and it looks like Gitlab would share many dependencies. So that would make my job easier :) You might want to check out my wiki page [1]. I'm trying to keep it up to date so that I can have a visual indicator of the scope of the work. If nothing else, it might help you brainstorm how to track your own dependencies.
In packaging Gitorious, there are four, fast-moving pieces:
1. Fedora (eg. the Ruby guidelines change almost every other release)
2. Ruby (eg. Ruby 2.0 will soon land in F19)
3. Rails (eg. Rails 4.0 will not run on EL6's Ruby)
4. And then upstream Gitorious itself is also changing.
I've come to find that packaging Ruby on Rails web apps is a special kind of pain :) I'm still plugging along at it with the hope that it will get easier. It seems like the Ruby community has learned from the 1.8 -> 1.9 conversion, and from what I understand, 2.0 brings fewer backwards-incompatible changes. I'm hoping that this greater focus on backwards compatibility will trickle down to the Rubygems culture as well. In the mean time, it's still really challenging to make all the puzzle pieces fit together while following Fedora's update policy guidelines, etc, and I anticipate many more months of work.
It looks like Gitlab's Gemfile.lock is even bigger than Gitorious', so you must be really ambitious!
One thing that you may want to look into up front is the idea of running Gitlab's test suite inside mock. I was sad to find out that Gitorious' test suite requires a running MySQL server instance, which means that I can't run the test suite inside Koji. I wonder if Gitlab's test suite has the same requirement. You may want to bring this up with your upstream devs.
- Ken
On 03/04/2013 05:47 AM, Ken Dreyer wrote:
Hi Axilleas,
I think this would be great. I'm working on Gitorious myself, and it looks like Gitlab would share many dependencies. So that would make my job easier :) You might want to check out my wiki page [1]. I'm trying to keep it up to date so that I can have a visual indicator of the scope of the work. If nothing else, it might help you brainstorm how to track your own dependencies.
In packaging Gitorious, there are four, fast-moving pieces:
Fedora (eg. the Ruby guidelines change almost every other release)
Ruby (eg. Ruby 2.0 will soon land in F19)
Rails (eg. Rails 4.0 will not run on EL6's Ruby)
And then upstream Gitorious itself is also changing.
I've seen your page about Gitorious and I had in mind making one for Gitlab as well :) Well, these 4 pieces you mentioned, pretty much apply to Gitlab as well, so I'll keep them in mind.
In the mean time, it's still really challenging to make all the puzzle pieces fit together while following Fedora's update policy guidelines, etc, and I anticipate many more months of work.
It looks like Gitlab's Gemfile.lock is even bigger than Gitorious', so you must be really ambitious!
Yeap! Last time I checked the Gemfile.lockhad 500+ lines... It sure isn't a project that will end in 3 months work. It's an ongoing marathon and a challenge worth pursuing :)
One thing that you may want to look into up front is the idea of running Gitlab's test suite inside mock. I was sad to find out that Gitorious' test suite requires a running MySQL server instance, which means that I can't run the test suite inside Koji. I wonder if Gitlab's test suite has the same requirement. You may want to bring this up with your upstream devs.
Thanks for bringing mock to my attention :) I will check Gitlab's test suites and see if I'm in the same position as you are.
Thanks again for your time! Your help will be valuable, as this is my first time ever packaging for Fedora.
Cheers!
Hello everyone! It's been over a month since I last wrote to this list regarding the GitLab project. I managed to make a blog post of the story so far[0], any feedback welcomed :)
Cheers!
[0] http://axilleas.github.io/en/blog/2013/bringing-gitlab-in-fedora/
Dne 10.4.2013 19:59, Axilleas Pipinellis napsal(a):
Hello everyone! It's been over a month since I last wrote to this list regarding the GitLab project. I managed to make a blog post of the story so far[0], any feedback welcomed :)
Cheers!
[0] http://axilleas.github.io/en/blog/2013/bringing-gitlab-in-fedora/
Thank you for the writeup.
One note with regard to MariaDB. There should not be any issue at the moment, since they should be 100% compatible. It will change in the future though. Nevertheless, you should have no issues with PostgreSQL, since there seems to be support already (database.yml.postgresql).
And next question, since you already know the number of packages needed to get into Fedora, you probably have also list of them. Could you share it with us?
Thank you.
Vít
On 04/11/2013 08:04 AM, Vít Ondruch wrote:
Thank you for the writeup.
One note with regard to MariaDB. There should not be any issue at the moment, since they should be 100% compatible. It will change in the future though. Nevertheless, you should have no issues with PostgreSQL, since there seems to be support already (database.yml.postgresql).
Yes, it is said to be fully mysql-compatible, but I haven't tested any app to see for myself, that's why I put it in the list. I'm thinking to open an issue at gitlab's bugtracker and ask them to include it in the supported databases.
And next question, since you already know the number of packages needed to get into Fedora, you probably have also list of them. Could you share it with us?
Yes, see [0]. Forgot to include it, I'll update the post as well.
[0] https://github.com/axilleas/fedora-gitlab/blob/master/rubygems_missing
On Apr 11, 2013 3:46 AM, "Axilleas Pipinellis" axilleas@archlinux.gr wrote:
Yes, [MariaDB] is said to be fully mysql-compatible, but I haven't tested
any app to see for myself, that's why I put it in the list. I'm thinking to open an issue at gitlab's bugtracker and ask them to include it in the supported databases.
That's a great idea, definitely the right place to start.
-Dan
Absolutely fantastic write-up. This is exactly the type of update that starts the effort off on the right foot and gets the community behind it. Thank you for posting.
I just went through the process of packaging a few gems, so *if* I have time, I'd be happy to grab a few from the list and get them packaged or do reviews.
Best of luck!
-Dan
-- Sent from my CyanogenMod-powered Android device, an open platform for carriers, developers and consumers. On Apr 10, 2013 11:59 AM, "Axilleas Pipinellis" axilleas@archlinux.gr wrote:
Hello everyone! It's been over a month since I last wrote to this list regarding the GitLab project. I managed to make a blog post of the story so far[0], any feedback welcomed :)
Cheers!
[0] http://axilleas.github.io/en/blog/2013/bringing-gitlab-in-fedora/ _______________________________________________ ruby-sig mailing list ruby-sig@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/ruby-sig
On 04/11/2013 12:36 PM, Dan Allen wrote:
Absolutely fantastic write-up. This is exactly the type of update that starts the effort off on the right foot and gets the community behind it. Thank you for posting.
I just went through the process of packaging a few gems, so *if* I have time, I'd be happy to grab a few from the list and get them packaged or do reviews.
Best of luck!
-Dan
Thanks xD Any help maintaining them will be valuable since there are a lot :) If I have some free time during the weekend, I'll begin packaging one or two to see how the process rolls.
There are some of them already undergoing review:
awesome_print - https://bugzilla.redhat.com/show_bug.cgi?id=839650 backports - https://bugzilla.redhat.com/show_bug.cgi?id=816991 - though I am not sure if we should continue with this one, since it brings nothing new to Fedora bootstrap-sass - https://bugzilla.redhat.com/show_bug.cgi?id=920436 faraday - https://bugzilla.redhat.com/show_bug.cgi?id=820063 rails_best_practices - https://bugzilla.redhat.com/show_bug.cgi?id=839649 ruby-progressbar - There is rubygem-progressbar, which seems to be the same library: https://bugzilla.redhat.com/show_bug.cgi?id=642666 there was also rubygem-ruby-progressbar review: https://bugzilla.redhat.com/show_bug.cgi?id=737551 - Although this one was deferred, not sure if we should not name the gem by upstream spork - https://bugzilla.redhat.com/show_bug.cgi?id=588476 - This was deferred stringex - https://bugzilla.redhat.com/show_bug.cgi?id=728051 yajl-ruby - https://bugzilla.redhat.com/show_bug.cgi?id=823351 - This is blocked by upstream a bit :/
And there are some which are in Fedora already:
facter - https://admin.fedoraproject.org/pkgdb/acls/name/facter
And also others, which are not yet in Fedora, but they are packaged by their upstream, such as Foreman. The Katello Koji instance [1] might be good source for packages like that. Katello guys are eager to get their dependencies into Fedora as far as I know.
Some of them should not be needed at all: libv8 - This is used by therubyracer as far as I know. The system libv8 is used for therubyracer instead and if there is other need, it should be used there as well. rb-fsevent - This is Mac specific gem. It is useless on Linux. And that is exactly the thing which is problematic for collaboration with Ruby community :/ It is very probably dependency of Spork, so it should not be needed anyway. In the worst case, Spork can use polling by default.
And also, please note that you are listing also the development dependencies, such as Spork. Spork is no way needed for run-time nor build-time. It is pure development tool, which allows you to run your test suite as soon as you save some changes. We should eliminate such gems from the list. Not that we don't want them in Fedora, but just because they are out of scope for this project IMO.
Vít
[1] http://koji.katello.org/koji/packages
Dne 11.4.2013 11:36, Dan Allen napsal(a):
Absolutely fantastic write-up. This is exactly the type of update that starts the effort off on the right foot and gets the community behind it. Thank you for posting.
I just went through the process of packaging a few gems, so *if* I have time, I'd be happy to grab a few from the list and get them packaged or do reviews.
Best of luck!
-Dan
-- Sent from my CyanogenMod-powered Android device, an open platform for carriers, developers and consumers.
On Apr 10, 2013 11:59 AM, "Axilleas Pipinellis" <axilleas@archlinux.gr mailto:axilleas@archlinux.gr> wrote:
Hello everyone! It's been over a month since I last wrote to this list regarding the GitLab project. I managed to make a blog post of the story so far[0], any feedback welcomed :) Cheers! [0] http://axilleas.github.io/en/blog/2013/bringing-gitlab-in-fedora/ _______________________________________________ ruby-sig mailing list ruby-sig@lists.fedoraproject.org <mailto:ruby-sig@lists.fedoraproject.org> https://admin.fedoraproject.org/mailman/listinfo/ruby-sig
ruby-sig mailing list ruby-sig@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/ruby-sig
<snip>
Some of them should not be needed at all: libv8 - This is used by therubyracer as far as I know. The system libv8 is used for therubyracer instead and if there is other need, it should be used there as well.
</snip> This is unfortunately not true for new therubyracer versions. therubyracer now depends on libv8 again and libv8 can either use its bundled version of v8 or the system one [1]. So I guess we will end up with rubygem-libv8 eventually.
Vít
Dne 11.4.2013 13:00, Bohuslav Kabrda napsal(a):
<snip>
Some of them should not be needed at all: libv8 - This is used by therubyracer as far as I know. The system libv8 is used for therubyracer instead and if there is other need, it should be used there as well.</snip> This is unfortunately not true for new therubyracer versions. therubyracer now depends on libv8 again and libv8 can either use its bundled version of v8 or the system one [1]. So I guess we will end up with rubygem-libv8 eventually.
Ah, ok ... thank you for clarification.
Vít
There is a way around this if therubyracer is being pulled in by execjs. You can set an environment variable to use the system js binary, which is in Fedora by default.
EXECJS_RUNTIME=SpiderMonkey
Then you don't need the racer.
In general, though, execution of JavaScript from Ruby is annoying. I have high hopes that DynJS or Nashhorn (JVM implementations) are going to help solve this problem. Easier to invoke Java (esp in JRuby) than deal with compiling libv8 cross platform.
-Dan
-- Sent from my CyanogenMod-powered Android device, an open platform for carriers, developers and consumers. On Apr 11, 2013 5:00 AM, "Bohuslav Kabrda" bkabrda@redhat.com wrote:
<snip>
Some of them should not be needed at all: libv8 - This is used by therubyracer as far as I know. The system libv8 is used for therubyracer instead and if there is other need, it should be used there as well.
</snip> This is unfortunately not true for new therubyracer versions. therubyracer now depends on libv8 again and libv8 can either use its bundled version of v8 or the system one [1]. So I guess we will end up with rubygem-libv8 eventually.
Vít
-- Regards, Bohuslav "Slavek" Kabrda.
[1] https://github.com/cowboyd/therubyracer/commit/fa2787ab4e8380df6eb02c05cb678...
ruby-sig mailing list ruby-sig@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/ruby-sig
Dne 11.4.2013 20:11, Dan Allen napsal(a):
There is a way around this if therubyracer is being pulled in by execjs. You can set an environment variable to use the system js binary, which is in Fedora by default.
EXECJS_RUNTIME=SpiderMonkey
SpiderMonkey is not recomended by execjs upstream anymore. There are some unresolved unicode issues as far as I remember.
Then you don't need the racer.
It is not that much what we need of want, but what upstream is using and what they have in their Gemfile. Yes, we can patch it, or have ours Gemfile, but carrying such changes in our packages is pain.
Vít
Hi all!
Long time no see, so I thought I dropped a line of what is achieved so far about this project. GSoC comes to an end later this month but that doesn't mean the effort is over. I will continue to work on this until it gets officially in Fedora.
I managed to deploy GitLab on a Fedora 19 machine using only installed packaged gems. Below you will find some more info as well as the url of the testing environment. Please use/test it and report any issues here [0]. If anyone needs an admin account for further testing just let me know. Just bare in mind that you might see some 500 errors as I will be trying some things :)
-------------------- OS: Fedora 19 Temp deploy: https://fedora.axilleas.me Enviromnent: using only installed packaged gems either from the official repos or my custom one [1]. Gemfile diffs: [2] and [3] Systemd services: Running without using bundler [4] --------------------
There are a lot to be done yet but that's a start.
TODO ----
*Short term - Write gitlab.spec that will glue all the dependencies together
*Long term - Commit as many specs as possible to BZ. - GitLab forks: one option is to patch upstream with GitLab's changes. second but rather avoided is to to ask FPC for an exception and package the forks as they are. - Coordinate efforts with Debian ruby team (I already contacted them) [5] - Deploy on rawhide: when GitLab supports rails 4. That depends on many dependencies gems as well.
*Longer term - port to EPEL
[0] https://github.com/axilleas/gsoc/issues [1] http://repos.fedorapeople.org/repos/axilleas/gitlab/fedora-19/ [2] https://github.com/axilleas/gsoc/blob/master/Gemfile.diff [3] https://github.com/axilleas/gsoc/blob/master/Gemfile.lock.diff [4] https://github.com/axilleas/gsoc/tree/master/systemd [5] http://debian.2.n7.nabble.com/Regarding-gitlab-td2843993.html
On 04/11/2013 01:52 PM, Vít Ondruch wrote:
There are some of them already undergoing review:
awesome_print - https://bugzilla.redhat.com/show_bug.cgi?id=839650 backports - https://bugzilla.redhat.com/show_bug.cgi?id=816991- though I am not sure if we should continue with this one, since it brings nothing new to Fedora bootstrap-sass - https://bugzilla.redhat.com/show_bug.cgi?id=920436 faraday - https://bugzilla.redhat.com/show_bug.cgi?id=820063 rails_best_practices - https://bugzilla.redhat.com/show_bug.cgi?id=839649 ruby-progressbar - There is rubygem-progressbar, which seems to be the same library: https://bugzilla.redhat.com/show_bug.cgi?id=642666there was also rubygem-ruby-progressbar review: https://bugzilla.redhat.com/show_bug.cgi?id=737551- Although this one was deferred, not sure if we should not name the gem by upstream spork - https://bugzilla.redhat.com/show_bug.cgi?id=588476- This was deferred stringex - https://bugzilla.redhat.com/show_bug.cgi?id=728051 yajl-ruby - https://bugzilla.redhat.com/show_bug.cgi?id=823351- This is blocked by upstream a bit :/
Are these listed somewhere, or you manually search for them?
And there are some which are in Fedora already:
facter - https://admin.fedoraproject.org/pkgdb/acls/name/facter
Ah yes, I searched only packages that contained the rubygem prefix. I'll fix that.
And also others, which are not yet in Fedora, but they are packaged by their upstream, such as Foreman. The Katello Koji instance [1] might be good source for packages like that. Katello guys are eager to get their dependencies into Fedora as far as I know.
Some of them should not be needed at all: libv8 - This is used by therubyracer as far as I know. The system libv8 is used for therubyracer instead and if there is other need, it should be used there as well. rb-fsevent - This is Mac specific gem. It is useless on Linux. And that is exactly the thing which is problematic for collaboration with Ruby community :/ It is very probably dependency of Spork, so it should not be needed anyway. In the worst case, Spork can use polling by default.
Thanks for the koji link, I'll check it.
And also, please note that you are listing also the development dependencies, such as Spork. Spork is no way needed for run-time nor build-time. It is pure development tool, which allows you to run your test suite as soon as you save some changes. We should eliminate such gems from the list. Not that we don't want them in Fedora, but just because they are out of scope for this project IMO.
Vít
oops, you're right. I'll exclude devel packages as well. Nice catch :) I didn't take the Gemfile into account, that's why it ended up listing these deps.
Dne 11.4.2013 14:00, Axilleas Pipinellis napsal(a):
On 04/11/2013 01:52 PM, Vít Ondruch wrote:
There are some of them already undergoing review:
awesome_print - https://bugzilla.redhat.com/show_bug.cgi?id=839650 backports - https://bugzilla.redhat.com/show_bug.cgi?id=816991- though I am not sure if we should continue with this one, since it brings nothing new to Fedora bootstrap-sass - https://bugzilla.redhat.com/show_bug.cgi?id=920436 faraday - https://bugzilla.redhat.com/show_bug.cgi?id=820063 rails_best_practices - https://bugzilla.redhat.com/show_bug.cgi?id=839649 ruby-progressbar - There is rubygem-progressbar, which seems to be the same library: https://bugzilla.redhat.com/show_bug.cgi?id=642666there was also rubygem-ruby-progressbar review: https://bugzilla.redhat.com/show_bug.cgi?id=737551- Although this one was deferred, not sure if we should not name the gem by upstream spork - https://bugzilla.redhat.com/show_bug.cgi?id=588476- This was deferred stringex - https://bugzilla.redhat.com/show_bug.cgi?id=728051 yajl-ruby - https://bugzilla.redhat.com/show_bug.cgi?id=823351- This is blocked by upstream a bit :/
Are these listed somewhere, or you manually search for them?
I have saved query in BZ for rubygem- reviews:
https://bugzilla.redhat.com/query.cgi?component=Package%20Review&product...
Or you can search Fedora's package-review ML.
And also, please note that you are listing also the development dependencies, such as Spork. Spork is no way needed for run-time nor build-time. It is pure development tool, which allows you to run your test suite as soon as you save some changes. We should eliminate such gems from the list. Not that we don't want them in Fedora, but just because they are out of scope for this project IMO.
Vít
oops, you're right. I'll exclude devel packages as well. Nice catch :) I didn't take the Gemfile into account, that's why it ended up listing these deps.
Some of development packages are needed to run test suite during package builds, which is always good idea, so be careful.
Vít
I plan on doing faye-websocket in the near future (within a month or two). I'm ok if someone else grabs before I get to it, but if not, I'll grab that one.
Troy
On 04/11/2013 05:52 AM, Vít Ondruch wrote:
There are some of them already undergoing review:
awesome_print - https://bugzilla.redhat.com/show_bug.cgi?id=839650 backports - https://bugzilla.redhat.com/show_bug.cgi?id=816991 - though I am not sure if we should continue with this one, since it brings nothing new to Fedora bootstrap-sass - https://bugzilla.redhat.com/show_bug.cgi?id=920436 faraday - https://bugzilla.redhat.com/show_bug.cgi?id=820063 rails_best_practices - https://bugzilla.redhat.com/show_bug.cgi?id=839649 ruby-progressbar - There is rubygem-progressbar, which seems to be the same library: https://bugzilla.redhat.com/show_bug.cgi?id=642666 there was also rubygem-ruby-progressbar review: https://bugzilla.redhat.com/show_bug.cgi?id=737551 - Although this one was deferred, not sure if we should not name the gem by upstream spork - https://bugzilla.redhat.com/show_bug.cgi?id=588476 - This was deferred stringex - https://bugzilla.redhat.com/show_bug.cgi?id=728051 yajl-ruby - https://bugzilla.redhat.com/show_bug.cgi?id=823351 - This is blocked by upstream a bit :/
And there are some which are in Fedora already:
facter - https://admin.fedoraproject.org/pkgdb/acls/name/facter
And also others, which are not yet in Fedora, but they are packaged by their upstream, such as Foreman. The Katello Koji instance [1] might be good source for packages like that. Katello guys are eager to get their dependencies into Fedora as far as I know.
Some of them should not be needed at all: libv8 - This is used by therubyracer as far as I know. The system libv8 is used for therubyracer instead and if there is other need, it should be used there as well. rb-fsevent - This is Mac specific gem. It is useless on Linux. And that is exactly the thing which is problematic for collaboration with Ruby community :/ It is very probably dependency of Spork, so it should not be needed anyway. In the worst case, Spork can use polling by default.
And also, please note that you are listing also the development dependencies, such as Spork. Spork is no way needed for run-time nor build-time. It is pure development tool, which allows you to run your test suite as soon as you save some changes. We should eliminate such gems from the list. Not that we don't want them in Fedora, but just because they are out of scope for this project IMO.
Vít
[1] http://koji.katello.org/koji/packages
Dne 11.4.2013 11:36, Dan Allen napsal(a):
Absolutely fantastic write-up. This is exactly the type of update that starts the effort off on the right foot and gets the community behind it. Thank you for posting.
I just went through the process of packaging a few gems, so *if* I have time, I'd be happy to grab a few from the list and get them packaged or do reviews.
Best of luck!
-Dan
-- Sent from my CyanogenMod-powered Android device, an open platform for carriers, developers and consumers.
On Apr 10, 2013 11:59 AM, "Axilleas Pipinellis" <axilleas@archlinux.gr mailto:axilleas@archlinux.gr> wrote:
Hello everyone! It's been over a month since I last wrote to this list regarding the GitLab project. I managed to make a blog post of the story so far[0], any feedback welcomed :) Cheers! [0] http://axilleas.github.io/en/blog/2013/bringing-gitlab-in-fedora/
Hey guys,
FYI the project got accepted in this year's GSoC :)
Cheers!
ruby-sig@lists.fedoraproject.org
-
Achilleas Pipinellis -
Axilleas Pipinellis -
Bohuslav Kabrda -
Dan Allen -
Ken Dreyer -
Troy Dawson -
Vít Ondruch