In bug #976330[1], there's been some discussion of the desire to upgrade Gradle from 1.0 to a new version (currently 1.8), and that there is significant hassle to actually do so. Some of this hassle I see in the bugs blocking 976330 - upgrading to Aether, newer Plexus containers, and maybe Polyglot Maven.
What are the key issues that make it difficult to upgrade and maintain Gradle in Fedora? The likely ones I see:
- Incompatible dependencies (Gradle may require package versions no longer shipped in Fedora) - Bootstrapping problems (the Gradle 1.8 sources won't build with Gradle 1.8, they seem to require 1.7) - Possibly library embedding (although the Gradle sources do seem to pull in the dependencies from Maven)
1. https://bugzilla.redhat.com/show_bug.cgi?id=976330
On 10/21/2013 05:11 PM, Michael Ekstrand wrote:
In bug #976330[1], there's been some discussion of the desire to upgrade Gradle from 1.0 to a new version (currently 1.8), and that there is significant hassle to actually do so. Some of this hassle I see in the bugs blocking 976330 - upgrading to Aether, newer Plexus containers, and maybe Polyglot Maven.
What are the key issues that make it difficult to upgrade and maintain Gradle in Fedora? The likely ones I see:
- Incompatible dependencies (Gradle may require package versions no longer shipped in Fedora)
Gradle used to use (very) old versions of some dependencies, for example legacy pre-1.0 alpha version of Plexus, released ~10 years ago. It also used some Maven 2 APIs (Fedora ships Maven 3 only). This was improved recently IIRC.
Gradle also uses Sonatype Aether, which is not in Fedora any longer. Porting it to Eclipse Aether should be fairly straightforward, but still, needs to be done.
- Bootstrapping problems (the Gradle 1.8 sources won't build with Gradle 1.8, they seem to require 1.7)
These are IMO two separate problems.
1. Gradle breaks compatibility between releases. If package builds with Gradle version X then it not necessarily builds with Gradle version Y, Y
X. This is a serious problem as Fedora may have to maintain multiple
versions of Gradle to build different projects. Other build systems may have some incompatibilities too, but in case of Gradle they are causing more problems. Additionally, Gradle build scripts are written in fully-featured programming language and may not be so easy to port to different version of Gradle.
2. Gradle is built with itself (as many build systems are), so it needs bootstrapping. Upstream of majority of other build systems (including Ant or Maven) are maintaining secondary ways of building them (Ant can be built with javac, Maven can be built with Ant etc.), but Gradle upstream does not provide any way to bootstrap Gradle. Normally bootstrapping would need to be performed only once, but because of 1. bootstrapping may be required more often, worst case with every release.
Fedora would probably need to create and maintain a separate build script for Gradle (for example using Ant or Maven) or obtain an exception from FPC to build using prebuilt binaries. Currently Fedora ships Ant build.xml for Gradle, but it requires substantial amount of manual work to synchronize it with upstream build. (I would prefer to use Maven here. It would not only ease dependency management, but also allow easier installation of Gradle artifacts.)
- Possibly library embedding (although the Gradle sources do seem to pull in the dependencies from Maven)
That's a generic problem and it's not really Gradle-specific. But yes, there are some version problems, most notably Objectweb ASM. Different Gradle dependencies use versions 3 and 4 (shaded to avoid namespace conflicts). Fedora does not allow bundled libraries, which causes conflict between ASM 3 and 4. (Porting from ASM 3 to ASM 4 is possible, but non-trivial as there were major changes. This would again require some work.)
To sum up, Gradle maintainence requires substantial amount of work. Currently only few packages in Fedora are using Gradle which means that maintenance costs of Gradle outweight costs of porting other packages to different build systems.
Hi,
How about converting graddle.build files to pom.xml files [1] ? This could be the bootstrap we need for the build part, but I have no idea whether it would actually be simpler.
Dridi
[1] http://www.gradle.org/docs/current/userguide/maven_plugin.html#sec:maven_pom...
On Mon, Oct 21, 2013 at 5:55 PM, Mikolaj Izdebski mizdebsk@redhat.com wrote:
On 10/21/2013 05:11 PM, Michael Ekstrand wrote:
In bug #976330[1], there's been some discussion of the desire to upgrade Gradle from 1.0 to a new version (currently 1.8), and that there is significant hassle to actually do so. Some of this hassle I see in the bugs blocking 976330 - upgrading to Aether, newer Plexus containers, and maybe Polyglot Maven.
What are the key issues that make it difficult to upgrade and maintain Gradle in Fedora? The likely ones I see:
- Incompatible dependencies (Gradle may require package versions no longer shipped in Fedora)
Gradle used to use (very) old versions of some dependencies, for example legacy pre-1.0 alpha version of Plexus, released ~10 years ago. It also used some Maven 2 APIs (Fedora ships Maven 3 only). This was improved recently IIRC.
Gradle also uses Sonatype Aether, which is not in Fedora any longer. Porting it to Eclipse Aether should be fairly straightforward, but still, needs to be done.
- Bootstrapping problems (the Gradle 1.8 sources won't build with Gradle 1.8, they seem to require 1.7)
These are IMO two separate problems.
- Gradle breaks compatibility between releases. If package builds with
Gradle version X then it not necessarily builds with Gradle version Y, Y
X. This is a serious problem as Fedora may have to maintain multiple
versions of Gradle to build different projects. Other build systems may have some incompatibilities too, but in case of Gradle they are causing more problems. Additionally, Gradle build scripts are written in fully-featured programming language and may not be so easy to port to different version of Gradle.
- Gradle is built with itself (as many build systems are), so it needs
bootstrapping. Upstream of majority of other build systems (including Ant or Maven) are maintaining secondary ways of building them (Ant can be built with javac, Maven can be built with Ant etc.), but Gradle upstream does not provide any way to bootstrap Gradle. Normally bootstrapping would need to be performed only once, but because of 1. bootstrapping may be required more often, worst case with every release.
Fedora would probably need to create and maintain a separate build script for Gradle (for example using Ant or Maven) or obtain an exception from FPC to build using prebuilt binaries. Currently Fedora ships Ant build.xml for Gradle, but it requires substantial amount of manual work to synchronize it with upstream build. (I would prefer to use Maven here. It would not only ease dependency management, but also allow easier installation of Gradle artifacts.)
- Possibly library embedding (although the Gradle sources do seem to pull in the dependencies from Maven)
That's a generic problem and it's not really Gradle-specific. But yes, there are some version problems, most notably Objectweb ASM. Different Gradle dependencies use versions 3 and 4 (shaded to avoid namespace conflicts). Fedora does not allow bundled libraries, which causes conflict between ASM 3 and 4. (Porting from ASM 3 to ASM 4 is possible, but non-trivial as there were major changes. This would again require some work.)
To sum up, Gradle maintainence requires substantial amount of work. Currently only few packages in Fedora are using Gradle which means that maintenance costs of Gradle outweight costs of porting other packages to different build systems.
-- Mikolaj Izdebski IRC: mizdebsk
PS. Integration of Gradle with Fedora is a completely different story...
java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
On 10/21/2013 06:06 PM, Dridi Boukelmoune wrote:
How about converting graddle.build files to pom.xml files [1] ? This could be the bootstrap we need for the build part, but I have no idea whether it would actually be simpler.
I am aware that Gradle generate POMs, but they contain only minimal metadata, like identifiers and dependencies. There are no build instructions, so these POMs are not suitable for building projects, only for using them as dependencies of other projects.
For example of POM generated by Gradle see [1]. It has no parent and no build instructions.
[1] http://repo1.maven.org/maven2/org/hibernate/hibernate-core/4.3.0.Beta5/hiber...
On 10/21/2013 11:55 AM, Mikolaj Izdebski wrote:
yes, there are some version problems, most notably Objectweb ASM. Different Gradle dependencies use versions 3 and 4 (shaded to avoid namespace conflicts). Fedora does not allow bundled libraries, which causes conflict between ASM 3 and 4. (Porting from ASM 3 to ASM 4 is possible, but non-trivial as there were major changes. This would again require some work.)
You said exactly what I was thinking about gradle breaking API compatibility between every release and its being a full language. I am not sure what it says about a project like this when it can't even be used to build itself. I have discussed the same things with others at Red Hat.
I am not sure what you mean about asm. Fedora may not allow bundled libraries, but it may allow two versions of a library in the case of major incompatibility. The maven coordinates changed from 3 to 4, but unfortunately most of the class names did not, leading to serious problems if both versions are on the classpath at the same time. But, this doesn't mean that it has to be bundled.
Currently only few packages in Fedora are using Gradle which means that maintenance costs of Gradle outweight costs of porting other packages to different build systems.
Let's hope it stays that way. The response from certain gradle-using projects was to just use the bundled `gradlew' (which goes online, so really the gradle zip that it downloads). The fact that a private copy of gradle is needed for every project release is not seen as a drawback. Luckily, I only have to deal with two or three projects using gradle, and I hope for your sake that it is the same in Fedora.
I think that although developers do not understand the type of build issues that Fedora and Red Hat have to deal with, they do understand that using gradle makes both their build dependencies and build code very unmaintainable.
On Mon, 21 Oct 2013 15:23:28 -0400 David Walluck david@zarb.org wrote:
Let's hope it stays that way. The response from certain gradle-using projects was to just use the bundled `gradlew' (which goes online, so really the gradle zip that it downloads). The fact that a private copy of gradle is needed for every project release is not seen as a drawback. Luckily, I only have to deal with two or three projects using gradle, and I hope for your sake that it is the same in Fedora.
Does it use a per-project private copy? I thought it would, but then tried it and it seems to be downloading Gradle to ~/.gradle, so you should only have one copy per Gradle version used. Which is annoying, but not *quite* as bad.
- Michael
On 10/21/2013 04:15 PM, Michael Ekstrand wrote:
Does it use a per-project private copy? I thought it would, but then tried it and it seems to be downloading Gradle to ~/.gradle, so you should only have one copy per Gradle version used. Which is annoying, but not *quite* as bad.
When building in Fedora, one should set the .gradle dir to be inside the build tree. But, the .gradle tree should be shareable, as it is with .m2, because the dependencies are versioned and supposed to be unique.
I was speaking of a copy of the gradle build tool itself plus all of its dependencies. All gradle builds tend to ship with the `gradlew' shell script which specifies a specific distro of gradle. Gradle also bundles a full copy of maven with itself.
Keep in mind that gradle cannot even necessarily build itself, nevermind build version 1.(x+1) with version 1.x. I don't know of another piece of Java software that is this bad.
On 10/21/2013 09:23 PM, David Walluck wrote:
On 10/21/2013 11:55 AM, Mikolaj Izdebski wrote:
yes, there are some version problems, most notably Objectweb ASM. Different Gradle dependencies use versions 3 and 4 (shaded to avoid namespace conflicts). Fedora does not allow bundled libraries, which causes conflict between ASM 3 and 4. (Porting from ASM 3 to ASM 4 is possible, but non-trivial as there were major changes. This would again require some work.)
I am not sure what you mean about asm. Fedora may not allow bundled libraries, but it may allow two versions of a library in the case of major incompatibility. The maven coordinates changed from 3 to 4, but unfortunately most of the class names did not, leading to serious problems if both versions are on the classpath at the same time. But, this doesn't mean that it has to be bundled.
Gradle uses some libraries, which use different incompatible versions of ASM (3 and 4). For upstream that's not a problem as one copy of ASM is shaded (bundled with namespace changed). In Fedora ASM cannot be shaded as this would be against Fedora policy. This causes problems as two versions of ASM end up being on Gradle classpath.
On 10/22/2013 04:29 AM, Mikolaj Izdebski wrote:
Gradle uses some libraries, which use different incompatible versions of ASM (3 and 4). For upstream that's not a problem as one copy of ASM is shaded (bundled with namespace changed). In Fedora ASM cannot be shaded as this would be against Fedora policy. This causes problems as two versions of ASM end up being on Gradle classpath.
Now I understand. Yes, the two major versions cannot be on the same classpath.
I was seeing a similar issue with xbean, since newer versions of that use asm4, even though there is a jar called xbean-asm{,4}-shaded. As you said, you disallow such jars altogether as they contain essentially private copies of the asm code that is now immune to upgrades to the system asm packages.
Has this been discussed upstream ?
I've found one unrelated reference [1] to Fedora on the github repository, and couldn't find gradle's issue tracker in less than 2 minutes (so I gave up).
Dridi
[1] https://github.com/gradle/gradle/pull/76
On Tue, Oct 22, 2013 at 4:23 PM, David Walluck david@zarb.org wrote:
On 10/22/2013 04:29 AM, Mikolaj Izdebski wrote:
Gradle uses some libraries, which use different incompatible versions of ASM (3 and 4). For upstream that's not a problem as one copy of ASM is shaded (bundled with namespace changed). In Fedora ASM cannot be shaded as this would be against Fedora policy. This causes problems as two versions of ASM end up being on Gradle classpath.
Now I understand. Yes, the two major versions cannot be on the same classpath.
I was seeing a similar issue with xbean, since newer versions of that use asm4, even though there is a jar called xbean-asm{,4}-shaded. As you said, you disallow such jars altogether as they contain essentially private copies of the asm code that is now immune to upgrades to the system asm packages.
-- java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
Mikolaj,
Thanks for taking the time to reply. I was in part seeking to find out if the problems with packaging Gradle are something that I could spend some time to help address, but it looks like they're significant enough that I don't have the time to work on them right now.
Some more comments inline.
On Mon, 21 Oct 2013 17:55:47 +0200 Mikolaj Izdebski mizdebsk@redhat.com wrote:
On 10/21/2013 05:11 PM, Michael Ekstrand wrote:
- Incompatible dependencies (Gradle may require package versions no longer shipped in Fedora)
Gradle used to use (very) old versions of some dependencies, for example legacy pre-1.0 alpha version of Plexus, released ~10 years ago. It also used some Maven 2 APIs (Fedora ships Maven 3 only). This was improved recently IIRC.
Gradle also uses Sonatype Aether, which is not in Fedora any longer. Porting it to Eclipse Aether should be fairly straightforward, but still, needs to be done.
It seems like that is potentially problematic - what if a build script (unfortunately) depends on the fact that Gradle uses Sonatype Aether - but also unavoidable. I would hope that the set of projects that would be affected by this is minimal.
- Bootstrapping problems (the Gradle 1.8 sources won't build with Gradle 1.8, they seem to require 1.7)
These are IMO two separate problems.
- Gradle breaks compatibility between releases. If package builds
with Gradle version X then it not necessarily builds with Gradle version Y, Y
X. This is a serious problem as Fedora may have to maintain multiple
versions of Gradle to build different projects. Other build systems may have some incompatibilities too, but in case of Gradle they are causing more problems. Additionally, Gradle build scripts are written in fully-featured programming language and may not be so easy to port to different version of Gradle.
- Gradle is built with itself (as many build systems are), so it
needs bootstrapping. Upstream of majority of other build systems (including Ant or Maven) are maintaining secondary ways of building them (Ant can be built with javac, Maven can be built with Ant etc.), but Gradle upstream does not provide any way to bootstrap Gradle. Normally bootstrapping would need to be performed only once, but because of 1. bootstrapping may be required more often, worst case with every release.
If the general pattern is followed of Gradle X building with Gradle X-1, then it shouldn't be too bad - once gradle18 is bootstrapped and in, then gradle19 can Build-Require gradle18. Assuming multiple Gradle versions, which seems nasty.
If this isn't the general pattern, then more bootstrapping with backwards compatibility problems.
This does indeed feel like a colossal mess.
To sum up, Gradle maintainence requires substantial amount of work. Currently only few packages in Fedora are using Gradle which means that maintenance costs of Gradle outweight costs of porting other packages to different build systems.
Thanks for elucidating just how much work. It does indeed seem to be not worth the effort.
My personal interest is primarily in having Gradle available for use in automating my various projects, but it looks like continuing to just use Gradle's binary distributions is the best path forward for that. Along with avoiding using Gradle as the (sole) build system for software that I distribute with intent to see packaged in distributions.
- Michael
As a follow-up to this mail:
The only package that required Gradle was Hibernate. Recently (thanks to Gil) I migrated it from Gradle back to Maven. It works perfectly and is way easier to maintain now. You can see the spec file here:
http://pkgs.fedoraproject.org/cgit/hibernate.git/tree/hibernate.spec
I've opened a bug report to retire Grade in Rawhide and in Fedora 20:
https://bugzilla.redhat.com/show_bug.cgi?id=1029534
--Marek
On 21.10.2013 17:11, Michael Ekstrand wrote:
In bug #976330[1], there's been some discussion of the desire to upgrade Gradle from 1.0 to a new version (currently 1.8), and that there is significant hassle to actually do so. Some of this hassle I see in the bugs blocking 976330 - upgrading to Aether, newer Plexus containers, and maybe Polyglot Maven.
What are the key issues that make it difficult to upgrade and maintain Gradle in Fedora? The likely ones I see:
- Incompatible dependencies (Gradle may require package versions no longer shipped in Fedora)
- Bootstrapping problems (the Gradle 1.8 sources won't build with Gradle 1.8, they seem to require 1.7)
- Possibly library embedding (although the Gradle sources do seem to pull in the dependencies from Maven)
Il 12/11/2013 15:45, Marek Goldmann ha scritto:
As a follow-up to this mail:
The only package that required Gradle was Hibernate. Recently (thanks to Gil) I migrated it from Gradle back to Maven. It works perfectly and is way easier to maintain now. You can see the spec file here:
http://pkgs.fedoraproject.org/cgit/hibernate.git/tree/hibernate.specI've opened a bug report to retire Grade in Rawhide and in Fedora 20:
https://bugzilla.redhat.com/show_bug.cgi?id=1029534--Marek
thanks to you work on this now... regards gil p.s. there is also latest groovy release (2.1.x) need gradle (circular dep ...?...)
On 21.10.2013 17:11, Michael Ekstrand wrote:
In bug #976330[1], there's been some discussion of the desire to upgrade Gradle from 1.0 to a new version (currently 1.8), and that there is significant hassle to actually do so. Some of this hassle I see in the bugs blocking 976330 - upgrading to Aether, newer Plexus containers, and maybe Polyglot Maven.
What are the key issues that make it difficult to upgrade and maintain Gradle in Fedora? The likely ones I see:
- Incompatible dependencies (Gradle may require package versions no longer shipped in Fedora)
- Bootstrapping problems (the Gradle 1.8 sources won't build with Gradle 1.8, they seem to require 1.7)
- Possibly library embedding (although the Gradle sources do seem to pull in the dependencies from Maven)
and newer release use objectweb-asm4, but only for Fedora (maybe also for other distros e.g. Mageia) is not possible use groovy without bundled objectweb-asm3 commos-cli and antlr libraries (groovy-all see Debian package)
-- java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
java-devel@lists.fedoraproject.org
-
David Walluck -
Dridi Boukelmoune -
gil -
Marek Goldmann -
Michael Ekstrand -
Mikolaj Izdebski