Some time ago, I started an effort to track API (and ABI) breakages in OpenJDK releases:
https://bugzilla.redhat.com/show_bug.cgi?id=1094780 https://bugzilla.redhat.com/show_bug.cgi?id=1095309 https://bugzilla.redhat.com/show_bug.cgi?id=1095328
Is something like this useful?
These reports were based on a database which contains all Java method/class/field references from Java class files contained in Fedora RPMs. This means that once we know about a removal and have written the required query, reports can be automated.
There is also a weird form of static linking (copying of JAR files into other JAR files):
https://bugzilla.redhat.com/show_bug.cgi?id=1098237
I think we discussed this before, but the discussion never came to a real conclusion.
On 24/09/15 11:56, Florian Weimer wrote:
Some time ago, I started an effort to track API (and ABI) breakages in OpenJDK releases:
https://bugzilla.redhat.com/show_bug.cgi?id=1094780 https://bugzilla.redhat.com/show_bug.cgi?id=1095309 https://bugzilla.redhat.com/show_bug.cgi?id=1095328
Is something like this useful?
These reports were based on a database which contains all Java method/class/field references from Java class files contained in Fedora RPMs. This means that once we know about a removal and have written the required query, reports can be automated.
I like the idea and I do not want to discourage you, but I see 2 issues with that:
1. With current overuse of reflection, there is no compile-time safety any more, and this approach can miss multiple API-level breaks.
2. There are some libraries that may cause huge amount of false positives, for example pretty much all logging toolkits, which find the implementation by reflection or by analysing some runtime configuration, they may have special implementations for outdated backends or for backends not on the classpath that you'll see as error, while they are not loaded and not used in runtime.
There is also a weird form of static linking (copying of JAR files into other JAR files):
https://bugzilla.redhat.com/show_bug.cgi?id=1098237
I think we discussed this before, but the discussion never came to a real conclusion.
Ouch... Is there some concrete projects to blame? And instead of finding tricky workarounds and encouraging developers to continue with that approach, will it not be better to add some build options not to do that and use normal classpath?
On 09/24/2015 12:08 PM, Stanislav Baiduzhyi wrote:
I like the idea and I do not want to discourage you, but I see 2 issues with that:
- With current overuse of reflection, there is no compile-time safety
any more, and this approach can miss multiple API-level breaks.
Understood.
- There are some libraries that may cause huge amount of false
positives, for example pretty much all logging toolkits, which find the implementation by reflection or by analysing some runtime configuration, they may have special implementations for outdated backends or for backends not on the classpath that you'll see as error, while they are not loaded and not used in runtime.
I don't think this can happen with Fedora because classes must be compiled from source, and the sources will no longer compile.
There is also a weird form of static linking (copying of JAR files into other JAR files):
https://bugzilla.redhat.com/show_bug.cgi?id=1098237
I think we discussed this before, but the discussion never came to a real conclusion.
Ouch... Is there some concrete projects to blame? And instead of finding tricky workarounds and encouraging developers to continue with that approach, will it not be better to add some build options not to do that and use normal classpath?
The issue is complex. Today, if you package a Java application, you have to collect the list of JAR files which need to be on the class path manually, and put that into the wrapper script which launches the JVM. If any of your dependent JARs changes and introduces a new dependency, your application will break. Copying class files avoids this problem and can even be automated with tools, so it is very attractive from a packager perspective.
I think this should be fixed with the Class-Path manifest attribute. People have claimed that it prevents overriding JAR files using the -classpath argument, but my tests show that this is not true at all:
https://github.com/fweimer/classpath-manifest-override
I would like to see Fedora to revisit its stance on the Class-Path attribute and eventually require that all JAR files in /usr/share/java have one.
On 24/09/15 13:00, Florian Weimer wrote:
I don't think this can happen with Fedora because classes must be compiled from source, and the sources will no longer compile.
Ah yes, that is true.
There is also a weird form of static linking (copying of JAR files into other JAR files):
https://bugzilla.redhat.com/show_bug.cgi?id=1098237
I think we discussed this before, but the discussion never came to a real conclusion.
Ouch... Is there some concrete projects to blame? And instead of finding tricky workarounds and encouraging developers to continue with that approach, will it not be better to add some build options not to do that and use normal classpath?
The issue is complex. Today, if you package a Java application, you have to collect the list of JAR files which need to be on the class path manually, and put that into the wrapper script which launches the JVM. If any of your dependent JARs changes and introduces a new dependency, your application will break. Copying class files avoids this problem and can even be automated with tools, so it is very attractive from a packager perspective.
Well that is still error-prone approach. Classpath scanning goes with "first found wins", which means that the version of your jar will not be loaded if there is similar class earlier on the classpath. BEA WebLogic used to do updated based on this principle. So I do not believe this approach helps anyone. As we can see on bigger projects, proper applications are just packaging everything they need with them under single root, or into some specific osgi bundles, and then using * classpathes.
I think it would be amazing to create some new type of container for Java apps, but I know I'll get beaten for saying that on any distribution packaging mailing list.
I think this should be fixed with the Class-Path manifest attribute. People have claimed that it prevents overriding JAR files using the -classpath argument, but my tests show that this is not true at all:
https://github.com/fweimer/classpath-manifest-override
I would like to see Fedora to revisit its stance on the Class-Path attribute and eventually require that all JAR files in /usr/share/java have one.
Just tested that, I'm little surprised that they are actually loaded recursively. Thinking of possible corner-cases, again taking logging toolkits as example, there may be lots of compile-time dependencies, but most of them may not be needed in run-time (or even not wanted), how will it behave then if all of them will be declared in manifest? Or even how will it be handled by RPM?
On 24/09/15 12:00, Florian Weimer wrote:
I would like to see Fedora to revisit its stance on the Class-Path attribute and eventually require that all JAR files in /usr/share/java have one.
Eventually we'll have the Java module system. I can't see the point of changing this now.
Andrew.
On 10/01/2015 10:33 AM, Andrew Haley wrote:
On 24/09/15 12:00, Florian Weimer wrote:
I would like to see Fedora to revisit its stance on the Class-Path attribute and eventually require that all JAR files in /usr/share/java have one.
Eventually we'll have the Java module system. I can't see the point of changing this now.
How will Fedora deal with conflicting JAR files? This has been brought up as a counter-argument to using Class-Path attributes, after all.
“ If the module system cannot fulfill a particular dependence with an artifact from a module path, or if it encounters two artifacts defining modules of the same name but different versions in some directory of a module path, then resolution will fail and the compiler or virtual machine will report an error and exit. ”
http://openjdk.java.net/projects/jigsaw/spec/sotms/
This suggests to me it is not possible to have a single, system-wide module path. And if applications still need to collect the module paths of all their dependencies, we face a problem similar to constructing the class path today.
Florian
On 10/06/2015 09:07 AM, Florian Weimer wrote:
On 10/01/2015 10:33 AM, Andrew Haley wrote:
On 24/09/15 12:00, Florian Weimer wrote:
I would like to see Fedora to revisit its stance on the Class-Path attribute and eventually require that all JAR files in /usr/share/java have one.
Eventually we'll have the Java module system. I can't see the point of changing this now.
How will Fedora deal with conflicting JAR files?
Conflicting in what way? Different versions of the same jar?
This has been brought up as a counter-argument to using Class-Path attributes, after all.
“ If the module system cannot fulfill a particular dependence with an artifact from a module path, or if it encounters two artifacts defining modules of the same name but different versions in some directory of a module path, then resolution will fail and the compiler or virtual machine will report an error and exit. ”
http://openjdk.java.net/projects/jigsaw/spec/sotms/
This suggests to me it is not possible to have a single, system-wide module path.
Why, exactly?
And if applications still need to collect the module paths of all their dependencies, we face a problem similar to constructing the class path today.
We always have to figure out the dependencies of a jarfile. But we don't have to use the Class-Path atribute.
Andrew.
Hi,
On 09/24/2015 11:56 AM, Florian Weimer wrote:
<snip>
There is also a weird form of static linking (copying of JAR files into other JAR files):
https://bugzilla.redhat.com/show_bug.cgi?id=1098237
I think we discussed this before, but the discussion never came to a real conclusion.
AFAIK we (Java SIG) put these cases on the same level as bundling and treat them accordingly. So all of them are genuine bugs (in Fedora) and tracking them definitely makes sense. Upstream projects sometimes use these techniques to provide so called "uber" JARs, e.g. [1], or to overcome ABI breakages in popular libraries [2], [3] - by including package-renamed classes from dependencies in resulting JAR.
Michal
[1]: https://wiki.eclipse.org/Jetty/Howto/Using_Jetty_Runner [2]: http://spring.io/blog/2007/06/11/asm-version-incompatibilities-using-spring-... [3]: https://weblogs.java.net/blog/kohsuke/archive/2010/02/12/asm-incompatible-ch...
java-devel@lists.fedoraproject.org
-
Andrew Haley -
Florian Weimer -
Michal Srb -
Stanislav Baiduzhyi