The eclipse PTP project that I package is starting to ship an executable in one of its plugins. The executable is packed into a jar file. This prevents debuginfo creation for the executable.
I started taking a look at how this might be handled elsewhere and found:
/usr/lib64/eclipse/dropins/cdt/eclipse/plugins/org.eclipse.cdt.core.linux.x86_64_5.2.0.201302132326.jar: 13327 Defl:X 3624 73% 02-13-2013 23:21 326488c7 os/linux/x86_64/libpty.so 23720 Defl:X 7151 70% 02-13-2013 23:21 c26c2c94 os/linux/x86_64/libspawner.so
but no debuginfo package.
Thoughts, comments?
On 04/22/2013 11:01 PM, Orion Poplawski wrote:
The eclipse PTP project that I package is starting to ship an executable in one of its plugins. The executable is packed into a jar file. This prevents debuginfo creation for the executable.
I started taking a look at how this might be handled elsewhere and found:
/usr/lib64/eclipse/dropins/cdt/eclipse/plugins/org.eclipse.cdt.core.linux.x86_64_5.2.0.201302132326.jar: 13327 Defl:X 3624 73% 02-13-2013 23:21 326488c7 os/linux/x86_64/libpty.so 23720 Defl:X 7151 70% 02-13-2013 23:21 c26c2c94 os/linux/x86_64/libspawner.so
but no debuginfo package.
Thoughts, comments?
This is pretty obviously a breach of packaging guidelines. We shouldn't be hiding DSOs in archives.
Andrew.
Quoting Andrew Haley (2013-04-26 11:11:41)
On 04/22/2013 11:01 PM, Orion Poplawski wrote:
The eclipse PTP project that I package is starting to ship an executable in one of its plugins. The executable is packed into a jar file. This prevents debuginfo creation for the executable.
I started taking a look at how this might be handled elsewhere and found:
/usr/lib64/eclipse/dropins/cdt/eclipse/plugins/org.eclipse.cdt.core.linux.x86_64_5.2.0.201302132326.jar: 13327 Defl:X 3624 73% 02-13-2013 23:21 326488c7 os/linux/x86_64/libpty.so 23720 Defl:X 7151 70% 02-13-2013 23:21 c26c2c94 os/linux/x86_64/libspawner.so
but no debuginfo package.
Thoughts, comments?
This is pretty obviously a breach of packaging guidelines. We shouldn't be hiding DSOs in archives.
Hmm, I am not aware of any guideline touching this topic (perhaps because it's not that common). But from practical perspective these jars are extremely practical because they don't have to deal with JNI file locations and similar things which ten to be quite variable.
I don't consider this approach hiding, but we could surely improve the tooling support for it.
WRT original question, I am not sure how to proceed. Even if you generate debuginfo packages I have to wonder how easy will it be to use in a debugger?
On 04/26/2013 10:39 AM, Stanislav Ochotnicky wrote:
Quoting Andrew Haley (2013-04-26 11:11:41)
On 04/22/2013 11:01 PM, Orion Poplawski wrote:
The eclipse PTP project that I package is starting to ship an executable in one of its plugins. The executable is packed into a jar file. This prevents debuginfo creation for the executable.
I started taking a look at how this might be handled elsewhere and found:
/usr/lib64/eclipse/dropins/cdt/eclipse/plugins/org.eclipse.cdt.core.linux.x86_64_5.2.0.201302132326.jar: 13327 Defl:X 3624 73% 02-13-2013 23:21 326488c7 os/linux/x86_64/libpty.so 23720 Defl:X 7151 70% 02-13-2013 23:21 c26c2c94 os/linux/x86_64/libspawner.so
but no debuginfo package.
Thoughts, comments?
This is pretty obviously a breach of packaging guidelines. We shouldn't be hiding DSOs in archives.
Hmm, I am not aware of any guideline touching this topic (perhaps because it's not that common). But from practical perspective these jars are extremely practical because they don't have to deal with JNI file locations and similar things which ten to be quite variable.
I don't understand your point. It's a package, so the JNI file location is fixed. What can be variable?
I don't consider this approach hiding, but we could surely improve the tooling support for it.
It surely is. One of the things that packaging does is scour packages for DSOs to make sure that the debuginfo is present. By putting the package in a DSO we've hidden it from the package.
WRT original question, I am not sure how to proceed. Even if you generate debuginfo packages I have to wonder how easy will it be to use in a debugger?
Well, quite. The right thing to do is not to hide the DSOs. Then everything will just work. The alternative is to put the debuginfo in the jar with the DSO, but IMO we should not be considering this option.
Andrew.
OSGi guidelines allow for packing *.so in jars. Bundles with native code can be installed as directories (I think it is a preferred way) or as jars (and then are extracted at runtime into ~/.eclipse [1]).
I'd suggest checking what is the shape of the bundle (jar or dir) in ptp installed from upstream (which will be processed by Eclipse P2 installation mechanism) and then either asking upstream to ship the bundle as jar or unpacking the jar during building.
There is no support for installation preprocessing in Fedora Eclipse yet (if you don't extract the bundle during build time it will not be extracted at runtime).
In case of questions, ping me on irc.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=832053
-- Krzysztof Daniel
On 04/26/2013 12:11 PM, Krzysztof Daniel wrote:
OSGi guidelines allow for packing *.so in jars. Bundles with native code can be installed as directories (I think it is a preferred way) or as jars (and then are extracted at runtime into ~/.eclipse [1]).
Is this extraction something the JVM can do automatically, or is it necessary to implement it from scratch each time you need it?
(I recently came across one such DSO extractor and wondered if there's a better way.)
On Mon, 2013-04-29 at 11:22 +0200, Florian Weimer wrote:
On 04/26/2013 12:11 PM, Krzysztof Daniel wrote:
OSGi guidelines allow for packing *.so in jars. Bundles with native code can be installed as directories (I think it is a preferred way) or as jars (and then are extracted at runtime into ~/.eclipse [1]).
Is this extraction something the JVM can do automatically, or is it necessary to implement it from scratch each time you need it?
OSGi framework takes care about extracting *.so at runtime.
On 04/29/2013 10:22 AM, Florian Weimer wrote:
On 04/26/2013 12:11 PM, Krzysztof Daniel wrote:
OSGi guidelines allow for packing *.so in jars. Bundles with native code can be installed as directories (I think it is a preferred way) or as jars (and then are extracted at runtime into ~/.eclipse [1]).
Is this extraction something the JVM can do automatically, or is it necessary to implement it from scratch each time you need it?
(I recently came across one such DSO extractor and wondered if there's a better way.)
Right, but even if the JVM could do so, it's irrelevant to Fedora, for the reasons that mark Wielaard enumerated. We just can't do it.
Andrew.
On 04/26/2013 10:52 AM, Andrew Haley wrote:
On 04/26/2013 10:39 AM, Stanislav Ochotnicky wrote:
Quoting Andrew Haley (2013-04-26 11:11:41)
On 04/22/2013 11:01 PM, Orion Poplawski wrote:
The eclipse PTP project that I package is starting to ship an executable in one of its plugins. The executable is packed into a jar file. This prevents debuginfo creation for the executable.
I started taking a look at how this might be handled elsewhere and found:
/usr/lib64/eclipse/dropins/cdt/eclipse/plugins/org.eclipse.cdt.core.linux.x86_64_5.2.0.201302132326.jar: 13327 Defl:X 3624 73% 02-13-2013 23:21 326488c7 os/linux/x86_64/libpty.so 23720 Defl:X 7151 70% 02-13-2013 23:21 c26c2c94 os/linux/x86_64/libspawner.so
but no debuginfo package.
Thoughts, comments?
This is pretty obviously a breach of packaging guidelines. We shouldn't be hiding DSOs in archives.
Hmm, I am not aware of any guideline touching this topic (perhaps because it's not that common). But from practical perspective these jars are extremely practical because they don't have to deal with JNI file locations and similar things which ten to be quite variable.
I don't understand your point. It's a package, so the JNI file location is fixed. What can be variable?
I don't consider this approach hiding, but we could surely improve the tooling support for it.
It surely is. One of the things that packaging does is scour packages for DSOs to make sure that the debuginfo is present. By putting the package in a DSO we've hidden it from the package.
WRT original question, I am not sure how to proceed. Even if you generate debuginfo packages I have to wonder how easy will it be to use in a debugger?
Well, quite. The right thing to do is not to hide the DSOs. Then everything will just work. The alternative is to put the debuginfo in the jar with the DSO, but IMO we should not be considering this option.
Hmm, I just found this in https://fedoraproject.org/wiki/Packaging:Java?rd=Packaging/Java#Packaging_JA...
"JAR files using JNI or containing JNI shared objects themselves MUST be placed in %{_jnidir} and CAN BE symlinked to %{_libdir}/%{name}."
So, it seems as though the practice of putting DSOs into jarfiles is legitimate. I am appalled that this suggestion got past Fedora packaging, but I can't deny that it's there.
The only sensible thing to do, if we must put DSOs into jarfiles, is to generate the DSO with proper debuginfo.
Andrew.
On Fri, 2013-04-26 at 10:52 +0100, Andrew Haley wrote:
On 04/26/2013 10:39 AM, Stanislav Ochotnicky wrote:
WRT original question, I am not sure how to proceed. Even if you
generate debuginfo packages I have to wonder how easy will it be to use in a debugger?
Well, quite. The right thing to do is not to hide the DSOs. Then everything will just work. The alternative is to put the debuginfo in the jar with the DSO, but IMO we should not be considering this option.
Indeed. It means the fedora autoqa processes don't see it, the build-ids and source files don't get collected (making profiling, tracing, debugging, automatic crash detection and core file inspection tricky), prelink doesn't see it, the shared library cannot actually be shared by multiple programs using it, the selinux file contexts don't apply to it, etc.
I don't think it makes much sense to hide an executable file inside a jar file where all the standard distribution and packaging tools basically cannot see it.
Cheers,
Mark
On 04/26/2013 03:39 AM, Stanislav Ochotnicky wrote:
Quoting Andrew Haley (2013-04-26 11:11:41)
On 04/22/2013 11:01 PM, Orion Poplawski wrote:
The eclipse PTP project that I package is starting to ship an executable in one of its plugins. The executable is packed into a jar file. This prevents debuginfo creation for the executable.
I started taking a look at how this might be handled elsewhere and found:
/usr/lib64/eclipse/dropins/cdt/eclipse/plugins/org.eclipse.cdt.core.linux.x86_64_5.2.0.201302132326.jar: 13327 Defl:X 3624 73% 02-13-2013 23:21 326488c7 os/linux/x86_64/libpty.so 23720 Defl:X 7151 70% 02-13-2013 23:21 c26c2c94 os/linux/x86_64/libspawner.so
but no debuginfo package.
Thoughts, comments?
This is pretty obviously a breach of packaging guidelines. We shouldn't be hiding DSOs in archives.
Hmm, I am not aware of any guideline touching this topic (perhaps because it's not that common). But from practical perspective these jars are extremely practical because they don't have to deal with JNI file locations and similar things which ten to be quite variable.
I don't consider this approach hiding, but we could surely improve the tooling support for it.
WRT original question, I am not sure how to proceed. Even if you generate debuginfo packages I have to wonder how easy will it be to use in a debugger?
The workaround that I am employing for the moment in PTP is to install a copy of the binary in %{_libdir}/ptp so that rpm can find it an generate debuginfo. As for how useful that might be, I don't know. I *think* gdb matches things up via hashes these days so it might work, and might for the cdt shared libraries as well.
Whatever the solution, I think we need to improve our tooling to check inside of archives for binaries. I'm CC'ing the packaging list as well.
Orion Poplawski wrote:
Whatever the solution, I think we need to improve our tooling to check inside of archives for binaries. I'm CC'ing the packaging list as well.
Please keep in mind that jarfiles get repackaged inside other jarfiles (.war, .ear etc), and I've also seen applications that built the files with .zip extensions (though I haven't touched Java packaging for some time now). To be sure you've got everything you can't rely on file extensions, you have to open every file in the RPM (and every file in every zipfile you find in there, ad infinitum) and look for the zipfile magic code. It's also best not to rely on finding the solibs in any specific place, because the chances are there's yet another .*ar format you haven't heard of that puts them somewhere else ;)
Cheers, Gary
java-devel@lists.fedoraproject.org
-
Andrew Haley -
Florian Weimer -
Gary Benson -
Krzysztof Daniel -
Mark Wielaard -
Orion Poplawski -
Stanislav Ochotnicky -
Ville Skyttä