Re: OCaml and static linking (was old thread: Re: [Fedora-packaging] Issues with Ocaml and Static Linking)
David Woodhouse wrote:
On Sat, 2007-05-19 at 17:36 +0100, Richard W.M. Jones wrote:
I suspect it's unlikely that upstream will do (a), ever. There's a technical issue. OCaml really doesn't have a concept of an ABI. It does a kind of whole-program optimisation where even changes to the internal implementation of a library can affect the resulting binary. Moreover even if you "fixed" that, any change whatsoever to the library's signature or the version of compiler it was built with (even bugfix releases which have the same version number) will make the library incompatible.
Our current package scheme doesn't handle this at all, does it? Should our ocaml-*-devel packages have runtime Requires: on the precise n-v-r of ocaml used to build them?
Yes definitely. In fact any other behaviour is broken. (All packages should have this Requires -- I'm not sure why you elected for just the -devel packages). Furthermore, if one library or program depends on another library, then it must contain a dependency on the precise n-v-r of the library.
The only reason I didn't do it for the four packages I just put up for review is that I couldn't work out _how_ to do it in the spec file :-(
[My packages for review: http://tinyurl.com/2rl4w6]
Example:
ocaml-3.09.3-1
ocaml-pcre-5.11.4-1 Requires: ocaml = 3.09.3-1
ocaml-extlib-1.5-1 Requires: ocaml = 3.09.3-1
cocanwiki-1.4.3-1 Requires: ocaml-pcre = 5.11.4-1, ocaml-extlib = 1.5-1, ocaml = 3.09.3-1
The final package requires the precise versions of the two libraries it was built against, plus the precise version of OCaml that it was built with.
I don't know how well this will interact with the Fedora build system. For instance if a new version of a fairly fundamental library (eg. ocaml itself, or something like ocaml-pcre) is released, everything which depends on that has to be recompiled. This is something of a perpetual problem for the Debian folks.
At one point I remember that OCaml in Ubuntu was really broken, apparently because they'd taken packages from upstream Debian half way through a transition like this, so some packages were compiled against one version of ocaml-pcre, and others against another version, with the result that you couldn't use certain combinations of libraries if the libraries depended on different ocaml-pcre.
Rich.
Richard W.M. Jones wrote:
David Woodhouse wrote:
On Sat, 2007-05-19 at 17:36 +0100, Richard W.M. Jones wrote:
I suspect it's unlikely that upstream will do (a), ever. There's a technical issue. OCaml really doesn't have a concept of an ABI. It does a kind of whole-program optimisation where even changes to the internal implementation of a library can affect the resulting binary. Moreover even if you "fixed" that, any change whatsoever to the library's signature or the version of compiler it was built with (even bugfix releases which have the same version number) will make the library incompatible.
Our current package scheme doesn't handle this at all, does it? Should our ocaml-*-devel packages have runtime Requires: on the precise n-v-r of ocaml used to build them?
Yes definitely. In fact any other behaviour is broken. (All packages should have this Requires -- I'm not sure why you elected for just the -devel packages). Furthermore, if one library or program depends on another library, then it must contain a dependency on the precise n-v-r of the library.
The only reason I didn't do it for the four packages I just put up for review is that I couldn't work out _how_ to do it in the spec file :-(
Looking at this a bit closer, seems like we need an OCaml version of "find-requires" and "find-provides"?
There is a program called objinfo (packaged in Debian as "ocamlobjinfo"). This isn't part of our RPM, but it should be. You can get this program by building OCaml from the current SRPM, then: cd ~/rpmbuild/BUILD/ocaml-3.09.3 cd tools make objinfo cp objinfo ~/bin/ocamlobjinfo
Now, a bit of background first. When OCaml builds a module, it takes a MD5 hash over the interface and some of the internals. It also stores in the module the MD5 hashes of any modules that it depends upon. At link time the MD5 hashes are compared, and the link is only allowed to proceed if they match.
You can use ocamlobjinfo to find the hashes of modules (and what they depend upon).
For example:
$ ocamlobjinfo /usr/lib64/ocaml/calendar/calendar.cma
gives output like this (heavily cut down):
Unit name: Time_Zone Interfaces imported: 71f888453b0f26895819460a72f07493 Pervasives 1876ce678cfa5a1961fac21850f71efd Unix 8d84727c6f398e1a8b710d8161e26479 Time_Zone
This means that the library file calendar.cma contains a module called Time_Zone. Time_Zone depends on two modules from the stdlib (Pervasives and Unix) and gives the expected MD5 hashes of those modules. It also gives the MD5 hash of the Time_Zone module itself. Confusingly the hashes of the module and its dependencies are all mixed up... Parsing the output of ocamlobjinfo is annoying.
So my first idea was to write a "ocaml-find-requires.sh" script which would look at the output of ocamlobjinfo and try to work out which files (and therefore RPMs) a library depends upon. This turned out to be quite difficult because of the problem of what if a library is already installed. Anyhow I had (I think) a much better idea ...
How about we store the actual module names and MD5 sums as actual provides and requires? So for example the base ocaml module would provide:
ocaml-Pervasives-71f888453b0f26895819460a72f07493 ocaml-Unix-1876ce678cfa5a1961fac21850f71efd
and the ocaml-calendar module would require the above, as well as providing:
ocaml-Time_Zone-8d84727c6f398e1a8b710d8161e26479
(It would still need to depend on the exact n-v-r of the ocaml compiler).
So RPM enforces the exact same dependency requirements as OCaml itself. Hopefully with such a scheme it would be impossible to install incompatible OCaml modules, at least not without forcing them.
Attached are two candidate ocaml-find-requires.sh and ocaml-find-provides.sh scripts.
$ echo /usr/lib64/ocaml/calendar/calendar.cma | ./ocaml-find-requires.sh ocaml-Array-a904b798dd9665c2d3635636d293403c ocaml-Buffer-45f466ce46f213dae41be77dd8505f5f ocaml-Format-67f81fa527012cf0f70f6f6a24f07417 ocaml-Lazy-27baa9469b2986a6ccbba3e85275ecf1 ocaml-List-5a0e3217fc356bd18f60bff31861dfd3 ocaml-Period-d99e051c4c63bb80cd2586ea99584429 ocaml-Pervasives-71f888453b0f26895819460a72f07493 ocaml-Str-8a11c5ef144a995903cc9e1bac5e353c ocaml-String-fec8292bb1a02d2c7b8b4ba7b83a7d8b ocaml-Unix-1876ce678cfa5a1961fac21850f71efd ocaml = 3.09.3-1.fc6
$ echo /usr/lib64/ocaml/calendar/calendar.cma | ./ocaml-find-provides.sh ocaml-Calendar-514a56b1c3e9c1e5139e81e0e2736ab8 ocaml-Date-0b9d8d46ec722919ef4a2b4f7576d8f1 ocaml-Printer-de9afc53dc6db958fdf5927dc86df9aa ocaml-Time-1b657b86bb0e7ba36acf02910897b2eb ocaml-Time_Zone-8d84727c6f398e1a8b710d8161e26479
Rich.
packaging@lists.fedoraproject.org
-
Richard W.M. Jones