Hi,
I was thinking of having a 'aot-compile' command that would perform all tasks related to aot compilation.
For instance:
aot-compile
by itself would perform the additional compilation steps necessary to generate the .so files etc., as currently done.
aot-compile --rebuilddb
would do the rebuild-gcj-db thing
aot-compile --install <install arguments>
would allow a conditional installation step
and so forth.
In addition to that, 'aot-compile' would either work or be a no-op, depending on certain conditions, like the presence of GCJ in the system, and/or some system-wide or local override configuration mechanism.
This way, aot-compile statements can be added to an RPM spec file and that same spec file can be used to build bytecode or pre-compiled RPMs, depending on the settings. Adding just this script to jpackage-utils would allow us to eliminate differences between the spec files upstream and in other distros, except for having to remove the "BuildArch: noarch" line.
It seems that there are a few cases to consider:
1) gcj is not installed ==> automatic ==> do not precompile
2) gcj is installed, but we don't want to pre-compile ==> override with either system or local config (we could test for "BuildArch: noarch"...) ==> do not precompile
3) gcj is installed, and we want to pre-compile ==> automatic ==> precompile
There are perhaps some other conditions, like you want to build the bytecode with another Java and then use gcj to pre-compile...
IMPORTANT: please note that aot-compile should not be tied to GCJ, but instead be a characteristic of any Java that is capable of pre-compilation. QUESTION: how to accommodate the differences in pre-compilation mechanisms? In any case, we should try and keep the command names generic so if necessary one day the JVMs that are AOT-capable can provide alternatives for those.
Regards to all, Fernando
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Fernando Nasser wrote:
IMPORTANT: please note that aot-compile should not be tied to GCJ, but instead be a characteristic of any Java that is capable of pre-compilation. QUESTION: how to accommodate the differences in pre-compilation mechanisms? In any case, we should try and keep the command names generic so if necessary one day the JVMs that are AOT-capable can provide alternatives for those.
I am glad you realized this. I was going to mention to the FC guys that, for example, kaffe has talked about supporting aot. If you have kaffe, you don't necessary need/want to use gcj.
With the integration into rpm, would it be possible to produce the gcj support as a separate package such that upgrades from JPackage to not trump it. All we need for this to work is to make sure we don't have any %{version}-%{release} requirements in JPackage and instead stick to %{version}. At least using this method release updates could be accomplished. I don't know enough about the situation of differing versions at this time.
- -- Sincerely,
David Walluck david@zarb.org
David Walluck wrote:
Fernando Nasser wrote:
IMPORTANT: please note that aot-compile should not be tied to GCJ, but instead be a characteristic of any Java that is capable of pre-compilation.
I am glad you realized this. I was going to mention to the FC guys that, for example, kaffe has talked about supporting aot. If you have kaffe, you don't necessary need/want to use gcj.
Well, if anything the aot supported by Kaffe will be GCJ's, so you'll need GCJ for compilation at any rate.
With the integration into rpm, would it be possible to...
Sadly, close integration with rpm can not happen without a serious change of heart on behalf of the rpm maintainers.
Cheers, Gary
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Gary Benson wrote:
Well, if anything the aot supported by Kaffe will be GCJ's, so you'll need GCJ for compilation at any rate.
At install time on the packagers machines, yes. At runtime on the user's machine, I don't think so. I just wanted to be sure that a user would not be forced to install gcj, which make sense, as gcj-dbtool and gcj are normally in separate rpms.
Anyway, there is no argument here except that aot-compile was going to end up one place and perhaps rebuild-gcj-db was going to end up another place (upstream). It could come into play depending on how they were split up.
- -- Sincerely,
David Walluck david@zarb.org
The way this traditionally works in UNIX is by using `make'.
For example, to rebuild the mail database you simply go to /etc/mail and type
make
which does whatever is necessary.
Why not do it this way with gcj's dependencies? Just go into /var/lib/gcj<blah> and run make? The makefile can then do everything that is necessary, using make's dependency analysis.
Andrew.
Andrew Haley wrote:
The way this traditionally works in UNIX is by using `make'.
For example, to rebuild the mail database you simply go to /etc/mail and type
makewhich does whatever is necessary.
Why not do it this way with gcj's dependencies? Just go into /var/lib/gcj<blah> and run make? The makefile can then do everything that is necessary, using make's dependency analysis.
That's fine for sendmail, where everything in /etc/mail is owned by sendmail. GCJ can't know in advance what will be in /usr/lib/gcj, so such a makefile would need to be edited or generated by something. Instead of having a script to build a database, we'd end up with a script to build a makefile that'd build a database.
Cheers, Gary
Gary Benson writes:
Andrew Haley wrote:
The way this traditionally works in UNIX is by using `make'.
For example, to rebuild the mail database you simply go to /etc/mail and type
makewhich does whatever is necessary.
Why not do it this way with gcj's dependencies? Just go into /var/lib/gcj<blah> and run make? The makefile can then do everything that is necessary, using make's dependency analysis.
That's fine for sendmail, where everything in /etc/mail is owned by sendmail. GCJ can't know in advance what will be in /usr/lib/gcj, so such a makefile would need to be edited or generated by something.
I don't think so. All the makefile has to know is that the master db depends on *.db and *.so in some directory.
Andrew.
Andrew Haley wrote:
Gary Benson writes:
Andrew Haley wrote:
The way this traditionally works in UNIX is by using `make'.
For example, to rebuild the mail database you simply go to /etc/mail and type
makewhich does whatever is necessary.
Why not do it this way with gcj's dependencies? Just go into /var/lib/gcj<blah> and run make? The makefile can then do everything that is necessary, using make's dependency analysis.
That's fine for sendmail, where everything in /etc/mail is owned by sendmail. GCJ can't know in advance what will be in /usr/lib/gcj, so such a makefile would need to be edited or generated by something.
I don't think so. All the makefile has to know is that the master db depends on *.db and *.so in some directory.
How would it cope with file deletions? If we uninstall a rpm then its entries should be removed from the master database.
Cheers, Gary
Gary Benson writes:
Andrew Haley wrote:
Gary Benson writes:
Andrew Haley wrote:
The way this traditionally works in UNIX is by using `make'.
For example, to rebuild the mail database you simply go to /etc/mail and type
makewhich does whatever is necessary.
Why not do it this way with gcj's dependencies? Just go into /var/lib/gcj<blah> and run make? The makefile can then do everything that is necessary, using make's dependency analysis.
That's fine for sendmail, where everything in /etc/mail is owned by sendmail. GCJ can't know in advance what will be in /usr/lib/gcj, so such a makefile would need to be edited or generated by something.
I don't think so. All the makefile has to know is that the master db depends on *.db and *.so in some directory.
How would it cope with file deletions?
Use the directory as a dependency. Any deletions will cause the directory to be touched.
------------------------------------------------------------------------ bar: foo echo "update" touch bar ------------------------------------------------------------------------ $ mkdir foo $ touch foo/baz $ make echo "update" update touch bar $ make make: `bar' is up to date. $ rm foo/baz $ make echo "update" update touch bar
Andrew.
Fernando Nasser wrote:
I was thinking of having a 'aot-compile' command that would perform all tasks related to aot compilation.
For instance:
aot-compile
by itself would perform the additional compilation steps necessary to generate the .so files etc., as currently done.
[snip]
aot-compile --install <install arguments>
would allow a conditional installation step
and so forth.
Firstly, a minor point. I guess you want this so that the former can go in %build and the latter can go in %install? I wanted to do this anyway, but separating the process turns out to be a nightmare because what's in the build tree after %build is almost never anything like what's in the install tree after %install. Not everything is copied over, and what is is almost always renamed.
FWIW katana _was_ split in this way: packages frequently required tens of options just to tell it what was going to be installed where. It was awful...
In addition to that, 'aot-compile' would either work or be a no-op, depending on certain conditions, like the presence of GCJ in the system, and/or some system-wide or local override configuration mechanism.
The fact that aot-compiled rpms have more files than their uncompiled equivalents will make this far less neat than you perhaps think. Each package will need a conditional block in its %files list. It's not something that can be done simply with globs.
That aside, having the database rebuilding as an alternative (so possibly a no-op) would also cause problems. Imagine this:
1. User has GCJ as their JVM. 2. a) User installs some Fedora packages. b) GCJ database is rebuilt into a consistent state. 3. User switches to some other JVM. 4. a) User installs some more Fedora packages. b) GCJ database is not rebuilt. 5. User switches to GCJ as their JVM.
The user has ended up with a broken database. His applications will be slower, and in some (admittedly broken) cases will suddenly start to fail.
Rather than being an alternative, the rebuild command would need to rebuild all databases that could be installed on the system. This could be done by for example having all JVMs drop scripts in some database, and having 'aot-compile --rebuild' or whatever call them all in turn.
It seems that there are a few cases to consider:
gcj is not installed ==> automatic ==> do not precompile
gcj is installed, but we don't want to pre-compile ==> override with either system or local config (we could test for "BuildArch: noarch"...) ==> do not precompile
That's certainly possible: aot-compile-rpm already checks your BuildArch.
IMPORTANT: please note that aot-compile should not be tied to GCJ, but instead be a characteristic of any Java that is capable of pre-compilation.
Sure. This doesn't preclude us from putting the present, GCJ-specific aot-compile-rpm into the gcc-java rpm, and I still think we should proceed with this.
In any case, we should try and keep the command names generic so if necessary one day the JVMs that are AOT-capable can provide alternatives for those.
How about making them even more generic and mandating that all JPackage rpms call certain scripts at certain points. At the end of %install, for example, you could require that all packages invoke jpackage-install, and similarly %post would have jpackage-post and %postun would have jpackage-postun.
Each script could do both generic JPackage stuff and call specific JVM stuff. Under your system jpackage-install would call aot-compile-rpm if GCJ was selected as your JVM, and jpackage-post and -postun would do the database rebuilding. The scripts could do allsorts: the %prep script jpackage-prep could check for bundled jars and classes for example.
Cheers, Gary
Gary Benson writes:
That aside, having the database rebuilding as an alternative (so possibly a no-op) would also cause problems. Imagine this:
- User has GCJ as their JVM.
- a) User installs some Fedora packages. b) GCJ database is rebuilt into a consistent state.
- User switches to some other JVM.
- a) User installs some more Fedora packages. b) GCJ database is not rebuilt.
- User switches to GCJ as their JVM.
The user has ended up with a broken database. His applications will be slower, and in some (admittedly broken) cases will suddenly start to fail.
Rather than being an alternative, the rebuild command would need to rebuild all databases that could be installed on the system. This could be done by for example having all JVMs drop scripts in some database, and having 'aot-compile --rebuild' or whatever call them all in turn.
Use make. If any .db files are added or altered, the .db gets rebuilt. If they aren't, it isn't. The makefile contains the commands that do the right thing.
make is designed to solve this kind of problem.
Andrew.
Andrew Haley wrote:
Use make. If any .db files are added or altered, the .db gets rebuilt. If they aren't, it isn't. The makefile contains the commands that do the right thing.
make is designed to solve this kind of problem.
Can you write a makefile to do this? I've never been able to figure out how to do multiple directory stuff :-/
Cheers, Gary
Gary Benson writes:
Andrew Haley wrote:
Use make. If any .db files are added or altered, the .db gets rebuilt. If they aren't, it isn't. The makefile contains the commands that do the right thing.
make is designed to solve this kind of problem.
Can you write a makefile to do this? I've never been able to figure out how to do multiple directory stuff :-/
Yes, but I'm not about to do it right now until we've agreed precisely what it is supposed to do. Besides, make runes are tromey's department. :-)
Andrew.
Gary Benson writes:
Andrew Haley wrote:
Use make. If any .db files are added or altered, the .db gets rebuilt. If they aren't, it isn't. The makefile contains the commands that do the right thing.
make is designed to solve this kind of problem.
Can you write a makefile to do this? I've never been able to figure out how to do multiple directory stuff :-/
OK, I just checked with tromey, and we think we can write a makefile that allows you to install a stanadrd jpackage and then do a "make".
For example, this rule does the building, where needed:
%.so: %.jar gcj -o $@ blah blah $<
%.db: %.so gcj-dbtool -o $@ db/$(basename $<).jar $<
... etc.
Andrew.
Andrew Haley wrote:
OK, I just checked with tromey, and we think we can write a makefile that allows you to install a stanadrd jpackage and then do a "make".
For example, this rule does the building, where needed:
%.so: %.jar gcj -o $@ blah blah $<
%.db: %.so gcj-dbtool -o $@ db/$(basename $<).jar $<
... etc.
Building .so files and per-package .db files is not the issue. What is needed is the merging of /usr/lib/gcj/*/*.db into /var/lib/gcj/classmap.db.
Cheers, Gary
Gary Benson writes:
Andrew Haley wrote:
OK, I just checked with tromey, and we think we can write a makefile that allows you to install a stanadrd jpackage and then do a "make".
For example, this rule does the building, where needed:
%.so: %.jar gcj -o $@ blah blah $<
%.db: %.so gcj-dbtool -o $@ db/$(basename $<).jar $<
... etc.
Building .so files and per-package .db files is not the issue.
It certainly was mentioned in the message I was replying to.
What is needed is the merging of /usr/lib/gcj/*/*.db into /var/lib/gcj/classmap.db.
So all you want is a command that does the merging we do at the moment, but only when necessary?
Andrew.
Andrew Haley wrote:
Gary Benson writes:
What is needed is the merging of /usr/lib/gcj/*/*.db into /var/lib/gcj/classmap.db.
So all you want is a command that does the merging we do at the moment, but only when necessary?
I think all he wanted is the command that does the merging we do at the moment full stop. He just wanted what rebuild-gcj-db does in gcj-dbtool. I'm quite happy to write it if you like.
Cheers, Gary
Gary Benson writes:
Andrew Haley wrote:
Gary Benson writes:
What is needed is the merging of /usr/lib/gcj/*/*.db into /var/lib/gcj/classmap.db.
So all you want is a command that does the merging we do at the moment, but only when necessary?
I think all he wanted is the command that does the merging we do at the moment full stop. He just wanted what rebuild-gcj-db does in gcj-dbtool. I'm quite happy to write it if you like.
Writing it is easy; I want to make sure it does the right thing.
Andrew.
Andrew Haley wrote:
Gary Benson writes:
Andrew Haley wrote:
Gary Benson writes:
What is needed is the merging of /usr/lib/gcj/*/*.db into /var/lib/gcj/classmap.db.
So all you want is a command that does the merging we do at the moment, but only when necessary?
I think all he wanted is the command that does the merging we do at the moment full stop. He just wanted what rebuild-gcj-db does in gcj-dbtool. I'm quite happy to write it if you like.
Writing it is easy; I want to make sure it does the right thing.
Using make worries me because I frequently downgrade versions of things while I'm testing. Downgrading will give the .db files an older timestamp, and the system database will not be rebuilt.
There are occasions when beehive will downgrade what's in its buildroot. Fixing broken a database on my machine is easy, fixing it on 20+ beehive machines is not.
That aside, rebuilding the databases takes no time at all. Using make seems to me to be adding an additional layer of complexity for no perceptible gain.
Cheers, Gary
Gary Benson writes:
Andrew Haley wrote:
Gary Benson writes:
Andrew Haley wrote:
Gary Benson writes:
What is needed is the merging of /usr/lib/gcj/*/*.db into /var/lib/gcj/classmap.db.
So all you want is a command that does the merging we do at the moment, but only when necessary?
I think all he wanted is the command that does the merging we do at the moment full stop. He just wanted what rebuild-gcj-db does in gcj-dbtool. I'm quite happy to write it if you like.
Writing it is easy; I want to make sure it does the right thing.
Using make worries me because I frequently downgrade versions of things while I'm testing. Downgrading will give the .db files an older timestamp, and the system database will not be rebuilt.
It'll give the directory a newer timestamp, so the system database will be rebuilt.
There are occasions when beehive will downgrade what's in its buildroot. Fixing broken a database on my machine is easy, fixing it on 20+ beehive machines is not.
That aside, rebuilding the databases takes no time at all. Using make seems to me to be adding an additional layer of complexity for no perceptible gain.
So why worry about making rebuild an alternative? If it's no big deal, why not always do it?
Andrew.
Andrew Haley wrote:
Gary Benson writes:
Using make worries me because I frequently downgrade versions of things while I'm testing. Downgrading will give the .db files an older timestamp, and the system database will not be rebuilt.
It'll give the directory a newer timestamp, so the system database will be rebuilt.
I need to see a makefile which does this.
That aside, rebuilding the databases takes no time at all. Using make seems to me to be adding an additional layer of complexity for no perceptible gain.
So why worry about making rebuild an alternative? If it's no big deal, why not always do it?
There's two issues here which I think you are confusing:
1) Presently, rebuild-gcj-db and aot-compile-rpm _are_ alternatives, though not between JVMs: they're shared between versions of java-1.4.2-gcj*-compat. Tom Fitzsimmons wants them not to be alternatives, but to be in gcc subpackages (libgcj for r-g-d, and gcc-java for a-c-r). Almost incidentally, Tom pointed out that rebuild-gcj-db is so small its functionality could easily be incorporated in gcj-dbtool.
2) On the other hand, Fernando wants the two scripts to remain alternatives, but shared between all JVMs (not just GCJ ones). My opinion is that something like this would be a good idea, but that acquiring the GCJ-specific command names for it is the wrong thing to do (not least because GCJ's database should be rebuilt whenever an rpm with GCJ-precompiled stuff is installed regardless of what JVM alternative is in force). This needs more discussion (on JPackage lists so the relevant people see it) but the result of that discussion should not stop us from making the GCJ-specific changes to the GCJ-specific rpms that we need for future development.
Cheers, Gary
Gary Benson writes:
Andrew Haley wrote:
Gary Benson writes:
Using make worries me because I frequently downgrade versions of things while I'm testing. Downgrading will give the .db files an older timestamp, and the system database will not be rebuilt.
It'll give the directory a newer timestamp, so the system database will be rebuilt.
I need to see a makefile which does this.
That aside, rebuilding the databases takes no time at all. Using make seems to me to be adding an additional layer of complexity for no perceptible gain.
So why worry about making rebuild an alternative? If it's no big deal, why not always do it?
There's two issues here which I think you are confusing:
I don't think so!
- Presently, rebuild-gcj-db and aot-compile-rpm _are_ alternatives, though not between JVMs: they're shared between versions of java-1.4.2-gcj*-compat. Tom Fitzsimmons wants them not to be alternatives, but to be in gcc subpackages (libgcj for r-g-d, and gcc-java for a-c-r). Almost incidentally, Tom pointed out that rebuild-gcj-db is so small its functionality could easily be incorporated in gcj-dbtool.
Right. No argument there. I have no disagreement with any of this. Making gcj-dbtool do the right thing is easy, and I'll do it.
- On the other hand, Fernando wants the two scripts to remain alternatives, but shared between all JVMs (not just GCJ ones). My opinion is that something like this would be a good idea, but that acquiring the GCJ-specific command names for it is the wrong thing to do (not least because GCJ's database should be rebuilt whenever an rpm with GCJ-precompiled stuff is installed regardless of what JVM alternative is in force).
This is the issue that I am addressing.
My suggestion solves this problem:
GCJ's database should be rebuilt whenever an rpm with GCJ-precompiled stuff is installed regardless of what JVM alternative is in force
... without penalizing users who aren't installing gcj packages. It also abstracts away from the RPM specfiles the nitpicky gcj details. This is better than having "gcj-dbtool --rebuild" or its equivalent in each specfile.
This needs more discussion (on JPackage lists so the relevant people see it) but the result of that discussion should not stop us from making the GCJ-specific changes to the GCJ-specific rpms that we need for future development.
Andrew.
Andrew Haley wrote:
Gary Benson writes:
- Presently, rebuild-gcj-db and aot-compile-rpm _are_ alternatives, though not between JVMs: they're shared between versions of java-1.4.2-gcj*-compat. Tom Fitzsimmons wants them not to be alternatives, but to be in gcc subpackages (libgcj for r-g-d, and gcc-java for a-c-r). Almost incidentally, Tom pointed out that rebuild-gcj-db is so small its functionality could easily be incorporated in gcj-dbtool.
Right. No argument there. I have no disagreement with any of this. Making gcj-dbtool do the right thing is easy, and I'll do it.
Oh, ok, cool. In that case, you should know that you only need to scan /usr/lib/gcj. rebuild-gcj-db also scans $dblocation (aka /usr/lib/gcj-x.y.z/classmap.db.d) but that was just a workaround for FC4 and can be removed.
- On the other hand, Fernando wants the two scripts to remain alternatives, but shared between all JVMs (not just GCJ ones). My opinion is that something like this would be a good idea, but that acquiring the GCJ-specific command names for it is the wrong thing to do (not least because GCJ's database should be rebuilt whenever an rpm with GCJ-precompiled stuff is installed regardless of what JVM alternative is in force).
This is the issue that I am addressing.
My suggestion solves this problem:
GCJ's database should be rebuilt whenever an rpm with GCJ-precompiled stuff is installed regardless of what JVM alternative is in force... without penalizing users who aren't installing gcj packages. It also abstracts away from the RPM specfiles the nitpicky gcj details. This is better than having "gcj-dbtool --rebuild" or its equivalent in each specfile.
When will it be invoked if not in every specfile?
Cheers, Gary
Gary Benson writes:
Andrew Haley wrote:
Gary Benson writes:
- Presently, rebuild-gcj-db and aot-compile-rpm _are_ alternatives, though not between JVMs: they're shared between versions of java-1.4.2-gcj*-compat. Tom Fitzsimmons wants them not to be alternatives, but to be in gcc subpackages (libgcj for r-g-d, and gcc-java for a-c-r). Almost incidentally, Tom pointed out that rebuild-gcj-db is so small its functionality could easily be incorporated in gcj-dbtool.
Right. No argument there. I have no disagreement with any of this. Making gcj-dbtool do the right thing is easy, and I'll do it.
Oh, ok, cool. In that case, you should know that you only need to scan /usr/lib/gcj. rebuild-gcj-db also scans $dblocation (aka /usr/lib/gcj-x.y.z/classmap.db.d) but that was just a workaround for FC4 and can be removed.
OK.
- On the other hand, Fernando wants the two scripts to remain alternatives, but shared between all JVMs (not just GCJ ones). My opinion is that something like this would be a good idea, but that acquiring the GCJ-specific command names for it is the wrong thing to do (not least because GCJ's database should be rebuilt whenever an rpm with GCJ-precompiled stuff is installed regardless of what JVM alternative is in force).
This is the issue that I am addressing.
My suggestion solves this problem:
GCJ's database should be rebuilt whenever an rpm with GCJ-precompiled stuff is installed regardless of what JVM alternative is in force... without penalizing users who aren't installing gcj packages. It also abstracts away from the RPM specfiles the nitpicky gcj details. This is better than having "gcj-dbtool --rebuild" or its equivalent in each specfile.
When will it be invoked if not in every specfile?
It will be invoked from the specfile, but by either
a. Going to some java dir and running make, or b. Running some VM-agnostic script that does a.
That way, should there ever be some other VM that does precompilation, we need only to add a make rule for whatever it needs.
Andrew.
Andrew Haley wrote:
Gary Benson writes:
Andrew Haley wrote:
Gary Benson writes:
- On the other hand, Fernando wants the two scripts to remain alternatives, but shared between all JVMs (not just GCJ ones). My opinion is that something like this would be a good idea, but that acquiring the GCJ-specific command names for it is the wrong thing to do (not least because GCJ's database should be rebuilt whenever an rpm with GCJ-precompiled stuff is installed regardless of what JVM alternative is in force).
This is the issue that I am addressing.
My suggestion solves this problem:
GCJ's database should be rebuilt whenever an rpm with GCJ-precompiled stuff is installed regardless of what JVM alternative is in force
... without penalizing users who aren't installing gcj packages. It also abstracts away from the RPM specfiles the nitpicky gcj details. This is better than having "gcj-dbtool --rebuild" or its equivalent in each specfile.
When will it be invoked if not in every specfile?
It will be invoked from the specfile, but by either
a. Going to some java dir and running make, or b. Running some VM-agnostic script that does a.
That way, should there ever be some other VM that does precompilation, we need only to add a make rule for whatever it needs.
Ok. FWIW I perfer B, but it's really something for JPackage not Fedora to decide. As long as Fedora rpms can continue to call 'rebuild-gcj-db' or 'gcj-dbtool --rebuild' until JPackage figure out what they're doing then that's fine.
Cheers, Gary
Gary Benson writes:
Andrew Haley wrote:
Gary Benson writes:
Andrew Haley wrote:
Gary Benson writes:
- On the other hand, Fernando wants the two scripts to remain alternatives, but shared between all JVMs (not just GCJ ones). My opinion is that something like this would be a good idea, but that acquiring the GCJ-specific command names for it is the wrong thing to do (not least because GCJ's database should be rebuilt whenever an rpm with GCJ-precompiled stuff is installed regardless of what JVM alternative is in force).
This is the issue that I am addressing.
My suggestion solves this problem:
GCJ's database should be rebuilt whenever an rpm with GCJ-precompiled stuff is installed regardless of what JVM alternative is in force
... without penalizing users who aren't installing gcj packages. It also abstracts away from the RPM specfiles the nitpicky gcj details. This is better than having "gcj-dbtool --rebuild" or its equivalent in each specfile.
When will it be invoked if not in every specfile?
It will be invoked from the specfile, but by either
a. Going to some java dir and running make, or b. Running some VM-agnostic script that does a.
That way, should there ever be some other VM that does precompilation, we need only to add a make rule for whatever it needs.
Ok. FWIW I perfer B, but it's really something for JPackage not Fedora to decide. As long as Fedora rpms can continue to call 'rebuild-gcj-db' or 'gcj-dbtool --rebuild' until JPackage figure out what they're doing then that's fine.
Well, OK: I don't mind. Going through every RPM changing "rebuild-gcj-db" to "gcj-dbtool --rebuild" and then changing it again some time in the future sounds like a PITA to me ... but then I won't be doing it!
Andrew.
Andrew Haley wrote:
Gary Benson writes:
Andrew Haley wrote:
It will be invoked from the specfile, but by either
a. Going to some java dir and running make, or b. Running some VM-agnostic script that does a.
That way, should there ever be some other VM that does precompilation, we need only to add a make rule for whatever it needs.
Ok. FWIW I perfer B, but it's really something for JPackage not Fedora to decide. As long as Fedora rpms can continue to call 'rebuild-gcj-db' or 'gcj-dbtool --rebuild' until JPackage figure out what they're doing then that's fine.
Well, OK: I don't mind. Going through every RPM changing "rebuild-gcj-db" to "gcj-dbtool --rebuild" and then changing it again some time in the future sounds like a PITA to me ... but then I won't be doing it!
It'd probably be best to leave them calling "rebuild-gcj-db" for now, since we won't be able to get rid of /usr/bin/rebuild-gcj-db until we stop caring about people upgrading from FC4.
Cheers, Gary
Andrew Haley wrote:
Gary Benson writes:
Andrew Haley wrote:
Gary Benson writes:
- Presently, rebuild-gcj-db and aot-compile-rpm _are_ alternatives, though not between JVMs: they're shared between versions of java-1.4.2-gcj*-compat. Tom Fitzsimmons wants them not to be alternatives, but to be in gcc subpackages (libgcj for r-g-d, and gcc-java for a-c-r). Almost incidentally, Tom pointed out that rebuild-gcj-db is so small its functionality could easily be incorporated in gcj-dbtool.
Right. No argument there. I have no disagreement with any of this. Making gcj-dbtool do the right thing is easy, and I'll do it.
Oh, ok, cool. In that case, you should know that you only need to scan /usr/lib/gcj. rebuild-gcj-db also scans $dblocation (aka /usr/lib/gcj-x.y.z/classmap.db.d) but that was just a workaround for FC4 and can be removed.
OK.
Oh, and something else. Even if we switch new rpms over to do something like 'gcj-dbtool --rebuild' we must retain something at /usr/bin/rebuild-gcj-db or it will become impossible to upgrade or uninstall the FC4 packages.
My preference would be to have /usr/bin/rebuild-gcj-db a symbolic link to gcj-dbtool and have gcj-dbtool check its argv[0] (if that's possible under Java).
Also note that while FC5 and the newest FC4 packages call rebuild-gcj-db with no arguments, the majority of FC4 packages will call 'rebuild-gcj-db /usr/lib' or 'rebuild-gcj-db /usr/lib64'. Any replacement must not choke on this.
Cheers, Gary
Gary Benson writes:
Andrew Haley wrote:
Gary Benson writes:
Andrew Haley wrote:
Gary Benson writes:
- Presently, rebuild-gcj-db and aot-compile-rpm _are_ alternatives, though not between JVMs: they're shared between versions of java-1.4.2-gcj*-compat. Tom Fitzsimmons wants them not to be alternatives, but to be in gcc subpackages (libgcj for r-g-d, and gcc-java for a-c-r). Almost incidentally, Tom pointed out that rebuild-gcj-db is so small its functionality could easily be incorporated in gcj-dbtool.
Right. No argument there. I have no disagreement with any of this. Making gcj-dbtool do the right thing is easy, and I'll do it.
Oh, ok, cool. In that case, you should know that you only need to scan /usr/lib/gcj. rebuild-gcj-db also scans $dblocation (aka /usr/lib/gcj-x.y.z/classmap.db.d) but that was just a workaround for FC4 and can be removed.
OK.
Oh, and something else. Even if we switch new rpms over to do something like 'gcj-dbtool --rebuild' we must retain something at /usr/bin/rebuild-gcj-db or it will become impossible to upgrade or uninstall the FC4 packages.
My preference would be to have /usr/bin/rebuild-gcj-db a symbolic link to gcj-dbtool and have gcj-dbtool check its argv[0] (if that's possible under Java).
OK. That seems to me like a good idea.
Also note that while FC5 and the newest FC4 packages call rebuild-gcj-db with no arguments, the majority of FC4 packages will call 'rebuild-gcj-db /usr/lib' or 'rebuild-gcj-db /usr/lib64'. Any replacement must not choke on this.
OK. Is there somewhere an exact description of what arguemnts rebuild-gcj-db will take?
Andrew.
Andrew Haley wrote:
Gary Benson wrote:
Also note that while FC5 and the newest FC4 packages call rebuild-gcj-db with no arguments, the majority of FC4 packages will call 'rebuild-gcj-db /usr/lib' or 'rebuild-gcj-db /usr/lib64'. Any replacement must not choke on this.
OK. Is there somewhere an exact description of what arguemnts rebuild-gcj-db will take?
The top of rebuild-gcj-db.in says:
if [ $# != 0 ]; then if [ $# != 1 -o $1 != @LIBDIR@ ]; then # emit usage message and die fi fi
@LIBDIR@ is either /usr/lib or /usr/lib64, depending.
Cheers, Gary
Gary Benson writes:
Andrew Haley wrote:
Gary Benson wrote:
Also note that while FC5 and the newest FC4 packages call rebuild-gcj-db with no arguments, the majority of FC4 packages will call 'rebuild-gcj-db /usr/lib' or 'rebuild-gcj-db /usr/lib64'. Any replacement must not choke on this.
OK. Is there somewhere an exact description of what arguemnts rebuild-gcj-db will take?
The top of rebuild-gcj-db.in says:
if [ $# != 0 ]; then if [ $# != 1 -o $1 != @LIBDIR@ ]; then # emit usage message and die fi fi
@LIBDIR@ is either /usr/lib or /usr/lib64, depending.
I just came across a nasty problem with all this.
If you build a copy of gcj from CVS it won't pick up the system-wide database, because that is in /usr/lib/gcj-4.1.0/, not wherever gcj has been installed. We need a way to rebuild the database into the right place for the installed gcj.
Having gcj-dbtool --rebuild in gcj itself will solve this very nicely because it will rebuild the installed copy, wherever that happens to be.
Andrew.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Andrew Haley wrote:
It will be invoked from the specfile, but by either
a. Going to some java dir and running make, or b. Running some VM-agnostic script that does a.
It happens so quick. Why not have `java' be a script that invokes gij, and then just run it from there? I am not clear why this should be a script that every JVM runs, since only gij (right now) makes use of it.
- -- Sincerely,
David Walluck david@zarb.org
David Walluck wrote:
Andrew Haley wrote:
It will be invoked from the specfile, but by either
a. Going to some java dir and running make, or b. Running some VM-agnostic script that does a.
It happens so quick. Why not have `java' be a script that invokes gij, and then just run it from there?
It needs to happen as root.
Though, I wonder if per-session databases might not be a better idea. Andrew?
I am not clear why this should be a script that every JVM runs, since only gij (right now) makes use of it.
Fernando wants it to be generalised, in case some other JVM acquires the ability to precompile.
Cheers, Gary
Gary Benson writes:
David Walluck wrote:
Andrew Haley wrote:
It will be invoked from the specfile, but by either
a. Going to some java dir and running make, or b. Running some VM-agnostic script that does a.
It happens so quick. Why not have `java' be a script that invokes gij, and then just run it from there?
Rebuild every time? It's fast, but it's not _that_ fast. The speed of rebuilding depends how many libraries you have installed, and scalability was my first design goal. (The time taken to do the rebuild is O(N) in theory, but likely slightly worse in practice because of the page cache.)
It needs to happen as root.
Though, I wonder if per-session databases might not be a better idea. Andrew?
It's a bad idea IMO. The libraries that are installed are global on the system, and the database should be too.
The database is in memory that's shared between all processes, and for that to be secure only root has write access to it. There's no reason not to have per-session databases in some special applications, but it is *not* a good choice for a default policy.
Andrew.
Andrew Haley wrote:
Gary Benson writes:
David Walluck wrote:
Andrew Haley wrote:
It will be invoked from the specfile, but by either
a. Going to some java dir and running make, or b. Running some VM-agnostic script that does a.
It happens so quick. Why not have `java' be a script that invokes gij, and then just run it from there?
Rebuild every time? It's fast, but it's not _that_ fast. The speed of rebuilding depends how many libraries you have installed, and scalability was my first design goal. (The time taken to do the rebuild is O(N) in theory, but likely slightly worse in practice because of the page cache.)
It needs to happen as root.
Though, I wonder if per-session databases might not be a better idea. Andrew?
It's a bad idea IMO. The libraries that are installed are global on the system, and the database should be too.
The database is in memory that's shared between all processes, and for that to be secure only root has write access to it. There's no reason not to have per-session databases in some special applications, but it is *not* a good choice for a default policy.
Fair enough.
Cheers, Gary
Hi,
On Fri, 2005-11-11 at 13:00 +0000, Gary Benson wrote:
[...]
That aside, having the database rebuilding as an alternative (so possibly a no-op) would also cause problems. Imagine this:
- User has GCJ as their JVM.
- a) User installs some Fedora packages. b) GCJ database is rebuilt into a consistent state.
- User switches to some other JVM.
- a) User installs some more Fedora packages. b) GCJ database is not rebuilt.
- User switches to GCJ as their JVM.
The user has ended up with a broken database. His applications will be slower, and in some (admittedly broken) cases will suddenly start to fail.
Right, aot-compile should definitely not be an alternative. The idea is that "aot-compile --rebuilddb" decides what to do based on three criteria:
1) a config file, maybe /etc/java/java.conf, specifies the default behaviour -- to run gcj-dbtool or to be a no-op 2) an environment variable can override this default on a per-run basis 3) if gcj-dbtool is not installed, "aot-compile --rebuilddb" is automatically a no-op
[...]
IMPORTANT: please note that aot-compile should not be tied to GCJ, but instead be a characteristic of any Java that is capable of pre-compilation.
Sure. This doesn't preclude us from putting the present, GCJ-specific aot-compile-rpm into the gcc-java rpm, and I still think we should proceed with this.
Agreed -- aot-compile-rpm is still needed here (and in fact, "aot- compile" is a confusing name to use in this proposal for a new jpackage- neutral script).
In any case, we should try and keep the command names generic so if necessary one day the JVMs that are AOT-capable can provide alternatives for those.
How about making them even more generic and mandating that all JPackage rpms call certain scripts at certain points. At the end of %install, for example, you could require that all packages invoke jpackage-install, and similarly %post would have jpackage-post and %postun would have jpackage-postun.
Each script could do both generic JPackage stuff and call specific JVM stuff. Under your system jpackage-install would call aot-compile-rpm if GCJ was selected as your JVM, and jpackage-post and -postun would do the database rebuilding. The scripts could do allsorts: the %prep script jpackage-prep could check for bundled jars and classes for example.
This sounds excellent. Perhaps we should post a proposal to the jpackage list, along with an example of a converted package.
Tom
java-devel@lists.fedoraproject.org
-
Andrew Haley -
David Walluck -
Fernando Nasser -
Gary Benson -
Thomas Fitzsimmons