I logged a bz (https://bugzilla.redhat.com/show_bug.cgi?id=990588) to request that the tomcat shell scripts (catalina.sh and the rest) be packaged. I've had some discussions with the person the bz is assigned to and few others, but no one knows why the scripts were not packaged. Nor, it seems, does anyone see a problem with them being packaged as far as I can tell.
Does anyone know some history here?
Rob
Hi,
The catalina.sh script work with the $CATALINA_HOME (tomcat binaries) and $CATALINA_BASE (tomcat instances) directories. My guess is that tomcat is packaged as a native (previously sysvinit, and now systemd) service, and that instances wouldn't make sense. Other scripts like startup.sh are just sugar wrappers to the catalina.sh script.
When I say it wouldn't make sense, I mean that it was probably packaged to feel like any other server: sudo service tomcat start or sudo systemctl start tomcat
The package probably owns directories in /var (or somewhere else) that would make it multi-instance unfriendly.
The catalina.sh script also expects sub-directories in $CATALINA_HOME and $CATALINA_BASE. I suspect that tomcat explodes the directory layout (in /usr, /var, maybe /etc) in order to be FHS compliant, which would probably break catalina.sh and its friends.
I'll install tomcat and take a look at the package ASAP.
Dridi
On Mon, Oct 21, 2013 at 9:07 PM, Robert Rati rrati@redhat.com wrote:
I logged a bz (https://bugzilla.redhat.com/show_bug.cgi?id=990588) to request that the tomcat shell scripts (catalina.sh and the rest) be packaged. I've had some discussions with the person the bz is assigned to and few others, but no one knows why the scripts were not packaged. Nor, it seems, does anyone see a problem with them being packaged as far as I can tell.
Does anyone know some history here?
Rob
java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
The lack of the tomcat shell scripts is causing issue with hadoop and some of their ecosystem packages. Some are webapps with custom configuration. The maven builds all create a tomcat install area with their custom configurations. It's not too hard to take that and adapt to fedora's tomcat if the schell scripts were packaged. Then these services could be stopped/started with systemd like other services.
That's assuming catalina.sh and friends are present and functional. :)
Rob
On 10/21/2013 04:42 PM, Dridi Boukelmoune wrote:
Hi,
The catalina.sh script work with the $CATALINA_HOME (tomcat binaries) and $CATALINA_BASE (tomcat instances) directories. My guess is that tomcat is packaged as a native (previously sysvinit, and now systemd) service, and that instances wouldn't make sense. Other scripts like startup.sh are just sugar wrappers to the catalina.sh script.
When I say it wouldn't make sense, I mean that it was probably packaged to feel like any other server: sudo service tomcat start or sudo systemctl start tomcat
The package probably owns directories in /var (or somewhere else) that would make it multi-instance unfriendly.
The catalina.sh script also expects sub-directories in $CATALINA_HOME and $CATALINA_BASE. I suspect that tomcat explodes the directory layout (in /usr, /var, maybe /etc) in order to be FHS compliant, which would probably break catalina.sh and its friends.
I'll install tomcat and take a look at the package ASAP.
Dridi
On Mon, Oct 21, 2013 at 9:07 PM, Robert Rati rrati@redhat.com wrote:
I logged a bz (https://bugzilla.redhat.com/show_bug.cgi?id=990588) to request that the tomcat shell scripts (catalina.sh and the rest) be packaged. I've had some discussions with the person the bz is assigned to and few others, but no one knows why the scripts were not packaged. Nor, it seems, does anyone see a problem with them being packaged as far as I can tell.
Does anyone know some history here?
Rob
java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
I've installed tomcat 7 on my machine to take a quick look at how it's packaged: - exploded FHS-compliant layout - systemd-friendly equivalent to catalina.sh - default configuration in /etc
Now if you want to run a tomcat instance (by instance I mean $CATALINA_BASE) /usr/sbin/tomcat seems to be the best candidate. Unlike catalina.sh, it expects a value for all the CATALINA_* variables in its environment, while catalina.sh has fall-backs relative to $CATALINA_HOME. Simply using /usr/sbin/tomcat as a substitute to catalina.sh wouldn't work of course.
Could you please post an example of what maven produces ? This would help see what could be done with simple maven configuration (eg. -Dsystem=properties) and what would require a patch (and help estimate the amount of work).
Dridi
On Tue, Oct 22, 2013 at 4:59 PM, Robert Rati rrati@redhat.com wrote:
The lack of the tomcat shell scripts is causing issue with hadoop and some of their ecosystem packages. Some are webapps with custom configuration. The maven builds all create a tomcat install area with their custom configurations. It's not too hard to take that and adapt to fedora's tomcat if the schell scripts were packaged. Then these services could be stopped/started with systemd like other services.
That's assuming catalina.sh and friends are present and functional. :)
Rob
On 10/21/2013 04:42 PM, Dridi Boukelmoune wrote:
Hi,
The catalina.sh script work with the $CATALINA_HOME (tomcat binaries) and $CATALINA_BASE (tomcat instances) directories. My guess is that tomcat is packaged as a native (previously sysvinit, and now systemd) service, and that instances wouldn't make sense. Other scripts like startup.sh are just sugar wrappers to the catalina.sh script.
When I say it wouldn't make sense, I mean that it was probably packaged to feel like any other server: sudo service tomcat start or sudo systemctl start tomcat
The package probably owns directories in /var (or somewhere else) that would make it multi-instance unfriendly.
The catalina.sh script also expects sub-directories in $CATALINA_HOME and $CATALINA_BASE. I suspect that tomcat explodes the directory layout (in /usr, /var, maybe /etc) in order to be FHS compliant, which would probably break catalina.sh and its friends.
I'll install tomcat and take a look at the package ASAP.
Dridi
On Mon, Oct 21, 2013 at 9:07 PM, Robert Rati rrati@redhat.com wrote:
I logged a bz (https://bugzilla.redhat.com/show_bug.cgi?id=990588) to request that the tomcat shell scripts (catalina.sh and the rest) be packaged. I've had some discussions with the person the bz is assigned to and few others, but no one knows why the scripts were not packaged. Nor, it seems, does anyone see a problem with them being packaged as far as I can tell.
Does anyone know some history here?
Rob
java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
Here's a listing of the directory structure hadoop and similar bits produce in their builds:
http://paste.fedoraproject.org/48849/13825327
There's a some stuff in there that's can obviously be paired down. Here's the script used to start/stop the service:
http://paste.fedoraproject.org/48852/38253290
It should be noted that upstream hadoop, and its ecosystem, use tomcat 6.x and as part of packaging it we've moved forward to tomcat 7.x.
Rob
On 10/22/2013 12:27 PM, Dridi Boukelmoune wrote:
I've installed tomcat 7 on my machine to take a quick look at how it's packaged:
- exploded FHS-compliant layout
- systemd-friendly equivalent to catalina.sh
- default configuration in /etc
Now if you want to run a tomcat instance (by instance I mean $CATALINA_BASE) /usr/sbin/tomcat seems to be the best candidate. Unlike catalina.sh, it expects a value for all the CATALINA_* variables in its environment, while catalina.sh has fall-backs relative to $CATALINA_HOME. Simply using /usr/sbin/tomcat as a substitute to catalina.sh wouldn't work of course.
Could you please post an example of what maven produces ? This would help see what could be done with simple maven configuration (eg. -Dsystem=properties) and what would require a patch (and help estimate the amount of work).
Dridi
On Tue, Oct 22, 2013 at 4:59 PM, Robert Rati rrati@redhat.com wrote:
The lack of the tomcat shell scripts is causing issue with hadoop and some of their ecosystem packages. Some are webapps with custom configuration. The maven builds all create a tomcat install area with their custom configurations. It's not too hard to take that and adapt to fedora's tomcat if the schell scripts were packaged. Then these services could be stopped/started with systemd like other services.
That's assuming catalina.sh and friends are present and functional. :)
Rob
On 10/21/2013 04:42 PM, Dridi Boukelmoune wrote:
Hi,
The catalina.sh script work with the $CATALINA_HOME (tomcat binaries) and $CATALINA_BASE (tomcat instances) directories. My guess is that tomcat is packaged as a native (previously sysvinit, and now systemd) service, and that instances wouldn't make sense. Other scripts like startup.sh are just sugar wrappers to the catalina.sh script.
When I say it wouldn't make sense, I mean that it was probably packaged to feel like any other server: sudo service tomcat start or sudo systemctl start tomcat
The package probably owns directories in /var (or somewhere else) that would make it multi-instance unfriendly.
The catalina.sh script also expects sub-directories in $CATALINA_HOME and $CATALINA_BASE. I suspect that tomcat explodes the directory layout (in /usr, /var, maybe /etc) in order to be FHS compliant, which would probably break catalina.sh and its friends.
I'll install tomcat and take a look at the package ASAP.
Dridi
On Mon, Oct 21, 2013 at 9:07 PM, Robert Rati rrati@redhat.com wrote:
I logged a bz (https://bugzilla.redhat.com/show_bug.cgi?id=990588) to request that the tomcat shell scripts (catalina.sh and the rest) be packaged. I've had some discussions with the person the bz is assigned to and few others, but no one knows why the scripts were not packaged. Nor, it seems, does anyone see a problem with them being packaged as far as I can tell.
Does anyone know some history here?
Rob
java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
I'm pretty sure I've seen a tomcat6 package too. I'll take a look at your links once I get my hands on my laptop.
-- On my phone, Dridi
On Wednesday, October 23, 2013, Robert Rati rrati@redhat.com wrote:
Here's a listing of the directory structure hadoop and similar bits
produce in their builds:
http://paste.fedoraproject.org/48849/13825327
There's a some stuff in there that's can obviously be paired down. Here's
the script used to start/stop the service:
http://paste.fedoraproject.org/48852/38253290
It should be noted that upstream hadoop, and its ecosystem, use tomcat
6.x and as part of packaging it we've moved forward to tomcat 7.x.
Rob
On 10/22/2013 12:27 PM, Dridi Boukelmoune wrote:
I've installed tomcat 7 on my machine to take a quick look at how it's
packaged:
- exploded FHS-compliant layout
- systemd-friendly equivalent to catalina.sh
- default configuration in /etc
Now if you want to run a tomcat instance (by instance I mean $CATALINA_BASE) /usr/sbin/tomcat seems to be the best candidate. Unlike catalina.sh, it expects a value for all the CATALINA_* variables in its environment, while catalina.sh has fall-backs relative to $CATALINA_HOME. Simply using /usr/sbin/tomcat as a substitute to catalina.sh wouldn't work of course.
Could you please post an example of what maven produces ? This would help see what could be done with simple maven configuration (eg. -Dsystem=properties) and what would require a patch (and help estimate the amount of work).
Dridi
On Tue, Oct 22, 2013 at 4:59 PM, Robert Rati rrati@redhat.com wrote:
The lack of the tomcat shell scripts is causing issue with hadoop and
some
of their ecosystem packages. Some are webapps with custom
configuration.
The maven builds all create a tomcat install area with their custom configurations. It's not too hard to take that and adapt to fedora's
tomcat
if the schell scripts were packaged. Then these services could be stopped/started with systemd like other services.
That's assuming catalina.sh and friends are present and functional. :)
Rob
On 10/21/2013 04:42 PM, Dridi Boukelmoune wrote:
Hi,
The catalina.sh script work with the $CATALINA_HOME (tomcat binaries) and $CATALINA_BASE (tomcat instances) directories. My guess is that tomcat is packaged as a native (previously sysvinit, and now systemd) service, and that instances wouldn't make sense. Other scripts like startup.sh are just sugar wrappers to the catalina.sh script.
When I say it wouldn't make sense, I mean that it was probably packaged to feel like any other server: sudo service tomcat start or sudo systemctl start tomcat
The package probably owns directories in /var (or somewhere else) that would make it multi-instance unfriendly.
The catalina.sh script also expects sub-directories in $CATALINA_HOME and $CATALINA_BASE. I suspect that tomcat explodes the directory layout (in /usr, /var, maybe /etc) in order to be FHS compliant, which would probably break catalina.sh and its friends.
I'll install tomcat and take a look at the package ASAP.
Dridi
On Mon, Oct 21, 2013 at 9:07 PM, Robert Rati rrati@redhat.com wrote:
I logged a bz (https://bugzilla.redhat.com/show_bug.cgi?id=990588) to request that the tomcat shell scripts (catalina.sh and the rest) be packaged. I've had some discussions with the person the bz is
assigned
to and few others, but no one knows why the scripts were not packaged.
Nor,
it seems, does anyone see a problem with them being packaged as far as I
can
tell.
Does anyone know some history here?
Rob
java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
On Wed, Oct 23, 2013 at 3:28 PM, Mikolaj Izdebski mizdebsk@redhat.com wrote:
On 10/23/2013 04:20 PM, Dridi Boukelmoune wrote:
I'm pretty sure I've seen a tomcat6 package too. I'll take a look at your links once I get my hands on my laptop.
Tomcat 6 is retired in Fedora 19 and later.
I see dead packages… Some of them scare me.
-- Mikolaj Izdebski IRC: mizdebsk -- java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
On Wed, Oct 23, 2013 at 1:57 PM, Robert Rati rrati@redhat.com wrote:
Here's a listing of the directory structure hadoop and similar bits produce in their builds:
Looks like tomcat is bundled, which itself is an issue.
There's a some stuff in there that's can obviously be paired down. Here's the script used to start/stop the service:
This whole script could be replaced by a in-house script just like /usr/sbin/tomcat, and it should be fairly easy.
A few things I've spotted: - PRG=$(readlink -f $0) could replace the whole while loop line 19 It's not portable, but we don't care since it'd work on Fedora - The so-called bug mentioned line 51 is actually documented [1] in catalina.sh It works like this by design, I'll notify upstream
It should be noted that upstream hadoop, and its ecosystem, use tomcat 6.x and as part of packaging it we've moved forward to tomcat 7.x.
Unless hadoop's code uses tomcat internals (a valve for instance) this should not be a problem. I don't have time right now do check that.
Dridi
[1] https://github.com/apache/tomcat/blob/d88ad9e/bin/catalina.sh#L36
Rob
On 10/22/2013 12:27 PM, Dridi Boukelmoune wrote:
I've installed tomcat 7 on my machine to take a quick look at how it's packaged:
- exploded FHS-compliant layout
- systemd-friendly equivalent to catalina.sh
- default configuration in /etc
Now if you want to run a tomcat instance (by instance I mean $CATALINA_BASE) /usr/sbin/tomcat seems to be the best candidate. Unlike catalina.sh, it expects a value for all the CATALINA_* variables in its environment, while catalina.sh has fall-backs relative to $CATALINA_HOME. Simply using /usr/sbin/tomcat as a substitute to catalina.sh wouldn't work of course.
Could you please post an example of what maven produces ? This would help see what could be done with simple maven configuration (eg. -Dsystem=properties) and what would require a patch (and help estimate the amount of work).
Dridi
On Tue, Oct 22, 2013 at 4:59 PM, Robert Rati rrati@redhat.com wrote:
The lack of the tomcat shell scripts is causing issue with hadoop and some of their ecosystem packages. Some are webapps with custom configuration. The maven builds all create a tomcat install area with their custom configurations. It's not too hard to take that and adapt to fedora's tomcat if the schell scripts were packaged. Then these services could be stopped/started with systemd like other services.
That's assuming catalina.sh and friends are present and functional. :)
Rob
On 10/21/2013 04:42 PM, Dridi Boukelmoune wrote:
Hi,
The catalina.sh script work with the $CATALINA_HOME (tomcat binaries) and $CATALINA_BASE (tomcat instances) directories. My guess is that tomcat is packaged as a native (previously sysvinit, and now systemd) service, and that instances wouldn't make sense. Other scripts like startup.sh are just sugar wrappers to the catalina.sh script.
When I say it wouldn't make sense, I mean that it was probably packaged to feel like any other server: sudo service tomcat start or sudo systemctl start tomcat
The package probably owns directories in /var (or somewhere else) that would make it multi-instance unfriendly.
The catalina.sh script also expects sub-directories in $CATALINA_HOME and $CATALINA_BASE. I suspect that tomcat explodes the directory layout (in /usr, /var, maybe /etc) in order to be FHS compliant, which would probably break catalina.sh and its friends.
I'll install tomcat and take a look at the package ASAP.
Dridi
On Mon, Oct 21, 2013 at 9:07 PM, Robert Rati rrati@redhat.com wrote:
I logged a bz (https://bugzilla.redhat.com/show_bug.cgi?id=990588) to request that the tomcat shell scripts (catalina.sh and the rest) be packaged. I've had some discussions with the person the bz is assigned to and few others, but no one knows why the scripts were not packaged. Nor, it seems, does anyone see a problem with them being packaged as far as I can tell.
Does anyone know some history here?
Rob
java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
On 10/23/2013 01:09 PM, Dridi Boukelmoune wrote:
On Wed, Oct 23, 2013 at 1:57 PM, Robert Rati rrati@redhat.com wrote:
Here's a listing of the directory structure hadoop and similar bits produce in their builds:
Looks like tomcat is bundled, which itself is an issue.
I've got a patch that pulls out the d/ling and bundling of tomcat. That's not a problem.
There's a some stuff in there that's can obviously be paired down. Here's the script used to start/stop the service:
This whole script could be replaced by a in-house script just like /usr/sbin/tomcat, and it should be fairly easy.
A few things I've spotted:
- PRG=$(readlink -f $0) could replace the whole while loop line 19 It's not portable, but we don't care since it'd work on Fedora
- The so-called bug mentioned line 51 is actually documented [1] in catalina.sh It works like this by design, I'll notify upstream
I'd prefer to avoid fedora specific changes. We'd like to push as much as possible upstream and that's difficult enough. Upstream moves fast and the recent rebase to 2.2.0 was quite a bit of work. If what upstream provides will work then I'd like to avoid adding more work for future rebases.
It should be noted that upstream hadoop, and its ecosystem, use tomcat 6.x and as part of packaging it we've moved forward to tomcat 7.x.
Unless hadoop's code uses tomcat internals (a valve for instance) this should not be a problem. I don't have time right now do check that.
Hadoop uses jasper and jspc and we've ported all that stuff forward to the latest in fedora. There's no need to dive into this mess unless you're curious or like pain. :)
Rob
I should mention that I'd actually tested the functionality and done all the work needed for this piece of functionality to make it into hadoop. The missing piece is the tomcat shell scripts. If those are packaged then I just need to do a little work and I can include the functionality in hadoop. The same work will be needed for other parts of the hadoop ecosystem.
Rob
On 10/23/2013 01:58 PM, Robert Rati wrote:
On 10/23/2013 01:09 PM, Dridi Boukelmoune wrote:
On Wed, Oct 23, 2013 at 1:57 PM, Robert Rati rrati@redhat.com wrote:
Here's a listing of the directory structure hadoop and similar bits produce in their builds:
Looks like tomcat is bundled, which itself is an issue.
I've got a patch that pulls out the d/ling and bundling of tomcat. That's not a problem.
There's a some stuff in there that's can obviously be paired down. Here's the script used to start/stop the service:
This whole script could be replaced by a in-house script just like /usr/sbin/tomcat, and it should be fairly easy.
A few things I've spotted:
- PRG=$(readlink -f $0) could replace the whole while loop line 19 It's not portable, but we don't care since it'd work on Fedora
- The so-called bug mentioned line 51 is actually documented [1] in
catalina.sh It works like this by design, I'll notify upstream
I'd prefer to avoid fedora specific changes. We'd like to push as much as possible upstream and that's difficult enough. Upstream moves fast and the recent rebase to 2.2.0 was quite a bit of work. If what upstream provides will work then I'd like to avoid adding more work for future rebases.
It should be noted that upstream hadoop, and its ecosystem, use tomcat 6.x and as part of packaging it we've moved forward to tomcat 7.x.
Unless hadoop's code uses tomcat internals (a valve for instance) this should not be a problem. I don't have time right now do check that.
Hadoop uses jasper and jspc and we've ported all that stuff forward to the latest in fedora. There's no need to dive into this mess unless you're curious or like pain. :)
Rob
java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
On Wed, Oct 23, 2013 at 7:03 PM, Robert Rati rrati@redhat.com wrote:
I should mention that I'd actually tested the functionality and done all the work needed for this piece of functionality to make it into hadoop. The
What have you tested exactly ? Have you manually added catalina.sh &co somewhere on your system and tested your package ?
missing piece is the tomcat shell scripts. If those are packaged then I just need to do a little work and I can include the functionality in hadoop.
How little ? Have you tried to replace catalina.sh in httpfs.sh [1] ?
-exec ${HTTPFS_CATALINA_HOME}/bin/catalina.sh "$@" +exec /usr/sbin/tomcat "$@"
And maybe set a proper TOMCAT_CFG environment variable pointing to the config for this very package.
The same work will be needed for other parts of the hadoop ecosystem.
Rob
Where can I find a spec with your current work (and a testing procedure) to understand better the problem ? Btw, I'm not the tomcat maintainer, just a regular tomcat user, this is just my opinion. As I said earlier, the catalina.sh file is part of the upstream all-in-one-dir bundles. I don't think catalina.sh is in Fedora's tomcat package, it's also not in Debian's [2] tomcat6 and tomcat7 packages.
Dridi
[1] https://github.com/apache/hadoop-common/blob/1f2a21f/hadoop-hdfs-project/had... [2] http://packages.debian.org/wheezy/all/tomcat6/filelist http://packages.debian.org/wheezy/all/tomcat7/filelist
On 10/23/2013 01:58 PM, Robert Rati wrote:
On 10/23/2013 01:09 PM, Dridi Boukelmoune wrote:
On Wed, Oct 23, 2013 at 1:57 PM, Robert Rati rrati@redhat.com wrote:
Here's a listing of the directory structure hadoop and similar bits produce in their builds:
Looks like tomcat is bundled, which itself is an issue.
I've got a patch that pulls out the d/ling and bundling of tomcat. That's not a problem.
There's a some stuff in there that's can obviously be paired down. Here's the script used to start/stop the service:
This whole script could be replaced by a in-house script just like /usr/sbin/tomcat, and it should be fairly easy.
A few things I've spotted:
- PRG=$(readlink -f $0) could replace the whole while loop line 19 It's not portable, but we don't care since it'd work on Fedora
- The so-called bug mentioned line 51 is actually documented [1] in
catalina.sh It works like this by design, I'll notify upstream
I'd prefer to avoid fedora specific changes. We'd like to push as much as possible upstream and that's difficult enough. Upstream moves fast and the recent rebase to 2.2.0 was quite a bit of work. If what upstream provides will work then I'd like to avoid adding more work for future rebases.
It should be noted that upstream hadoop, and its ecosystem, use tomcat 6.x and as part of packaging it we've moved forward to tomcat 7.x.
Unless hadoop's code uses tomcat internals (a valve for instance) this should not be a problem. I don't have time right now do check that.
Hadoop uses jasper and jspc and we've ported all that stuff forward to the latest in fedora. There's no need to dive into this mess unless you're curious or like pain. :)
Rob
java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
On Fri, Oct 25, 2013 at 11:32 PM, Dridi Boukelmoune dridi.boukelmoune@gmail.com wrote:
On Wed, Oct 23, 2013 at 7:03 PM, Robert Rati rrati@redhat.com wrote:
[...]
bundles. I don't think catalina.sh is in Fedora's tomcat package, it's also not in Debian's [2] tomcat6 and tomcat7 packages.
I meant to say: I don't think catalina.sh is relevant in Fedora's tomcat package
On 10/25/2013 06:32 PM, Dridi Boukelmoune wrote:
On Wed, Oct 23, 2013 at 7:03 PM, Robert Rati rrati@redhat.com wrote:
I should mention that I'd actually tested the functionality and done all the work needed for this piece of functionality to make it into hadoop. The
What have you tested exactly ? Have you manually added catalina.sh &co somewhere on your system and tested your package ?
I've just tested the setup/deployment for my use case. I tested with the shell scripts from the version of tomcat d/led by the hadoop build, and the Fedora tomcat rpm bits. Once I setup the dir structure properly and setup the hadoop httpfs config, I was able to start the service through systemd and access it as expected.
missing piece is the tomcat shell scripts. If those are packaged then I just need to do a little work and I can include the functionality in hadoop.
How little ? Have you tried to replace catalina.sh in httpfs.sh [1] ?
-exec ${HTTPFS_CATALINA_HOME}/bin/catalina.sh "$@" +exec /usr/sbin/tomcat "$@"
And maybe set a proper TOMCAT_CFG environment variable pointing to the config for this very package.
I'd really prefer not to have to come up with some non-upstreamable implementation here. What's the harm in packaging the shell scripts and allowing the projects to run tomcat webapps as they wish?
Where can I find a spec with your current work (and a testing procedure) to understand better the problem ? Btw, I'm not the tomcat maintainer, just a regular tomcat user, this is just my opinion. As I said earlier, the catalina.sh file is part of the upstream all-in-one-dir bundles. I don't think catalina.sh is in Fedora's tomcat package, it's also not in Debian's [2] tomcat6 and tomcat7 packages.
Hadoop has been packaged for F20. You can find it in koji. To build the bits I'm talking about you'll need to build the httpfs component (currently disabled in the spec, but I think it should build/install cleanly although I haven't tested that since 2.0.5-8 or so). You'll also need to not apply the patch that prevents tomcat from being downloaded by the maven build process. That patch will eventually be needed if building in koji and the tomcat shell scripts get packaged.
Rob
So what is the decision of community? Is it good to have some /usr/share/*/bin/*.sh files? It is not needed for tomcat package itself but some, not so systemd'ed, stuff like hadoop's httpfs may be happy to use it.
Should we package original upstream catalina.sh or we should create some "service tomcat $@" emulation of it?
2013/10/31 Robert Rati rrati@redhat.com
On 10/25/2013 06:32 PM, Dridi Boukelmoune wrote:
On Wed, Oct 23, 2013 at 7:03 PM, Robert Rati rrati@redhat.com wrote:
I should mention that I'd actually tested the functionality and done all the work needed for this piece of functionality to make it into hadoop. The
What have you tested exactly ? Have you manually added catalina.sh &co somewhere on your system and tested your package ?
I've just tested the setup/deployment for my use case. I tested with the shell scripts from the version of tomcat d/led by the hadoop build, and the Fedora tomcat rpm bits. Once I setup the dir structure properly and setup the hadoop httpfs config, I was able to start the service through systemd and access it as expected.
missing piece is the tomcat shell scripts. If those are packaged then I
just need to do a little work and I can include the functionality in hadoop.
How little ? Have you tried to replace catalina.sh in httpfs.sh [1] ?
-exec ${HTTPFS_CATALINA_HOME}/bin/catalina.sh "$@" +exec /usr/sbin/tomcat "$@"
And maybe set a proper TOMCAT_CFG environment variable pointing to the config for this very package.
I'd really prefer not to have to come up with some non-upstreamable implementation here. What's the harm in packaging the shell scripts and allowing the projects to run tomcat webapps as they wish?
Where can I find a spec with your current work (and a testing
procedure) to understand better the problem ? Btw, I'm not the tomcat maintainer, just a regular tomcat user, this is just my opinion. As I said earlier, the catalina.sh file is part of the upstream all-in-one-dir bundles. I don't think catalina.sh is in Fedora's tomcat package, it's also not in Debian's [2] tomcat6 and tomcat7 packages.
Hadoop has been packaged for F20. You can find it in koji. To build the bits I'm talking about you'll need to build the httpfs component (currently disabled in the spec, but I think it should build/install cleanly although I haven't tested that since 2.0.5-8 or so). You'll also need to not apply the patch that prevents tomcat from being downloaded by the maven build process. That patch will eventually be needed if building in koji and the tomcat shell scripts get packaged.
Rob
-- java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
On Wed, Nov 6, 2013 at 6:58 PM, Ivan Afonichev ivan.afonichev@gmail.com wrote:
So what is the decision of community?
Hi,
I've taken a look at the hadoop spec, and builded the httpfs sub-package. It is packaged as a classic all-in-one-dir "catalina base". I believe this goes against the guidelines [1] which state that "Fedora packages must follow the FHS".
Is it good to have some /usr/share/*/bin/*.sh files? It is not needed for tomcat package itself but some, not so systemd'ed, stuff like hadoop's httpfs may be happy to use it.
As I understand it, we cannot follow hadoop upstream's packaging just like the tomcat package doesn't. Also a "standard" java WAR (unpacked here) contains its JARs in its WEB-INF/lib directory, which seems to also go against java packaging guidelines [2]. It states that "All architecture-independent JAR files MUST go into %{_javadir} or [...] %{_javadir}-*".
Should we package original upstream catalina.sh or we should create some "service tomcat $@" emulation of it?
There actually is a systemd service [3] in the tomcat package. And hadoop has a similar service [4], the difference is that tomcat doesn't stick to upstream's scripts because the guidelines don't allow to work the way they work.
Dridi
[1] https://fedoraproject.org/wiki/Packaging:Guidelines#Filesystem_Layout [2] https://fedoraproject.org/wiki/Packaging:Java#Installation_directory [3] http://pkgs.fedoraproject.org/cgit/tomcat.git/tree/tomcat-7.0.service?h=f19 [4] http://pkgs.fedoraproject.org/cgit/hadoop.git/tree/hadoop-httpfs.service
2013/10/31 Robert Rati rrati@redhat.com
On 10/25/2013 06:32 PM, Dridi Boukelmoune wrote:
On Wed, Oct 23, 2013 at 7:03 PM, Robert Rati rrati@redhat.com wrote:
I should mention that I'd actually tested the functionality and done all the work needed for this piece of functionality to make it into hadoop. The
What have you tested exactly ? Have you manually added catalina.sh &co somewhere on your system and tested your package ?
I've just tested the setup/deployment for my use case. I tested with the shell scripts from the version of tomcat d/led by the hadoop build, and the Fedora tomcat rpm bits. Once I setup the dir structure properly and setup the hadoop httpfs config, I was able to start the service through systemd and access it as expected.
missing piece is the tomcat shell scripts. If those are packaged then I just need to do a little work and I can include the functionality in hadoop.
How little ? Have you tried to replace catalina.sh in httpfs.sh [1] ?
-exec ${HTTPFS_CATALINA_HOME}/bin/catalina.sh "$@" +exec /usr/sbin/tomcat "$@"
And maybe set a proper TOMCAT_CFG environment variable pointing to the config for this very package.
I'd really prefer not to have to come up with some non-upstreamable implementation here. What's the harm in packaging the shell scripts and allowing the projects to run tomcat webapps as they wish?
Where can I find a spec with your current work (and a testing procedure) to understand better the problem ? Btw, I'm not the tomcat maintainer, just a regular tomcat user, this is just my opinion. As I said earlier, the catalina.sh file is part of the upstream all-in-one-dir bundles. I don't think catalina.sh is in Fedora's tomcat package, it's also not in Debian's [2] tomcat6 and tomcat7 packages.
Hadoop has been packaged for F20. You can find it in koji. To build the bits I'm talking about you'll need to build the httpfs component (currently disabled in the spec, but I think it should build/install cleanly although I haven't tested that since 2.0.5-8 or so). You'll also need to not apply the patch that prevents tomcat from being downloaded by the maven build process. That patch will eventually be needed if building in koji and the tomcat shell scripts get packaged.
Rob
-- java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
Hi,
On Thu, 2013-11-07 at 08:29 +0100, Dridi Boukelmoune wrote:
On Wed, Nov 6, 2013 at 6:58 PM, Ivan Afonichev ivan.afonichev@gmail.com wrote:
So what is the decision of community?
Hi,
I've taken a look at the hadoop spec, and builded the httpfs sub-package. It is packaged as a classic all-in-one-dir "catalina base". I believe this goes against the guidelines [1] which state that "Fedora packages must follow the FHS".
This sounds strange. What's wrong with shipping CATALINA_BASE==/usr/share/hadoop/httpfs/tomcat where actual directories are symlinks to the right locations in %{_localstatedir} etc.? I guess what you wanted to point out is that he shouldn't directly install there, but use symlinks instead?
Since I've come along systemd's tomcat@foo service file[1] recently while packaging thermostat in rawhide, what is preventing you from using this for hadoop httpfs? It worked quite nicely for us, since we didn't have to ship our own systemd service file. We now simply use $ systemctl start tomcat@thermostat for firing up tomcat with a CATALINA_BASE in /var/lib/tomcats/thermostat.
Please forgive me if I've missed an important point.
Cheers, Severin
[1] http://pkgs.fedoraproject.org/cgit/tomcat.git/tree/tomcat-named.service
On Thu, Nov 7, 2013 at 10:41 AM, Severin Gehwolf sgehwolf@redhat.com wrote:
Hi,
On Thu, 2013-11-07 at 08:29 +0100, Dridi Boukelmoune wrote:
On Wed, Nov 6, 2013 at 6:58 PM, Ivan Afonichev ivan.afonichev@gmail.com wrote:
So what is the decision of community?
Hi,
I've taken a look at the hadoop spec, and builded the httpfs sub-package. It is packaged as a classic all-in-one-dir "catalina base". I believe this goes against the guidelines [1] which state that "Fedora packages must follow the FHS".
This sounds strange. What's wrong with shipping CATALINA_BASE==/usr/share/hadoop/httpfs/tomcat where actual directories are symlinks to the right locations in %{_localstatedir} etc.?
I didn't notice those were symlinks with just rpmls.
I guess what you wanted to point out is that he shouldn't directly install there, but use symlinks instead?
I'm sorry I don't understand your question. But the fact that the all-in-one-dir catalina base actually links to FHS-compliant places sounds good to me.
Since I've come along systemd's tomcat@foo service file[1] recently while packaging thermostat in rawhide, what is preventing you from using this for hadoop httpfs? It worked quite nicely for us, since we didn't have to ship our own systemd service file. We now simply use $ systemctl start tomcat@thermostat for firing up tomcat with a CATALINA_BASE in /var/lib/tomcats/thermostat.
It looks very elegant to me! But I haven't found in the systemd.service(5) man how you're achieving this, can you show me please ? Also you've convinced me even more that catalina.sh and its friends are less relevant to native packages.
Dridi
Please forgive me if I've missed an important point.
Cheers, Severin
[1] http://pkgs.fedoraproject.org/cgit/tomcat.git/tree/tomcat-named.service
-- java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
On Thu, 2013-11-07 at 11:05 +0100, Dridi Boukelmoune wrote:
On Thu, Nov 7, 2013 at 10:41 AM, Severin Gehwolf sgehwolf@redhat.com wrote:
Hi,
On Thu, 2013-11-07 at 08:29 +0100, Dridi Boukelmoune wrote:
On Wed, Nov 6, 2013 at 6:58 PM, Ivan Afonichev ivan.afonichev@gmail.com wrote:
So what is the decision of community?
Hi,
I've taken a look at the hadoop spec, and builded the httpfs sub-package. It is packaged as a classic all-in-one-dir "catalina base". I believe this goes against the guidelines [1] which state that "Fedora packages must follow the FHS".
This sounds strange. What's wrong with shipping CATALINA_BASE==/usr/share/hadoop/httpfs/tomcat where actual directories are symlinks to the right locations in %{_localstatedir} etc.?
I didn't notice those were symlinks with just rpmls.
I guess what you wanted to point out is that he shouldn't directly install there, but use symlinks instead?
I'm sorry I don't understand your question. But the fact that the all-in-one-dir catalina base actually links to FHS-compliant places sounds good to me.
Since I've come along systemd's tomcat@foo service file[1] recently while packaging thermostat in rawhide, what is preventing you from using this for hadoop httpfs? It worked quite nicely for us, since we didn't have to ship our own systemd service file. We now simply use $ systemctl start tomcat@thermostat for firing up tomcat with a CATALINA_BASE in /var/lib/tomcats/thermostat.
It looks very elegant to me! But I haven't found in the systemd.service(5) man how you're achieving this, can you show me please ? Also you've convinced me even more that catalina.sh and its friends are less relevant to native packages.
Have a look at the thermostat.spec[1] on master. The webapp subpackage is doing this.
Cheers, Severin
[1] http://pkgs.fedoraproject.org/cgit/thermostat.git/tree/thermostat.spec#n652
Dridi
Please forgive me if I've missed an important point.
Cheers, Severin
[1] http://pkgs.fedoraproject.org/cgit/tomcat.git/tree/tomcat-named.service
-- java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
On 11/07/2013 05:22 AM, Severin Gehwolf wrote:
Since I've come along systemd's tomcat@foo service file[1] recently while packaging thermostat in rawhide, what is preventing you from using this for hadoop httpfs? It worked quite nicely for us, since we didn't have to ship our own systemd service file. We now simply use $ systemctl start tomcat@thermostat for firing up tomcat with a CATALINA_BASE in /var/lib/tomcats/thermostat.
It looks very elegant to me! But I haven't found in the systemd.service(5) man how you're achieving this, can you show me please ? Also you've convinced me even more that catalina.sh and its friends are less relevant to native packages.
Have a look at the thermostat.spec[1] on master. The webapp subpackage is doing this.
I've finally gotten the time to look at this solution and it looks like there could be issues with multiple webapps deploying the way thermostat does. I'm not a tomcat expert so I'm probably missing something, but here are the concerns I have:
1) With all webapps using the same configuration files, wouldn't the logs become a mess? From my testing of thermostat, it places all logs in /usr/share/tomcat/logs with no unique extensions save a single exception (thermostat-web-storage). If multiple webapps launch this way, won't the logs be incomprehensible?
2) What ports end up being used by these webapps if they all use the same configuration? It seems like there'd be a logjam on 8005. What about webapps that want to use different ports? Is that needed/allowed? Can this be done with the config file in sysconfig?
Rob
On 01/09/2014 10:34 AM, Robert Rati wrote:
On 11/07/2013 05:22 AM, Severin Gehwolf wrote:
Since I've come along systemd's tomcat@foo service file[1] recently while packaging thermostat in rawhide, what is preventing you from using this for hadoop httpfs? It worked quite nicely for us, since we didn't have to ship our own systemd service file. We now simply use $ systemctl start tomcat@thermostat for firing up tomcat with a CATALINA_BASE in /var/lib/tomcats/thermostat.
This will work for hadoop's httpfs service and obviously works for thermostat, but I don't think is the direction that should be taken by most webapps. The problem with the tomcat@foo service as used by thermostat is that it fires up an instance of tomcat and deploys the webapp. That works great if: A) there is only 1 webapp that needs to run at any time with tomcat@foo B) all webapps use the same port C) the system tomcat isn't running
If A or C is false, tomcat@foo won't work. The problem is that all tomcat instances use the same server.xml file, so all try to use the same Connector setup. Whichever tomcat instance starts first will work fine. Try to launch a second and you get port collisions and tomcat goes down taking the previously running instance with it.
If B is false you end up running webapps that listen on ports other than what upstream uses, potentially causing confusion and/or breakage with whatever might interact with the webapp.
I've finally gotten the time to look at this solution and it looks like there could be issues with multiple webapps deploying the way thermostat does. I'm not a tomcat expert so I'm probably missing something, but here are the concerns I have:
- With all webapps using the same configuration files, wouldn't the
logs become a mess? From my testing of thermostat, it places all logs in /usr/share/tomcat/logs with no unique extensions save a single exception (thermostat-web-storage). If multiple webapps launch this way, won't the logs be incomprehensible?
This is what happens. All catalina logging ends up in the /var/log/tomcat directory (symlinked to from /usr/share/tomcat/logs). The mess isn't as bad as it could be because only 1 instance can be running at at time. Try to launch two and these log files become a record of the tomcat errors.
- What ports end up being used by these webapps if they all use the
same configuration? It seems like there'd be a logjam on 8005. What about webapps that want to use different ports? Is that needed/allowed? Can this be done with the config file in sysconfig?
There is a logjam on port 8080 (not 8005). Every tomcat instance uses the same configuration files with the same Connector configuration resulting in all instances listening on the same port.
Looking at other webapps besides thermostat, many seem to install themselves into /usr/share/tomcat/webapps and are run by starting tomcat itself. I would think this is the preferred way to deploy webapps.
Unfortunately installing into /usr/share/tomcat/webapps won't work for httpfs because httpfs uses system.getProperty to retrieve system properties to configure the service. I don't see a way to get around this requirement and as far as I've found there is no way to provide these values on a per-webapp basis. The only way I've found to provide the values is as options passed to tomcat on invocation.
The tomcat@foo allows you to define options that will be passed to tomcat on invocation, but there seems to be no co-ordination/plan for how tomcat@foo1 will run along side tomcat@foo2 or the system tomcat.
Should the goal be to allow these various methods of deploying a webapp to run on the same machine?
Rob
Since I've come along systemd's tomcat@foo service file[1] recently while packaging thermostat in rawhide, what is preventing you from using this for hadoop httpfs? It worked quite nicely for us, since we didn't have to ship our own systemd service file. We now simply use $ systemctl start tomcat@thermostat for firing up tomcat with a CATALINA_BASE in /var/lib/tomcats/thermostat.
I'll take a look at this and see if it'll work for us. Hadoop isn't the only service that runs this way. It seems the hadoop ecosystem tends to like to deploy this way and atm I need to look into what you've suggested to know if it'll work for hadoop and/or other projects.
Unless I missed something, I don't see any reason why the shell scripts can't be packaged. If the above solution is viable this could just be a through exercise, but what is wrong with providing the shell scripts?
Rob
On 11/07/2013 02:29 AM, Dridi Boukelmoune wrote:
On Wed, Nov 6, 2013 at 6:58 PM, Ivan Afonichev ivan.afonichev@gmail.com wrote:
So what is the decision of community?
Hi,
I've taken a look at the hadoop spec, and builded the httpfs sub-package. It is packaged as a classic all-in-one-dir "catalina base". I believe this goes against the guidelines [1] which state that "Fedora packages must follow the FHS".
The packaging is following the FHS guidelines and putting all files in the correct locations and symlinking. The tomcat package does this as well (look at /usr/share/tomcat).
Is it good to have some /usr/share/*/bin/*.sh files? It is not needed for tomcat package itself but some, not so systemd'ed, stuff like hadoop's httpfs may be happy to use it.
As I understand it, we cannot follow hadoop upstream's packaging just like the tomcat package doesn't. Also a "standard" java WAR (unpacked here) contains its JARs in its WEB-INF/lib directory, which seems to also go against java packaging guidelines [2]. It states that "All architecture-independent JAR files MUST go into %{_javadir} or [...] %{_javadir}-*".
The unpackaged WAR has it's jars replaced with symlinks and tomcat is told to follow symlinks for that service.
Should we package original upstream catalina.sh or we should create some "service tomcat $@" emulation of it?
There actually is a systemd service [3] in the tomcat package. And hadoop has a similar service [4], the difference is that tomcat doesn't stick to upstream's scripts because the guidelines don't allow to work the way they work.
The systemd script that is written for httpfs seems to work just fine and afaik is following Fedora's guidelines.
Rob
java-devel@lists.fedoraproject.org
-
Dridi Boukelmoune -
Ivan Afonichev -
Mikolaj Izdebski -
Robert Rati -
Severin Gehwolf