I'm one of the developers working on the next version of Dogtag, the Public Key Infrastructure packages (pki-*) which is, as far as I can tell, the only Java Web app that actually ships in Fedora.
PKI is an old project, and it has vestiges of many aspects of Java development that are just done differently today. We are looking to close the gap between where it is and where the Standard lies. As such, we have to figure out how to attack packaging the application.
Part of that, for Fedora, is to establish the standard in the first place.
Dogtag ships with a utility called pkicreate. This is an application that creates a Tomcat instance in (by default) /var/lib/pki-* and sets the ports, the SELinux policy, the config, etc. While I think it does more than it should, certain aspects of what it does are essential. For example, it handles all of the SELinux configuration for the Tomcat instance. Tomcat ships with SSL disabled, and without an AJP connector to apache. Thus, Tomcat can't serve over port 80 or 443, meaning that they can't be viewed by many people behind firewalls. pkicreate configures mod_proxy_ajp support. It performs the SELinux configuration for the HTTP ports (if they are different from 8080 and 8443 which should be handled by system policy), as well as the AJP port.
Russ Alberry, on the Debian Java team, has propsed a deployment utility similar in scope to what pkicreate does, but for the general case:
http://dep.debian.net/deps/dep7/
Fedora 17 and later will have JBoss, which has a different set of deployment rules. Geronimo is already part of Fedora, although I have to admit ignorance as far as how it would interact with Tomcat. Resin is a possiblity as well. So we don't want to hardcode the deployment approach to the Tomcat server. But, the Tomcat install should be supported first.
For a directory standard, I would say that a Fedora Web app should not have to provide a .war file, but should be able to use the tool to produce one .
Many web applications allow for post install customization. THe simplest case is that a Company wants to theme the application with its own images and CSS. If the web application is delivered in a .war file and expects to deploy this way on the Servlet engine, there is no way to customize it. Thus, Russ' approach listed above seems right on. The one thing I would adjust is that deployment as a war file should be an option, but not necessarily the prefereed option. MOst application servers have to unzip war files anyway, and it seems a waste and troublesome to zip up the file only to unzip it. I say war is the exception but not the rule.
pkicreate is a Perl application. As such, I don't think it will gain wide acceptance in the Java community. But, it can server as a prototype for A Java based java-deploy-webapp.
Here's a short list of functions that I think the deployment tool should handle beyond what is listed in Russ' propsal:
1. Configuring Security Realms 2. Configuring JDBC connections (It would be nice if it configured Postgresql and MySQL for JDBC, too....) 3. Configuring LDAP connections 4. Opening additional ports and providing SELinux configuration for them 5. Creating Symlinks for shared libraries to /usr/share/java and /usr/lib[64]/java 6. Registering generic Global scoped components that can be accessed via JNDI names.
It is possible that the workflow for java-deploy-webapp could best be done using either Ant or Maven. However, I would not like to require a .pom or build.xml file as part of the deployment: Web apps should follow the standard, and requiring additional configuration should be supported but not the norm. Perhaps we look for something like /etc/webapp/name/pom.xml and, if it doesn't exist, generate it from a skeleton. Or, we can say that if they provide the pom.xml, then we support deploying this way, otherwise you are on your own.
Excerpts from Adam Young's message of Wed Sep 28 16:13:06 +0200 2011:
I'm one of the developers working on the next version of Dogtag, the Public Key Infrastructure packages (pki-*) which is, as far as I can tell, the only Java Web app that actually ships in Fedora.
PKI is an old project, and it has vestiges of many aspects of Java development that are just done differently today. We are looking to close the gap between where it is and where the Standard lies. As such, we have to figure out how to attack packaging the application.
Part of that, for Fedora, is to establish the standard in the first place.
I like your approach. It's great that you are trying to get the packaging updated instead of working around it.
Dogtag ships with a utility called pkicreate. This is an application that creates a Tomcat instance in (by default) /var/lib/pki-* and sets the ports, the SELinux policy, the config, etc. While I think it does more than it should, certain aspects of what it does are essential. For example, it handles all of the SELinux configuration for the Tomcat instance. Tomcat ships with SSL disabled, and without an AJP connector to apache. Thus, Tomcat can't serve over port 80 or 443, meaning that they can't be viewed by many people behind firewalls. pkicreate configures mod_proxy_ajp support. It performs the SELinux configuration for the HTTP ports (if they are different from 8080 and 8443 which should be handled by system policy), as well as the AJP port.
I once prepared a tomcat webapp[1] for review and even though it never got in in the end because I backed off...it should still be a good starting point. Especially tomcat6 subpackage that installs one simple XML into Catalina directory and that's all that's needed for tomcat to recognize new webapp.
I can't really comment on different ports and such, but if you are right then I guess system policy should be modified either when installing tomcat itself or globally. I really don't think a second tomcat instance is the way to go.
Russ Alberry, on the Debian Java team, has propsed a deployment utility similar in scope to what pkicreate does, but for the general case:
You might want to have a look at one of our SIG meeting logs[2] from December of last year. We discussed said deployment utility and java webapps in general there (though we never finished the discussion).
Fedora 17 and later will have JBoss, which has a different set of deployment rules. Geronimo is already part of Fedora, although I have to admit ignorance as far as how it would interact with Tomcat. Resin is a possiblity as well. So we don't want to hardcode the deployment approach to the Tomcat server. But, the Tomcat install should be supported first.
I'd be careful about saying "Fedora 17 will have JBoss". Marek Goldmann is trying hard and he's doing a great job, but there's still a lot of work ahead of us.
For a directory standard, I would say that a Fedora Web app should not have to provide a .war file, but should be able to use the tool to produce one .
Many web applications allow for post install customization. THe simplest case is that a Company wants to theme the application with its own images and CSS. If the web application is delivered in a .war file and expects to deploy this way on the Servlet engine, there is no way to customize it. Thus, Russ' approach listed above seems right on. The one thing I would adjust is that deployment as a war file should be an option, but not necessarily the prefereed option. MOst application servers have to unzip war files anyway, and it seems a waste and troublesome to zip up the file only to unzip it. I say war is the exception but not the rule.
pkicreate is a Perl application. As such, I don't think it will gain wide acceptance in the Java community. But, it can server as a prototype for A Java based java-deploy-webapp.
Here's a short list of functions that I think the deployment tool should handle beyond what is listed in Russ' propsal:
- Configuring Security Realms
- Configuring JDBC connections (It would be nice if it configured
Postgresql and MySQL for JDBC, too....) 3. Configuring LDAP connections 4. Opening additional ports and providing SELinux configuration for them 5. Creating Symlinks for shared libraries to /usr/share/java and /usr/lib[64]/java 6. Registering generic Global scoped components that can be accessed via JNDI names.
I believe our initial idea for deployment tool was much more simple. Maybe we just never understood the real needs. I believe the discussion could/should be started again. Getting in touch with Russ would be a good idea too I guess.
It is possible that the workflow for java-deploy-webapp could best be done using either Ant or Maven. However, I would not like to require a .pom or build.xml file as part of the deployment: Web apps should follow the standard, and requiring additional configuration should be supported but not the norm. Perhaps we look for something like /etc/webapp/name/pom.xml and, if it doesn't exist, generate it from a skeleton. Or, we can say that if they provide the pom.xml, then we support deploying this way, otherwise you are on your own.
I don't think an ant/maven approach is viable, though I might be wrong. Deployment using pom.xml is fine for upstreams, but we'd have to create our own anyway because we are not really deploying to upstream servers.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=693646 [2] http://meetbot.fedoraproject.org/fedora-meeting-1/2010-12-08/fedora-meeting-...
-- Stanislav Ochotnicky sochotnicky@redhat.com Software Engineer - Base Operating Systems Brno
PGP: 7B087241 Red Hat Inc. http://cz.redhat.com
2011/9/29 Stanislav Ochotnicky:
Excerpts from Adam Young's message of Wed Sep 28 16:13:06 +0200 2011:
I once prepared a tomcat webapp[1] for review and even though it never got in in the end because I backed off...it should still be a good starting point. Especially tomcat6 subpackage that installs one simple XML into Catalina directory and that's all that's needed for tomcat to recognize new webapp.
That's true if you want to set some non-default paths, just dropping the war in webapps dir is enough to make tomcat be aware of that and try to start the new context. I think (but im not sure about that) that putting xml files in Catalina directory, in particular when not inside 'localhost', requires a tomcat restart.
I can't really comment on different ports and such, but if you are right then I guess system policy should be modified either when installing tomcat itself or globally. I really don't think a second tomcat instance is the way to go.
Agree with that, but what do you have in mind for packaging webapps, creating a single -tomcat | -jboss | -jetty | -whatever rpm during build for each provided/supported container built from the webapp srpm?
Many web applications allow for post install customization. THe simplest case is that a Company wants to theme the application with its own images and CSS. If the web application is delivered in a .war file and expects to deploy this way on the Servlet engine, there is no way to customize it. Thus, Russ' approach listed above seems right on. The one thing I would adjust is that deployment as a war file should be an option, but not necessarily the prefereed option. MOst application servers have to unzip war files anyway, and it seems a waste and troublesome to zip up the file only to unzip it. I say war is the exception but not the rule.
My knoledge of JBoss is still on v4.3. On that one, i'd say that war/ear IS actually the rule and not the exception, as JBoss does extract the archive in a working directory without custom configuration, and for istance there are other discrepancies (allowed by specification), as getServletContent().getRealPath() returning the actual absolute path in tomcat, and just null in JBoss.
Relying on the fact that wars have to be exploded in the context-s directory is also very bad for clustering in my opinion.
pkicreate is a Perl application. As such, I don't think it will gain wide acceptance in the Java community. But, it can server as a prototype for A Java based java-deploy-webapp.
Here's a short list of functions that I think the deployment tool should handle beyond what is listed in Russ' propsal:
- Configuring Security Realms
- Configuring JDBC connections (It would be nice if it configured
Postgresql and MySQL for JDBC, too....) 3. Configuring LDAP connections 4. Opening additional ports and providing SELinux configuration for them 5. Creating Symlinks for shared libraries to /usr/share/java and /usr/lib[64]/java 6. Registering generic Global scoped components that can be accessed via JNDI names.
I believe our initial idea for deployment tool was much more simple. Maybe we just never understood the real needs. I believe the discussion could/should be started again. Getting in touch with Russ would be a good idea too I guess.
I dont think sysadministrators will likely rely on a script or gui to edit their configuration files. But, a deploy manager could be very useful for the casual user (I mean, anyone like developers or final users of applications written as webapps).
That's my hypothetical use-case: 1- in my fedora system i installed tomcat7 and jboss 7 2- i install a mywebapp rpm using yum (just like installing phpmyadmin, but in this case the war is just sitting somewhere, no deploy) 3a- i install a mywebapp-tomcat7 bridge rpm using yum that does most configuration, but not everything or 3b- use a newly written deploy manager to deploy whatever in wherever
That could rise a lot of dependant questions: - WEB-INF/lib: the proposal of linking to system jars just wont work so often, as many webapps are written to particular library versions (of commons- or just log4j) - many webapps in the past needed endorsed libs or configuration of bootstrap or jvm keystore - most webapps need a database and come with hibernate, so which backing db engine to configure to be used with hibernate? - how to make the user aware that the user/pass used for database creation have to be changed? and you still have to get somewhere the grant for creating such user - what about specific settings as -Xms or -XX:PermGen for the jvm which could be in conflict among webapps
Imho, a fully automated deploy manager is just too complex to implement, and nevertheless, that would probably not let you avoid small manual configurations. What could be really a gift, is an automated tool which does a maximum common set of configurations, or even better, some sort of wizard that in a few steps could let you have running some webapp, that you just had sitting somewhere after installing the relevant packages.
On 09/29/2011 07:06 AM, Guido Grazioli wrote:
2011/9/29 Stanislav Ochotnicky:
Excerpts from Adam Young's message of Wed Sep 28 16:13:06 +0200 2011:
I once prepared a tomcat webapp[1] for review and even though it never got in in the end because I backed off...it should still be a good starting point. Especially tomcat6 subpackage that installs one simple XML into Catalina directory and that's all that's needed for tomcat to recognize new webapp.
That's true if you want to set some non-default paths, just dropping the war in webapps dir is enough to make tomcat be aware of that and try to start the new context. I think (but im not sure about that) that putting xml files in Catalina directory, in particular when not inside 'localhost', requires a tomcat restart.
Restarting Tomcat for initial deploy is probably OK. For hot deploy scenarios, people will likely want JBoss, and we shouldn't be trying to reimplement that here. Lets focus on the simple cases first.
I can't really comment on different ports and such, but if you are right then I guess system policy should be modified either when installing tomcat itself or globally. I really don't think a second tomcat instance is the way to go.
Agree with that, but what do you have in mind for packaging webapps, creating a single -tomcat | -jboss | -jetty | -whatever rpm during build for each provided/supported container built from the webapp srpm?
No. I don't think this should be done at RPM package time. I see three supported scenarios
1. RPM creates a war file and that is what the tool deploys as is. 2. RPM create an exploded war directory. Tool merges in config files from /etc/tomcat6/<webapp> with the code from /usr/share/java/webapps and creates a war file and deploys it 3. Same as 2 but code is deployed in exploded form.
For 2, copies of dependancy lbraries are put into WEB-INF/lib, for 3, it is done with symlinks.
Many web applications allow for post install customization. THe simplest case is that a Company wants to theme the application with its own images and CSS. If the web application is delivered in a .war file and expects to deploy this way on the Servlet engine, there is no way to customize it. Thus, Russ' approach listed above seems right on. The one thing I would adjust is that deployment as a war file should be an option, but not necessarily the prefereed option. MOst application servers have to unzip war files anyway, and it seems a waste and troublesome to zip up the file only to unzip it. I say war is the exception but not the rule.
My knoledge of JBoss is still on v4.3. On that one, i'd say that war/ear IS actually the rule and not the exception, as JBoss does extract the archive in a working directory without custom configuration, and for istance there are other discrepancies (allowed by specification), as getServletContent().getRealPath() returning the actual absolute path in tomcat, and just null in JBoss.
Relying on the fact that wars have to be exploded in the context-s directory is also very bad for clustering in my opinion.
Agreed. We should support both exploded and packaged deployments.
pkicreate is a Perl application. As such, I don't think it will gain wide acceptance in the Java community. But, it can server as a prototype for A Java based java-deploy-webapp.
Here's a short list of functions that I think the deployment tool should handle beyond what is listed in Russ' propsal:
- Configuring Security Realms
- Configuring JDBC connections (It would be nice if it configured
Postgresql and MySQL for JDBC, too....) 3. Configuring LDAP connections 4. Opening additional ports and providing SELinux configuration for them 5. Creating Symlinks for shared libraries to /usr/share/java and /usr/lib[64]/java 6. Registering generic Global scoped components that can be accessed via JNDI names.
I believe our initial idea for deployment tool was much more simple. Maybe we just never understood the real needs. I believe the discussion could/should be started again. Getting in touch with Russ would be a good idea too I guess.
I dont think sysadministrators will likely rely on a script or gui to edit their configuration files. But, a deploy manager could be very useful for the casual user (I mean, anyone like developers or final users of applications written as webapps).
A good sysadmin should be using something like puppet to ensure that all servers are kept in sync, and so on. Automated deployments should provide a reasonable set of defaults for a web application.It may be that we want to "wizardize it" for some choices, like "you have both postgres and mysql installed, which do you want to use"
That's my hypothetical use-case: 1- in my fedora system i installed tomcat7 and jboss 7 2- i install a mywebapp rpm using yum (just like installing phpmyadmin, but in this case the war is just sitting somewhere, no deploy) 3a- i install a mywebapp-tomcat7 bridge rpm using yum that does most configuration, but not everything or 3b- use a newly written deploy manager to deploy whatever in wherever
That could rise a lot of dependant questions:
- WEB-INF/lib: the proposal of linking to system jars just wont work so often,
as many webapps are written to particular library versions (of commons- or just log4j)
In exploded view do the linking, for a war view, do a copy. This is for Fedora applications, so they should be deploying with the Fedora versions of standard libraries. A web app that wants to deploy with F17 and uses log4j needs to work with the version deployed with F17 as well. We can provide a way to override this for people that want to use the tool for non-standard library versions, but that is not the goal, as I see it.
- many webapps in the past needed endorsed libs or configuration of bootstrap
or jvm keystore
- most webapps need a database and come with hibernate, so which
backing db engine to configure to be used with hibernate?
- how to make the user aware that the user/pass used for database creation
have to be changed? and you still have to get somewhere the grant for creating such user
- what about specific settings as -Xms or -XX:PermGen for the jvm which could
be in conflict among webapps
Imho, a fully automated deploy manager is just too complex to implement, and nevertheless, that would probably not let you avoid small manual configurations.
I don't see it being fully automated. More of a guide-you-through-it with-reasonable-defaults tool. A user should be able to run in in unattend mode by providing certain command line switches...or, conversely, if we wnat the default to be run it unattended, provide a switch --interactive and prompt them. If run in unattended mode, and one of the values is invalid, the whole thing should fail as a unit.
We should also track the files we modify, provide copies of the originals, and clean up on uninstall of the webapp.
What could be really a gift, is an automated tool which does a maximum common set of configurations, or even better, some sort of wizard that in a few steps could let you have running some webapp, that you just had sitting somewhere after installing the relevant packages.
Note that whey I suggested using maven, I did not mean that we should use the same pom as is used to build the application. What would make more sense is to provide a fragment of a pom used just for deployment. I'd see it something like this:
in /etc/jwebapp/myproject.pom (hey. why not! seriously, though, where? Should not be /etc/tomcat6...yes, I did this in JSON, to make it easier to read )
{ {common: {format:war}, # as opposed to exploded {libs:[commons-codec,commons-logging]}, {realm: simple}, {tomcat6: ... } {jbossas5: ... } }
and then
/usr/bin/jwebapp myproject --deploy --server tomcat6
Or something. Probably should keep these .poms in /usr/share/java/webapps, but again allow users to customize them.
I wrote this up, in an attempt to solicit comments from a wider audience.
http://adam.younglogic.com/2011/10/java-web-applications-fedora/
java-devel@lists.fedoraproject.org
-
Adam Young -
Guido Grazioli -
Stanislav Ochotnicky