I am having problems with plague-server dying on me. I am starting plague-server like: plague-server -c /etc/plague/server/plague-server.cfg -d -l /var/log/ plague/plague.log my.ip.address.here
After some amount of time (hours? I'm not sure exactly, but I know it was up all most of the day yesterday and then died sometime in the night) the process disappears, and nothing is left in the log file except for the startup and build messages.
Is there anything I can do to have it print out more information? I am running the server on an EL4 box, might that have something to do with it?
plague-server is CVS from a few days ago (using mysql back-end patch). python version is python-2.3.4-14.1
I also have plague-builder running on the same box, and it has been running with no problem.
Thanks, Jeff
On Oct 26, 2005, at 6:48 AM, Jeff Sheltren wrote:
I am having problems with plague-server dying on me. I am starting plague-server like: plague-server -c /etc/plague/server/plague-server.cfg -d -l /var/ log/plague/plague.log my.ip.address.here
After some amount of time (hours? I'm not sure exactly, but I know it was up all most of the day yesterday and then died sometime in the night) the process disappears, and nothing is left in the log file except for the startup and build messages.
Is there anything I can do to have it print out more information? I am running the server on an EL4 box, might that have something to do with it?
plague-server is CVS from a few days ago (using mysql back-end patch). python version is python-2.3.4-14.1
I also have plague-builder running on the same box, and it has been running with no problem.
Thanks, Jeff
Died again this morning after about 30 minutes - no jobs were submitted or run in this time. The only access was via the web interface to check if it was up. I was using it with a remote mysql server, so I'm going to try it with mysql on localhost and see if that works any better.
-Jeff
On Wed, 2005-10-26 at 07:42 -0400, Jeff Sheltren wrote:
On Oct 26, 2005, at 6:48 AM, Jeff Sheltren wrote:
I am having problems with plague-server dying on me. I am starting plague-server like: plague-server -c /etc/plague/server/plague-server.cfg -d -l /var/ log/plague/plague.log my.ip.address.here
After some amount of time (hours? I'm not sure exactly, but I know it was up all most of the day yesterday and then died sometime in the night) the process disappears, and nothing is left in the log file except for the startup and build messages.
Is there anything I can do to have it print out more information? I am running the server on an EL4 box, might that have something to do with it?
plague-server is CVS from a few days ago (using mysql back-end patch). python version is python-2.3.4-14.1
I also have plague-builder running on the same box, and it has been running with no problem.
Thanks, Jeff
Died again this morning after about 30 minutes - no jobs were submitted or run in this time. The only access was via the web interface to check if it was up. I was using it with a remote mysql server, so I'm going to try it with mysql on localhost and see if that works any better.
Hmm. Can you try to run the python process under gdb? Essentially:
gdb /usr/bin/python set args /usr/bin/plague-server -c /etc/plague/server/plague-server.cfg ... run
Then just let it run for a while, and see where it segfaults. It would also be helpful to install python-debuginfo and pyOpenSSL-debuginfo too so that the gdb backtrace is more useful.
Come to think of it, are you using SSL in your buildsystem at all, and if so, have you patched your copy of pyOpenSSL? This is probably something that should be documented :) FC4 and rawhide are already patched, but RHEL4 is most likely not patched.
Dan
On Oct 26, 2005, at 8:02 AM, Dan Williams wrote:
Hmm. Can you try to run the python process under gdb? Essentially:
gdb /usr/bin/python set args /usr/bin/plague-server -c /etc/plague/server/plague- server.cfg ... run
Then just let it run for a while, and see where it segfaults. It would also be helpful to install python-debuginfo and pyOpenSSL-debuginfo too so that the gdb backtrace is more useful.
Come to think of it, are you using SSL in your buildsystem at all, and if so, have you patched your copy of pyOpenSSL? This is probably something that should be documented :) FC4 and rawhide are already patched, but RHEL4 is most likely not patched.
Dan
Yes, I am using SSL for builders and for users. I haven't touched pyOpenSSL - could you direct me to the needed patch(es)?
I'll try running it under gdb to see what happens in the meantime.
Thanks, Jeff
On Wed, 2005-10-26 at 08:09 -0400, Jeff Sheltren wrote:
On Oct 26, 2005, at 8:02 AM, Dan Williams wrote:
Hmm. Can you try to run the python process under gdb? Essentially:
gdb /usr/bin/python set args /usr/bin/plague-server -c /etc/plague/server/plague- server.cfg ... run
Then just let it run for a while, and see where it segfaults. It would also be helpful to install python-debuginfo and pyOpenSSL-debuginfo too so that the gdb backtrace is more useful.
Come to think of it, are you using SSL in your buildsystem at all, and if so, have you patched your copy of pyOpenSSL? This is probably something that should be documented :) FC4 and rawhide are already patched, but RHEL4 is most likely not patched.
Dan
Yes, I am using SSL for builders and for users. I haven't touched pyOpenSSL - could you direct me to the needed patch(es)?
Ok, that's probably the issue. pyOpenSSL does not do proper Python locking and such for certificate verification callbacks, and because plague makes heavy use of threads, things fall over quite quickly.
You can get the patch here:
http://cvs.fedora.redhat.com/viewcvs/*checkout*/extras-buildsys/pyOpenSSL-th...
And you can find an SRPM for RHEL4 with the patch included here:
http://people.redhat.com/dcbw/pyOpenSSL/pyOpenSSL-0.6-1.p23.1.el4.src.rpm
Should be able to just rebuild that SRPM (if you have any problems with that, don't hesitate to ask) and you'll be good to go.
Dan
buildsys@lists.fedoraproject.org