Hi all,
I'm following up from ticket #226, which is tracking improvements to the log analyzer system. This would be what analyzers the logs on lockbox, which is the syslog host for infrastructure machines: https://hosted.fedoraproject.org/projects/fedora-infrastructure/ticket/226
I wanted to capture what we wanted the new analyzer to do. Main feedback I had from discussion in #fedora-admin was a need for more signal, less noise: the current 'analyzed' logs were too verbose and had too much cruft.
Did I capture that requirement? Are there other requirements besides improving the presentation? Anything else that people feel they need from the log analyzer that they aren't getting?
Currently Epylog is used - I did some looking around, and I'm not seeing something that looks like its any better. If someone knows another open source log analyzer they think would be much better, I'd like to hear. Else, my plan is to continue with Epylog, reconfigure it... and if really needed to get what we need, patch it and contribute upstream.
Thanks all, hope everyone is having a good weekend.
Cheers, Michael
Michael Yingbull wrote:
Hi all,
I'm following up from ticket #226, which is tracking improvements to the log analyzer system. This would be what analyzers the logs on lockbox, which is the syslog host for infrastructure machines: https://hosted.fedoraproject.org/projects/fedora-infrastructure/ticket/226
I wanted to capture what we wanted the new analyzer to do. Main feedback I had from discussion in #fedora-admin was a need for more signal, less noise: the current 'analyzed' logs were too verbose and had too much cruft.
Did I capture that requirement?
I think this is the biggest thing. Obviously we don't want to /dev/null log lines but at the same time the current format is pretty useless to us. I guess it might be best to do as much cleanup as possible and then see where things are.
-Mike
On Mon, 2007-11-26 at 08:33 -0600, Mike McGrath wrote:
Michael Yingbull wrote:
Hi all,
I'm following up from ticket #226, which is tracking improvements to the log analyzer system. This would be what analyzers the logs on lockbox, which is the syslog host for infrastructure machines: https://hosted.fedoraproject.org/projects/fedora-infrastructure/ticket/226
I wanted to capture what we wanted the new analyzer to do. Main feedback I had from discussion in #fedora-admin was a need for more signal, less noise: the current 'analyzed' logs were too verbose and had too much cruft.
Did I capture that requirement?
I think this is the biggest thing. Obviously we don't want to /dev/null log lines but at the same time the current format is pretty useless to us. I guess it might be best to do as much cleanup as possible and then see where things are.
Actually, there's a huge portion of what is in the current logs that needs to either: 1. be dumped out by epylog's weeder 2. be stopped from occurring on the system generating the message.
Michael, if you need any assistance with this, let me know, I have a fair bit of experience adding weedlists to epylog.
-sv
seth vidal wrote:
On Mon, 2007-11-26 at 08:33 -0600, Mike McGrath wrote:
Michael Yingbull wrote:
Hi all,
I'm following up from ticket #226, which is tracking improvements to the log analyzer system. This would be what analyzers the logs on lockbox, which is the syslog host for infrastructure machines: https://hosted.fedoraproject.org/projects/fedora-infrastructure/ticket/226
I wanted to capture what we wanted the new analyzer to do. Main feedback I had from discussion in #fedora-admin was a need for more signal, less noise: the current 'analyzed' logs were too verbose and had too much cruft.
Did I capture that requirement?
I think this is the biggest thing. Obviously we don't want to /dev/null log lines but at the same time the current format is pretty useless to us. I guess it might be best to do as much cleanup as possible and then see where things are.
Actually, there's a huge portion of what is in the current logs that needs to either:
- be dumped out by epylog's weeder
- be stopped from occurring on the system generating the message.
Michael, if you need any assistance with this, let me know, I have a fair bit of experience adding weedlists to epylog.
2) would be more favored by me where possible.
-Mike
On Mon, 2007-11-26 at 08:42 -0600, Mike McGrath wrote:
- would be more favored by me where possible.
No problem. From todays' report a couple of things we can do:
1. remove all user failure reports. They don't do us any good and they're always ssh bruteforce attacks. Denyhosts will do its thing, or not, but we can't be told about them all the time.
2. weed out pretty much everything beginning with: rsyncd - informational messages about rsync processes - not useful puppetd - notices on what is or is not done - not useful, either - if we can turn off the syslog component of this and only have this in the local puppet logs that'd be fine ntpd - garbage noise - not useful for a log report git-daemon - do I really need to explain why we can nuke this?
3. all of these lines: crond[19403]: pam_unix(crond:session): session closed for user root
iirc, there is a new login module which handles these
4. puppetmasterd* - these appear to be errors/warnings from puppetmasterd - these need to be fixed.
pruning out the items in 2 alone will nuke the better part of this logreport.
-sv
infrastructure@lists.fedoraproject.org