On Tue, Nov 25, 2014 at 07:24:32AM -0700, Kevin Fenzi wrote:
The OOM killer on bapp02 has terminated a few mirrormanager crawler processes. It seems it needs more memory or the number of parallel crawlers has to be further limited.
Well, it's got 16GB now... I can bump it to 24 without too much trouble. Will of course need a freeze break...
We are currently doing 60 threads. We could cut it down, but I guess I'd say lets try more memory first.
Another good idea would be to limit the duration of the rsync crawls in /etc/mirrormanager/prod.cfg to maybe one day (--timeout=86400) to avoid stale rsync processes
Good idea. Whats the config directive there? Seems we are not setting it at all currently.
I just found out that the crawler is not configurable. I was thinking about update-master-directory-list.
To add a timeout to the rsync crawls it would be necessary to add it directly to /usr/share/mirrormanager/server/crawler_perhost:497
So maybe something for the mirrormanager rewrite.
Adrian