infrastructure March 2010

infrastructure@lists.fedoraproject.org

41 participants
42 discussions

Trac logs..
by Mike McGrath 25 Mar '10

25 Mar '10

So our trac syncing has been failing lately. After some debugging I discovered that A) some projects are logging to projects/*/log/trac.log and B) we're doing so with debug enabled. Some of these files are HUGE. Pungi's is over 1.1G. So here's what I'd like to propose. 1) Remove all of those log files. 2) Disable logging (only a small handful have enabled it) 3) Set the default log level on all future and current projects to ERROR instead of DEBUG 4) After the freeze look at how to properly log rotate these files. Hosted1 is currently frozen but seeing as how this is impacting backups, can I get two +1's ? -Mike

5 4

[PATCH] Ensure that rsync backups are only running one at a time
by Mike McGrath 25 Mar '10

25 Mar '10

--- configs/db/backup-dbs | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/configs/db/backup-dbs b/configs/db/backup-dbs index 83ed6c8..7af66fb 100755 --- a/configs/db/backup-dbs +++ b/configs/db/backup-dbs @@ -20,6 +20,8 @@ mv $DEST/$HOSTNAME.new $DEST/$HOSTNAME # Sync out for host in db01 db02 db03; do + # Sleep if any other rsyncs are already + while ssh $host "pgrep rsync" | grep -q [0-9]; do sleep 10; done if [ "$host" != $HOSTNAME ]; then su - dbbackup -c "ssh $host mkdir -p $DEST/$HOSTNAME" su - dbbackup -c "rsync -azr --bwlimit=5000 -e ssh $DEST/$HOSTNAME/* $host:$DEST/$HOSTNAME/" -- 1.6.6.1

4 5

Change request: smolt_new db
by Mike McGrath 24 Mar '10

24 Mar '10

I forgot to cleanup after myself on db1 a few weeks back. as a result there's a smolt_new database that doesn't need to be there. 2 +1's to drop it? -Mike

3 2

Mohammed Sikandar has invited you to Dropbox
by Dropbox 23 Mar '10

23 Mar '10

We're excited to let you know that Mohammed Sikandar has invited you to Dropbox! Mohammed Sikandar has been using Dropbox to sync and share files online and across computers, and thought you'd find it useful too. Visit http://www.dropbox.com/link/20.b92tVamdCP/NjExOTI1ODAzNw to get started. - The Dropbox Team ____________________________________________________ To stop receiving invites from Dropbox, please go to http://www.dropbox.com/bl/f738aa5b138b/fedora-infrastructure-list%40redhat.…

1 0

[PATCH] rate limiting rsync copies
by Mike McGrath 23 Mar '10

23 Mar '10

This is causing disk contention without a limit --- configs/db/backup-dbs | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/configs/db/backup-dbs b/configs/db/backup-dbs index a3a12fa..83ed6c8 100755 --- a/configs/db/backup-dbs +++ b/configs/db/backup-dbs @@ -22,7 +22,7 @@ mv $DEST/$HOSTNAME.new $DEST/$HOSTNAME for host in db01 db02 db03; do if [ "$host" != $HOSTNAME ]; then su - dbbackup -c "ssh $host mkdir -p $DEST/$HOSTNAME" - su - dbbackup -c "rsync -azr -e ssh $DEST/$HOSTNAME/* $host:$DEST/$HOSTNAME/" + su - dbbackup -c "rsync -azr --bwlimit=5000 -e ssh $DEST/$HOSTNAME/* $host:$DEST/$HOSTNAME/" fi done -- 1.5.5.6

3 2

Freeze tomorrow
by Mike McGrath 22 Mar '10

22 Mar '10

Just a reminder... we freeze tomorrow! -Mike

1 0

Outage: fedorahosted.org - 2010-03-21 08:00:00 UTC
by Mike McGrath 21 Mar '10

21 Mar '10

There is an outage going on that started at 2010-03-21 08:00:00 UTC, which will last an unknown amount of time. To convert UTC to your local time, take a look at http://fedoraproject.org/wiki/Infrastructure/UTCHowto or run: date -d '2010-03-21 08:00:00 UTC' Reason for outage: Serverbeach4, which hosts *.fedorahosted.org is hanging shortly after boot. The hosting provider has been notified and are looking into it. Affected Services: Fedora Hosted - https://fedorahosted.org/ Unaffected Services: BFO - http://boot.fedoraproject.org/ Bodhi - https://admin.fedoraproject.org/updates/ Buildsystem - http://koji.fedoraproject.org/ CVS / Source Control DNS - ns1.fedoraproject.org, ns2.fedoraproject.org Docs - http://docs.fedoraproject.org/ Email system Fedora Account System - https://admin.fedoraproject.org/accounts/ Fedora Community - https://admin.fedoraproject.org/community/ Fedora People - http://fedorapeople.org/ Fedora Talk - http://talk.fedoraproject.org/ Main Website - http://fedoraproject.org/ Mirror List - https://mirrors.fedoraproject.org/ Mirror Manager - https://admin.fedoraproject.org/mirrormanager/ Package Database - https://admin.fedoraproject.org/pkgdb/ Smolt - http://smolts.org/ Spins - http://spins.fedoraproject.org/ Start - http://start.fedoraproject.org/ Torrent - http://torrent.fedoraproject.org/ Translation Services - http://translate.fedoraproject.org/ Wiki - http://fedoraproject.org/wiki/ Ticket Link: N/A since fedorahosted.org is down... Contact Information: Please join #fedora-admin in irc.freenode.net or respond to this email to track the status of this outage.

1 0

Possibility of Fedora sponsoring a Freenode server
by Nick Bebout 19 Mar '10

19 Mar '10

What is everyone's thoughts about us sponsoring a Freenode server? Their general requirements are noted on http://freenode.net/hosting_ircd.shtml and I've discussed them with mmcgrath and he has said they seem doable, probably on telia1 or osuosl1. Basically we'd be providing them a xen guest with about 1.5 or 2 GB ram, client servers should expect to sustain around 1Mbps of traffic, with much higher peaks. Total usage for client servers is usually around 320 gigabytes per month, while hidden hubs are generally much lower overall. We'd be giving them at least a user account on it, they prefer sudo if possible. They handle the day-to-day administering of the ircd, either scp'ing new binaries or compiling the ircd on the server itself, and copying new config files to there. I've talked to their sponsor liaison and they seem to be interested. Mike and I wanted to discuss it on here before I send the actual "proposal" email. I think it would be a good idea, especially since we use them so much for our IRC channels, it'd be nice to contribute a server back to the network. Nick

10 9

Re: main problems of fedora?
by Jose Manimala 17 Mar '10

17 Mar '10

On 3/18/2010 12:10 AM, Mike McGrath wrote: > On Wed, 17 Mar 2010, Gergely Buday wrote: > >> Hi there, >> >> is there a list of known deficiencies of fedora - I mean those which >> can be cured by developing new software, rather than fixing bugs? Or, >> if such list does not exist, can you name a few missing item from >> fedora? Think of relatively small would-be software, without a GUI. >> > > One of the new initatives in Fedora is called Fedora Engineering Services > - http://fedoraproject.org/wiki/Fedora_Engineering_Services > > If you're a fast learner who just wants to work on various technical and > other issues but don't care what, sign up. We can always use more help. > > -Mike Wow... this is the kind of work I was looking for... Can I sign up? -- Kind Regards Jose M Manimala http://josemanimala.eu.org Ph: +919884065040 GPGkeyID: 1BE49F39

2 1

[PATCH] expose MM crawler logs on bapp01
by Matt Domsch 17 Mar '10

17 Mar '10

My goal here is to expose the mirrormanager crawler logs, which are on bapp01:/var/log/mirrormanager/crawler/ via the web URL: http://admin.fedoraproject.org/mirrormanager/crawler/ Does this look right to do that? I can't easily test this in staging, so want an additional set of eyes. Thanks, Matt >From d6f45bf2fa9165712f9ce9032050c51c68e75db0 Mon Sep 17 00:00:00 2001 From: Matt Domsch <mdomsch(a)puppet01.phx2.fedoraproject.org> Date: Wed, 17 Mar 2010 15:47:17 +0000 Subject: [PATCH] MM: expose crawler logs on bapp01 --- manifests/servergroups/proxy.pp | 6 ++++++ modules/mirrormanager/manifests/init.pp | 16 ++++++++++++++++ .../templates/mirrormanager-crawlerlogs.conf.erb | 4 ++++ 3 files changed, 26 insertions(+), 0 deletions(-) create mode 100644 modules/mirrormanager/templates/mirrormanager-crawlerlogs.conf.erb diff --git a/manifests/servergroups/proxy.pp b/manifests/servergroups/proxy.pp index 33f83f4..8efbd45 100644 --- a/manifests/servergroups/proxy.pp +++ b/manifests/servergroups/proxy.pp @@ -679,6 +679,12 @@ class proxy { proxyurl => "http://localhost:10008", } + mirrormanager::proxy::crawlerlogs { "admin.fedoraproject.org/mirrormanager": + website => "admin.fedoraproject.org", + path => "/mirrormanager/crawler", + proxyurl => "http://bapp01", + } + mirrormanager::proxy::mirrorlist { "mirrors.fedoraproject.org": website => "mirrors.fedoraproject.org", proxyurl => "http://localhost:10002", diff --git a/modules/mirrormanager/manifests/init.pp b/modules/mirrormanager/manifests/init.pp index 4666231..979170a 100644 --- a/modules/mirrormanager/manifests/init.pp +++ b/modules/mirrormanager/manifests/init.pp @@ -278,6 +278,22 @@ define mirrormanager::proxy::mirrorlist( } } +define mirrormanager::proxy::crawlerlogs( + $website, + $path, + $proxyurl +) { + include httpd::base + + file { "/etc/httpd/conf.d/$website/mirrormanager-crawlerlogs.conf": + owner => "root", + group => "root", + mode => 0644, + content => template("mirrormanager/mirrormanager-crawlerlogs.conf.erb"), + notify => Service["httpd"], + require => Httpd::Website[$website], + } +} class mirrormanager::sync { package { 'mirrormanager-client': ensure => present } diff --git a/modules/mirrormanager/templates/mirrormanager-crawlerlogs.conf.erb b/modules/mirrormanager/templates/mirrormanager-crawlerlogs.conf.erb new file mode 100644 index 0000000..e9b8769 --- /dev/null +++ b/modules/mirrormanager/templates/mirrormanager-crawlerlogs.conf.erb @@ -0,0 +1,4 @@ +RewriteEngine On + +RewriteRule ^<%= path %>(.+) <%= proxyurl %><%= path %>$1 [P,QSA,L] +ProxyPassReverse / <%= proxyurl %>/ -- 1.5.5.6 -- Matt Domsch Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux

3 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure March 2010