infrastructure May 2010

infrastructure@lists.fedoraproject.org

42 participants
72 discussions

Change Request: Race condition syncing static websites to proxies
by Toshio Kuratomi 11 May '10

11 May '10

mmcgrath noted that we have a race condition in our syncStatic.sh script that is leading to proxies not having copies of the website. Here's a patch to the syncStatic script to fix that: diff --git a/modules/fedora-web/files/syncStatic.sh b/modules/fedora-web/files/syncStatic.sh index 23f5181..9c1b7ac 100644 --- a/modules/fedora-web/files/syncStatic.sh +++ b/modules/fedora-web/files/syncStatic.sh @@ -26,7 +26,7 @@ cd fedora-web pushd fedoraproject.org > /dev/null make > /dev/null 2>&1 -rsync -qa --delete out/* /srv/web/fedoraproject.org/ +rsync -qa --delete out/ /srv/web/fedoraproject.org/ popd > /dev/null # Make sure everything else builds from master. @@ -34,26 +34,22 @@ popd > /dev/null /usr/bin/git checkout -q master || exit 1 pushd spins.fedoraproject.org > /dev/null -/bin/rm -rf /srv/web/spins.fedoraproject.org/* make > /dev/null 2>&1 -rsync -qa --delete out/* /srv/web/spins.fedoraproject.org/ +rsync -qa --delete out/ /srv/web/spins.fedoraproject.org/ popd > /dev/null pushd talk.fedoraproject.org > /dev/null -/bin/rm -rf /srv/web/talk.fedoraproject.org/* make > /dev/null 2>&1 -rsync -qa --delete out/* /srv/web/talk.fedoraproject.org/ +rsync -qa --delete out/ /srv/web/talk.fedoraproject.org/ popd > /dev/null pushd boot.fedoraproject.org > /dev/null -/bin/rm -rf /srv/web/boot.fedoraproject.org/* make > /dev/null 2>&1 -rsync -qa --delete out/* /srv/web/boot.fedoraproject.org/ +rsync -qa --delete out/ /srv/web/boot.fedoraproject.org/ popd > /dev/null pushd start.fedoraproject.org > /dev/null -/bin/rm -rf /srv/web/start.fedoraproject.org/* make > /dev/null 2>&1 -rsync -qa --delete out/* /srv/web/start.fedoraproject.org/ +rsync -qa --delete out/ /srv/web/start.fedoraproject.org/ popd > /dev/null The patch does two things: 1) Removes the rm -rf step as that is causing the race (when proxies sync data after the rm -rf and before hte rsync, they get an empty directory) 2) Removes the wildcard from the rsync. The wildcard was preventing the --delete from functioning as expected. I believe (but ricky can confirm) that the --delete not removing files that are no longer needed from the final output directory is why we had the rm -rf in the first place. Could I get two +1's for this change? -Toshio

4 5

proxy07 logs sync additions (need review and +1)
by Stephen John Smoogen 10 May '10

10 May '10

Proxy07 was not set up with rsync added. This means the http logs for a while have not been syncing. Fix is to copy, add, commit and push rsync conf files that proxy[123456] have already. Please review and +1 or -1 before I push. commit 347fa93b0bcf3e8b6f7f865177a728079b6d2904 Author: Stephen Smoogen <smooge(a)redhat.com> Date: Mon May 10 23:47:41 2010 +0000 Add in proxy07 so sync's actually work diff --git a/modules/rsync/files/rsync.proxy07 b/modules/rsync/files/rsync.proxy07 new file mode 100644 index 0000000..f02e097 --- /dev/null +++ b/modules/rsync/files/rsync.proxy07 @@ -0,0 +1,14 @@ +# default: off +# description: The rsync server is a good addition to an ftp server, as it \ +# allows crc checksumming etc. +service rsync +{ + disable = no + socket_type = stream + wait = no + user = root + server = /usr/bin/rsync + server_args = --daemon + log_on_failure += USERID +} + diff --git a/modules/rsync/files/rsyncd.conf.proxy07 b/modules/rsync/files/rsyncd.conf.proxy07 new file mode 100644 index 0000000..eb2c13d --- /dev/null +++ b/modules/rsync/files/rsyncd.conf.proxy07 @@ -0,0 +1,16 @@ +pid file = /var/run/rsyncd.pid +syslog facility = daemon +dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.bz2 *.iso +use chroot = false +transfer logging = false +exclude = .snapshot/ .~tmp~/ /.private/ /.private/** **/.nfs* + +# For distributing applications +[log] +comment = Server Logs +path = /var/log +uid = root +gid = root +read only = yes +hosts allow = 10.5.126.28 192.168.1.35 -- Stephen J Smoogen. “The core skill of innovators is error recovery, not failure avoidance.” Randy Nelson, President of Pixar University. "We have a strategic plan. It's called doing things."" — Herb Kelleher, founder Southwest Airlines

3 2

[PATCH] add recoreds for all the new hosts
by Dennis Gilmore 10 May '10

10 May '10

From: Dennis Gilmore <ausil(a)puppet01.phx2.fedoraproject.org> --- modules/bind/files/master/125.5.10.in-addr.arpa | 34 ++++++++++---------- modules/bind/files/master/127.5.10.in-addr.arpa | 36 ++++++++++---------- modules/bind/files/master/phx2.fedoraproject.org | 37 ++++++++++++++++++++-- 3 files changed, 69 insertions(+), 38 deletions(-) diff --git a/modules/bind/files/master/125.5.10.in-addr.arpa b/modules/bind/files/master/125.5.10.in-addr.arpa index ad1b19b..5f905dc 100644 --- a/modules/bind/files/master/125.5.10.in-addr.arpa +++ b/modules/bind/files/master/125.5.10.in-addr.arpa @@ -2,7 +2,7 @@ $TTL 86400 @ IN SOA 125.5.10.in-addr.arpa. hostmaster.fedoraproject.org. ( ; Make SURE to update the serial # when changes are made. Format is YYYYMMDDNN ; (NN is just a counter to handle multiple changes in one day). - 2010031800 ; Serial + 2010051000 ; Serial 28800 ; Refresh 14400 ; Retry 1000000 ; Expire @@ -39,8 +39,8 @@ $TTL 86400 28 IN PTR ppc08.phx2.fedoraproject.org. 29 IN PTR ppc09.phx2.fedoraproject.org. 30 IN PTR ppc10.phx2.fedoraproject.org. -31 IN PTR unused. -32 IN PTR unused. +31 IN PTR ppc11.phx2.fedoraproject.org. +32 IN PTR ppc12.phx2.fedoraproject.org. 33 IN PTR unused. 34 IN PTR unused. 35 IN PTR unused. @@ -88,19 +88,19 @@ $TTL 86400 77 IN PTR unused. 78 IN PTR unused. 79 IN PTR unused. -80 IN PTR unused. -81 IN PTR unused. -82 IN PTR unused. -83 IN PTR unused. -84 IN PTR unused. -85 IN PTR unused. -86 IN PTR unused. -87 IN PTR unused. -88 IN PTR unused. -89 IN PTR unused. -90 IN PTR unused. -91 IN PTR unused. -92 IN PTR unused. +80 IN PTR x86-10.phx2.fedoraproject.org. +81 IN PTR x86-11.phx2.fedoraproject.org. +82 IN PTR x86-12.phx2.fedoraproject.org. +83 IN PTR x86-13.phx2.fedoraproject.org. +84 IN PTR x86-14.phx2.fedoraproject.org. +85 IN PTR x86-15.phx2.fedoraproject.org. +86 IN PTR x86-16.phx2.fedoraproject.org. +87 IN PTR x86-17.phx2.fedoraproject.org. +88 IN PTR x86-18.phx2.fedoraproject.org. +89 IN PTR x86-19.phx2.fedoraproject.org. +90 IN PTR x86-20.phx2.fedoraproject.org. +91 IN PTR x86-21.phx2.fedoraproject.org. +92 IN PTR x86-22.phx2.fedoraproject.org. 93 IN PTR unused. 94 IN PTR unused. 95 IN PTR unused. @@ -128,7 +128,7 @@ $TTL 86400 117 IN PTR unused. 118 IN PTR unused. 119 IN PTR unused. -120 IN PTR unused. +120 IN PTR bxen04.phx2.fedoraproject.org. 121 IN PTR bxen03.phx2.fedoraproject.org. 122 IN PTR bxen02.phx2.fedoraproject.org. 123 IN PTR bxen01.phx2.fedoraproject.org. diff --git a/modules/bind/files/master/127.5.10.in-addr.arpa b/modules/bind/files/master/127.5.10.in-addr.arpa index fd94075..c2650e0 100644 --- a/modules/bind/files/master/127.5.10.in-addr.arpa +++ b/modules/bind/files/master/127.5.10.in-addr.arpa @@ -2,7 +2,7 @@ $TTL 86400 @ IN SOA 127.5.10.in-addr.arpa. hostmaster.fedoraproject.org. ( ; Make SURE to update the serial # when changes are made. Format is YYYYMMDDNN ; (NN is just a counter to handle multiple changes in one day). - 2010041601 ; Serial + 2010051000 ; Serial 28800 ; Refresh 14400 ; Retry 1000000 ; Expire @@ -88,7 +88,7 @@ $TTL 86400 77 IN PTR x86-07-nfs.phx2.fedoraproject.org. 78 IN PTR x86-08-nfs.phx2.fedoraproject.org. 79 IN PTR x86-09-nfs.phx2.fedoraproject.org. -80 IN PTR xenbuilder04-nfs.phx2.fedoraproject.org. +80 IN PTR xb-01-nfs.phx2.fedoraproject.org. 81 IN PTR xen01-nfs.phx2.fedoraproject.org. 82 IN PTR xen02-nfs.phx2.fedoraproject.org. 83 IN PTR xen03-nfs.phx2.fedoraproject.org. @@ -132,15 +132,15 @@ $TTL 86400 121 IN PTR bxen01-nfs.phx2.fedoraproject.org. 122 IN PTR bxen02-nfs.phx2.fedoraproject.org. 123 IN PTR bxen03-nfs.phx2.fedoraproject.org. -124 IN PTR unused. +124 IN PTR bxen04-nfs.phx2.fedoraproject.org. 125 IN PTR unused. 126 IN PTR unused. 127 IN PTR unused. 128 IN PTR unused. 129 IN PTR unused. 130 IN PTR unused. -131 IN PTR unused. -132 IN PTR unused. +131 IN PTR ppc11-nfs.phx2.fedoraproject.org. +132 IN PTR ppc12-nfs.phx2.fedoraproject.org. 133 IN PTR unused. 134 IN PTR unused. 135 IN PTR unused. @@ -148,19 +148,19 @@ $TTL 86400 137 IN PTR unused. 138 IN PTR unused. 139 IN PTR unused. -140 IN PTR unused. -141 IN PTR unused. -142 IN PTR unused. -143 IN PTR unused. -144 IN PTR unused. -145 IN PTR unused. -146 IN PTR unused. -147 IN PTR unused. -148 IN PTR unused. -149 IN PTR unused. -150 IN PTR unused. -151 IN PTR unused. -152 IN PTR unused. +140 IN PTR x86-10-nfs.phx2.fedoraproject.org. +141 IN PTR x86-11-nfs.phx2.fedoraproject.org. +142 IN PTR x86-12-nfs.phx2.fedoraproject.org. +143 IN PTR x86-13-nfs.phx2.fedoraproject.org. +144 IN PTR x86-14-nfs.phx2.fedoraproject.org. +145 IN PTR x86-15-nfs.phx2.fedoraproject.org. +146 IN PTR x86-16-nfs.phx2.fedoraproject.org. +147 IN PTR x86-17-nfs.phx2.fedoraproject.org. +148 IN PTR x86-18-nfs.phx2.fedoraproject.org. +149 IN PTR x86-19-nfs.phx2.fedoraproject.org. +150 IN PTR x86-20-nfs.phx2.fedoraproject.org. +151 IN PTR x86-21-nfs.phx2.fedoraproject.org. +152 IN PTR x86-22-nfs.phx2.fedoraproject.org. 153 IN PTR unused. 154 IN PTR unused. 155 IN PTR unused. diff --git a/modules/bind/files/master/phx2.fedoraproject.org b/modules/bind/files/master/phx2.fedoraproject.org index 8a7dc9c..003656b 100644 --- a/modules/bind/files/master/phx2.fedoraproject.org +++ b/modules/bind/files/master/phx2.fedoraproject.org @@ -3,7 +3,7 @@ $TTL 3600 ; Make SURE to update the serial # when changes are made. Format is ; YYYYMMDDNN (NN is just a counter to handle multiple changes in one ; day). - 2010031701 ; Serial + 2010051000 ; Serial 28800 ; Refresh 14400 ; Retry 1000000 ; Expire @@ -133,6 +133,7 @@ bxen02 IN A 10.5.125.122 bxen2 IN CNAME bxen02 bxen03 IN A 10.5.125.121 bxen3 IN CNAME bxen03 +bxen04 IN A 10.5.125.120 xen01 IN A 10.5.126.101 xen1 IN CNAME xen01 xen02 IN A 10.5.126.102 @@ -181,7 +182,19 @@ x86-7 IN CNAME x86-07 x86-08 IN A 10.5.125.18 x86-8 IN CNAME x86-08 x86-09 IN A 10.5.125.19 -x86-9 IN CNAME x86-09 +x86-10 IN A 10.5.125.80 +x86-11 IN A 10.5.125.81 +x86-12 IN A 10.5.125.82 +x86-13 IN A 10.5.125.83 +x86-14 IN A 10.5.125.84 +x86-15 IN A 10.5.125.85 +x86-16 IN A 10.5.125.86 +x86-17 IN A 10.5.125.87 +x86-18 IN A 10.5.125.88 +x86-19 IN A 10.5.125.89 +x86-20 IN A 10.5.125.90 +x86-21 IN A 10.5.125.91 +x86-22 IN A 10.5.125.92 ppc01 IN A 10.5.125.21 ppc1 IN CNAME ppc01 ppc02 IN A 10.5.125.22 @@ -201,6 +214,8 @@ ppc8 IN CNAME ppc08 ppc09 IN A 10.5.125.29 ppc9 IN CNAME ppc09 ppc10 IN A 10.5.125.30 +ppc11 IN A 10.5.125.31 +ppc12 IN A 10.5.125.32 compose-x86-01 IN A 10.5.125.41 compose-x86-1 IN CNAME compose-x86-01 compose-x86-02 IN A 10.5.126.126 @@ -294,6 +309,8 @@ ppc07-nfs IN A 10.5.127.47 ppc08-nfs IN A 10.5.127.48 ppc09-nfs IN A 10.5.127.49 ppc10-nfs IN A 10.5.127.50 +ppc11-nfs IN A 10.5.127.131 +ppc12-nfs IN A 10.5.127.132 puppet01-nfs IN A 10.5.127.51 puppet02-nfs IN A 10.5.127.52 releng01-nfs IN A 10.5.127.53 @@ -316,7 +333,20 @@ x86-06-nfs IN A 10.5.127.76 x86-07-nfs IN A 10.5.127.77 x86-08-nfs IN A 10.5.127.78 x86-09-nfs IN A 10.5.127.79 -xenbuilder04-nfs IN A 10.5.127.80 +x86-10-nfs IN A 10.5.127.140 +x86-11-nfs IN A 10.5.127.141 +x86-12-nfs IN A 10.5.127.142 +x86-13-nfs IN A 10.5.127.143 +x86-14-nfs IN A 10.5.127.144 +x86-15-nfs IN A 10.5.127.145 +x86-16-nfs IN A 10.5.127.146 +x86-17-nfs IN A 10.5.127.147 +x86-18-nfs IN A 10.5.127.148 +x86-19-nfs IN A 10.5.127.149 +x86-20-nfs IN A 10.5.127.150 +x86-21-nfs IN A 10.5.127.151 +x86-22-nfs IN A 10.5.127.152 +xb-01-nfs IN A 10.5.127.80 xen01-nfs IN A 10.5.127.81 xen02-nfs IN A 10.5.127.82 xen03-nfs IN A 10.5.127.83 @@ -339,6 +369,7 @@ xen19-nfs IN A 10.5.127.99 bxen01-nfs IN A 10.5.127.121 bxen02-nfs IN A 10.5.127.122 bxen03-nfs IN A 10.5.127.123 +bxen04-nfs IN A 10.5.127.124 ;; app01.stg IN A 10.5.126.81 app02.stg IN A 10.5.126.82 -- 1.5.5.6

3 9

Re: Hello Everyone
by Abhishek Bose 10 May '10

10 May '10

>> Hi, >> ? >> I am Abhishek Bose from India.I have been using Fedora since Fedora 10 (pretty new in the group I guess). I am a Redhat Certified Engineer (RHCE) and working in Redhat >> Linux, Solaris, Netbackup since last?2.5 years but my main area of focus is Redhat Linux. I am working for?a company as a Linux Admin where I have to build and maintain >> servers along with several applications running on it. I would like to participate in?developing and helping in Fedora whenever and wherever possible. >> ? >Welcome Abhishek! Was there a particular area in infrastructure you were >interested in participating in? > -Mike Hi Mike Right now I want to overlook the proceedings of the community and in the meantime if there are some backend system administration jobs needs to be done I will be more than happy to help. Regards, /Abhishek

1 0

[PATCH] handle Amazon EC2 client yum misconfiguration
by Matt Domsch 10 May '10

10 May '10

EC2 clients with malformed yum URLs are hitting us a lot. Kill those requests at the proxy servers to keep them from hitting MM. diff --git a/modules/mirrormanager/templates/mirrormanager-mirrorlist.conf.erb b/modules/mirrormanager/templates/mirrormanager-mirrorlist.conf.erb index bd576ce..3b7fdcc 100644 --- a/modules/mirrormanager/templates/mirrormanager-mirrorlist.conf.erb +++ b/modules/mirrormanager/templates/mirrormanager-mirrorlist.conf.erb @@ -13,6 +13,9 @@ RequestHeader set CP-Location /mirrormanager Alias /releases.txt /srv/web/mirrors/releases.txt RewriteEngine On +# BEGIN hack for misconfigured Amazon EC2 clients +RewriteRule ^/mirrorlist?repo=epel-5&arch=\$basea\$ - [F] +# END hack RewriteRule ^/publiclist(.*) <%= proxyurl %>/publiclist/$1 [P,L] RewriteRule ^/mirrorlist(.*) <%= proxyurl %>/mirrorlist$1 [P,L] RewriteRule ^/metalink(.*) <%= proxyurl %>/metalink$1 [P,L] -- Matt Domsch Technology Strategist Dell | Office of the CTO

3 2

[PATCH] bump mirrorlist processes
by Matt Domsch 10 May '10

10 May '10

We're running into a problem where we're seeing 30 requests/second for the mirrorlists, but we only have 6 processes available for apache to send those to. This bumps up the number of processes, from 6 to 30, and cuts the number of threads from 25 to 1 (it's a single-threaded requestor app, and there's the python GIL still to deal with). We may need to do more on the mirrorlist-server.py process to handle the load (it fork()s on every connection), but this is a start... diff --git a/modules/mirrormanager/files/mirrorlist-server.conf b/modules/mirrormanager/files/mirrorlist-server.conf index a26e4b3..ba8e7ab 100644 --- a/modules/mirrormanager/files/mirrorlist-server.conf +++ b/modules/mirrormanager/files/mirrorlist-server.conf @@ -7,7 +7,7 @@ Alias /publiclist /var/lib/mirrormanager/mirrorlists/publiclist/ ExpiresDefault "modification plus 1 hour" </Directory> -WSGIDaemonProcess mirrorlist user=apache processes=6 threads=25 display-name=mirrorlist maximum-requests=1000 +WSGIDaemonProcess mirrorlist user=apache processes=30 threads=1 display-name=mirrorlist maximum-requests=1000 WSGIScriptAlias /metalink /usr/share/mirrormanager/mirrorlist-server/mirrorlist_client.wsgi WSGIScriptAlias /mirrorlist /usr/share/mirrormanager/mirrorlist-server/mirrorlist_client.wsgi -- Matt Domsch Technology Strategist Dell | Office of the CTO

2 1

Hello
by Ed Smith 08 May '10

08 May '10

Hello, All- My name is Ed and I am interested in getting involved with the Fedora project. To give you some of my background, I have a Bachelor's degree in Electrical Engineering but have spent the majority of my professional career in the realm of scientific application programming, system administration and network security. I have contributed to open source projects in the past mostly with code contributions that included random patch submissions for the FreeBSD project. I am adept at most compiled/scripted languages to include C, C++, ObjC, Python, Bash, etc with the exclusion of Java/C#. After several years I am again in a position where I can contribute again to the F/OSS community. In the past I mostly used the BSD flavor of UNIX OSes, Debian and Solaris but I recently tried Redhat again after a number of years and I am quite impressed with the latest iterations of the package. I'll be lurking on the list and on the IRC channel from time to time if anyone is interested in a somewhat veteran hand. As a note, in the past I had an array of hardware and processor architectures that I could test various code on but at the moment I am limited to one x86 desktop machine for any development and testing. Thank you for your time and have a nice day. V/R Ed

3 4

Hello
by John Stanton 08 May '10

08 May '10

Hello, I have been following the infrastructure list for sometime now. I am a system admin and I have some basic scripting knowledge with Perl/Python. I would like to help out with some of the sysadmin groups, such as sysadmin-noc. How can I get involved? I also have the RHCE and I am thinking about trying to get some additional Red Hat certs. -- John Stanton, RHCE john(a)stantonnet.org <mailto:john@stantonnet.org>

1 0

Another Hello
by Rob Marti 08 May '10

08 May '10

I've been lurking in #fedora-admin for a few days now, and figured I'd introduce myself. My name is Rob Marti, I have my RHCE, and after using Red Hat at my job for the past few years, I decided to get involved with the Fedora Project. Since I'm not a really strong coder, development is out, and being partially colorblind means web design stuff is out. I'm okay with that, however, because I really like dealing with the backend system administration stuff a lot more. I think I'd like to be involved in sysadmin-noc at first, and maybe expand from there into hosted and tools.

2 1

Key-value store comparison
by Jorge Gallegos 08 May '10

08 May '10

So, last week a conversation about key/value store engines sparked in #fedora-admin, and today a friend of mine passed this along. I thought it was a good piece to share with you guys: http://blog.medallia.com/2010/05/choosing_a_keyvalue_storage_sy.html Cheers

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure May 2010