infrastructure December 2009

infrastructure@lists.fedoraproject.org

35 participants
46 discussions

Bastion
by Mike McGrath 15 Dec '09

15 Dec '09

I've updated the bastion.fedoraproject.org ns record to point to bastion3. We still have a lot of work to do but we'll go ahead and stick with this ssh key: bastion,bastion.fedoraproject.org,209.132.182.51 ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuy7kT2nhJRYyi4Af28KzHJIgFzD84TE9lzDhGB6a9XvCpdI5n0RGZ2bznXR+SRbLjYrseVD+S7ScDFB6+TrXwZpWSM3aJiTJPCzGpI5SRZIaJ4IfAsY9t6n4K3QMmXh+BPGYrUb7ohwtyZADy+7/4E4YOc4cEMxzBEML2D06ERSmCybX01rAHw81Rx/4KN5oH05ynqAcfjF9f1tesDZIZ/NOMkVtMLBWwLf9iKkOKj8dAGBJy/uhidOkVm8bfI9X3NyKwjXC/V8RCeCv/n77y40yvEKYUy8iKOyHQHCVXmTck+6bX1Ity9XYFlljOLaoXPmrO2r6VlyzL0tDKbqtkw== -Mike

1 0

Mea Culpa and Apology
by Stephen John Smoogen 15 Dec '09

15 Dec '09

One of the big problems with the move from PHX1 to PHX2 has been the renaming of hosts. This was a big mistake on my part and made life very difficult for the Fedora people who worked over the weekend to get it working and running into constant headaches. I apologize and owe everyone a lot. Stephen -- Stephen J Smoogen. Ah, but a man's reach should exceed his grasp. Or what's a heaven for? -- Robert Browning

3 2

Voting app offline time?
by Paul W. Frields 15 Dec '09

15 Dec '09

Has the voting app been offline during the move? If so, how long? The answer to those questions will determine whether and how long we need to extend the voting period. -- Paul W. Frields http://paul.frields.org/ gpg fingerprint: 3DA6 A0AC 6D58 FEC4 0233 5906 ACDB C937 BD11 3717 http://redhat.com/ - - - - http://pfrields.fedorapeople.org/ irc.freenode.net: stickster @ #fedora-docs, #fedora-devel, #fredlug

5 9

Major dns issues
by Mike McGrath 15 Dec '09

15 Dec '09

So I woke up today and we're still having dns issues on at least one of my hosts. Could everyone that has access please do a dig fedoraproject.org on all their hosts and tell me if any of them cannot resolve? -Mike

9 8

Re: mirrors.fedoraproject.org problem?
by itamar 14 Dec '09

14 Dec '09

>>> >>> same problem for me, I have created a ipv6 tunnel and worked fine. > >> I can't get to the page either via yum or via the browser. However, I >> can connect to other websites with no problem. Very strange. > > Fedora Proj were moving *stuff* between Data Centres over the weekend > - is it possible the IP has changed and your DNS hasn't updated? > > Where are you going? > > [sam@samlap ~]$ host mirrors.fedoraproject.org > mirrors.fedoraproject.org is an alias for wildcard.fedoraproject.org. > wildcard.fedoraproject.org has address 80.239.156.215 > wildcard.fedoraproject.org has address 152.46.7.222 > wildcard.fedoraproject.org has address 66.35.62.166 > wildcard.fedoraproject.org has IPv6 address 2610:28:200:1::fed0:2 > > -- > Sam > even if you don't have a ipv6 address the 2610:28:200:1::fed0:2 have preference, because this you're not able to download updates. -- ------------ Itamar Reis Peixoto e-mail/msn/google talk/sip: itamar(a)ispbrasil.com.br skype: itamarjp icq: 81053601 +55 11 4063 5033 +55 34 3221 8599

1 0

Outage Notification - 2009-12-12 11:00 UTC
by Mike McGrath 11 Dec '09

11 Dec '09

There will be an outage starting at 2009-12-12 11:00 UTC, which will last approximately 48 hours. To convert UTC to your local time, take a look at http://fedoraproject.org/wiki/Infrastructure/UTCHowto or run: date -d '2009-12-12 11:00 UTC' Affected Services: Buildsystem CVS / Source Control Translation Services Websites Unaffected Services: Database DNS Fedora Hosted Fedora People Fedora Talk Mail Mirror System Torrent Ticket Link: https://fedorahosted.org/fedora-infrastructure/ticket/1845 Reason for Outage: This is the official outage notification that was mentioned days ago. The ticket link above will have the most up to date information and we will be coordinating the outage in #fedora-admin on irc.freenode.net. At the time specified above we will be powering down hosts, moving them on to a truck, unloading them re-racking and re-cabling and powering on. Contact Information: Please join #fedora-admin in irc.freenode.net or respond to this email to track the status of this outage.

1 0

Outage Notification - date -d '2009-12-11 02:00:00 UTC'
by Mike McGrath 11 Dec '09

11 Dec '09

There will be an outage starting at date -d '2009-12-11 02:00:00 UTC', which will last approximately 2 hours. To convert UTC to your local time, take a look at http://fedoraproject.org/wiki/Infrastructure/UTCHowto or run: date -d 'date -d '2009-12-11 02:00:00 UTC'' Affected Services: Database Fedora Hosted (Just auth against trac) Translation Services Websites Unaffected Services: Buildsystem CVS / Source Control DNS Fedora Talk Fedora People Mail Mirror System Torrent Ticket Link: https://fedorahosted.org/fedora-infrastructure/ticket/1845 Reason for Outage: Our temporary DB hosts are in PHX are ready to take on load. We're going to shut down db1 and db2, do an rsync then bring them up. During this time we'll also be configuring our new VPN. Also, a reminder for those who don't read the link, we'll be having a massive outage of many Fedora services this weekend while we are moving our servers to a new location. Contact Information: Please join #fedora-admin in irc.freenode.net or respond to this email to track the status of this outage.

5 7

[ANNOUNCEMENT] Red Hat Bugzilla 3.4 Upgrade Public Beta 2
by James Laska 11 Dec '09

11 Dec '09

I am sending this on behalf of Dave Lawrence and the bugzilla team at Red Hat. Please forward this on to any appropriate lists that were missed. ======== Greetings, The Red Hat Bugzilla team is happy to announce the second public beta release of the next version of Red Hat Bugzilla based on the upstream 3.4 code base. We have not yet received a lot of reports or feedback from our last public beta, so please take some time to try out our latest code release so that we will be sure that this is as stable as possible when we deploy live. Please test drive at: https://partner-bugzilla.redhat.com Over the years Red Hat has made substantial customizations to Bugzilla to fit into the Engineering tool chain. Over time the upstream has incorporated some of these customizations or solved them in different ways. Upgrading reduces our customization footprint (and thus maintenance) while bringing many bug fixes & enhancements. The main area of focus for our public betas are stability. Functionality that currently works in our 3.2 code base should continue to work as expected in the new 3.4 release. These include various ajax optimizations, needinfo actor support, frontpage.cgi, product browser, several various UI enhancements, and of course the XMLRPC API. Please feel free to point your various scripts and third party applications that use the XMLRPC API at the test server to make sure they continue to function properly. There are numerous other changes behind the scenes that we haven't listed. The goal is to make sure that functionality that people have come to expect in 3.2 is possible in the new system. There are also numerous new features/fixes that are part of the upstream 3.4 release. For more detailed information on what has changed since the last release, check out release notes here: https://partner-bugzilla.redhat.com/page.cgi?id=release-notes.html The database is a recent snapshot of the live database so should be useful for testing to make sure the information is displayed properly and changeable. Also with a full snapshot it is possible to test for any performance related issues. Email has been disabled so that unnecessary spam is not sent out. So feel free to make changes to bugs to verify proper working order. We are asking for everyone to get involved as much as possible with testing and feedback on the beta releases to help us make this the most robust and stable release possible. Please file any enhancement requests or bug reports in our current Bugzilla system go here: https://bugzilla.redhat.com/enter_bug.cgi?product=Bugzilla&version=3.4 To see current list of 3.4 related bugs as well as blockers for release, go here: https://bugzilla.redhat.com/buglist.cgi?product=Bugzilla&version=3.4 Thanks The Red Hat Bugzilla Team

1 0

Need a mentor in developing for fedora.
by TEE JIA HEN 11 Dec '09

11 Dec '09

Hi , my name is jia hen from Malaysia. I'm have some knowledge networking(socket programming) , some simple micro-controller programming and website programming(php). I know some basic of C++ , Java , but the language that i like most is python. i'm willing to help , please give me some guidance or some advice for me or some order ... haha .. -- Add my facebook! http://www.facebook.com/wizz.tee?ref=profile

1 0

Meeting Log - 2009-12-10
by Ricky Zhou 10 Dec '09

10 Dec '09

20:00 < skvidal> meeting? 20:00 * skvidal awaits 20:00 * dgilmore is here 20:00 < dgilmore> mmcgrath: 20:00 < dgilmore> mmcgrath: 20:00 < dgilmore> mmcgrath: 20:00 < mmcgrath> #startmeeting 20:00 < zodbot> Meeting started Thu Dec 10 20:00:40 2009 UTC. The chair is mmcgrath. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:00 < zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 20:00 < mmcgrath> #topic Who's here? 20:00 * mmcgrath is 20:00 -!- zodbot changed the topic of #fedora-meeting to: Who's here? 20:00 * dgilmore 20:00 < mmcgrath> smooge: ping 20:00 -!- a-k [n=akistler@2002:638e:1ba0:3:20d:56ff:fe10:bb8d] has joined #fedora-meeting 20:01 < smooge> here 20:01 * a-k is 20:01 < smooge> putting clothes in dryer 20:01 < mmcgrath> So the outage is this weekend, I figured we should all go over the timelines and such 20:01 < mmcgrath> jwb: This might interest you 20:01 < mmcgrath> Oxf13: ^^ 20:02 * skvidal is here 20:02 * jwb is semi here 20:02 < mmcgrath> So here's the basics 20:02 < mmcgrath> #action - Tonight we'll migrate db1, db2 and the vpn to PHX2 20:02 * nirik is sitting in the back in the cheap seats. 20:02 < a-k> mmcgrath: #topic? 20:02 < mmcgrath> #action - Sometime before Sat. Morning mail will start to go through bastion3 in PHX2 20:02 < mmcgrath> #topic Timeline 20:02 -!- zodbot changed the topic of #fedora-meeting to: Timeline 20:02 < smooge> #topic movement to PHX 20:02 < mmcgrath> #action - Tonight we'll migrate db1, db2 and the vpn to PHX2 20:03 < mmcgrath> #action - Sometime before Sat. Morning mail will start to go through bastion3 in PHX2 20:03 * mmcgrath doesn't get this bot stuff sometimes 20:03 < mmcgrath> The mail part is still in an unknown state to me 20:03 < mmcgrath> I've got a ticket open, I've pinged people about it's priority, I'm waiting to hear back. 20:03 < mmcgrath> #action - Smooge is working on some resolvers for us, those should be in place ASAP but isn't a blocker for the move. 20:03 < mmcgrath> it is a blocker to start turning things on though. 20:04 < dgilmore> mmcgrath: do we have the new bastion host up? 20:04 < mmcgrath> dgilmore: yes, bastion3.fedoraproject.org is up and running 20:04 < mmcgrath> Ok, this takes us to Friday morning. 20:04 < smooge> #action - Smooge will email out named scheme for PHX2 resolve domain 20:04 < mmcgrath> #action jwb/ someone in releng will tell me as soon as they are done writing to /mnt 20:05 < mmcgrath> #action at that time I'll mount the public mirrors as read only. 20:05 < jwb> mmcgrath, will likely be two sets of things there 20:05 < jwb> updates, and rawhide 20:05 < mmcgrath> jwb: totally fine, just let me know when it's done. 20:05 < jwb> k 20:05 < mmcgrath> Ok, this is where the coordination stuff happens. 20:05 < mmcgrath> I'll be flying to PHX at noon on Friday. 20:05 < mmcgrath> smooge: when do you leave? 20:06 < smooge> I leave at ~0900 tomorrow morning 20:06 < mmcgrath> that's localtime? 20:06 < dgilmore> smooge: MST 20:06 -!- Sonar_Guy [i=Who@fedora/sonarguy] has quit Read error: 104 (Connection reset by peer) 20:06 < mmcgrath> k 20:07 < mmcgrath> We both have access to PHX1 and PHX2 from Friday to Wed I believe 20:07 * SmootherFrOgZ is around 20:07 < smooge> sorry 0800 tomorrow mornign ariving at ~0900 MST 20:07 < mmcgrath> k 20:07 < mmcgrath> So I'll be heading to PHX1 in the afternoon to give everything a final look over 20:07 < mmcgrath> this is still Friday. 20:08 -!- Sonar_Guy [i=Who@fedora/sonarguy] has joined #fedora-meeting 20:08 < smooge> mmcgrath, when do you get there 20:08 < mmcgrath> I'll probably shut down stuff like db1 since it's not being used at that point. 20:08 < smooge> nm.. see now 20:08 < mmcgrath> I get in at 3:30 20:08 < smooge> 03:30 or 15:30 20:08 < mmcgrath> 15 20:08 < skvidal> what's the plan for mail at this point? 20:09 * mdomsch is here finally 20:09 < mmcgrath> skvidal: I said that already :) 20:09 < skvidal> oh 20:09 < skvidal> it had just scrolled off 20:09 < skvidal> so I missed it on the divider :( 20:09 < skvidal> sorry 20:09 < mmcgrath> skvidal: I've got a ticket open and have pinged people about it's priority 20:09 < mmcgrath> no worries. 20:09 < mmcgrath> I'm hoping for that to get done tonight, if it's not done by then I'm going to need to hand that off to someone to keep bugging people until it gets done because I won't be around on Friday 20:09 < mmcgrath> skvidal: but the good news is we aren't the blockers on that AFAIK. 20:10 < mmcgrath> bastion3 is setup and listening and ready to forward mail. 20:10 < mmcgrath> Ok, so here's what is left 20:10 < mmcgrath> #action Friday night they'll be configuring the new netapp head unit in PHX2 20:10 < mmcgrath> Then sleep 20:10 < mmcgrath> #action Smooge and Mike will meet at PHX1 at 05:15 am 20:11 < mmcgrath> #action Disable nagios 20:11 < mmcgrath> #action make a final backup of sigul and a few other bits (this will probably be done the night before, just forgot to meniton it) 20:11 < mmcgrath> #action power everything down 20:11 < mmcgrath> #action movers will arrive at 05:45 to move 20:11 < jwb> action pray 20:11 < jwb> :) 20:12 < mmcgrath> The movers will actually be helping us unhook cables and such. 20:12 < mmcgrath> oh 20:12 < dgilmore> mmcgrath: buildsys we should disable friday night 20:12 < mmcgrath> #action I'll be taking a sigul drive, and smooge will have the other. 20:12 < jwb> ah, good 20:12 < mmcgrath> dgilmore: we can do it that morning if we want or the night before 20:12 < smooge> mmcgrath, make sure its got both as being bootable. 20:12 < mmcgrath> The movers themselves will actually be loading the entire racks onto the truck. 20:12 < smooge> sorry thats meant more as I need to ... 20:13 < smooge> mmcgrath, they will be? 20:13 < mmcgrath> yep, I have no idea how that'll work but I bet it's going to be wicked awesome. 20:13 < smooge> we aren't getting new racks in PHX or they will move them from one to another there? 20:13 < dgilmore> mmcgrath: lets do it at night make sure all in progress builds complete 20:13 < mmcgrath> dgilmore: sure 20:13 < mmcgrath> #action meet the movers at PHX2 where they'll unload the racks 20:14 < mmcgrath> #action we'll then be moving servers from the old racks (which I guess are now sitting near the new racks) and re-racking everything. 20:14 * mmcgrath will bring bandaids 20:14 < mmcgrath> now this is where part of our story has changed a bit 20:14 < mmcgrath> we were under the impression contractors would be on site to re-wire everything 20:14 < mmcgrath> that's not true 20:14 < pjones> bandaids? consider stopping by autozone and getting some work gloves. 20:15 < mmcgrath> Jonathan (the guy normally onsite at PHX2) is going to be pre-wiring some of the racks. 20:15 < mmcgrath> depending on how far he gets we may have a lot of cabling to do ourselves or none. 20:15 < smooge> pjones, I have had a rack go through teflon armored gloves.. they need blood to work 20:15 < mmcgrath> :) 20:15 < mdomsch> as electronics need smoke 20:15 < pjones> surely you mean kevlar? 20:15 < mmcgrath> so lets see where that leaves us... 20:16 < mmcgrath> #action start powering services back on 20:16 < mmcgrath> let me get the priority list... 20:16 < pjones> (and in any case; knives go through kevlar more easily than they go through canvas. the high velocity of something like a bullet is where kevlar gets its strength from...) 20:16 < mmcgrath> #action get the app servers online first 20:16 < mmcgrath> #action then the buildsystem 20:17 < mmcgrath> I only put them in that order because the app servers should be very straight forward. 20:17 < dgilmore> do we have dhcp? 20:17 < mmcgrath> dgilmore: yeah, and we run it 20:17 -!- sijis_afk [n=sijis(a)70.131.83.26] has joined #fedora-meeting 20:17 < dgilmore> should we reconfigure all the builders first for the new networks 20:17 * mmcgrath is skipping the network config stuff for a later part of the meeting 20:17 < dgilmore> ok 20:17 < mmcgrath> because some of it is quite a bit different 20:17 < mmcgrath> but we'll get to that. 20:17 < mmcgrath> Now it's while we're re-racking this stuff that we will need help from people. 20:18 < mmcgrath> I've already worked on a checklist via CSI 20:18 < mmcgrath> basically I'm looking for people to re-certify these hosts 20:18 < mmcgrath> what does that mean? 20:18 < smooge> pjones, I meant kevlar.. but teflon would expect why I couldn't hold anything 20:18 < mmcgrath> verify remote management, power and cyclades works, as well as network. 20:18 < mmcgrath> this is stuff anyone in sysadmin-main can do. 20:19 < mmcgrath> Ok, so any questions about that? 20:19 < smooge> mmcgrath, pointer to checklist 20:19 < mmcgrath> smooge: I'm going to be adding that info to the ticket in a moment - 20:19 < mmcgrath> https://fedorahosted.org/fedora-infrastructure/ticket/1845 20:19 < smooge> ok cool 20:19 < mmcgrath> Ok, so as far as timeline goes and the physical work of things are there any questions? 20:20 < mmcgrath> Sometime after the move we'll be moving the db hosts back to their normal machines. 20:20 < mmcgrath> Ok, so the next topic is about what the PHX2 world will look like. 20:20 < mmcgrath> #topic PHX2 - what does it look like? 20:20 -!- zodbot changed the topic of #fedora-meeting to: PHX2 - what does it look like? 20:21 < mmcgrath> So in PHX2 we're going to have 3 networks and might add a 4th one later. 20:21 < mmcgrath> The 3 networks are as follows 20:21 -!- rdieter [n=rdieter(a)ip174-71-102-133.om.om.cox.net] has joined #fedora-meeting 20:21 < mmcgrath> 1) public network - 10.5.126 20:21 < mmcgrath> 2) build network - 10.5.125 20:21 < mmcgrath> 2) storage network - 10.5.127 20:21 < mmcgrath> the majority of our services will end up on the buidl network 20:21 < mmcgrath> that should be 3) storage :) 20:22 < mmcgrath> and the majority of our services will end up on the public network, not build. 20:22 * mmcgrath needs more sleep. 20:22 < mmcgrath> the build network will be mostly for releng 20:22 < mmcgrath> and the storage network will be for nfs traffic and possibly backups. 20:22 < mmcgrath> I'm going to be doing tests to find out where backups best work and have the least impact on other things. 20:23 < mmcgrath> This also means that several of our hosts will now have multiple IP addresses. 20:23 -!- notting [n=notting@redhat/notting] has joined #fedora-meeting 20:23 < mmcgrath> for example relengX will be on the build and storage network. 20:23 < mmcgrath> we won't be routing them. 20:23 -!- sandorlev [n=sandorle(a)87.97.110.134.pool.invitel.hu] has joined #fedora-meeting 20:23 < mmcgrath> Anyone have any questions on this? 20:23 < sandorlev> hello 20:24 < mmcgrath> Ok. so that's really it. 20:24 < mmcgrath> I'm working on a spreadsheet right now that I'd appreciate some second looks at. 20:24 < mdomsch> I presume we'll only have DNS A records for the inbound service IPs on those dual-homed boxes 20:25 < mmcgrath> mdomsch: I believe so. 20:25 < mdomsch> e.g. build1's IP on network 3 won't have an A record 20:25 < mmcgrath> we'll be running our own DNS servers in PHX2, smooge is working on that now. 20:25 < mmcgrath> mdomsch: correct but it will probably have a reverse. 20:25 < mmcgrath> smooge: what do you think? 20:25 < smooge> i am going to have two boxes eventually but at the moment just one.. ns001.phx2.fedoraproject.org 20:26 < mdomsch> mmcgrath, makes sense 20:26 < mdomsch> reverse is good to keep track of allocated addresses at least 20:27 < smooge> the hosts will just be on the phx2 subdomain which will be a 'hidden' domain (not on public dns since it will contain 192 and 10.x ips) 20:27 < mmcgrath> smooge: also on the meeting yesterday I think it was requested that we RH to still be able to do lookups and reverse lookups so we might be stuck doing fedora.phx2.redhat.com 20:27 < mmcgrath> unless there's an easy way to just mask it for them? 20:27 * mmcgrath hasn't thought about that much. 20:27 < smooge> I am going to keep it simple in version one because we don't ahve a lot of time.. but dual homed would be later 20:28 < smooge> mmcgrath, I think in either case they will be able to do zone transfers from our servers 20:28 < mmcgrath> smooge: wise idea 20:28 < mmcgrath> Ok, so any questions about this? 20:28 < smooge> so we could call it fedoraproject.int and it should be good 20:29 < mmcgrath> at present there is no firewall preventing access to or from any of the 3 networks I've listed. 20:29 < mmcgrath> we'll be doing that later, just ran out of time. 20:29 < nb> i would suggest not using .int because .int is a real TLD 20:29 < smooge> nb it was a joke sorry 20:29 -!- nucleo [n=nucleo@fedora/nucleo] has quit Remote closed the connection 20:29 < nb> smooge, oh ok :) 20:29 < mmcgrath> Ok, so anyone else have any questions about this? 20:29 < mmcgrath> I really have no estimate for how long this will take. 20:30 < mmcgrath> I've never moved this many machines before and I have no idea how much is already done and how many people we'll have helping 20:30 < smooge> mmcgrath, you are leaving on Tuesday correct? 20:30 < smooge> smooge will be leaving on Wednesday so we wont lose coverage afterwords 20:30 < smooge> and we have problems like "Oh crap that disk drive didn't make it." 20:30 < mmcgrath> smooge: I'm not sure actually I need to look at my trip info. 20:31 < smooge> in case of real emergency I will just get back in the car and drive back til we are stable. 20:31 < mmcgrath> so that's really all I have to discuss 20:31 < mmcgrath> anyone else have any questions or comments? 20:31 < mmcgrath> if not I'll get back to my IP list and let smooge get back to DNS stuff. 20:32 -!- nucleo [n=nucleo@fedora/nucleo] has joined #fedora-meeting 20:32 < nb> anything non-main people can do to help? 20:32 < mmcgrath> #topic open floor 20:32 -!- zodbot changed the topic of #fedora-meeting to: open floor 20:32 < mmcgrath> nb: I'm hoping the non-main people will help with logistics and troubleshooting 20:32 < nb> ok 20:32 < mmcgrath> when the services start coming back online and interacting with eachother I suspect there will be lots of little bugs 20:32 < mmcgrath> like say, app2's network is behaving poorly 20:33 < mmcgrath> nb: but also keeping watch in #fedora-admin and #fedora 20:33 * stickster wants to say a big "thank you" from all the people in Fedora that won't know to whom they should say it... for working so hard to keep us in business during the move. 20:34 < mmcgrath> nb: but yeah, when stuff comes back online smooge and I will probably say stuff like "the wiki is back" 20:34 < mmcgrath> we'll need verification of that :) 20:34 -!- sandorlev [n=sandorle(a)87.97.110.134.pool.invitel.hu] has quit "Have to go" 20:34 < nb> ok 20:34 < sijis_afk> i know i've missed most of the meeting.. but its this weekend, right? 20:34 < mmcgrath> sijis_afk: correct. 20:34 < mmcgrath> stickster: :) 20:34 < mmcgrath> any other questions? 20:34 < sijis_afk> i should be around and help if possible 20:34 -!- sijis_afk is now known as sijis 20:35 < mmcgrath> when I'm done with this IP list I could use some extra eyes 20:35 * nb will be around saturday and sunday afternoon/evening 20:35 < mmcgrath> Anyone that'll be around in the next hour or so just look, try to find duplicates or where I've done something stupid. 20:35 < nb> mmcgrath, ok 20:35 < mmcgrath> If no one has anything else, I'll close the meeting in 30 20:36 < mmcgrath> alllrighty 20:36 < mmcgrath> #endmeeting 20:36 -!- zodbot changed the topic of #fedora-meeting to: Channel is used by various Fedora groups and committees for their regular meetings | Note that meetings often get logged | For questions about using Fedora please ask in #fedora | See http://fedoraproject.org/wiki/Communicate/FedoraMeetingChannel for meeting schedule 20:36 < zodbot> Meeting ended Thu Dec 10 20:36:46 2009 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . 20:36 < zodbot> Minutes: http://meetbot.fedoraproject.org/fedora-meeting/2009-12-10/fedora-meeting.2… 20:36 < zodbot> Minutes (text): http://meetbot.fedoraproject.org/fedora-meeting/2009-12-10/fedora-meeting.2… 20:36 < zodbot> Log: http://meetbot.fedoraproject.org/fedora-meeting/2009-12-10/fedora-meeting.2…

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure December 2009