infrastructure April 2008

infrastructure@lists.fedoraproject.org

52 participants
89 discussions

torrent migration, reversed
by Seth Vidal 03 Apr '08

03 Apr '08

Hi folks, I attempted to migrate the torrent to the new server today. That didn't pan out First, the new torrent version didn't work well under any significant load. It worked okay in the tests I had run, but not when hit by all of our users. I worked around this by playing with the version a bit. It was working for a while. Then it died b/c, apparently, the machine it is hosted on died. I've migrated the hostname back to the old machine and brought the old torrent back online. I've also requested the hardware be replaced in the new machine. As soon as we can get a stable machine I'll attempt the migration, again. Thanks, -sv -- I only speak for me.

1 0

Meeting Log - 2008-04-03
by Ricky Zhou 03 Apr '08

03 Apr '08

15:02 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Who's here? 15:03 * dgilmore is here 15:03 * daMaestro is here 15:03 * f13 15:04 -!- fugolini [n=francesc(a)host86-200-dynamic.7-87-r.retail.telecomitalia.it] has quit Remote closed the connection 15:05 * iWolf lurks 15:06 < mmcgrath> pretty low turnout 15:06 * mmcgrath pings some people 15:07 < mdomsch> hi 15:07 < dgilmore> mmcgrath: can we add our CA discussion to the meeting 15:07 < mmcgrath> dgilmore: sure 15:07 < abadger1999> hEY GUYS 15:07 < mmcgrath> lets start with the tickets 15:07 < dgilmore> gday abadger1999 15:07 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Tickets 15:07 < mmcgrath> .tiny https://fedorahosted.org/fedora-infrastructure/query?status=new&status=assi… 15:07 < zodbot> mmcgrath: http://tinyurl.com/2hyyz6 15:07 < mmcgrath> Audio streaming for conference calls 15:07 < mmcgrath> .ticket 395 15:07 < zodbot> mmcgrath: #395 (Audio Streaming of Fedora Board Conference Calls) - Fedora Infrastructure - Trac - https://fedorahosted.org/projects/fedora-infrastructure/ticket/395 15:08 < mmcgrath> .any jcollie 15:08 < zodbot> mmcgrath: jcollie was last seen in #fedora-meeting 6 days, 7 hours, 20 minutes, and 27 seconds ago: *** jcollie has quit IRC ("Ex-Chat") 15:08 < mmcgrath> jcollie's not been around in a bit 15:08 < mmcgrath> so we'll skip that one for now. 15:08 < mmcgrath> .ticket 398 15:08 < zodbot> mmcgrath: #398 (elfutils `monotone' (mtn) error) - Fedora Infrastructure - Trac - https://fedorahosted.org/projects/fedora-infrastructure/ticket/398 15:08 * lmacken is here 15:08 < mmcgrath> someone in #fedora-devel mentioned there's a fix for this but I've not seen it yet 15:08 < mmcgrath> .any roland 15:08 < zodbot> mmcgrath: I have not seen roland. 15:09 < mmcgrath> roland was recently added to the sysadmin-hosted group. He may be able to better assist in this 15:09 < dgilmore> .any rmcgrath 15:09 < zodbot> dgilmore: I have not seen rmcgrath. 15:09 < mmcgrath> we'll move on for th 15:09 < mmcgrath> .ticket 446 15:09 < zodbot> mmcgrath: #446 (Possibility to add external links on spins page) - Fedora Infrastructure - Trac - https://fedorahosted.org/projects/fedora-infrastructure/ticket/446 15:09 < mmcgrath> skvidal: ping 15:09 -!- sebastian^ [n=sebastia(a)Ya1e5.y.pppool.de] has joined #fedora-meeting 15:10 < sebastian^> hi ho all 15:10 < mmcgrath> I never really heard anything more on this. 15:10 -!- buggbot [n=supybot(a)landfill.bugzilla.org] has quit Read error: 104 (Connection reset by peer) 15:10 < mmcgrath> IIRC we talked about adding a link to the spins.fp.o page that links to the wiki 15:10 -!- geroldka [n=Gerold@fedora/geroldka] has joined #fedora-meeting 15:10 < mmcgrath> is that what others recall? 15:10 < dgilmore> i think thats the best option 15:10 < mmcgrath> we can always go back and look at the meeting minutes. 15:11 < dgilmore> thats what we talked about 15:11 < mmcgrath> <nod> 15:11 < mmcgrath> dgilmore: would you mind getting that wiki page setup? 15:11 < dgilmore> mmcgrath: can do 15:11 < mdomsch> is spins.fp.o that hard to edit now? 15:11 < mmcgrath> solid. 15:11 -!- buggbot [n=supybot(a)landfill.bugzilla.org] has joined #fedora-meeting 15:11 < mmcgrath> mdomsch: not really hard, no. 15:12 < mmcgrath> alrighty, anyone have anything else on that? 15:12 < mmcgrath> next ticket 15:12 < mmcgrath> .ticket 485 15:12 < zodbot> mmcgrath: #485 (Schedule Koji DB vacuum) - Fedora Infrastructure - Trac - https://fedorahosted.org/projects/fedora-infrastructure/ticket/485 15:12 < mmcgrath> abadger1999: ^^ 15:13 < mdomsch> mmcgrath, then I just am curious why do it in the wiki instead of just letting the right folks edit the page directly 15:13 < mdomsch> oh well moving on 15:13 < mmcgrath> mdomsch: well, I think there were a couple of reasons. At least one of which is technical. 15:13 < abadger1999> So we just need to do a complete vacuumof koji. 15:13 < mmcgrath> the other was more about hosting and stuff. How do we know the difference between an "official" spin and a non official one? Do we care? 15:13 < mmcgrath> abadger1999: how long will it take? 15:14 < abadger1999> It will probably make kojia bit slow. 15:14 < abadger1999> Unknown how long it will take but 15:14 < mdomsch> mmcgrath, official ones are blessed by the board, hosting is determined by rel-eng based on space available determined by infra 15:14 < abadger1999> Lasttime we attemptedthis, did it go for longerthan 24 hours? 15:14 < mmcgrath> abadger1999: and it does not require a lock right? 15:14 < abadger1999> Correct. 15:15 < mmcgrath> mdomsch: but how do the users figure it out if they're all listed in the same place? 15:15 < abadger1999> But it could slow things down. 15:15 < mmcgrath> abadger1999: is that something we could just start on Sat? 15:15 < mmcgrath> and hope its done by monday? 15:15 < abadger1999> We can definitely startit. 15:15 < mmcgrath> f13: ping 15:15 < mmcgrath> you guys have any major rebuilds planned for this weekend? 15:15 < mdomsch> listed != hosted :-) 15:16 < dgilmore> mmcgrath: i moved the sparc postgres to 8.3 this week 15:16 < mmcgrath> dgilmore: how'd that go? 15:16 < dgilmore> mmcgrath: ill see how it works since autovacuum is supposed to work 15:16 < f13> mmcgrath: this weekend? Not that I know of, except that it's the last weekend before the Final Freeze for Fedora 9 15:17 < mmcgrath> dgilmore: excellent. 15:17 < dgilmore> mmcgrath: it transitioned smoothly, you have to do a dump restore to change versions 15:17 < mmcgrath> f13: we're thinking about running a vacuum on the db so we stop getting notices about it. 15:17 < mmcgrath> and because we're running out of transactions 15:17 < dgilmore> mmcgrath: i think this weekend is fine 15:17 < f13> hrm. 15:18 < f13> it 15:18 < mmcgrath> dgilmore: <nod> 15:18 < f13> it'll just make things a bit slower right, not break anything? 15:18 < dgilmore> f13: yes 15:18 < mmcgrath> f13: correct. 15:18 < abadger1999> f13: Correct. 15:18 < f13> ok... 15:18 < f13> I reserve the right to be bitchy if something falls over and holds up Fedora 9 builds (: 15:19 * abadger1999 notes that the warnings start when we hit half way so we could go for a bit longer if we have to. 15:19 < mmcgrath> abadger1999: does this weekend work for you or would you rather me do it? 15:19 < mmcgrath> abadger1999: and how the hell did we run out of FAS tranasctions so quick :) ? 15:19 -!- spoleeba [n=one@fedora/Jef] has quit "Leaving" 15:20 < abadger1999> I can do it this weekend 15:20 < abadger1999> FAS is kinda worrisome but at leastit's a small db. 15:20 < mmcgrath> yeah 15:20 < mmcgrath> abadger1999: solid, well let me know if you need anything 15:21 < mbonnet> abadger1999: remember to disable the other vacuums while it's going 15:21 < abadger1999> Right. mbonnet -- only for that db, though, correct? 15:21 < mbonnet> abadger1999: correct 15:21 < abadger1999> Cool. I think we're all set on that then. 15:22 < mbonnet> abadger1999: also, don't bother with the analyze, since that can be done on a table-by-table basis later 15:22 < abadger1999> <nod> 15:22 -!- Southern_Gentlem [n=notfred@unaffiliated/southerngentlem/x-2894754] has joined #fedora-meeting 15:22 < mmcgrath> cool 15:22 < mmcgrath> alrighty, anyone have anything else on that? 15:23 < dgilmore> not I 15:23 < mmcgrath> cool, then we're done with the tickets 15:23 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Certificates 15:23 -!- spoleeba [n=one@fedora/Jef] has joined #fedora-meeting 15:23 < mmcgrath> dgilmore: want to take the floor on that one? 15:23 < dgilmore> mmcgrath: :) yep 15:24 < dgilmore> so looking at all the options for setting up and managing our certificate infrastructure 15:24 < dgilmore> all of them have definete downsides 15:25 < dgilmore> I really dont want to write our own 15:25 < dgilmore> i think its too much effort for too little gain 15:25 < mdomsch> dgilmore, agreed 15:25 -!- kital [n=Joerg_Si@fedora/kital] has joined #fedora-meeting 15:25 < dgilmore> i'm thinking that we should try make dogfood work 15:26 < dgilmore> i have spoken with the devs of it and they commited to helping us 15:26 < skvidal> mmcgrath: pong 15:26 < dgilmore> dog tag 15:26 < skvidal> sorry, was in another desktop 15:26 < mmcgrath> skvidal: I forget what I was going to ask :) 15:26 < skvidal> torrent? 15:26 < mmcgrath> dgilmore: is it in Fedora already? 15:26 < mmcgrath> skvidal: maybe 15:26 < dgilmore> mmcgrath: not yet 15:27 < dgilmore> mmcgrath: however none of the options is 15:27 < dgilmore> one of them requires jboss 15:27 < dgilmore> so it will be a huge task to get in 15:27 < dgilmore> dogtag has some pieces in fedora 15:27 < dgilmore> it uses fedora-ds for storage 15:27 -!- fab_away [n=bellet(a)bellet.info] has quit No route to host 15:28 < dgilmore> the option using a relational db uses jboss 15:28 < dgilmore> i think it was openca 15:28 < dgilmore> one of the other options uses openldap 15:28 < mmcgrath> So whats the scope of this project? We're looking to give and revoke certificates to our users. 15:28 < mmcgrath> will this also sign our VPN and builder certs? 15:28 < dgilmore> yes 15:29 < mmcgrath> will this have anything to do with package signing? 15:29 < dgilmore> nothing to do with packag signing 15:29 < dgilmore> so we will need to setup the ca 15:29 < mmcgrath> Solid, so what do you need to get a proof of concept setup? 15:30 < dgilmore> test that when we revoke certs apps honour the revokation 15:30 < dgilmore> probably will need 2 or 3 vm's 15:30 < mmcgrath> k, the german box telia1 is up 15:30 < dgilmore> setup as many apps as we can 15:30 < mmcgrath> .ping telia1.fedoraproject.org 15:30 < zodbot> pong 15:31 < mmcgrath> We're going to stick one app and one proxy server out there, the rest can be used for test. 15:31 < mmcgrath> we've got a /28 for it. 15:31 < dgilmore> :) nice 15:31 < mmcgrath> if you want to virt-inst some guests over there, have at it. 15:31 < dgilmore> ok will do 15:32 < dgilmore> I'm going to need some help getting things setup and running 15:32 < dgilmore> and testing 15:32 -!- geroldka [n=Gerold@fedora/geroldka] has quit "Leaving" 15:32 < dgilmore> in the end all users will need new certs. as will all builders and vpn endpoints 15:32 < mmcgrath> I'm a little confused on how this will tie in to FAS2 (if it even will) 15:32 < mmcgrath> Like, where will users go to actually get their cert? 15:33 < dgilmore> they will go to fas2 15:33 < mmcgrath> perhaps we'll have a better view after we get it all setup 15:33 < dgilmore> but fas2 will request it from the ca 15:33 < iWolf> dgilmore: i might have time to help with some setup if you need it. 15:33 < dgilmore> iWolf: thanks 15:34 < mmcgrath> solid. 15:34 < mmcgrath> dgilmore: you have anything else on that front? 15:34 < dgilmore> so thats how i think we should procede 15:34 < dgilmore> mmcgrath: i think thats it 15:35 < mmcgrath> solid 15:35 < mmcgrath> dgilmore: sounds good, let me know if you need anything 15:35 < mmcgrath> Movin on 15:35 < dgilmore> mmcgrath: willd o 15:35 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Torrent 15:35 < mmcgrath> skvidal: mind giving a quick overview of what we did, why we did it and how much space we've got left? 15:36 < skvidal> yes 15:36 < skvidal> okay 15:36 < skvidal> 1. eventually got torrent1.fp.o to stay up and online 15:36 < skvidal> 2. allocated 400gb of the 425gb available to /srv of torrent 15:36 < skvidal> 3. installed bittorrent 5.2.0 - had it fall over every 20 minutes or so, 15:37 < skvidal> 4. cursed a lot, attempted to fix 15:37 < skvidal> 5. found it was just not reliable 15:37 < skvidal> 6. rebuilt bt 4.4.0 and installed it 15:37 < skvidal> 7. had it fall over , while not a lot, a fair bit, 15:37 < skvidal> 8. figured out that 4.4.0 and modern python-twisted don't get along 15:37 < skvidal> 9. disable twisted 15:37 < skvidal> 10. restart bttrack 15:37 < skvidal> 11. profit 15:38 < mmcgrath> all in all though we're in good shape now? 15:38 < skvidal> 12. removed zod and bordeaux (among other crap like fedora7-rc2) from torrents 15:38 < mmcgrath> eww 15:38 < skvidal> currently there are 203G remaining 15:38 < skvidal> and it appears to be running correctly 15:38 < skvidal> time and f9 will tell if it is truly reliable 15:38 < mmcgrath> <nod> 15:38 < mmcgrath> skvidal: thanks! 15:39 < skvidal> but atm it is more reliable than it was before 15:39 < mmcgrath> excellent. 15:39 < mmcgrath> anything else? 15:39 < skvidal> I'm updating the SOP, now 15:39 < skvidal> oh and 15:39 < skvidal> if anyone wants a project that will make them instantly excellent 15:39 < skvidal> fork bt 4.4.0 and maintain it/make it sane 15:40 < skvidal> hell, even if you only made it a server (bttrack/bttseed) you'd be a hit 15:40 < skvidal> that's all 15:40 < mmcgrath> :) 15:40 < mmcgrath> skvidal isn't kidding btw :) 15:40 < mmcgrath> alrighty. Movin on 15:40 < dgilmore> skvidal: what is the deal with bt 5.x? 15:41 < skvidal> dgilmore: it would either run out of files (even when I increased it to ludicrous amounts) or it would stop sending the /announce page, at all. 15:41 < skvidal> and then 5.2.0 is the dead end 15:41 < skvidal> bittorrent.com stopped the open source releases 15:41 < dgilmore> :( didnt they change the license? 15:41 < skvidal> oh they're chock full of evil 15:42 < mmcgrath> pure evil 15:42 < mmcgrath> ok, movin on 15:42 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Me gone tomorrow and saturday. 15:43 < mmcgrath> I've got a conference where I'm representin' Red Hat and Fedora. 15:43 < mmcgrath> I'll be back on Sunday. 15:43 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Open Floor 15:43 < mdomsch> enjoy 15:43 < mmcgrath> Ok, so thats really all we had planned for this meeting. 15:43 < mmcgrath> The wiki migration is moving along but there's still TONS to be done for those of you python coders interested in helping, the migration script could always use work. 15:44 < mmcgrath> I'm also working on getting the docbooko stuff all figured out. 15:44 < mmcgrath> ricky: you around by chance? 15:45 < mmcgrath> guess not. 15:45 < mmcgrath> we're still blocking a bit on FAS2 and OpenID but I'm pretty confident that'll be in good shape. 15:45 < mmcgrath> Alrighty, anyone have anything else they'd like to discuss? If not I'll close the meeting. 15:45 < dgilmore> mmcgrath: making fas a openid provider? 15:46 * dgilmore is done 15:46 < mmcgrath> dgilmore: yeah 15:46 < poelcat> mmcgrath: what is the targeted golive date for the new wiki? 15:46 < iWolf> mmcgrath: where is the migration script for the wiki? 15:46 < mmcgrath> poelcat: don't have one yet but it will be after F9 ships. Hopefully by a week or two. 15:46 < poelcat> okay 15:46 < mmcgrath> iWolf: git.fedorahosted.org/git/fedora-infrastructure.git/scripts/ 15:47 < iWolf> mmcgrath: thanks! 15:47 < mmcgrath> alrighty, if no one has anything else we'll close the meeting in 30 15:47 < mmcgrath> 15 15:47 < mdomsch> mmcgrath, when is freeze for f9? 15:47 < mmcgrath> mdomsch: ahh, good question. 15:48 * mmcgrath looks. 15:48 * mdomsch sneaks in under the wire 15:48 < mmcgrath> mdomsch: well last time we did a 2 week freeze IIRC. 15:48 -!- gkrpan [n=greg(a)c-71-56-252-37.hsd1.co.comcast.net] has joined #fedora-meeting 15:49 < mmcgrath> f13: did you want a longer change freeze this time around? 15:49 < mdomsch> yes, and that seemed to work pretty well 15:49 < mmcgrath> we could do a partial change freeze on the 8th followed by a full freeze on the 15th? 15:49 * mmcgrath doesn't know what a partial change freeze is but we could come up with something. 15:49 < mmcgrath> otherwise I'm fine with just doing the full freeze on the 15th. 15:49 < mdomsch> everyone's got the 29th free for an all-hands-on-deck, right? 15:50 < mmcgrath> mdomsch: I'll send a reminder out to everyone and about the change freeze. 15:50 < f13> mmcgrath: I think jsut full freeze on the 15th is fine, but I would like veto powers on any proposed changes leading up to then 15:50 < f13> well, I'd like releng to thave that, not me specifically 15:51 < mmcgrath> f13: hmm, I don't see any reason that wouldn't work. veto / revert powers. Plenty of changes go through without approval. 15:51 < f13> I think those two sentences contradict eachother 15:52 -!- giallu [n=giallu(a)81-174-46-50.dynamic.ngi.it] has joined #fedora-meeting 15:52 < mmcgrath> f13: they do, the fact is you can't have veto power on something that doesn't go through an approval process. But I'm happy to give it to you :) 15:52 < mmcgrath> f13: If you've got a problem with a change though bring it to the list, I'm sure we can all work something out. 15:52 < mdomsch> mmcgrath, one more - soc status on download1 routing? 15:53 < mmcgrath> mdomsch: you're guess is as good as mine on that one. They don't tell me anything / keep me up to date on anything. 15:53 < mdomsch> mgalgoci was going to stir up trouble on Monday morning 15:53 < mmcgrath> I can follow up in the ticket and let tom know. 15:53 < mdomsch> as it stands, we won't be able to feed our tier 1 mirrors, much less anyone else 15:53 < mmcgrath> mdomsch: do you know when we lost that link yet? 15:53 < mdomsch> weeks ago 15:53 < mmcgrath> I requested that information from the soc and heard nothing. 15:53 < mdomsch> I remember seeing a note at the time, thinking it would be fixed in a few days 15:54 < mmcgrath> ok. I'm going to bring it to my manager because A) its constant bs with the soc and I'm tired of it. and B) they never bothered to notify us. 15:54 < mdomsch> check mirror-list-d archives, they may have sent something there... 15:54 < mmcgrath> k, I'll double check there then. 15:54 < mdomsch> in any case, we've _got_ to get the bits to ibiblio, else we're scrod 15:54 < mmcgrath> That I'm sure we can do. 15:54 < mdomsch> (past pluperfect of screw) 15:54 < f13> mmcgrath: ok, so maybe the concept of a soft freeze is where you ask before making any changes, and at that point I'd have the ability to say no. 15:55 < f13> mmcgrath: at the hard freeze date, just no changes at all unless it's fixing things 15:55 < mmcgrath> f13: why not just stick with the system we put in place last release? It seemed to work just fine? 15:55 < f13> *shrug* sure, we could do that 15:55 * mmcgrath just doesn't want to complicate it unless there's a reason to. We've got a good team of people there. 15:56 < mmcgrath> Alrighty, anyone have anything else to discuss? If not we'll close in 30 15:56 < mmcgrath> 15 15:57 < f13> nod 15:57 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Meeting Closed!

1 0

Change Freeze (IMPORTANT)
by Mike McGrath 03 Apr '08

03 Apr '08

Just a reminder to everyone that the change freeze will be taking place on April 15th. What does this mean? It means you don't make _ANY_ changes to our infrastructure without running it by the list first with an explanation of why it can't wait until after the freeze is over (April 30th) These dates are dependent on if the release slips and such. Just like last time this is a STRICTLY enforced rule. If you've got any questions. Let us know. -Mike

1 0

service accounts for bz and FAS
by Matt Domsch 03 Apr '08

03 Apr '08

Per http://fedoraproject.org/wiki/MattDomsch/FTBFS, I created an account in bugzilla, ftbfs(a)fp.o, which I will use for filing automated bug reports. In aliases.template.erb I'm routing it's mail to /dev/null. :-) However, it dawns on me that someone _could_ come along and try to create a username of "ftbfs" in FAS, which is undesirable. Should I create a new account in FAS too, to "claim" it? This account in bugzilla also needs to be in the bugzilla "editbugs" group, in order to properly file bugs (you can't set bug_state, blocked, or dependson fields via xmlrpc without being in editbugs). I've asked dkl to grant editbugs to this account, but if I created the account in FAS that would be another way to get that set. Except for the whole "signing of the CLA" for what isn't a real person's account, it's a "service" account. Which strikes me as odd. Advice on the best way to handle this, and future similar account needs? Thanks, Matt -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux

1 0

Looking for team reps
by Mike McGrath 02 Apr '08

02 Apr '08

I'm looking for team reps from Ambassadors and translators for the wiki migration. Any volunteers? -Mike

4 6

Nagios down?
by susmit shannigrahi 01 Apr '08

01 Apr '08

Hi, Looks like nagios is down. https://admin.fedoraproject.org/nagios/ "Error: Could not read host and service status information!" Host down or being rebuilt? -- Regards, Susmit. ============================================= ssh 0x86DD170A http://www.fedoraproject.org/wiki/SusmitShannigrahi =============================================

1 0

Outage Notification Torrent.fedoraproject.org - 2008-04-03 13:00 UTC
by Seth Vidal 01 Apr '08

01 Apr '08

There will be an outage starting at 2008-04-03 13:00 UTC, which will last approximately 2 hours. To convert UTC to your local time, take a look at http://fedoraproject.org/wiki/Infrastructure/UTCHowto or run: date -d '2008-04-03 13:00 UTC UTC' Affected Services: Torrent Unaffected Services: Everything that is not torrent. Ticket Link: https://fedorahosted.org/fedora-infrastructure/ticket/263 Reason for Outage: New box. Contact Information: Seth Vidal <skvidal(a)fedoraproject.org> Please join #fedora-admin in irc.freenode.net or respond to this email to track the status of this outage. -sv

1 0

FAS2 problems
by Michael Schwendt 01 Apr '08

01 Apr '08

1) > Please submit bugs to https://fedorahosted.org/fas2/ gives just: Environment not found 2) https://admin.fedoraproject.org/accounts/dump-group.cgi 500 Internal Server Error

6 10

xen7 networking issue?
by Toshio Kuratomi 31 Mar '08

31 Mar '08

Someone just reported that the wiki was down and I thought xen7 might have crashed again. when I tried to ssh to it my connection timed out. ping xen7 was fine. I serial consoled in and then immediately tried to ssh to xen7 again. it worked. uptime showed that xen7 has been up since this morning and the app servers are all running, etc. There are no iscsi errors in /var/log/messages. Is there a possibility that we're experiencing some sort of networking issue with xen7? Maybe that issue is exacerbating the iscsi bug that causes our xen hosts to crash? -Toshio

3 3

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure April 2008