============================================ #fedora-meeting: Infrastructure (2014-08-28) ============================================

Meeting started by nirik at 18:00:11 UTC. The full logs are available at http://meetbot.fedoraproject.org/fedora-meeting/2014-08-28/infrastructure.20... .

Meeting summary --------------- * aloha (nirik, 18:00:11)

* New folks introductions and Apprentice tasks. (nirik, 18:01:38)

* Freeze reminder (nirik, 18:12:14)

* Applications status / discussion (nirik, 18:13:52) * fedocal, pkgdb2 bugfix releases this week (nirik, 18:17:16) * nuancier release with lots of changes. (nirik, 18:17:27) * fedimg work ongoing (nirik, 18:17:39)

* Sysadmin status / discussion (nirik, 18:23:42) * pkgs01.stg now in ansible and running rhel7 (nirik, 18:24:53) * fixed a bunch of hosts monday that are still in puppet that were stuck. ;( (nirik, 18:25:52)

* nagios/alerts recap (nirik, 18:27:39)

* Upcoming Tasks/Items (nirik, 18:33:17) * LINK: https://apps.fedoraproject.org/calendar/list/infrastructure/ (nirik, 18:33:17)

* Open Floor (nirik, 18:36:03)

Meeting ended at 18:39:52 UTC.

Action Items ------------

Action Items, by person ----------------------- * **UNASSIGNED** * (none)

People Present (lines said) --------------------------- * nirik (70) * pingou (32) * oddshocks (18) * danielbruno (7) * mirek-hm (7) * lanica (6) * Neldogz (5) * zodbot (5) * smooge (3) * vgologuz (3) * threebean (2) * lmacken (1) * bwood09 (1) * danofsatx (1) * relrod (1) * puiterwijk (0) * abadger1999 (0) * mdomsch (0) * dgilmore (0) -- 18:00:11 <nirik> #startmeeting Infrastructure (2014-08-28) 18:00:11 <zodbot> Meeting started Thu Aug 28 18:00:11 2014 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:11 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:11 <nirik> #meetingname infrastructure 18:00:11 <zodbot> The meeting name has been set to 'infrastructure' 18:00:11 <nirik> #topic aloha 18:00:11 <nirik> #chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch threebean pingou puiterwijk 18:00:11 <zodbot> Current chairs: abadger1999 dgilmore lmacken mdomsch nirik pingou puiterwijk relrod smooge threebean 18:00:17 * pingou here 18:00:21 * bwood09 is here 18:00:32 * oddshocks here 18:00:34 * danielbruno here 18:00:35 * lmacken 18:00:51 * mirek-hm is here 18:00:58 <Neldogz> neldogz is here 18:01:30 * threebean is here 18:01:36 <nirik> cool. Lets go ahead and dive on in then... 18:01:38 <nirik> #topic New folks introductions and Apprentice tasks. 18:01:45 <nirik> any new folks want to intrduce themselves? 18:01:53 <nirik> or apprentices with questions/comments/ideas? 18:02:05 <smooge> here 18:02:21 <nirik> hey smooge 18:02:31 <danielbruno> I have some questions about the Apprentice tasks 18:02:35 * danofsatx present for accounting purposes 18:02:38 <nirik> danielbruno: fire away. ;) 18:02:41 <Neldogz> hello by any chance are there any openings in the Fedora Infrastructure team for volunteers? I have a few hours a week that I would love to dedicate to this team and the Fedora Project. 18:03:02 * lanica is here for the infra meeting. 18:03:29 <nirik> Neldogz: see http://fedoraproject.org/wiki/Infrastructure/GettingStarted and yes, always looking for folks to help out... are you more interested in sysadmin tasks? or application development? 18:03:30 <danielbruno> so, I read the documentations.. and I would like to know a good point to start 18:03:45 <danielbruno> if I have to ask in the channel 18:04:30 <Neldogz> hi nirik; mostly in systems administration. I do have experience with RHEL and Fedora. I would love to learn more about how these work in a corporate environment. I am pretty good with monitoring, documentation and up for anythign that is needed. 18:04:34 <nirik> danielbruno: well, best bets are to look at the easyfix tickets and pick something you are interested in and start working on it... and/or hang out in our channels and when you see an interesting to you discussion chime in and ask if you can help out. ;) 18:05:14 <nirik> if there's specific areas you are interested in we might be able to come up with some more targeted things... :) 18:05:38 <danielbruno> great! :) 18:05:45 <nirik> Neldogz: ok, welcome. we have an apprentice program (see link off that getting started page) if you are interested in it, I can add you after the meeting... see me in #fedora-admin. 18:06:06 <Neldogz> great! would love to join fi-apprentice 18:06:12 <Neldogz> see you in #fedora-admin 18:06:27 <nirik> cool. ;) 18:06:58 <nirik> I will also see about filing some more easyfix tickets... haven't done any new ones for a bit. 18:07:28 <danielbruno> nirik, I have looked at the easyfix and i'm interested in some tasks 18:07:28 <vgologuz> hi all 18:08:08 <nirik> danielbruno: cool. Do comment in those tickets and/or ask the person who filed them if they are around on irc... 18:08:15 <nirik> morning vgologuz 18:08:34 <vgologuz> I've recently joined Copr project, and would like (in some time) to learn about fedora infrastructure 18:08:41 <danielbruno> nirik, ok.. I will :) 18:09:01 <nirik> vgologuz: great. :) I can also add you to our apprentice group if you like... see me after the meeting in #fedora-admin. 18:09:37 <vgologuz> nirik, great ) 18:10:05 <mirek-hm> I will try to learn vgologuz as much as I know about ansible.git, playbooks and FedoraCloud 18:10:25 <mirek-hm> s/learn/teach/ 18:12:04 <nirik> great. 18:12:06 * nirik just had a laptop lockup. sorry for the delay. 18:12:14 <nirik> #topic Freeze reminder 18:12:44 <nirik> So, just a reminder that we are freeze now. Any frozen hosts should not be modified without a patch with 2 +1s from sysadmin-main or releng. 18:12:58 * pingou was the first one to forgot about it 18:13:04 <pingou> it has been so long since the last one :) 18:13:37 <nirik> yeah, I know. ;) 18:13:44 <nirik> see the mail on the list for more details. ;) 18:13:52 <nirik> #topic Applications status / discussion 18:14:04 <nirik> any applications news/status this week? 18:14:08 <lanica> Everything is frozen besides stg? 18:14:10 <lanica> Sorry for the lateness 18:14:34 <pingou> new fedocal released earlier this week, bug fix mainly 18:14:50 <pingou> new pkgdb2 released as well, bugfix again 18:14:59 <pingou> new nuancier released, quite some changes on that one :) 18:15:15 <oddshocks> Fedimg is installed on fedimg01.stg. Working out some Comcast internet problems this moment (I'm currently tethered to my phone 'net), but once they're resolved I'm gonna find out if it's properly doing the thing 18:15:20 <pingou> and elections just got a new election type: irc which offers +1/0/-1 18:15:29 <pingou> work from nerdsville 18:15:35 <oddshocks> (finally got permissions stuff with my sudo access to fedimg01.stg worked out.. I think) 18:15:47 <oddshocks> /s/i think// 18:16:01 <pingou> oddshocks: ansible? 18:16:50 <nirik> lanica: there's a script in ansible called 'freezelist' it will show the ones frozen/not. 18:17:02 <nirik> there are some non stg hosts that also aren't frozen. 18:17:03 <lanica> nirik: thanks! 18:17:16 <nirik> #info fedocal, pkgdb2 bugfix releases this week 18:17:27 <nirik> #info nuancier release with lots of changes. 18:17:39 <nirik> #info fedimg work ongoing 18:17:41 <oddshocks> pingou: what about it? I'm not 100% clear on what the issue was, nirik and smooge resolved it for me 18:18:07 <oddshocks> pingou: i didn't install fedimg with ansible yet, if that's what you're asking. just `rpm -i`'d the rpm 18:18:19 <pingou> oddshocks: that was my question :) 18:18:21 * relrod checks in late, sorry 18:18:27 <oddshocks> I need to figure this ansible stuff out though. I have little to no experience with it sad to say 18:18:31 <oddshocks> relrod: hey 18:18:39 <pingou> oddshocks: installing it via a playbook might have solved the problem there :) 18:18:40 <nirik> oddshocks: we can walk you through it. 18:18:45 <oddshocks> nirik: awesome, thanks :) 18:18:47 <oddshocks> pingou: ahh 18:19:00 <smooge> oddshocks, none of us know how it works. I have 3 voudou dolls which help me 18:19:05 <oddshocks> nirik: pingou: TBH I'm not even 100% clear on which things I should be doing with ansible, and what it can do for me 18:19:06 <pingou> oddshocks: feel free to ping wrt ansible 18:19:09 <oddshocks> smooge: haha 18:19:14 <oddshocks> pingou: cool 18:19:19 <pingou> oddshocks: install rpm, configuration file 18:19:41 <pingou> oddshocks: basically everything that's needed to drop the VM and build a new one working out-of-the-box 18:20:05 <pingou> oddshocks: getting one of those woudou dolls from smooge is a good idea ;-) 18:20:28 <nirik> basically if we had to reinstall the machine today, ansible would let us get it back to the same exact working state. 18:20:33 <oddshocks> pingou: ah that makes sense. 18:20:44 <oddshocks> nirik: right, so yeah just installing the fedimg rpm and putting fedimg.cfg in /etc/ 18:20:54 <pingou> nirik: are we doing cloud on kvm? :) 18:20:55 <oddshocks> and likely starting the fedimg relay if it doesn't do so already 18:21:10 <oddshocks> nirik: I'll also ping you soon this week about internal openstack uploading if you'd like 18:21:13 <pingou> oddshocks: sounds nice and simple, shouldn't be too hard to do with ansible :) 18:21:17 <nirik> oddshocks: sounds good. 18:21:24 <nirik> pingou: our cloud uses kvm, yes? 18:21:55 <mirek-hm> yes, OS use kvm 18:21:56 <pingou> nirik: I meant more our stg/prod boxes, we're always "ready" to turn one down and rebuild one, sounds cloudy :) 18:22:16 <nirik> pingou: kinda, but more persistent I think... 18:22:29 <pingou> hopefully :) 18:22:35 <nirik> and more decentralized 18:22:48 <pingou> that's very true 18:23:06 <nirik> any other application news? 18:23:24 <nirik> I'm gonna try and ping abompard next week when he's back about moving more forward on hyperkitty 18:23:35 * pingou looks forward 18:23:42 <nirik> #topic Sysadmin status / discussion 18:23:50 <nirik> lets see... 18:24:19 <nirik> I got bochecha's pkgs migration to ansible commited and made a new pkgs01.stg 18:24:31 <nirik> There's more work we need to do on it, but I think it's coming along nicely. 18:24:53 <nirik> #info pkgs01.stg now in ansible and running rhel7 18:25:08 <pingou> w/ it we're moving to gitolite3 18:25:39 * nirik nods. 18:25:40 <pingou> is that the remaining work you were thinking about or are there also other things? 18:25:52 <nirik> #info fixed a bunch of hosts monday that are still in puppet that were stuck. ;( 18:26:01 <nirik> yeah, mostly gitolite3 and lots of testing. 18:26:08 <pingou> +1 on testing 18:26:29 <nirik> cgit isn't working right for some reason yet. 18:26:30 <pingou> note: the gitolite configuration cron/task isn't on (or working) atm 18:27:02 <nirik> yeah. 18:27:25 <nirik> So, given the freeze we can work on stg stuff and documentation and planning. ;) 18:27:30 <pingou> \ó/ 18:27:39 <nirik> #topic nagios/alerts recap 18:27:40 <mirek-hm> I proceed with Openstack installatation. networking is hopefully fixed (although after reset it need manual intervention via console :( ) and I'm currently fighting with one problem in packstack. 18:27:59 <nirik> mirek-hm: great. ;) so it's getting close? 18:28:05 <mirek-hm> yes 18:28:19 <pingou> I've been working with releng to drop the pkgdb2branch script 18:28:20 <mirek-hm> it is my priority now 18:28:59 <nirik> awesome. we still need to hook in storage... but hopefully that won't be too bad 18:29:12 <pingou> the idea is to replace that script by another that checks all git for their branches vs the branches they should have according to pkgdb and adjust the git accordingly 18:29:30 <pingou> so that's one more thing changing with the move of pkgs01 to el7 :) 18:30:06 <nirik> .tiny https://admin.fedoraproject.org/nagios/cgi-bin//summary.cgi?report=1&dis... 18:30:14 <zodbot> nirik: http://tinyurl.com/m3v3a5x 18:30:59 <nirik> most of our alerts continue to be network related... but I've found that it's likely openvpn getting saturated. 18:31:13 <nirik> I've not isolated what traffic is causing it yet, but will keep looking. 18:31:21 <pingou> threebean: and I are thinking about giving a trial for datanommer/datagrepper on a mongodb backend, that might help with datagrepper's response time 18:32:17 <nirik> ok. I wonder also if moving that db to rhel7 and newer postgres could help 18:32:29 <pingou> could be also 18:33:17 <nirik> #topic Upcoming Tasks/Items 18:33:17 <nirik> https://apps.fedoraproject.org/calendar/list/infrastructure/ 18:33:26 <nirik> anything upcoming anyone would like to schedule or note? 18:34:00 <smooge> not from me 18:34:10 <threebean> pingou: yeah, we should really follow up with yograterol on that. he has actual experience with mongo and is interested in helping with FI 18:34:57 <nirik> we should make sure we know what we are getting into before we move to it, but trying things seems like a great first step . 18:35:30 <pingou> threebean: +1 but I haven't seen him on irc recently :( 18:36:03 <nirik> #topic Open Floor 18:36:10 <nirik> anyone have anything for open floor? 18:36:20 <nirik> questions, ideas, favorate cartoons? 18:37:38 <oddshocks> I like The Boondocks 18:37:42 <lanica> noc01/nagios mod_auth_openid issue -- looks to be a cobination of a bug in mod_auth_openid and libopkele. One patch, already in existance for mod_auth_openid fixes part of the issue. Once I can find some time, I can do some more testing and validate the fix for libopkele. I probably need to submit a bug report for libopkele if it does not already exist. 18:37:50 <lanica> DuckTales perhaps? 18:38:10 <nirik> lanica: awesome. Thanks for tracking that down... it's an anoying bug. ;) 18:38:17 * nirik likes bob's burgers. 18:39:07 <nirik> alright, will close out in a minute if nothing more. 18:39:49 <nirik> Thanks for coming everyone! 18:39:52 <nirik> #endmeeting