infrastructure June 2014

infrastructure@lists.fedoraproject.org

36 participants
24 discussions

Introduction - Ethan Hussong
by Ethan Hussong 22 Jun '14

22 Jun '14

Hello all. I've been lurking in IRC and digging through some of the publicly available Fedora pages for the last couple weeks in attempt to familiarize myself with the group and the environment and thought that it was the right time to introduce myself. My name is Ethan Hussong and my IRC nick is "Xirrin". I'm in the Chicago suburbs which puts me in the Central timezone in the US. I have recently made a transition over the last few years into IT and have found myself often excited about Linux and Open Source even though I am not a programmer. I would say that my Linux skills are probably around an RHCSA-level (I've been lightly pondering taking the test for the last couple of weeks). Most of my IT experience has come in the form of beginner level network and system administration on top of the help desk jobs that I have worked that have included system performance monitoring and basic remediation. I've attached a resume as well for the sake of helping folks understand my background. As I mentioned I have some simple network and systems administration experience. I have experience building PCs as well as enterprise servers and have a small test lab of my own at home. I would like to join the Fedora Infrastructure group primarily because of my excitement for the project. I have great respect for Red Hat as far as what it provides to the world and the IT community as a whole (standards, leadership in Linux, a face for Open Source, a great platform to share with both individuals as well as companies, the list goes on...). Fedora plays a significant part in the RHEL ecosystem and because of that, despite my meager credentials, I would very much like to contribute to the cause. My hope is that going forward I will be able to start small and grow within the community. As my current job is working in a NOC it seems most appropriate to start with system monitoring and trouble remediation tasks. Additionally it seems as though there may be some opportunities to clean up some of the Nagios configuration with tweaking some alert triggers as well as making sure all of the infrastructure is actually monitored (based on what I've seen from conversation the #fedora-noc channel). I would eventually like to help with the Ansible work that is being done and hope that I can eventually help with server deployment, configuration, and decommissioning as my skills expand. Currently I have an interesting work schedule that gives me a couple of hours of free time on one week and a large amount of free time the next. My availability on a specific week is irregular, but ranges between 5 and 25 hours depending on my schedule. All that being said I am looking for a sponsor/mentor here within the FIG and hope that someone may be willing to step up and guide me to my first steps within the group. Thank you! Ethan Hussong <http://protocol.by/ehussong>

1 0

Hello World
by Hans-Peter Romfeld 20 Jun '14

20 Jun '14

Hey, EXP: Monitoring: nagios. cacti, smokeping, logstash, graphite, shinken (want to go deeper into snmp and graphical monitoring, mostly doing event_handling if some service doesnt work right now) Configuration management: puppet (interest to learn also chef) Backup: mainly script to s3 with gpg encryption, a little bit amanda, Veeam Versioning: git, svn (want to lern webDAV) Usermgmt: windowsAD, openldap (want to lern Radius, kerberus and more) Firewalls: CISCO ASA virtualization: VMware, VirtualBox (highly interested into XEN lerning) Mail: Zimbra, iRedMail Clustering: Pacemaker (interested in everythin here othen then windows) Cloud: AWS (want to lern urgently openstack) Datastore: MySql, Couchbase, Mongo, NDBCLUSTER (i am huge fan of non-sql since i touched it) CI: jenkins/pylint/nose/.. python a little bit (huge interest to learn this more) ManagementSoftware: openERP, redmine, requesttracker, bugzilla web: nginx, apache, some loadbalancing(AWS more then some) scripting: bash, python python frameworks: django, cherrypy, falcon, ...(had to do some basic webdev for app api's....) other: DNS config(want also lern howto setup my own DNS server), Vlan/SECURITY MANAGEMENT, much about windows(dont want to touch windows much again: linux: wget http://www.domain.com/myCA.pem mac: curl -LH http://www.domain.com/myCA.pem windows: (new-object System.Net.WebClient).DownloadFile('http://www.domain.com/myCA.pem','C:\tmp\rootCA.pem').... maybe something more i didnt think about yet.. I learn fast and i am very interested, i work not even 2 years in IT and have no education background ;) Cheers, Peter

1 0

Meeting Agenda Item: Introduction Libert Schmidt (resend)
by Libert R Schmidt 20 Jun '14

20 Jun '14

(sorry about the other email) IRC: lrsz Skills: Proactive Curious Work Pressure troubleshooting I would like to learn how the Fedora community works to make possible the Fedora project up to running and I also want to expand my programming and my Linux System administrator Skills.

1 0

Meeting Agenda Item: Introduction Libert Schmidt
by Libert R Schmidt 20 Jun '14

20 Jun '14

- IRC: lrsz - Skills: - Proactive - What you would like to work on and quite possibly which outstanding issues <https://fedorahosted.org/fedora-infrastructure/report/1> you would like to help resolve - After you send your "Hello World!" to the Fedora Infrastructure Group a greeter should respond to you on the mailing list offering to help you get started. But don't feel like you have to wait for this message, you can also join the #fedora-admin channel on IRC and ask for a greeter to help you out!

1 0

Meeting Agenda Item: Introduction Phillip T. George
by Phillip T. George 20 Jun '14

20 Jun '14

Hey everybody. I'm looking to join the Fedora project to give something back to the community. I've been a Linux user for quite awhile, and Linux has been a big part of a career. Previous to my current role I was a Linux sysadmin/engineer for a couple of years. My client's environment is made up RHEL (75%), Solaris (<5%), and Windows (~20%). The total server count is around 5-6K -- that does include some virtuals, but it is primarily physicals. I definitely work for a large corporation, but I'm not here to represent them in any way at this point in time. I have been using Linux distros for nearly 20 years (probably since around 1997). My current role involves automation of existing processes and tools. Its a lot of trying to introduce people to common scripting practices as well as making systems communicate that previously did not (often through an orchestration engine). I have a lot of skills/knowledge/experiences, some significant, some not so much. Programming/scripting: PHP (w/ MySQL), JS, HTML, CSS, jQuery (mostly for UI), bash, C/C++, Java, Objective C, Perl OSes: Fedora, RHEL, Mac OS X, Ubuntu, Debian, Red Hat (pre-enterprise), Windows (server 2008, server 2003, 8, 7, XP, and older OSes) Projects: 1. Fedora customized build for Linux sysadmins/engineers at work (now its a retrofit script) 2. PHP-based front-end for a change management tool -- uses an orchestration engine to do the work, though all of the orchestration steps had to be created (SOAP based in this case, though there were some pre-fabricated operations that helped with this a bit) 3. PHP-based server build/configuration tool -- just for Linux but planning on using it for other OSes eventually. Has multiple levels/objects you can configure: Server, Profile (multiple), OS (w/ hierarchy), and Site/Location 4. PHP-based report/data customized viewer -- a perl script imports a massive CSV to allow updating of the data without having to push around CSV/spreadsheet files 5. iPhone note sharing app 6. Lots of miscellaneous partially finished projects -- mostly game development projects Unfortunately these projects are not open source, so I cannot share them with you. Though the iPhone note sharing app is not open source, I have full rights to it, though I doubt anyone here is interested in that :) Obviously I have many interests. The best summary of my interests would be: computer-based technology. A more focused summary would be: Linux and programming. Today, I am interesting in doing some sysadmin/engineer and possibly web development work. Tomorrow (figuratively speaking), it may be more than that. I'm willing to provide up to 5 hours a week of my time. As far as what I would specifically like to work on -- I have no idea. Right now, I'm hear to listen, learn, and absorb. Eventually I'll find something. I will be browsing the open items to see if there is indeed something I can try to solution, though not yet knowing much details of the infrastructure makes that somewhat difficult. A quick look at the Trac tickets reveals that some cleanup of the tickets might be needed. There are some items open since 2007, which implies they may no longer be valid (unless there is a review process going on to ensure they are indeed valid). IRC: lanica -Phillip

1 0

Plan for tomorrow's Fedora Infrastructure meeting (2014-06-19)
by Kevin Fenzi 19 Jun '14

19 Jun '14

The infrastructure team will be having it's weekly meeting tomorrow, 2014-06-19 at 18:00 UTC in #fedora-meeting on the freenode network. Suggested topics: #topic New folks introductions and Apprentice tasks. If any new folks want to give a quick one line bio or any apprentices would like to ask general questions, they can do so in this part of the meeting. Don't be shy! #topic Applications status / discussion Check in on status of our applications: pkgdb, fas, bodhi, koji, community, voting, tagger, packager, dpsearch, etc. If there's new releases, bugs we need to work around or things to note. #topic Sysadmin status / discussion Here we talk about sysadmin related happenings from the previous week, or things that are upcoming. #topic nagios/alerts recap Here we go over the last weeks alerts and see if we can find ways to make it so they don't happen again. #topic Upcoming Tasks/Items https://apps.fedoraproject.org/calendar/list/infrastructure/ #topic Open Floor Submit your agenda items, as tickets in the trac instance and send a note replying to this thread. More info here: https://fedoraproject.org/wiki/Infrastructure/Meetings#Meetings Thanks kevin

1 1

Re-enable Greek Language on ask.fedoraproject.org
by NikTh 17 Jun '14

17 Jun '14

May I ask you to enable (re-enable) the Greek Language support on askfedoraproject.org ? It was disabled because of poor participation, but we would like to be enabled (again) and thus any local users be able to post any questions/answers on relevant section(Greek) in their own language. Relevant link of recent post in Greek Language[1]. Also read the comments. [1] https://ask.fedoraproject.org/en/question/48968/to-fedora-20-upostirizetai-… Best Regards NikTh

2 2

Plan for tomorrow's Fedora Infrastructure meeting (2014-06-12)
by Kevin Fenzi 12 Jun '14

12 Jun '14

The infrastructure team will be having it's weekly meeting tomorrow, 2014-06-12 at 18:00 UTC in #fedora-meeting on the freenode network. Suggested topics: #topic New folks introductions and Apprentice tasks. If any new folks want to give a quick one line bio or any apprentices would like to ask general questions, they can do so in this part of the meeting. Don't be shy! #topic Applications status / discussion Check in on status of our applications: pkgdb, fas, bodhi, koji, community, voting, tagger, packager, dpsearch, etc. If there's new releases, bugs we need to work around or things to note. #topic Sysadmin status / discussion Here we talk about sysadmin related happenings from the previous week, or things that are upcoming. #topic nagios/alerts recap Here we go over the last weeks alerts and see if we can find ways to make it so they don't happen again. #topic Upcoming Tasks/Items https://apps.fedoraproject.org/calendar/list/infrastructure/ #topic Open Floor Submit your agenda items, as tickets in the trac instance and send a note replying to this thread. More info here: https://fedoraproject.org/wiki/Infrastructure/Meetings#Meetings Thanks kevin

1 1

pkgdb2 post-mortem and strategy for future deployments
by Toshio Kuratomi 06 Jun '14

06 Jun '14

This came up in a different venue and pingou and I have continued to talk about it. Seemed that this was the right place to bring the discussion though. Some observations: * Pkgdb2 and a call for testing in staging was announced well in advance of the deployment to production (good) but not everyone understood that we were going to be breaking API (bad). * There were people inside of fedora infrastructure and outside of infrastructure who were surprised by the API break. There were also some community members and infrastructure members who heeded the call for testing and both gave feedback and ported before the deployment. * There was a FAS2 update that pkgdb2 depended upon. That was also pending in stg for a long time and also had some minor API changes (IIRC, all unintentional. I hotfixed one of them that was simply a bug last week). These also caused issues for some scripts. * Unexpected problems: we had things that we didn't know used the pkgdb API, things that weren't tested in stg because stg couldn't replicate that part of production, and things that were ported but mistakes caused the ported scripts to not be deployed or to point at stg instead of production. I saw that we had the right people on IRC throughout the day working on analyzing and patching all of the broken things so. However, this was somewhat by accident and some of those people were surprised that they spent their day doing this. Some ideas for doing major deployments in the future: 1: We have to make people aware when a new deployment means API breaks. * Be clear that the new deployment means API breaks in every call for testing. Send announcements to infrastructure list and depending on the service to devel list. * Have a separate announcement besides the standard outage notification that says that an API breaking update is planned for $date * When we set a date for the new deployment, discuss it at least once in a weekly infrastructure meeting. * See also the solution in #3 below 2: It would be really nice for people to do more testing in stg. * Increase rube coverage. rube does end-to-end testing so it's better at catching cross-app issues where API changes better than unittests which try to be small and self-contained - A flock session where everyone/dev in infra gets to write one rube test so we get to know the framework * Run rube daily - Could we run rube in an Xvfb on an infrastructure host? * Continue to work towards a complete replica of production in the stg environment. 3: "Mean time to repair is more important than mean time between failure." It seems like anytime there's a major update there's unexpected things that break. Let's anticipate the unexpected happening. * Explicitly plan for everyone to spend their day firefighting when we make a major new deployment. If you've already found all the places your code is affected and pre-ported it and the deployment goes smoothly then hey, you've got 6 extra working hours to shift back to doing other things. If it's not smooth, then we've planned to have the attention of the right people for the unexpected difficulties that arise. * As part of this, we need to identify people outside of infrastructure that should also be ready for breakage. Reach out to rel-eng, docs, qa, cvsadmins, etc if there's a chance that they will be affected. 4: Related to the FAS release: Buggy code happens. How can we make it happen less? * More unittests would be good however we know from experience with bodhi that unittests don't catch a lot of things that are changes in behaviour rather than true "bugs". Unexpected API changes that cause people porting pain can be as simple as returning None instead of an empty list which causes a no-op iteration in running code to fail while the unittests survive because they're checking that "no results were returned". * Pingou has championed making API calls and WebUI calls into separate URL endpoints. I think that coding style makes it easier to control bugs related to updating the webui while trying to preserve the API so we probably want to move to that model as we move onto the next major version of our apps. * Not returning json-ified versions of internal data structures (like database tables) but instead parsing the results and returning a specific structure would also help divorce internal changes from external API. What should we apply this to? * Probably can skip if: - Things that we don't think have API breaks - Things that are minor releases (hopefully these would correlate with not having API breaks :-) - Leaf services that are not essential to releasing Fedora. + ask, nuancier, elections, easyfix, badges, paste, nuancier + There's a lot of boderline cases too -- is fedocal essential enough to warrant being under this policy? Since the wiki is used via its API should that fall under this as well? Comments, thoughts, other ideas? Do we need to "ratify" something like this at a meeting? What's the next app deploy where we'll want to enact this? Maybe bodhi2 ;-)? -Toshio

4 4

Code review on kerneltest-harness
by Pierre-Yves Chibon 06 Jun '14

06 Jun '14

Hi all, Justin and I have been working on a small application gathering logs of kernel tests and giving the kernel maintainers some stats about these results The idea is that there is a script running tests on a kernel. That script is ran for every kernel built by the kernel team but also anyone can run the tests and also submit their results (with a final idea to provide badges to the users that do). There are three possibilities to submit one's results - Via the UI of the application, just login, select the `upload` tab at the top and upload your result file - Via curl or any other tool that can make POST request - either with an API key that is kept secret This API key is used by the automatic testing tool used by the kernel team, it ensures these results are trusted - either via a public endpoint. This endpoint support both anonymous and logged-in (via openid) uploads The challenge is of course about allowing anonymous upload while not giving too much space for people to abuse it. So we are restricting the uploads by their mime type and size (10Kb by default). The mime type is however something that is easy to circumvent but at least that means they'll check the sources. (I was just thinking, maybe we could enforce a dedicated mime type for the CL upload) Also, any file that do no contain the expected pattern will be discarded. I am fairly confident about the application but I would not mind more eyes looking at the code and checking if we missed obvious solutions to limit the potential angles of attack. For those interested: The code is at: https://github.com/jmflinuxtx/kerneltest-harness (see the frontend folder) A sample input file is at: http://paste.fedoraproject.org/107782/02061911 Thanks in advance for your help :) Pierre

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure June 2014