With F19 almost out the door, it's time to shift attention to the next thing!
Here's what I'm thinking about right now; would love to hear your opinions and advice here, and also about other things in the cloud arena which we should be working on.
Cloud Image Generation ====================== - finally getting this automated weekly in koji, to a download site and to EC2. See notes for this at https://fedoraproject.org/wiki/Features/FirstClassCloudImages/KojiPlan - move to Oz (or other fully anaconda-based image generation)
Improving the Cloud Image ========================= - cloud-util improvement - subpackaging growpart means we don't need to pull in huge deps (bz #975497) - cloud-init improvements - can we remove the cheetah dependency? (bz #974327) - other dependency softening (eg bz #974329, bz #786488) - disable direct writes to the console, which will address several problems (bz #977952) - others? - disable the gettys by default? - look at the logging configuration (rsyslog/journald double-logging issue; look at memory and on-disk journald consumption) - more reductionism as reasonable -- maybe we can look at tackling the docs, translations, and kernel-modules size issues? burn sendmail with fire? get systemd-journal-gatewayd out of the image so it stops pulling in the libmicrohttpd web server and crypto libs which wouldn't be there otherwise (bz #908081)
Vagrant ======= - We're blocked from having "real" vagrant base boxes by a lack of Chef, which is blocked by bz #823351; hope to have that resolved and Chef as an F20 feature - Also, Vagrant is very Virtualbox-centric, and we don't have that in Fedora (out of tree kernel modules); work on a KVM provider for Vagrant should make that a non-issue -- https://github.com/adrahon/vagrant-kvm
Cloud Infrastructure Software ============================= - OpenNebula?
On 2013-06-26, Matthew Miller mattdm@fedoraproject.org wrote:
Improving the Cloud Image
- disable the gettys by default?
Well, start at least one, for those of us using these images outside of EC2 environments where we may actually have console access.
It would be nice if we could enable console output by default if that's not already happening (not running a getty on the console; just ensuring that kernel messages go out the serial port if one is available).
- look at the logging configuration (rsyslog/journald double-logging issue; look at memory and on-disk journald consumption)
I would love to see a systemd-only base configuration for cloud images. If I want remote logging I'll install rsyslog as part of the initial deployment.
----- Original Message -----
From: "Lars Kellogg-Stedman" lars@oddbit.com To: cloud@lists.fedoraproject.org Sent: Wednesday, June 26, 2013 11:50:51 AM Subject: Re: Fedora 20 cloud plans?
On 2013-06-26, Matthew Miller mattdm@fedoraproject.org wrote:
Improving the Cloud Image
- disable the gettys by default?
Well, start at least one, for those of us using these images outside of EC2 environments where we may actually have console access.
It would be nice if we could enable console output by default if that's not already happening (not running a getty on the console; just ensuring that kernel messages go out the serial port if one is available).
- look at the logging configuration (rsyslog/journald double-logging issue; look at memory and on-disk journald consumption)
I would love to see a systemd-only base configuration for cloud images. If I want remote logging I'll install rsyslog as part of the initial deployment.
It was mentioned elsewhere that Matt/Garrett had a thought to have systemd journal logs output to syslog and console; do you mean something like that, or do you just mean straight-up, no rsyslog?
(Warning: uneducated-about-logging comments start here)
My only question about doing that as the standard is whether or not many of the log analyzers or monitoring systems out there actually have journal logs as an input (for lack of a better term). TBH - I don't really know if "journal logs as an input" is an issue; I know there are plugins for journal like journal2gelf, but there are a zillion other things out there where I don't know how things would function (log4j 2, logstash, loggly, splunk, nagios, etc).
While I agree that it's good to push people towards the new technologies in Fedora - I think there's a fine line in situations like this where we want to make sure that we're at least being aware of end-user surrounding infrastructure and making sure other projects are going to be compatible.
But like I said, I am rather uneducated in this area so I could be worrying for nothing.
All that said: I did see a post from Johann on G+ about rsyslog gaining systemd journal support: https://plus.google.com/107796074242209146626/posts/ZhfYBR6b1cv
(Short version: rsyslog 7.4.0 released and gains systemd journal support
* omjournal writes messages to the journal * imjournal obtains messages including journal-specific meta data items from the journal)
... which also points to this rsyslog vs. journal paper which might be good food for thought: http://blog.gerhards.net/2013/05/rsyslog-vs-systemd-journal.html
-robyn
-- Lars Kellogg-Stedman lars@oddbit.com
cloud mailing list cloud@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/cloud
On 2013-06-28, Robyn Bergeron rbergero@redhat.com wrote:
I would love to see a systemd-only base configuration for cloud images. If I want remote logging I'll install rsyslog as part of the initial deployment.
It was mentioned elsewhere that Matt/Garrett had a thought to have systemd journal logs output to syslog and console; do you mean something like that, or do you just mean straight-up, no rsyslog?
I was referring to straight-up, no rsyslog. I've found that with the adoption of systemd/journald that I use journalctl much more often than "grep on logfiles", since journalctl has (a) access to the structured data collected by journald and (b) provides a rich set of query operations for filtering messages.
For the use case of "I just a need a simple cloud image" I'm not sure that rsyslog provides much value add. For the use case of "I'm deploying hundreds of nodes in the cloud and I need an integrated logging solution", I'm not going to be using the stock rsyslog configuration in any case, so I don't lose much by having to explicitly install it in addition to configuring it. And even in this case I may only want it for remote logging.
But like I said, I am rather uneducated in this area so I could be worrying for nothing.
I think you've raised a number of good points, and I'm not really pushing hard for ditching rsyslog. In my world it's not necessary, but I don't actually know how much my use cases intersect with the rest of the community.
On 06/26/2013 11:50 PM, Matthew Miller wrote:
With F19 almost out the door, it's time to shift attention to the next thing!
Here's what I'm thinking about right now; would love to hear your opinions and advice here, and also about other things in the cloud arena which we should be working on.
Cloud Image Generation
- finally getting this automated weekly in koji, to a download site and to EC2. See notes for this at https://fedoraproject.org/wiki/Features/FirstClassCloudImages/KojiPlan
- move to Oz (or other fully anaconda-based image generation)
Yes, it'd be nice to see Oz, I've been using it for a while, and can help here in any little way I can.
Improving the Cloud Image
- cloud-util improvement
- subpackaging growpart means we don't need to pull in huge deps (bz #975497)
- cloud-init improvements
- can we remove the cheetah dependency? (bz #974327)
- other dependency softening (eg bz #974329, bz #786488)
- disable direct writes to the console, which will address several problems (bz #977952)
- others?
- disable the gettys by default?
- look at the logging configuration (rsyslog/journald double-logging issue; look at memory and on-disk journald consumption)
- more reductionism as reasonable -- maybe we can look at tackling the docs, translations, and kernel-modules size issues? burn sendmail with fire? get systemd-journal-gatewayd out of the image so it stops pulling in the libmicrohttpd web server and crypto libs which wouldn't be there otherwise (bz #908081)
Some of the above indeed looks worth investigating. I'll see what I can do.
Thanks for writing this all up.
----- Original Message -----
From: "Matthew Miller" mattdm@fedoraproject.org To: "Fedora Cloud SIG" cloud@lists.fedoraproject.org Sent: Wednesday, June 26, 2013 11:20:55 AM Subject: Fedora 20 cloud plans?
With F19 almost out the door, it's time to shift attention to the next thing!
Here's what I'm thinking about right now; would love to hear your opinions and advice here, and also about other things in the cloud arena which we should be working on.
Cloud Image Generation
- finally getting this automated weekly in koji, to a download site and to EC2. See notes for this at https://fedoraproject.org/wiki/Features/FirstClassCloudImages/KojiPlan
- move to Oz (or other fully anaconda-based image generation)
Improving the Cloud Image
- cloud-util improvement
#975497)
- subpackaging growpart means we don't need to pull in huge deps (bz
- cloud-init improvements
- can we remove the cheetah dependency? (bz #974327)
- other dependency softening (eg bz #974329, bz #786488)
- disable direct writes to the console, which will address several problems (bz #977952)
- others?
- disable the gettys by default?
- look at the logging configuration (rsyslog/journald double-logging issue; look at memory and on-disk journald consumption)
- more reductionism as reasonable -- maybe we can look at tackling the docs, translations, and kernel-modules size issues? burn sendmail with fire? get systemd-journal-gatewayd out of the image so it stops pulling in the libmicrohttpd web server and crypto libs which wouldn't be there otherwise (bz #908081)
Vagrant
- We're blocked from having "real" vagrant base boxes by a lack of Chef, which is blocked by bz #823351; hope to have that resolved and Chef as an F20 feature
I talked to Sam Kottler about this a bit over the weekend when I saw him and I didn't exactly get the warm fuzzies about upstream movement here; maybe he can elaborate? :\ (But chef & real vagrant boxes would be awesome)
- Also, Vagrant is very Virtualbox-centric, and we don't have that in Fedora (out of tree kernel modules); work on a KVM provider for Vagrant should make that a non-issue -- https://github.com/adrahon/vagrant-kvm
To be clear: This is so that we can use vagrant boxes (of any OS flavor?) on a Fedora desktop, yes? Not an issue with "have a Fedora vagrant box to open on other Linux distros or mac" ?
Cloud Infrastructure Software
- OpenNebula?
Sure. I know Shawn Starr was working on this last year, and Jaime Melis had mailed the list a few times about it; the packaging got somewhere, though had some issues - https://bugzilla.redhat.com/show_bug.cgi?id=815001
I was also going to mention OwnCloud - which had been targeted for F18, was dropped, and now I see a F19 feature page for it, and it says it's 100% done, but isn't in the feature list (???) - http://fedoraproject.org/wiki/Features/OwnCloud - I'm guessing it didn't make the feature list since it doesn't seem to have been moved into the feature wrangler queue (still tagged as FeaturePageIncomplete) ... which is kind of a bummer since it would have been nice to feature. But is probably worthwhile adding at least to the cloud sig wiki that it's there.
-r
-- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mattdm@fedoraproject.org _______________________________________________ cloud mailing list cloud@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/cloud
On Fri, Jun 28, 2013 at 06:53:02AM -0400, Robyn Bergeron wrote:
- We're blocked from having "real" vagrant base boxes by a lack of Chef, which is blocked by bz #823351; hope to have that resolved and Chef as an F20 feature
I talked to Sam Kottler about this a bit over the weekend when I saw him and I didn't exactly get the warm fuzzies about upstream movement here; maybe he can elaborate? :\ (But chef & real vagrant boxes would be awesome)
He seemed more optimistic when I talked to him -- plan was to remove the need for YAJL, the blocking Ruby bit in the bug mentioned above, from Chef. Maybe that didn't go as smoothly as hoped?
- Also, Vagrant is very Virtualbox-centric, and we don't have that in Fedora (out of tree kernel modules); work on a KVM provider for Vagrant should make that a non-issue -- https://github.com/adrahon/vagrant-kvm
To be clear: This is so that we can use vagrant boxes (of any OS flavor?) on a Fedora desktop, yes? Not an issue with "have a Fedora vagrant box to open on other Linux distros or mac" ?
Right, these are not interdependent issues, but it's nice to be able to use the thing we're making on on Fedora itself.
Note that the vagrant box _has_ to drift from the generic image, because the recommended configuration is to have an account name, account password, and root password of "vagrant".
cloud@lists.stg.fedoraproject.org
-
Kashyap Chamarthy -
Lars Kellogg-Stedman -
Matthew Miller -
Robyn Bergeron