So - as we noodle around with cloud instances more the most obvious problem I have seen is getting a list of instance ids like:
i-F7AA3F96 i-77B34039 i-B1EB403D i-2C294684
and then trying to figure out which ones are jenkins slaves, the torrent seed test and the fedocal instance. And which can be nuked safely or not.
I'm thinking we need a tool that would poll the cloudlet(s), retrieve all the basic, available, info about the running instances.
Then admins could either add metadata to any given instance id to know whence things come.
Data I'd be interested in having: - who owns it - not just the account/tenant - what it is for - expected expiration date (if any) - who should have access to it (usernames from fas and or group names from fas, ideally) - this will make keeping ssh keys on it somewhat sane - what, if any, configuration script was run on it (eg: an ansible playbook) - published urls and where they should alias from?
Now we probably also need something that keeps a list of persistent instances we should always restart and register them.
for example: let's say we want one instance always running as a simple webserver - maybe as a touchstone to verify the cloud is always working. So we should be able to register this instance. Say which img it should use, what security group, etc and note that it should ALWAYS be running. Then when that instance is running its instance id/public ip should be registered in the db listed above.
We can use the data in the db to generate aliases, perhaps.
still fleshing out these ideas.
-sv
On Tue, 16 Oct 2012 17:02:00 -0400 (EDT) Seth Vidal skvidal@fedoraproject.org wrote:
So - as we noodle around with cloud instances more the most obvious problem I have seen is getting a list of instance ids like:
i-F7AA3F96 i-77B34039 i-B1EB403D i-2C294684
and then trying to figure out which ones are jenkins slaves, the torrent seed test and the fedocal instance. And which can be nuked safely or not.
I'm thinking we need a tool that would poll the cloudlet(s), retrieve all the basic, available, info about the running instances.
I agree.
Then admins could either add metadata to any given instance id to know whence things come.
Data I'd be interested in having:
- who owns it - not just the account/tenant
Ideally I'd like an email address we can use to contact the responsible party. Or perhaps a primary and a backup.
- what it is for
- expected expiration date (if any)
- who should have access to it (usernames from fas and or group names
from fas, ideally) - this will make keeping ssh keys on it somewhat sane
- what, if any, configuration script was run on it (eg: an ansible
playbook)
If this thing ends up having a database, perhaps we could just store history on it? when started, what playbooks, etc. We might also be able to get some accounting data... BW used, etc.
- published urls and where they should alias from?
Now we probably also need something that keeps a list of persistent instances we should always restart and register them.
for example: let's say we want one instance always running as a simple webserver - maybe as a touchstone to verify the cloud is always working. So we should be able to register this instance. Say which img it should use, what security group, etc and note that it should ALWAYS be running. Then when that instance is running its instance id/public ip should be registered in the db listed above.
Yeah, I think that would be nice/good.
We can use the data in the db to generate aliases, perhaps.
still fleshing out these ideas.
yeah, there's a ton of ways we could work things. I think we are just going to have to feel them out and adjust over time as we get better at seeing whats where and how we can use it.
kevin
On Tue, Oct 16, 2012 at 05:02:00PM -0400, Seth Vidal wrote:
I'm thinking we need a tool that would poll the cloudlet(s), retrieve all the basic, available, info about the running instances.
Unfortunately, neither Eucalpytus or OpenStack support 'tags' in the EC2 api. That wouldn't solve the entire problem, but it's the right way to do part of it.
http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/Using_Tags.html
I bring this up not just to be annoying :) but wanted to make sure you're aware of it when designing anything. (I think practically that means: store things as key value pair per resource, and try to keep below Amazon's arbitrary limit of 10 tags per resource.)
On Thu, 18 Oct 2012, Matthew Miller wrote:
On Tue, Oct 16, 2012 at 05:02:00PM -0400, Seth Vidal wrote:
I'm thinking we need a tool that would poll the cloudlet(s), retrieve all the basic, available, info about the running instances.
Unfortunately, neither Eucalpytus or OpenStack support 'tags' in the EC2 api. That wouldn't solve the entire problem, but it's the right way to do part of it.
http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/Using_Tags.html
I bring this up not just to be annoying :) but wanted to make sure you're aware of it when designing anything. (I think practically that means: store things as key value pair per resource, and try to keep below Amazon's arbitrary limit of 10 tags per resource.)
I can say right now that keeping below 10 tags per resource is unlikely to happen.
And I've talked to the euca folks about tags that's not available and unlikely to be in 3.2
-sv
On Thu, Oct 18, 2012 at 06:24:10PM -0400, Seth Vidal wrote:
I bring this up not just to be annoying :) but wanted to make sure you're aware of it when designing anything. (I think practically that means: store things as key value pair per resource, and try to keep below Amazon's arbitrary limit of 10 tags per resource.)
I can say right now that keeping below 10 tags per resource is unlikely to happen.
That's ten _keys_ per resource. You can have many more than ten values. Is that still a problem?
And I've talked to the euca folks about tags that's not available and unlikely to be in 3.2
Yeah, crickets for openstack too.
On Thu, 18 Oct 2012, Matthew Miller wrote:
On Thu, Oct 18, 2012 at 06:24:10PM -0400, Seth Vidal wrote:
I bring this up not just to be annoying :) but wanted to make sure you're aware of it when designing anything. (I think practically that means: store things as key value pair per resource, and try to keep below Amazon's arbitrary limit of 10 tags per resource.)
I can say right now that keeping below 10 tags per resource is unlikely to happen.
That's ten _keys_ per resource. You can have many more than ten values. Is that still a problem?
Ultimately, yes - and of course my other problem is I need this data available ACROSS reboots/rebuilds of the cloud cluster.
For our persistent instances plan I need to be able to know that I need to bring up 10 instances as m1.xlarge. Instance 1 needs to get this ip: allocate it accordingly, instance 1 needs this volume attached, etc, etc.
See where I'm going? If the instance is down that's not very functional
Yeah, crickets for openstack too.
maybe cloudstack? David was telling me about their options.
-sv
On Thu, Oct 18, 2012 at 11:57:26PM -0400, Seth Vidal wrote:
For our persistent instances plan I need to be able to know that I need to bring up 10 instances as m1.xlarge. Instance 1 needs to get this ip: allocate it accordingly, instance 1 needs this volume attached, etc, etc. See where I'm going? If the instance is down that's not very functional
Yeah. It's just particularly nice because people could easily tag their own instances with euca-create-tags.
Have you looked at Scalr? It might be overkill here -- I don't know how well it scales _down_.
On 2012-10-18 15:24, Seth Vidal wrote:
On Thu, 18 Oct 2012, Matthew Miller wrote:
Unfortunately, neither Eucalpytus or OpenStack support 'tags' in the EC2 api. That wouldn't solve the entire problem, but it's the right way to do part of it.
http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/Using_Tags.html
I bring this up not just to be annoying :) but wanted to make sure you're aware of it when designing anything. (I think practically that means: store things as key value pair per resource, and try to keep below Amazon's arbitrary limit of 10 tags per resource.)
I can say right now that keeping below 10 tags per resource is unlikely to happen.
And I've talked to the euca folks about tags that's not available and unlikely to be in 3.2
Indeed. It is slated for 3.3, AFAIK.
If these tags need to persist longer than instances do then resource tags on individual instances probably aren't the right solution anyway -- they evaporate when the instances do. :-\ Tags on auto-scaling groups can be more useful, but that isn't going to land until eucalyptus 3.3 either.
infrastructure@lists.fedoraproject.org