openlmi-devel February 2013

openlmi-devel@lists.fedorahosted.org

10 participants
8 discussions

Design of mounting in OpenLMI-Storage
by Jan Safranek 01 Mar '13

01 Mar '13

Hi, we should do some mounting in OpenLMI storage provider and I want to discuss its design. (sorry for HTML, is there any better way how to send such proposals?) * SMASH and DASH do not cover mounting of filesystems. * SMI-S has Filesystem Manipulation Profile, but it's closely tight to filesystem management, for example: o Filesystems are mounted using SNIA_CreateFileSystem method, i.e. it's possible to create & mount a FS in one call, but it's weird to call SNIA_CreateFileSystem to mount existing FS (but it's possible). o Filesystems can be mounted only once. o Only local filesystems can be mounted. CIM provides lot of classes to manage mounts: * CIM_Mount - association between a FS and a directory, where it's mounted. * CIM_FileImportService - service, which mounts remote filesystems (NFS, CIFS) + usual Setting & Capabilities classes There is no service which mounts local filesystems! So, looking at the CIM_FileImportService, we might get inspired by it and create following classes: * LMI_MountService o CreateMount(FS, type, mountpoint, goal, permanent) + FS: reference to either Local or RemoteFileSystem or NULL. + type: uint16, enum (nfs, cifs, xfs, ext2/3/4, vfat, ...). + mountpoint: string (or reference to LMI_Directory?). + goal: reference to LMI_MountSettings or NULL to use default. + permanent: boolean, whether to store in fstab. + if FS is NULL, type can be used to create tmpfs, sysfs, procfs, cgroups, ... + LMI_Mount is created with attached LMI_MountSetting. o Remote file systems are represented by RemoteFileSystem, but how to enumerate them when they are not mounted? We probably don't want to scan the network for available NFS/CIFS shares. We might add a method to create such RemoteFileSystem and mount it using CreateMount(), but I propose following function: o CreateRemoteMount(specifier, type, mountpoint, permanent) + specifier: string like 'foo:/home/jsafrane' for NFS or '//foo/home/jsafrane' for CIFS. + type, mountpoint, goal, permanent: same as above. + Appropriate CIM_RemoteFileSystem gets created. + LMI_Mount is created with attached LMI_MountSetting. o ModifyMount(mount, goal, permanent) + mount: reference to existing LMI_Mount. + goal: reference to new LMI_MountSettings of the mount. + permanent: whether to store this mount in fstab. # if the mount was permanent and is not anymore, it gets removed from fstab. # if the mount was not permanent and is now, it gets added to fstab. # if the mount was permanent and still is, fstab is edited. o DeleteMount(mount) + mount: reference to existing LMI_Mount. o All these methods return Job (mounting can take some time, right?) * LMI_Mount: inherited from CIM_Mount o Association between a CIM_FileSystem (local or remote) and CIM_Directory where it's mounted. o Has property if this mount is permanent (in fstab). o Has property if this mount is active (i.e. currently mounted). o There can be LMI_Mount, which does not exist on the system, but has entry in fstab! + e.g. when admin remounts a mount which is in fstab with different options, there will be two LMI_Mount instances. # One for the fstab mount (inactive, permanent). # One for the actual mount (active, not permanent). # OpenLMI can edit both. # One of the mount instances disappears when fstab and running system gets in sync. + /We must add new key to the association to //distinguish multiple mounts of one fs to the same directory, just with different options and permanent/non permanent flag./ + /There can be only one fstab entry for (device, directory) pair./ o If there is an entry in fstab but the filesystem (or block device) is not available, no LMI_Mount is created! + /How to edit/delete such leftovers? Maybe the device was just renamed./ * LMI_MountSetting o Configuration of a mount. o Has properties for selected mount options + ro + rw + Anything else? We may gradually extend this class and even make subclasses for each FS, e.g. have LMI_Ext2MountSetting, LMI_XFSMountSetting, LMI_NFSMountSetting, ... o Has generic array of strings of 'additional options' for all mount options, which are not covered by specific properties. o Is associated with LMI_Mount to shows its configuration. + /LMI_Mount is association, i.e. we have association between association and MountSetting, which is slightly confusing./ * LMI_MountCapabilities o Capabilities of the MountService, as usual in SMI-S and LMI storage model. UML class diagram This CIM API does not follow any DMTF or SMI-S standard, but follows the idea of it - have a Service, Setting and Capabilities classes and can be extended in the future. Later we still may implement CIM_FileImportService if someone wants, but it would be just a wrapper around our MountService. In Fedora19 I think we can do only 'transient' mounts, no fstab editing, Blivet doesn't support non-destructive fstab manipulation now. In addition, Blivet does support mounting of local filesystems on block devices for now (no cgroups, sysfs, procfs, tmpfs, ...), but even this limited functionality is IMHO quite lot of work on OpenLMI side. Any comments are welcome, especially the parts in /italics/ are quite disturbing to me. And links to any model which already covers mounting would be greatly appreciated. Jan

3 4

What is the best way to model a name/value pair in CIM?
by John Dennis 26 Feb '13

26 Feb '13

In the provider I'm writing lists of key/value pairs are needed in a few different contexts (as a parameter to a method, as an instance property, etc.). In CIM what is the best way to to model a name/value pair? In CIM I dont' think there is any mechanism by which the value can be of an arbitrary type (right?). Therefore lets assume both the name and value are strings. I didn't find much in the existing core schema for modeling name/value. There is the CIM_Configuration class but it's deprecated, replaced by CIM_Setting. But CIM_Setting has semantics inappropriate for a general name/value pair. Is the best approach to define a new class that has just a name and value property? (I can't imagine others haven't had the need for name/value pairs, so shouldn't such a fundamental class be part of the core schema? If so I haven't found it.) Such a class would not need a key property [1] see below for a requested clarification. If the best approach is to define a class with just a name and value property then is the best way to to have a multiple name/values is to define an array of references to the individual name/value instances? There doesn't seem to be any need or advantage to introducing a CIM_Collection for this purpose, or is there? Are there examples of other CIM classes that use sets of name/value pairs which would provide an example of best CIM practice for this? [1] I've been a bit confused by the term "concrete" as used in the context of CIM. One definition I've seen is a concrete class is any non-abstract class. Another definition I saw was a concrete class is an instantiated class instance. And yet a third definition I saw was a concrete class has to have a one or more key properties. So two questions: What is a concrete class? Is defining a class without any key properties problematic if there is no need to uniquely reference a specific class instance? -- John Dennis <jdennis(a)redhat.com> Looking to carve out IT costs? www.redhat.com/carveoutcosts/

3 4

Storage events
by Jan Safranek 25 Feb '13

25 Feb '13

Hello all, I've talked to David Lehman (Blivet devel) how OpenLMI can report various storage events to applications. Currently, Blivet does not consume any events from udev and it would be rather difficult to implement it - Blivet is a library and receiving events means it must have some sort of listening thread with a main loop. 1: http://pyudev.readthedocs.org/en/latest/ So, what events OpenLMI wants to report to applications (and what info it wants) [source of the event]: - device appeared/disappeared [udev] - device was changed (*what* changed - e.g. size, fs on it, RAID level, ...) [udev] - mount appeared/disappeared [/proc/mounts] - mount changed (list of changed options) [/proc/mounts] - RAID events - man mdamd, look for "MONITOR MODE". There might be netlink/inotify/dbus events for that. - LVM events (snapshot/thin LV/VG is running out of space; with configurable threshold) [none at the moment] As written on [2], select or poll on /proc/mounts or /proc/self/mountinfo can be used to get mounting events. 2: http://karelzak.blogspot.cz/2011/12/monitor-list-of-currently-mounted.html Question is, how Blivet and OpenLMI cooperate to generate these events. Option 1: OpenLMI listens for udev and mount and mdadm events (in C, not in Python). When an event occurs, it re-creates Blivet's device tree and compares old and new trees manually on its own. Benefit of this solution is that Blivet (Python) does not need to be in memory for the whole time. This would require some wrapping between CIMOM and OpenLMI, but I think that's doable. Option 2: OpenLMI tells Blivet to wait for events. When event occurs, Blivet calls OpenLMI callback. Add/remove events are simple, just tell OpenLMI what appeared/disappeared. Changes are more complex, but IMHO it might be enough to call the callback with old StorageDevice and new StorageDevice and we'll check what's changed. This solution can be used by more applications on top of Blivet, but it requires Blivet to run a main loop in a thread and be in memory for the whole time. Option 1.5: OpenLMI listens for the udev, mount and mdadm events (in C, not in Python), but passes them in some 'raw' form to Blivet. Blivet then check what's going on and calls OpenLMI callback as in Option 2. We don't need to have Blivet in memory for the whole time, but we need it to be possible to store Blivet state to a file and reload it when requested. Of course, we expect incremental development, let's start with udev first and gradually integrate all other sources of events. I personally vote for option 2, not because it's more work for Blivet and less for OpenLMI, but if Blivet is going to be generic storage library, this is one of core functionality. If option 2 show too resource-heavy, we might try to go for 1.5. Jan

1 0

Is enumerateInstances guaranteed to be called exactly once?
by John Dennis 22 Feb '13

22 Feb '13

I've been looking at the Konkret doc and the various OpenLMI providers and there seems to be an an inherent assumption that enumerateInstances will be called exactly once. There are no checks to see if an instance already exists during the enumeration. I went to the CMPI spec for a clarification, except it had nothing to say about this (at least as far I could find). My naive assumption for a function with the verb Enumerate in it's name followed by the noun Instance says you're suppose to return a list of existing Instances, not create them. Also you might be asked to enumerate those same Instances more than once (i.e. give me a list of what you've got). But it seems like the actual semantics are "create your initial set of default instances and tell me what they are" Hence it has more to do with creation than enumeration. Is this correct or do I have a misunderstanding? Is this actually spelled out someplace? If so can you point me at that doc? Thanks, John -- John Dennis <jdennis(a)redhat.com> Looking to carve out IT costs? www.redhat.com/carveoutcosts/

3 4

Konkret array counts
by John Dennis 06 Feb '13

06 Feb '13

Oddly there doesn't seem to be an interface (API) in Konkret to get the size of an array. This seems like a very odd omission. For instance, lets say you define a method that has an IN parameter which is an array of strings, for example sake let's call this array "Names", your Konkret generated method will have a parameter like this: const KStringA *Names To do anything with the Names array you have to know how many elements it has, this proved difficult to figure out. The definition of KStringA looks like this: typedef struct _KStringA { CMPIUint32 exists; CMPIUint32 null; const CMPIArray* value; char __padding1[16 - sizeof(CMPIArray*)]; CMPIUint32 count; char __padding2[4]; } KStringA; Notice it has a count member. If you were thinking there is an accessor defined as an inline or macro for the count you'd be wrong. If you were thinking you could access Names->count and get a valid value you would be wrong. Apparently it's never initialized. I verified that when my method is invoked via YAWN it's passing an array of strings, I can see it in the generated XML. I also called the XXX_Args_Print() function that Konkret generated. It also shows the expected array elements. I also called KStringA_Get(Names, 0) to retrieve the first element, and lo and behold got the first element. But nowhere can I find how many elements are in this array nor a way to iterate it. Apparently there is nothing in Konkret to give you this critical information. I resorted to calling CMPI directly (which seems like an API violation) by doing this: count = CMGetArrayCount(OptionNames->value, NULL); which works. But this seems wrong, I shouldn't be reaching in to CMPI data structures that Konkret is supposed to be hiding from me, *plus* what's the point of the count member of the KStringA struct? BTW all KArray types have a count field, by looking at the source code it appears none of the count fields are initialized, unless you create the array yourself via of of the K*A_Init() functions which calls KArray_Init(). Also, there doesn't seem to be any doc whatsoever in Konkret for working with arrays. That also seems odd. Are arrays a second thought in Konkret? Comments? Am I missing something? John -- John Dennis <jdennis(a)redhat.com> Looking to carve out IT costs? www.redhat.com/carveoutcosts/

3 5

Re: openlmi-devel Digest, Vol 6, Issue 1
by Russell Doty 04 Feb '13

04 Feb '13

On Mon, 2013-02-04 at 12:30 +0000, openlmi-devel-request(a)lists.fedorahosted.org wrote: > Send openlmi-devel mailing list submissions to > openlmi-devel(a)lists.fedorahosted.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.fedorahosted.org/mailman/listinfo/openlmi-devel > or, via email, send a message with subject or body 'help' to > openlmi-devel-request(a)lists.fedorahosted.org > > You can reach the person managing the list at > openlmi-devel-owner(a)lists.fedorahosted.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of openlmi-devel digest..." > > > Today's Topics: > > 1. Re: Proposed F19 Feature: systemd features (Stephen Gallagher) > 2. RFC: Blivet & mounting (Jan Safranek) > 3. Re: RFC: Blivet & mounting (Klaus Kaempf) > 4. Re: RFC: Blivet & mounting (Jan Safranek) > 5. Re: RFC: Blivet & mounting (Stephen Gallagher) > 6. Re: RFC: Blivet & mounting (Klaus Kaempf) > 7. About the firewall provider (Tomáš Smetana) > 8. Re: RFC: Blivet & mounting (Stephen C. Tweedie) > > > ---------------------------------------------------------------------- > > Date: Thu, 31 Jan 2013 15:57:40 +0100 > From: Jan Safranek <jsafrane(a)redhat.com> > To: David Lehman <dlehman(a)redhat.com> > Cc: openlmi-devel(a)lists.fedorahosted.org, Discussion of Development > and Customization of the Red Hat Linux Installer > <anaconda-devel-list(a)redhat.com> > Subject: RFC: Blivet & mounting > Message-ID: <510A8664.2070104(a)redhat.com> > Content-Type: text/plain; charset=UTF-8 > > Hello, > > OpenLMI project uses Blivet, former Anaconda storage library, to manage > storage. Management of storage also means management of mounts, i.e. > ability to mount/unmount/remount stuff, list existing mounts and > add/delete/modify/list entries in /etc/fstab. > > The installer somehow manipulates fstab and mounts stuff and there is > some code in Blivet for that, but I am not sure it can be easily > extended for generic use case. > > Is generic mounting/unmounting capability that something Blivet should > do, so it would be integrated there, or there should be separate library > for this? In both cases, we (=OpenLMI) would /probably/ provide the code. > > Is there any other potential Blivet user who needs mounting? Can the > installer benefit from more generic mounting? > > What OpenLMI needs: > - parse /proc/mounts and provide list of mounts with their options > - parse /etc/fstab and provide list of persistent mounts with their > options (which can be different to /proc/mounts, someone might re-mount > something with different options!) > - re-mount a mount with different options > - create new mount, both permanent (with fstab entry) and 'temporary' > - unmount anything (incl. removing entry from fstab). > > BTW, for fstab manipulation I am thinking about augeas-python. It does > not pull any big dependency except libxml2, still it's yet another > package needed in installer ramdisk if generic mounting is in Blivet. > > Jan I would strongly encourage putting all storage related code into Blivet. This gives us a single source for storage manipulation capabilities, the opportunity to solve integration issues in one place, and a better end user experience. If we fragment storage we will reduce the value and use of Blivet. > > > ------------------------------ > Message: 7 > Date: Mon, 4 Feb 2013 13:12:53 +0100 > From: Tomáš Smetana <tsmetana(a)redhat.com> > To: openlmi-devel(a)lists.fedorahosted.org > Subject: About the firewall provider > Message-ID: <20130204131253.324c102d(a)zaphod.usersys.redhat.com> > Content-Type: text/plain; charset=UTF-8 > > Hello, > I think it might be useful to dump my ideas about the firewall provider > here to see what the others think. > > I was looking at the DMTF standard models for the packet filtering: There is > a FilterList class to aggregate FilterEntryBase classes (or their > descendants). All the FilterEntryBase items are ANDed in the list (no notion > of "chain"). Given there is no "management" class that would allow altering > the filters and the lists we will need to create proprietary ones for this We aren't looking at creating a proprietary interface - we are looking at creating an open interface which is non-conformant with parts of the DMTF/CIM specification. Perhaps we should refer to these things as custom interfaces or Linux specific interfaces? > purpose. Since we've decided to rely on the firewalld as the backend the > management classes would probably need to copy the Firewalld API to some > extent. I'm inclined to think this provider is one of those where the Makes sense. > standard model is simply unsuitable. Sounds like it. > > The other problem is: our Trac says [0] (it's quite vague otherwise) the first > version of the firewall provider is scheduled to Milestone 1. Looking at the > firewalld development I'm almost sure whatever we create by then would have > to be rewritten anyway. Do we want to move the ticket deadline? Sounds like we have to. > > Regards,

1 0

RFC: Blivet & mounting
by Jan Safranek 04 Feb '13

04 Feb '13

Hello, OpenLMI project uses Blivet, former Anaconda storage library, to manage storage. Management of storage also means management of mounts, i.e. ability to mount/unmount/remount stuff, list existing mounts and add/delete/modify/list entries in /etc/fstab. The installer somehow manipulates fstab and mounts stuff and there is some code in Blivet for that, but I am not sure it can be easily extended for generic use case. Is generic mounting/unmounting capability that something Blivet should do, so it would be integrated there, or there should be separate library for this? In both cases, we (=OpenLMI) would /probably/ provide the code. Is there any other potential Blivet user who needs mounting? Can the installer benefit from more generic mounting? What OpenLMI needs: - parse /proc/mounts and provide list of mounts with their options - parse /etc/fstab and provide list of persistent mounts with their options (which can be different to /proc/mounts, someone might re-mount something with different options!) - re-mount a mount with different options - create new mount, both permanent (with fstab entry) and 'temporary' - unmount anything (incl. removing entry from fstab). BTW, for fstab manipulation I am thinking about augeas-python. It does not pull any big dependency except libxml2, still it's yet another package needed in installer ramdisk if generic mounting is in Blivet. Jan

4 5

About the firewall provider
by Tomáš Smetana 04 Feb '13

04 Feb '13

Hello, I think it might be useful to dump my ideas about the firewall provider here to see what the others think. I was looking at the DMTF standard models for the packet filtering: There is a FilterList class to aggregate FilterEntryBase classes (or their descendants). All the FilterEntryBase items are ANDed in the list (no notion of "chain"). Given there is no "management" class that would allow altering the filters and the lists we will need to create proprietary ones for this purpose. Since we've decided to rely on the firewalld as the backend the management classes would probably need to copy the Firewalld API to some extent. I'm inclined to think this provider is one of those where the standard model is simply unsuitable. The other problem is: our Trac says [0] (it's quite vague otherwise) the first version of the firewall provider is scheduled to Milestone 1. Looking at the firewalld development I'm almost sure whatever we create by then would have to be rewritten anyway. Do we want to move the ticket deadline? Regards, -- Tomáš Smetana Base OS Engineering, Red Hat [0] https://fedorahosted.org/openlmi/ticket/23

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

openlmi-devel February 2013