Hello list.
There was a discussion in irc about device filtering. I just wanted to summarize and post what was discussed so we can maybe get some valuable insight from the mailing list. This is more like an irc dump than a summary. But I hope it shows some of the topics that should/could be discussed
* What information to give to the filter? This is related to the UI. And this was the reason to begin the discussion. - Give a regular expression that identifies the devices - Specify with a checkbox if you want anaconda to use the regular expression. - Specify what that regular expression should do (Ignore or Include)
* On what would those regular expressions operate? - canonicalized path of the device link under /sys/block. - []$ readlink -f /sys/block/sda/device /sys/devices/pci0000:00/0000:00:05.0/host0/target0:0:0/0:0:0:0 - /dev/disk/by-path version is also possible. Could be useful.
* Why not have the possibility to refresh? (Refresh would work with the information contained in the kernel.) - The user tweaks the scanning list and hits refresh. - User does this until he/she is satisfied with the device list.
* Three possible scanning stages where identified. Each of these might be controlled by the filter. 1. Kernel scan 2. udev scan 3. devicetree.py:scan
* We actually want to be able to tell the kernel not to scan certain LUN's.
* Wish people would learn how to configure fibrechannel switch acl's so we would only see the lun's they actually want us to touch. Would it be a good idea to document the configuration setup that anaconda expects?
* How would udev filters aid us with the overall filtering.
Regards.
There was a discussion in irc about device filtering. I just wanted to summarize and post what was discussed so we can maybe get some valuable insight from the mailing list.
Someone did me the favor of unplugging the network cable from my computer while I was away, so I do not have a log of this discussion. Allow me to rehash things that have probably already come up.
- What information to give to the filter? This is related to the UI. And this was the reason to begin the discussion.
- Give a regular expression that identifies the devices
- Specify with a checkbox if you want anaconda to use the regular expression.
- Specify what that regular expression should do (Ignore or Include)
Designing a UI for this could very quickly become a widget explosion. Have you ever seen any of the regex building programs? Scary. I feel like we're either going to have to go down that route or have some lame text entry box where you enter a string. I don't see a lot of middle ground.
I think I prefer globs to regexes because I feel like it'd be easier to support, but I'm open to discussion.
- On what would those regular expressions operate?
- canonicalized path of the device link under /sys/block.
- []$ readlink -f /sys/block/sda/device /sys/devices/pci0000:00/0000:00:05.0/host0/target0:0:0/0:0:0:0
- /dev/disk/by-path version is also possible. Could be useful.
Peter and I discussed the possibility of using basically anything that udev provides us with for filtering. So you could filter on these paths, or sizes, or model, or whatever else. Given such flexibility, there'd also need to be the ability for you to use most of the comparison operators.
Keep in mind we're going to need all this in kickstart as well.
This introduces the problem of udev changing what information is available and screwing us over when we can no longer support something that was being relied upon. So maybe we only offer a subset of the udev options that are fairly basic and most important.
- Why not have the possibility to refresh? (Refresh would work with the information contained in the kernel.)
- The user tweaks the scanning list and hits refresh.
- User does this until he/she is satisfied with the device list.
Seems reasonable, though I'm not sure how well it will work in practice.
- Wish people would learn how to configure fibrechannel switch acl's so we would only see the lun's they actually want us to touch. Would it be a good idea to document the configuration setup that anaconda expects?
I think the response from customers here will be that anaconda should deal with whatever configuration it's given, whether it's reasonable or not. That's probably extreme, but we should definitely anticipate that people will not have their storage networks set up the polite way and will still want their data preserved.
- Chris
Just a few comments from the viewpoint of the s390 architecture, which could provide use cases since there are often many (hundreds or thousands of) devices, as we have already discussed at FUDCon.
On 06/19/2009 01:49 PM, Joel Granados wrote:
- What information to give to the filter? This is related to the UI. And this was the reason to begin the discussion.
- Give a regular expression that identifies the devices
Regexes seem way too complicated for anaconda which otherwise guides the user in a very helpful way. Wouldn't two entry fields to specify an interval/range do it? I mean, on systems with many (hundreds to thousands on s390) devices users would typically want to filter for device number ranges. E.g. activate all DASDs between and including 0.0.beef and 0.0.dead.
- On what would those regular expressions operate?
- canonicalized path of the device link under /sys/block.
- []$ readlink -f /sys/block/sda/device /sys/devices/pci0000:00/0000:00:05.0/host0/target0:0:0/0:0:0:0
# readlink -f /sys/block/dasda/device /sys/devices/css0/0.0.0003/0.0.eb1b # readlink -f /sys/block/sda/device /sys/devices/css0/0.0.0015/0.0.3c1b/host0/rport-0:0-24/target0:0:24/0:0:24:1089355792
- /dev/disk/by-path version is also possible. Could be useful.
/dev/disk/by-path/ccw-0.0.eb1b -> ../../dasda /dev/disk/by-path/ccw-0.0.3c1b-zfcp-0x5005XXXXXXXXXXXXX:0xXXXXXXXX00000000 -> ../../sda
By-path looks less confusing but I don't know whether users would want to filter for rport, target or other stuff in the canonicalized path.
Why not have the possibility to refresh? (Refresh would work with the information contained in the kernel.)
- The user tweaks the scanning list and hits refresh.
- User does this until he/she is satisfied with the device list.
Three possible scanning stages where identified. Each of these might be controlled by the filter.
- Kernel scan
On s390 we have a kernel command line option cio_ignore to prevent devices from even being sensed by the kernel, since on systems with thousands of devices already the pure sensing might consume too much time or memory (for the allocated in-kernel device objects).
- udev scan
- devicetree.py:scan
We actually want to be able to tell the kernel not to scan certain LUN's.
Wish people would learn how to configure fibrechannel switch acl's so we would only see the lun's they actually want us to touch. Would it be a good idea to document the configuration setup that anaconda expects?
On s390, only devices that have been sensed *and* are activated explicitly become visible to Linux. I agree with Chris that we should support all users no matter how they have configured their I/O subsystem. Even if users have setup their FCP storage network with ACLs but don't have NPIV in their hardware, they might see all disks of all virtual machines on the system. Due to massive virtualization this means n disks per VM times the number of VMs which might be hundreds and thus too many devices.
Even if this might mean that s390 does not need additional device filtering, it maybe advantageous to design it such that we can share code between generic filtering and the display filtering of devices in the advanced storage configuration subdialogs.
Steffen Maier
Linux on System z Development
IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martin Jetter Geschäftsführung: Erich Baier Sitz der Gesellschaft: Böblingen Registergericht: Amtsgericht Stuttgart, HRB 243294
On 30.06.2009 Steffen Maier wrote:
On 06/19/2009 01:49 PM, Joel Granados wrote:
- What information to give to the filter? This is related to the UI. And this was the reason to begin the discussion.
- Give a regular expression that identifies the devices
Regexes seem way too complicated for anaconda which otherwise guides the user in a very helpful way. Wouldn't two entry fields to specify an interval/range do it? I mean, on systems with many (hundreds to thousands on s390) devices users would typically want to filter for device number ranges. E.g. activate all DASDs between and including 0.0.beef and 0.0.dead.
Another great option is to make a "narrowing" filter -- I.e, when you start typing characters in the filter entry box, the list dynamically shrinks to the *subset* containing this sub-string (think awesome-bar in Firefox or the search boxes in kmail/kaddressbook).
Although regex are more powerful, the "narrowing" filter pros: * Very simple and intuitive for newbies. * No need for extra controls (Apply button, etc.) as the action is immediate as you type. * Add/remove/change filtering are all the same action (type/edit the string). * It's important to narrow by sub-string (not prefix), so it won't be very limited. * Making the matched sub-string bold in the list (as in FF awesome-bar) would make it even more intuitive.
However, if this is going to be implemented, it's important to have it for any list which may be long (for consistent look and feel).
On Fri, 2009-06-19 at 13:49 +0200, Joel Granados wrote:
Hello list.
There was a discussion in irc about device filtering. I just wanted to summarize and post what was discussed so we can maybe get some valuable insight from the mailing list. This is more like an irc dump than a summary. But I hope it shows some of the topics that should/could be discussed
This is very late, I know. Here is my take after an irc discussion with clumens.
- What information to give to the filter? This is related to the UI. And this was the reason to begin the discussion.
- Give a regular expression that identifies the devices
- Specify with a checkbox if you want anaconda to use the regular expression.
- Specify what that regular expression should do (Ignore or Include)
I think the regex approach is way more than we can (or want to) chew for F12. It also sounds really unwieldy for users.
Here's my proposal: a list of disks, each with a checkbox. The list items' information can be argued between device node name, size, model, whatever. We also provide a "select all" and "select none" button so users with 10,000 LUNs can hit "select none" and then click on their one local disk and proceed.
It has been pointed out that we still want a way to specify devices in kickstart that does not rely on device node names. I think the sysfs path stuff can be useful here. Perhaps the identifiers used for /dev/disk/by-id and /dev/disk/by-path.
I also keep thinking of checkboxes (which translate to switches in ks) like "select FCoE devices" or "select USB devices".
- Three possible scanning stages where identified. Each of these might be controlled by the filter.
- Kernel scan
I think we need to accept that this will have already happened by the time we have launched the graphical installer. Perhaps a separate mechanism can be developed to control this from the kernel command line? Maybe I'm just not familiar with the argument for needing this level of control, but to me it seems unnecessary.
- udev scan
- devicetree.py:scan
To me it breaks down like this:
1. get a list of disks from udev 2. present the list, allow users to edit it 3. once the list is decided upon, do the full device scan and populate the tree
I'm not sure where setup of isci/fcoe/zfcp devices fits into this but, given the matter of upgrades, it might be worth considering moving the entire storage configuration process ahead of the install/upgrade screen.
- We actually want to be able to tell the kernel not to scan certain LUN's.
Why?
Please do provide any feedback that occurs to you so we can continue to move this forward.
Dave
Hi,
On 07/07/2009 08:12 PM, David Lehman wrote:
<snip lots of good stuff I all agree with>
I'm not sure where setup of isci/fcoe/zfcp devices fits into this but, given the matter of upgrades, it might be worth considering moving the entire storage configuration process ahead of the install/upgrade screen.
Make that iscsi/fcoe/zfcp/dasd (*), anyways all these devices need to be setup by the user manually, iow when not specifically enabled they are not seen, so no need to filter here, it would be a bit strange to first add a disk to then filter it, this could happen though, when one add storage action results in multiple LUN's the user may want to filter some LUN's
The exception here is that we auto detect iscsi disks which are configured in the machines firmware (ie BIOS setup), but these disks are usually configured in the firmware because they are meant as local disks, so no problem here. The same (firmware based detection) will be added in the future for FCoE disks.
(*) Currently we only have the ability to add dasd disks from the special s390 rc.sysinit, when talking about device filtering with the IBM s390 guys at LinuxTag, it was mentioned that it would be nice (and consistent) to also have the ability to bring online dasd disks from the "advanced storage" / add storagre button menu. Not saying that this is going to be done in time for F-12, but this is something which was discussed.
Regards,
Hans
anaconda-devel@lists.stg.fedoraproject.org
-
Chris Lumens -
David Lehman -
Hans de Goede -
Joel Granados -
Oron Peled -
Steffen Maier