vdsm-devel May 2013

vdsm-devel@lists.stg.fedorahosted.org

28 participants
27 discussions

About vdsmd init script
by zhshzhou＠linux.vnet.ibm.com 03 Jun '13

03 Jun '13

Hi, I'm working on making VDSM run on Ubuntu. The current work item is to start vdsmd service on Ubuntu. After discussion with Alon Bar-Lev on one of my patches [1], I come up with an idea. I would like to listen to other developers' suggestions before submit a new patch set. Firstly the vdsmd.init.in should be split into pieces. I move the initialization operations out of vdsmd.init.in, and put them in respective scripts, for example, init_00_shutdown_conflicting_srv.sh init_04_gencerts.sh init_08_libvirtd_sysv2upstart.sh init_12_libvirtd_reconfigure.sh init_16_start_needed_srv.sh ... init_48_tune_system.sh Then I can write a master script named init_common.sh. The init_common.sh provides some environment variables and common functions, it sources init_*_*.sh and execute all of them in alphabetic order. Then in vdsmd.init.in, it calls init_common.sh before actually starts vdsm daemon. After move these things out of vdsmd.init.in, it slims to about 100 lines of code, and most of them are boilerplate. Then in Debian, we should write vdsmd_debian.init.in using Debian's boilerplate, which fits to Debian's SysV environment. Debians SysV environment is a bit different from Red Hat family's. Red Hat provides /etc/init.d/functions and Debian provides /lib/lsb/init-functions, the error logging function and daemon function are different. The VDSM SystemD unit calls vdmsd.init.in to start daemon, after we move initialization to init_common.sh and its slaves, we can call init_common.sh in vdsmd.service then start "/usr/bin/python /usr/share/vdsm/vdsm" directly. SystemD knows how to daemonize and respawn a process, so we can make use its native ability. As regard to Upstart on Ubuntu, we can write an Upstart job that calls init_common.sh, then use Upstart's native ability to daemonize and respawn vdsm service process -- just a few lines of declaration. In this way we relieve the headache of supporting a monolithic vdsmd.init.in in different systems. After I split initialization operations into init_*_*.sh scripts, I find they can be modularized. There are almost no dependency in these init_*_*.sh scripts. Supporting them in different systems is not hard, and we can add/remove initialization scripts when porting VDSM to a new system. We may has concerns on libvirt configuration. The libvirt configuration code is complicated in vdsmd.init.in. After the refactoring, it would be more complicated to improve it. I think the code is modular enough, I did not change the code when I moved it. I also have an idea on how to simplify it. If you take a look at configure_libvirt function in vdsmd.init.in, you will find the code is mixed with the data. The mechanism of configuration is mixed with what we'd like to configure. For example, set_if_default $lconf listen_addr \"0.0.0.0\" set_if_default $lconf unix_sock_group \"kvm\" set_if_default $lconf unix_sock_rw_perms \"0770\" ... set_if_default $lconf keepalive_interval -1 I think we can split the configuration settings into a template file, for example, listen_addr="0.0.0.0" unix_sock_group="kvm" unix_sock_rw_perms="0770" ... keepalive_interval=-1 Then write a function that reads this template and "merge" the settings into libvirtd.conf. Then for different systems we can ship VDSM with different template with various settings, and the code implementing configuring remains the same. The template can also support dynamic values, for example, host_uuid<="$(uuidgen)" Means it should eval the value before write it to libvirtd configure file. [1] Split vdsm init script into pieces http://gerrit.ovirt.org/#/c/14826/ -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshzhou(a)linux.vnet.ibm.com Telephone: 86-10-82454397

4 6

Re: [vdsm] Migration regression on master
by gvallare＠redhat.com 31 May '13

31 May '13

| ----- Original Message ----- | | From: "Assaf Muller" <amuller(a)redhat.com> | | To: "Michal Skrivanek" <michal.skrivanek(a)redhat.com> | | Cc: "vdsm-devel(a)lists.fedorahosted.org Development" | | <vdsm-devel(a)lists.fedorahosted.org> | | Sent: Thursday, May 23, 2013 2:12:47 PM | | Subject: Re: [vdsm] Migration regression on master | | | | As you can see in a previous patch set I checked if the alias attribute | | exists instead of assuming it exists. | | I then changed my mind with Dan's blessing, and decided to assume it does | | exist, exactly for cases like this. | | Even if we check if the alias exists, what do we do if we find out it | | doesn't? We're at a problem and need to understand why the alias doesn't | | exist because it should - For all devices. | | | | We definitely need to deal with this issue - Can you provide the domxml of | | the VM during creation, and during migration? Hi Assaf, Peter today has reproduced the same error and provided me the output log, you can find it here: http://etherpad.ovirt.org/p/migration-errors a previous patch blocked by this issue has been accepted (http://gerrit.ovirt.org/#/c/14203/) I've been able to do a migration using the engine. Feel free to ask if you need more details, I will look into this issue in the next few days. Giuseppe | | | | | ----- Original Message ----- | | From: "Michal Skrivanek" <michal.skrivanek(a)redhat.com> | | To: "Vinzenz Feenstra" <vfeenstr(a)redhat.com>, "Assaf Muller" | | <amuller(a)redhat.com> | | Cc: "vdsm-devel(a)lists.fedorahosted.org Development" | | <vdsm-devel(a)lists.fedorahosted.org>, "Dan Kenigsberg" <danken(a)redhat.com> | | Sent: Wednesday, May 22, 2013 3:56:22 PM | | Subject: Re: [vdsm] Migration regression on master | | | | | | On May 22, 2013, at 10:14 , Vinzenz Feenstra <vfeenstr(a)redhat.com> wrote: | | | | > Hi, | | > | | > During the verification of the refactoring in VDSM (libvirtvm + vm == vm) | | > we have encountered some issue. | | > There seems to be a regression on master branch regarding device aliases. | | > | | > On the target machine _updateDevicesDomxmlCache accesses dev.alias | | > without | | > the attribute being present. | | > | | > This happens to happen at least on balloon devices. | | > I am not sure why we do have this, however it is reproducible and I am | | > not | | > sure why we do this now. | | > | | > I know it has been introduced by http://gerrit.ovirt.org/#/c/13876 and | | > it's | | > actually in the commit message that we'd fail. | | :-) you mean: | | * We now crash horribly if an alias was not found for a device | | when updating devices domxml cache after migrating | | | | > Which is the case... So can somebody please enlighten me and tell me if | | > this is a configuration issue of the VM or if this really a regression. | | not all devices have alias so I think this is indeed a regression | | | | | | > It's possible that the tests aren't properly configured, however that | | > would | | > from my understanding mean that we potentially will run into troubles | | > during migrations from 3.2 -> 3.3 vdsm. | | > | | > PS: | | > This kind of blocks a bit the refactoring verification :/ | | > | | > -- | | > Regards, | | > | | > Vinzenz Feenstra | Senior Software Engineer | | > RedHat Engineering Virtualization R & D | | > Phone: +420 532 294 625 | | > IRC: vfeenstr or evilissimo | | > | | > Better technology. Faster innovation. Powered by community collaboration. | | > See how it works at redhat.com | | > | | > _______________________________________________ | | > vdsm-devel mailing list | | > vdsm-devel(a)lists.fedorahosted.org | | > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel | | | | _______________________________________________ | | vdsm-devel mailing list | | vdsm-devel(a)lists.fedorahosted.org | | https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel | | |

5 6

Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
by Vinzenz Feenstra 30 May '13

30 May '13

Please find the prettier version on the wiki: http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval Proposal VDSM - Engine Data Statistics Retrieval VDSM <=> Engine data retrieval optimization Motivation: Currently the RHEVM engine is polling the a lot of data from VDSM every 15 seconds. This should be optimized and the amount of data requested should be more specific. For each VM the data currently contains much more information than actually needed which blows up the size of the XML content quite big. We could optimize this by splitting the reply on the getVmStats based on the request of the engine into sections. For this reason Omer Frenkel and me have split up the data into parts based on their usage. This data can and usually does change during the lifetime of the VM. Rarely Changed: This data is change not very frequent and it should be enough to update this only once in a while. Most commonly this data changes after changes made in the UI or after a migration of the VM to another Host. *Status* = Running *acpiEnable* = true *vmType* = kvm *guestName* = W864GUESTAGENTT *displayType* = qxl *guestOs* = Win 8 *kvmEnable* = true #/*this should be constant and never changed*/ *pauseCode* = NOERR *monitorResponse* = 0 *session* = Locked # unused *netIfaces* = [{'name': 'Realtek RTL8139C+ Fast Ethernet NIC', 'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet': ['10.34.60.148'], 'hw': '00:1a:4a:22:3c:db'}] *appsList* = ['RHEV-Tools 3.2.4', 'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3', 'RHEV-Network64 3.2.2', 'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3', 'RHEV-Balloon64 3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2', 'RHEV-USB 3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2'] *pid* = 11314 *guestIPs* = 10.34.60.148 # duplicated info *displayIp* = 0 *displayPort* = 5902 *displaySecurePort* = 5903 *username* = user@W864GUESTAGENTT *clientIp* = *lastLogin* = 1361976900.67 Often Changed: This data is changed quite often however it is not necessary to update this data every 15 seconds. As this is cumulative data and reflects the current status, and it does not need to be snapshotted every 15 seconds to retrieve statistics. The data can be retrieved in much more generous time slices. (e.g. Every 5 minutes) *network* = {'vnet1': {'macAddr': '00:1a:4a:22:3c:db', 'rxDropped': '0', 'txDropped': '0', 'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '100', 'name': 'vnet1'}} *disksUsage* = [{'path': 'c:\\', 'total': '64055406592', 'fs': 'NTFS', 'used': '19223846912'}, {'path': 'd:\\', 'total': '3490912256', 'fs': 'UDF', 'used': '3490912256'}] *timeOffset* = 14422 *elapsedTime* = 68591 *hash* = 2335461227228498964 *statsAge* = 0.09 # unused Often Changed but unused This data does not seem to be used in the engine at all. It is *not* even used in the data warehouse. *memoryStats* = {'swap_out': '0', 'majflt': '0', 'mem_free': '1466884', 'swap_in': '0', 'pageflt': '0', 'mem_total': '2096736', 'mem_unused': '1466884'} *balloonInfo* = {'balloon_max': 2097152, 'balloon_cur': 2097152} *disks* = {'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '1754496', 'imageID': '28abb923-7b89-4638-84f8-1700f0b76482', 'flushLatency': '156549', 'readRate': '0.00', 'truesize': '18855059456', 'writeRate': '952.05'}, 'hdc': {'readLatency': '0', 'apparentsize': '0', 'writeLatency': '0', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '0', 'writeRate': '0.00'}} Very frequent uppdates needed by webadmin portal: This data is mostly needed for the webadmin portal and might be required to be updated quite often. An exception here is the statsAge field, which seems to be unused by the Engine. This data could be requested every 15 seconds to keep things as they are now. *cpuSys* = 2.32 *cpuUser* = 1.34 *memUsage* = 30 Proposed Solution for VDSM & Engine: We will introduce new optional parameters to getVmStats, getAllVmStats and list to allow a finer grained specification of data which should be included. *Parameter:* *statsType*=/*<string>*/ (getVmStats, getAllVmStats only) *Allowed values:* * full (default to keep backwards compatibility) * app-list (Just send the application list) * rare (include everything from rarely changed to very frequent) * often (include everything from often changed to very frequent) * frequent (only send the very frequently changed items) *Parameter:* *clientId*=*<string>* The client id is specified by the client and should be unique however constantly used. *Parameter:* *diff*=*<boolean>* In combination with the clientId VDSM will send only differences to the previous request from the named clientId. (if diff=true) Additional Change: Besides the introduction of the new parameters for list, getVmStats and getAllVmStats it might make sense to include a hash for the appList into the rarely changed section of the response which would allow to identify changes and avoid having to sent the complete appList every so often and only if the hash known to the client is outdated. *Note:* The appList (Application List) reported by the guest agent could be fully implemented on request only, as long as the guest agent installed supports this. As there seems to be a request to have the complete list of installed applications on all guests this data could be quite extensive and a huge list. On the other hand this data is only rarely visible and therefore it should not be requested all the time and only on demand. Improvement of the Guest Agent: As part of the proposed solution it is necessary to improve the guest agent as well. For the full application list there should be implemented a caching system which will be fully reactive and should not poll the application list for example all the time. The guest can create a prepared data file containing all data in the JSON format (as used for the communication with VDSM via VIO) and just have to read that file from disk and directly sends it to VDSM. However it is quite possible that this list is to big and it might have to be chunked into pieces. (Multiple messages, which would have to be supported by VDSM then as well) The solution for this is to make VDSM request this data and it will retrieve the data necessary on request only. -- Regards, Vinzenz Feenstra | Senior Software Engineer RedHat Engineering Virtualization R & D Phone: +420 532 294 625 IRC: vfeenstr or evilissimo Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com

8 23

Potential Bug in cpopen
by zhshzhou＠linux.vnet.ibm.com 29 May '13

29 May '13

Hi, Recently Jenkins unit test sometimes fails on PidStatTests.test, for example http://gerrit.ovirt.org/#/c/14670/ After it execCmd a "sleep" command with sync=False, in /proc/[xxxpid]/stat we should see the name is "sleep", but in this case we get "python", which means there is a possible race condition. The most possible situation is that execCmd returns before the child process execvp the "sleep", then the parent process reads the stat and sees the process name is still "python". cpopen is designed to avoid this kind of race. It uses a pipe to synchronize the child and parent. It sets the FD_CLOEXEC on the child side of the pipe, so that once execvp succeeds, the child pipe is closed. If execvp fails, child writes error code to the pipe. The parent reads the other end of the pipe like follow. if (read(errnofd[0], &childErrno, sizeof(int)) == sizeof(int)) { PyErr_SetString(PyExc_OSError, strerror(childErrno)); goto fail; } It assumes that when read returns, the return value is either sizeof(int), which indicates an error in the child side, or 0, which indicates the child side of pipe is closed and execvp succeeds. However this assumption may not always be true. If the parent process gets a signal, the read invocation would be interrupt, and the code treats this interruption the same as the case of execvp succeeds. If the system is very busy like our Jenkins slave concurrently executing jobs, it's possible the parent gets interrupted before the child execvp succeeds, so cpopen returns to execCmd and cause the race. To produce this problem, we can add a "sleep(10);" before the exec invocation in cpopen.c, it simulates a busy/slow system. Then in a Python interpreter, register a signal handler and calls execCmd. >>> from vdsm.utils import execCmd >>> from vdsm import utils >>> def handler(signum, frame): ... print 'Signal handler called with signal', signum ... >>> import signal >>> signal.signal(signal.SIGALRM, handler) >>> signal.signal(signal.SIGCHLD, handler) >>> p = execCmd(['sleep', '3'], sync=False, sudo=False); s = utils.pidStat(p.pid); print s; p.wait() Then kill -SIGCHLD or kill -SIGALRM to this Python interpreter process, we can see the output. Signal handler called with signal 17 (9541, 'python', 'S', 9509, 9509, 6843, 34817, 9509, 4218944, 66, 0, 0, 0, 0, 0, 0, 0, 20, 0, 1, 0, 663767, 217919488, 1676, 18446744073709551615L, 4194304, 4197004, 140735665131088, 140735665124456, 268226504400, 0, 0, 16781312, 73730, 18446744071579398713L, 0, 0, 17, 3, 0, 0, 0, 0, 0, 6294952, 6297616, 19152896, 140735665132425, 140735665132432, 140735665132432, 140735665135592, 0) True I have not found other ways to produce it, just found this method. Not sure it is a bug. Is it reasonable to check the read() return value and retry on EAGAIN/EINTR to fix this? -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshzhou(a)linux.vnet.ibm.com Telephone: 86-10-82454397

3 3

supervdsm broken on master
by Dan Kenigsberg 29 May '13

29 May '13

I've just taken Yaniv's http://gerrit.ovirt.org/11051 "Supervdsm as external service" to master, but unfortunately, I decided to test it myself only afterwards. Currently, supervdsm fails to start unless you manually create the directory /var/run/vdsm/ as root. The fix should not be complex, but I'm on the run. I'm confident that Yaniv would fix it soon. Regards, Dan.

2 1

oVirt updates - April 28th, 2013
by iheim＠redhat.com 28 May '13

28 May '13

1. From the Web - interview with Theron on oVirt (chinese) http://www.infoq.com/cn/news/2013/05/conrey-on-ovirt - interview with Dave Neary about his work (and oVirt) http://www.techradar.com/news/software/what-went-wrong-with-meego-nokia-los… - Nagios monitoring plugin check_rhev3 1.2 released http://lists.ovirt.org/pipermail/users/2013-May/014389.html - a blog on how to do HA for engine (written for rhev, should be relevant to oVirt as well) http://captainkvm.com/2013/05/providing-high-availability-for-rhev-m/ 2. Video - youtube available for IBM's session on connected Communities, Innovative Technologies: OpenStack, oVirt, and KVM http://www.youtube.com/watch?v=Pg7ShV-HvCE - fog/foreman by ohad levy (fog supports oVirt) http://www.youtube.com/watch?v=JgaQ_ekR2JA 3. Conferences - FOSDEM presentations page uploaded http://www.ovirt.org/FOSDEM_2013 - some of the Shangahi presentations uploaded http://www.ovirt.org/Intel_Workshop_May_2013 - upcoming - LinuxCon Japan oVirt session in LinuxCon Japan (this week) - upcoming - oVirt Developer days (with KVM Forum) Edinburgh, UK - October 21 - 23, 2013 4. Other - help test the new oVirt installer and developer setup environments http://www.ovirt.org/OVirtEngineDevelopmentEnvironment - RC for oVirt Node 3.0.0 is now available (but not compatible with ovirt-engine yet) Thanks, Itamar

1 0

Migration regression on master
by Vinzenz Feenstra 23 May '13

23 May '13

Hi, During the verification of the refactoring in VDSM (libvirtvm + vm == vm) we have encountered some issue. There seems to be a regression on master branch regarding device aliases. On the target machine _updateDevicesDomxmlCache accesses dev.alias without the attribute being present. This happens to happen at least on balloon devices. I am not sure why we do have this, however it is reproducible and I am not sure why we do this now. I know it has been introduced by http://gerrit.ovirt.org/#/c/13876 and it's actually in the commit message that we'd fail. Which is the case... So can somebody please enlighten me and tell me if this is a configuration issue of the VM or if this really a regression. It's possible that the tests aren't properly configured, however that would from my understanding mean that we potentially will run into troubles during migrations from 3.2 -> 3.3 vdsm. PS: This kind of blocks a bit the refactoring verification :/ -- Regards, Vinzenz Feenstra | Senior Software Engineer RedHat Engineering Virtualization R & D Phone: +420 532 294 625 IRC: vfeenstr or evilissimo Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com

3 2

vdsm sync meeting 2013-05-20
by asegurap＠redhat.com 21 May '13

21 May '13

VDSM sync meeting May 20th 2013 =============================== ## Build cleaning - Yaniv patch to take cpopen on its own rpm was merged. - We have to continue to move binary parts in their own packages. ## Refactorings - Vinzenz's vm.py and libvirtvm.py needs more testing resources to verify that the refactoring works well for merging. It is proposed to him to use jenkins jobs for it (although it is only for downstream, which might be an issue right now). - Yaniv's supervdsm split out has been +1ed by Mark Wu but it has not been yet merged. ## Feature and release management - Reminder to Federico about the update to release management that should include all the features for 3.3. ## Network refactoring - Mark submitted patches on top of Toni's for network reloaded that improve the modeling. Reviews ongoing. ## API - Adam to take a review some work that has been going on with the schema. - Saggi's java bindings were submitted for review last week.

2 1

VDSM Java Bindings
by smizrahi＠redhat.com 20 May '13

20 May '13

A preliminary version on the Java bindings is up on gerrit ready for review. It's been up for a while now but I noticed I forgot to notify the list. It currently only supports the TCP transport as the Java implementation of Proton gave me a lot of grief and I wanted to release things ASAP. Also, receiving unions is still a bit problematic. The will always arrive as the base form of the union since detection of the concrete type is not implemented yet. I also don't like how unions works ATM and would like to come up with a system that makes them more easily detectable. You can review the code here http://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:js… An example project showing how to set up a Java project and connect to a VDSM instance. https://github.com/ficoos/vdsmhello I would like to have more feedback about how the interface is exported and generated. Getting the interface right is much more important to me at the moment than performance tweaks and corner cases.

1 0

vdsClient setupNetworks command usage
by psebek＠redhat.com 20 May '13

20 May '13

Hi all, I've added setupNetworks command to vdsClient http://gerrit.ovirt.org/#/c/14132/ and I want to show you some example of how to use it. Syntax is following: setupNetworks [connectivityCheck=False(default)|True] [connectivityTimeout=<seconds>] [<option>=<value>] [networks='{<bridge>:{nic:<nic>,vlan:<number>,bonding:<bond>,...}}'] [bondings='{<bond>:{nics:<nic>[+<nic>],..}}'] Note that quotation marks "'" in networks='...' and bongings='...' are important, if they would not be there shell would make brace expansion and the arguments would not be in the form we want. Example of usage: # vdsClient 0 setupNetworks networks='{red:{bonding:bond4,STP:no,bridged:True},black:{nic:eth3,vlan:5,STP:no,bridged:true}}' bondings='{bond4:{nics:eth1+eth2,options:mode=4}}' Here I created two networks: red with bond4, no STP and bridge, black on nic eth3, vlan 5, with no STP and bridged. I also created bonding over nics eth1 and eth2 with mode 4. # vdsClient 0 setupNetworks networks='{red:{remove:true},black:{remove:true}}' bondings='{bond4:{remove:true}}' And here I've destroyed all created networks and bond. vdsClient setupNetworks command calls configNetwork.setupNetworks, that can be called with following parameters so you can use it: Add/Edit/Remove configuration for networks and bondings. Params: networks - dict of key=network, value=attributes where 'attributes' is a dict with the following optional items: vlan=<id> bonding="<name>" | nic="<name>" (bonding and nics are mutually exclusive) ipaddr="<ip>" netmask="<ip>" gateway="<ip>" bootproto="..." delay="..." onboot="yes"|"no" (other options will be passed to the config file AS-IS) -- OR -- remove=True (other attributes can't be specified) bondings - dict of key=bonding, value=attributes where 'attributes' is a dict with the following optional items: nics=["<nic1>" , "<nic2>", ...] options="<bonding-options>" -- OR -- remove=True (other attributes can't be specified) options - dict of options, such as: force=0|1 connectivityCheck=0|1 connectivityTimeout=<int> Notes: When you edit a network that is attached to a bonding, it's not necessary to re-specify the bonding (you need only to note the attachment in the network's attributes). Similarly, if you edit a bonding, it's not necessary to specify its networks. Regards, Petr.

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-devel May 2013