Nir Soffer has uploaded a new change for review.
Change subject: libvirtconnection: Replace assert with AssertionError
......................................................................
libvirtconnection: Replace assert with AssertionError
The code wrongly assumed that assert always exists. When running in
optimized mode, the check would be skipped, and instead of getting an
AssertionError, which is the expected error for programmer error
(starting the eventloop twice), we could get a confusing
RuntimeException or RuntimeError from Thread.start (depending on Python
version).
RuntimeError misused in the standard library for all kinds of errors
that do not have builtin errors. It is particularry bad option when used
for usage error.
Change-Id: Icf1564f81f4c1fbf77ccaff6d93c047a02d946da
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M lib/vdsm/libvirtconnection.py
1 file changed, 2 insertions(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/64/34364/1
diff --git a/lib/vdsm/libvirtconnection.py b/lib/vdsm/libvirtconnection.py
index 5430c82..009f8b7 100644
--- a/lib/vdsm/libvirtconnection.py
+++ b/lib/vdsm/libvirtconnection.py
@@ -37,7 +37,8 @@
self.__thread = None
def start(self):
- assert not self.run
+ if self.run:
+ raise AssertionError("EventLoop is running")
self.__thread = threading.Thread(target=self.__run,
name="libvirtEventLoop")
self.__thread.setDaemon(True)
--
To view, visit http://gerrit.ovirt.org/34364
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Icf1564f81f4c1fbf77ccaff6d93c047a02d946da
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Federico Simoncelli has uploaded a new change for review.
Change subject: vdsm: add eventfd and EventFile synchronization
......................................................................
vdsm: add eventfd and EventFile synchronization
Change-Id: I0d237f13c42b1f4505c90d30c6d3c3ecbd1e9fa7
Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com>
---
M lib/vdsm/Makefile.am
A lib/vdsm/eventfd.py
M tests/Makefile.am
A tests/eventfdTests.py
4 files changed, 251 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/87/33687/1
diff --git a/lib/vdsm/Makefile.am b/lib/vdsm/Makefile.am
index 4bebf28..e712cad 100644
--- a/lib/vdsm/Makefile.am
+++ b/lib/vdsm/Makefile.am
@@ -25,6 +25,7 @@
__init__.py \
compat.py \
define.py \
+ eventfd.py \
exception.py \
ipwrapper.py \
libvirtconnection.py \
diff --git a/lib/vdsm/eventfd.py b/lib/vdsm/eventfd.py
new file mode 100644
index 0000000..b2a7084
--- /dev/null
+++ b/lib/vdsm/eventfd.py
@@ -0,0 +1,140 @@
+#
+# Copyright 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+
+"""\
+This module provides the support for eventfd(2).
+
+More information about eventfd and usage examples can be found in the
+eventfd(2) man page.
+
+The EventFile class provides a single synchronization object exposing
+the python Event interface and associated eventfds.
+
+The eventfd() context manager returns a file descriptor that can be
+used to provide the event notice to select, poll and epoll, e.g.
+
+ import os
+ import sys
+ import select
+ import threading
+ import time
+ from vdsm.eventfd import EventFile, DATASIZE
+
+ e = EventFile()
+ p = select.epoll()
+
+ threading.Timer(5, e.set).start()
+
+ with e.eventfd() as efd:
+ p.register(efd, select.EPOLLIN)
+ p.register(sys.stdin.fileno(), select.EPOLLIN)
+
+ print "Echoing lines until event is received"
+ event_received = False
+
+ while not event_received:
+ for fileno, event in p.poll():
+ if not event & select.EPOLLIN:
+ continue
+
+ if fileno == efd:
+ os.read(efd, DATASIZE)
+ event_received = True
+ elif fileno == sys.stdin.fileno():
+ print os.read(sys.stdin.fileno(), 1024),
+
+ print "Event received!"
+
+
+The Event set() semantic is preserved in the eventfd context manager:
+if the event is set then the eventfd already contains the notification.
+This is both to maintain the semantic and to avoid possible races as:
+
+ if not e.is_set():
+ with e.eventfd() as efd:
+ ...
+"""
+
+import os
+import ctypes
+import threading
+
+from contextlib import contextmanager
+
+_libc = ctypes.CDLL('libc.so.6', use_errno=True)
+
+EFD_NONBLOCK = os.O_NONBLOCK
+EFD_CLOEXEC = 02000000 # os.O_CLOEXEC in python 3.3
+EFD_SEMAPHORE = 00000001
+
+DATASIZE = ctypes.sizeof(ctypes.c_ulonglong)
+
+
+def eventfd(initval, flags):
+ return _libc.eventfd(initval, flags)
+
+
+class EventFile(object):
+ def __init__(self, event=None):
+ self.__lock = threading.Lock()
+ self.__fds = set()
+ self.__event = event or threading.Event()
+
+ @staticmethod
+ def __fire_event(fd):
+ os.write(fd, ctypes.c_ulonglong(1))
+
+ def open_eventfd(self):
+ with self.__lock:
+ fd = eventfd(0, 0)
+
+ self.__fds.add(fd)
+
+ if self.__event.is_set():
+ self.__fire_event(fd)
+
+ return fd
+
+ @contextmanager
+ def eventfd(self):
+ fd = self.open_eventfd()
+
+ yield fd
+
+ with self.__lock:
+ self.__fds.remove(fd)
+ os.close(fd)
+
+ def isSet(self):
+ return self.__event.isSet()
+
+ is_set = isSet
+
+ def set(self):
+ with self.__lock:
+ self.__event.set()
+ for fd in self.__fds:
+ self.__fire_event(fd)
+
+ def clear(self):
+ self.__event.clear()
+
+ def wait(self, timeout=None, balancing=True):
+ self.__event.wait(timeout)
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 449d7b1..120712e 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -31,6 +31,7 @@
clientifTests.py \
configNetworkTests.py \
domainMonitorTests.py \
+ eventfdTests.py \
fileVolumeTests.py \
fileUtilTests.py \
fuserTests.py \
diff --git a/tests/eventfdTests.py b/tests/eventfdTests.py
new file mode 100644
index 0000000..be15248
--- /dev/null
+++ b/tests/eventfdTests.py
@@ -0,0 +1,109 @@
+#
+# Copyright 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+
+import os
+import select
+from vdsm.eventfd import EventFile, DATASIZE
+from nose.tools import timed, raises, TimeExpired
+
+TEST_TIMEOUT = 1
+WAIT_TIMEOUT = 2
+
+
+def test_set():
+ e = EventFile()
+ e.set()
+ assert e.is_set()
+ assert e.isSet()
+
+
+def text_clear():
+ e = EventFile()
+ e.set()
+ assert e.is_set()
+ e.clear()
+ assert not e.is_set()
+
+
+@timed(TEST_TIMEOUT)
+def test_wait_set():
+ e = EventFile()
+ e.set()
+ e.wait(WAIT_TIMEOUT)
+
+
+@raises(TimeExpired)
+@timed(TEST_TIMEOUT)
+def test_wait_noset():
+ e = EventFile()
+ e.wait(WAIT_TIMEOUT)
+
+
+@timed(TEST_TIMEOUT)
+def test_eventfd_earlyset():
+ e = EventFile()
+ e.set()
+ with e.eventfd() as fd:
+ assert len(__select_and_read(fd)) == DATASIZE
+
+
+@timed(TEST_TIMEOUT)
+def test_eventfd_lateset():
+ e = EventFile()
+ with e.eventfd() as fd:
+ e.set()
+ assert len(__select_and_read(fd)) == DATASIZE
+
+
+@raises(TimeExpired)
+@timed(TEST_TIMEOUT)
+def test_eventfd_noset():
+ e = EventFile()
+ with e.eventfd() as fd:
+ assert len(__select_and_read(fd)) != DATASIZE
+
+
+@timed(TEST_TIMEOUT)
+def test_eventfd_multiple():
+ e = EventFile()
+ e.set()
+ with e.eventfd() as fd1:
+ assert len(__select_and_read(fd1)) == DATASIZE
+ with e.eventfd() as fd2:
+ assert len(__select_and_read(fd2)) == DATASIZE
+ with e.eventfd() as fd3:
+ assert len(__select_and_read(fd3)) == DATASIZE
+
+
+@raises(TimeExpired)
+@timed(TEST_TIMEOUT)
+def test_eventfd_clear():
+ e = EventFile()
+ e.set()
+ e.clear()
+ with e.eventfd() as fd:
+ assert len(__select_and_read(fd)) != DATASIZE
+
+
+def __select_and_read(fd):
+ rd, wr, ex = select.select((fd,), (), (), WAIT_TIMEOUT)
+ if fd in rd:
+ return os.read(fd, DATASIZE)
+ return ''
--
To view, visit http://gerrit.ovirt.org/33687
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I0d237f13c42b1f4505c90d30c6d3c3ecbd1e9fa7
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: log: Use INFO log level as default
......................................................................
log: Use INFO log level as default
The current logs are much too verbose which cause trouble for users, and
make us look unprofessional. Mature project should not use debug log by
default.
To debug issues that are not clear enough using INFO logs, the relevant
logger level can be modified on a user machine as needed.
Change-Id: I767dcd9bad7b9fbeebb438e9ef13cb0ec3f042ee
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/logger.conf.in
1 file changed, 4 insertions(+), 4 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/04/32504/1
diff --git a/vdsm/logger.conf.in b/vdsm/logger.conf.in
index 64b154f..8e963dd 100644
--- a/vdsm/logger.conf.in
+++ b/vdsm/logger.conf.in
@@ -8,18 +8,18 @@
keys=long,simple,none,sysform
[logger_root]
-level=DEBUG
+level=INFO
handlers=syslog,logfile
propagate=0
[logger_vds]
-level=DEBUG
+level=INFO
handlers=syslog,logfile
qualname=vds
propagate=0
[logger_Storage]
-level=DEBUG
+level=INFO
handlers=logfile
qualname=Storage
propagate=0
@@ -31,7 +31,7 @@
propagate=1
[logger_connectivity]
-level=DEBUG
+level=INFO
handlers=connlogfile
qualname=connectivity
propagate=0
--
To view, visit http://gerrit.ovirt.org/32504
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I767dcd9bad7b9fbeebb438e9ef13cb0ec3f042ee
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: lvm: Exclude faulty devices from lvm long filter
......................................................................
lvm: Exclude faulty devices from lvm long filter
lvm commands use filter to limit access to relevant devices. When the
filter includes a faulty device, lvm commands may block for several
minutes (stuck in D state). We have seen getDevicesList command stuck
for up to 10 minutes because of faulty devices in the long filter.
We used to build the filter from all multipath devices. Now we build the
filter only from devices which have at least one active paths.
# multipath -ll
360060160f4a0300038ed7058b5e9e311 dm-0 DGC ,VRAID
size=15G features='0' hwhandler='1 emc' wp=rw
|-+- policy='service-time 0' prio=0 status=enabled
| `- 4:0:3:0 sdd 8:48 failed faulty running
`-+- policy='service-time 0' prio=0 status=enabled
`- 4:0:2:0 sdb 8:16 failed faulty running
360060160f4a030003268ab211002e411 dm-1 DGC ,VRAID
size=30G features='1 queue_if_no_path' hwhandler='1 emc' wp=rw
|-+- policy='service-time 0' prio=4 status=active
| `- 4:0:3:1 sde 8:64 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
`- 4:0:2:1 sdc 8:32 active ready running
Previously, both devices were included in the filter, now only
360060160f4a030003268ab211002e411 will be included in lvm filter.
A faulty device which became active again will be included in lvm filter
after the next refresh (every 5 minutes), or after trying edit or create
a new storage domain.
lvm uses also short filter, including devices used by the certain vg or
lv. It is possible that we also have to exclude such devices from the
short filter. This will be handled later if needed.
Change-Id: I6d7a973bcefa95813fdc289847760c0955aca30c
Bug-Url: https://bugzilla.redhat.com/880738
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/storage/lvm.py
M vdsm/storage/multipath.py
2 files changed, 13 insertions(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/75/31875/1
diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py
index 86edf55..9cfad01 100644
--- a/vdsm/storage/lvm.py
+++ b/vdsm/storage/lvm.py
@@ -244,7 +244,7 @@
if not self._filterStale:
return self._extraCfg
- self._extraCfg = _buildConfig(multipath.getMPDevNamesIter())
+ self._extraCfg = _buildConfig(multipath.getActiveMPDevNamesIter())
_updateLvmConf(self._extraCfg)
self._filterStale = False
diff --git a/vdsm/storage/multipath.py b/vdsm/storage/multipath.py
index ba98866..2b30995 100644
--- a/vdsm/storage/multipath.py
+++ b/vdsm/storage/multipath.py
@@ -382,6 +382,18 @@
yield os.path.join(devicemapper.DMPATH_PREFIX, name)
+def getActiveMPDevNamesIter():
+ status = devicemapper.getPathsStatus()
+ for dmId, guid in getMPDevsIter():
+ active = [slave for slave in devicemapper.getSlaves(dmId)
+ if status.get(slave) == "active"]
+ if not active:
+ log.warning("Skipping device %s - no active slave", guid)
+ continue
+ log.debug("Found device %s %s", guid, active)
+ yield os.path.join(devicemapper.DMPATH_PREFIX, guid)
+
+
def getMPDevsIter():
"""
Collect the list of all the multipath block devices.
--
To view, visit http://gerrit.ovirt.org/31875
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I6d7a973bcefa95813fdc289847760c0955aca30c
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Nir Soffer has uploaded a new change for review.
Change subject: lvm: Decrease number of retries before failing
......................................................................
lvm: Decrease number of retries before failing
Previously we configured lvm to stop accessing a device during an lvm
opoeration after 3 errors. This can cause lvm to block 3 times when
trying to access inaccessible device. With current iscsi settings, each
block can be 120 seconds, total 360 seconds. We have seen lvm block for
couple of minutes in such cases.
The retries seems uneeded when working with multiple paths, as multipath
already retry all available paths after SCSI errors on one path. However
when working with single path, multipath should fail after one try.
According to lvm developer this may decrease the time lvm is blocked
when devices are not accesible.
(Not tested yet)
Change-Id: I5d11abaaff45ce86e88c6589264e162318ac1f1d
Relates-To: https://bugzilla.redhat.com/880738
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/storage/lvm.py
1 file changed, 1 insertion(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/56/32356/1
diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py
index 86edf55..f760f7b 100644
--- a/vdsm/storage/lvm.py
+++ b/vdsm/storage/lvm.py
@@ -106,7 +106,7 @@
preferred_names = ["^/dev/mapper/"]
ignore_suspended_devices=1
write_cache_state=0
-disable_after_error_count=3
+disable_after_error_count=1
obtain_device_list_from_udev=0
%s
}
--
To view, visit http://gerrit.ovirt.org/32356
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I5d11abaaff45ce86e88c6589264e162318ac1f1d
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>