LNST-developers July 2014

lnst-developers@lists.fedorahosted.org

3 participants
28 discussions

[PATCH] PacketAssert: fix tcpdump output for daemonized lnst-slave
by Ondrej Lichtner 22 Jul '14

22 Jul '14

From: Ondrej Lichtner <olichtne(a)redhat.com> This patch fixes the issue of tcpdump not returning any output when lnst-slave is running as a daemon. We're not exactly sure what causes this problem but setting the file mode to 'a' seems to solve the issue. Thanks to Artem Savkov for proposing this workaround. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> --- test_modules/PacketAssert.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/test_modules/PacketAssert.py b/test_modules/PacketAssert.py index 0611179..3fb8293 100644 --- a/test_modules/PacketAssert.py +++ b/test_modules/PacketAssert.py @@ -81,7 +81,9 @@ class PacketAssert(TestGeneric): def _execute_tcpdump(self): """ Start tcpdump in the background """ cmd = self._cmd - tcpdump_file = tempfile.NamedTemporaryFile(delete=False) + #setting the mode to 'a' is temporary, the default mode 'w' doesn't work + #when lnst-slave is running as a daemon, but we don't know why exactly + tcpdump_file = tempfile.NamedTemporaryFile(mode='a+b', delete=False) self._tcpdump_capture_file = tcpdump_file.name proc = subprocess.Popen(cmd, shell=True, stdout=tcpdump_file, -- 1.9.3

1 1

[PATCH 2/2] NetTestController: fix url handling for python tasks
by Jan Tluka 21 Jul '14

21 Jul '14

The python task pathname was not handled properly if recipe was specified as url, e.g. lnst-ctl -d run http://example.net/recipe.xml The fix is to use abstract class RecipePath for python task pathnames in the recipe. Signed-off-by: Jan Tluka <jtluka(a)redhat.com> --- lnst/Controller/NetTestController.py | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/lnst/Controller/NetTestController.py b/lnst/Controller/NetTestController.py index 3c33d68..58a17b4 100644 --- a/lnst/Controller/NetTestController.py +++ b/lnst/Controller/NetTestController.py @@ -312,10 +312,10 @@ class NetTestController: if "python" in task_data: root = RecipePath(None, self._recipe_path).get_root() - path = "%s/%s" % (root, task_data["python"]) + path = RecipePath(root, task_data["python"]) task["python"] = path - if not os.path.isfile(path): + if not path.exists(): msg = "Task file '%s' not found." % path raise RecipeError(msg, task_data) @@ -635,8 +635,9 @@ class NetTestController: # Initialize the API handle Task.ctl = Task.ControllerAPI(self, self._machines) - name = os.path.basename(task["python"]).split(".")[0] - module = imp.load_source(name, task["python"]) + task_path = task["python"] + name = os.path.basename(task_path.abs_path()).split(".")[0] + module = imp.load_source(name, task_path.resolve()) #restore resource table self._resource_table = res_table_bkp -- 1.8.3.1

1 0

[PATCH 1/2] RecipePath: Add exists and resolve methods
by Jan Tluka 21 Jul '14

21 Jul '14

These two new methods, exists() and resolve() are needed for proper handling of python tasks if a recipe is specified as URL. The exists() method returns True if a file or URL exists, False otherwise. The resolve() method returns the filename containing the data. In case of ordinary file nothing special needs to be done. In case of URL, the file is downloaded and saved in temporary file. Signed-off-by: Jan Tluka <jtluka(a)redhat.com> --- lnst/Common/RecipePath.py | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/lnst/Common/RecipePath.py b/lnst/Common/RecipePath.py index 84c6acc..b976238 100644 --- a/lnst/Common/RecipePath.py +++ b/lnst/Common/RecipePath.py @@ -13,6 +13,7 @@ jtluka(a)redhat.com (Jan Tluka) import os from urlparse import urljoin from urllib2 import urlopen +from tempfile import NamedTemporaryFile def get_recipepath_class(root, path): if root == None: @@ -44,6 +45,12 @@ class RecipePath: def to_str(self): return self._recipepath_class.to_str() + def exists(self): + return self._recipepath_class.exists() + + def resolve(self): + return self._recipepath_class.resolve() + class RecipePathGeneric: def __init__(self, root, path): self._root = root @@ -59,6 +66,12 @@ class RecipePathGeneric: def to_str(self): pass + def exists(self): + pass + + def resolve(self): + pass + class FileRecipePath(RecipePathGeneric): def _load_file(self): f = open(self.abs_path(),'r') @@ -85,7 +98,14 @@ class FileRecipePath(RecipePathGeneric): def get_root(self): return os.path.dirname(self.abs_path()) + def exists(self): + return os.path.isfile(self.abs_path()) + + def resolve(self): + return self.abs_path() + class HttpRecipePath(RecipePathGeneric): + _file = None def _get_url(self): url = self.abs_path() @@ -113,3 +133,23 @@ class HttpRecipePath(RecipePathGeneric): def get_root(self): url = self.abs_path() return url.rpartition('/')[0] + + def exists(self): + url = self.abs_path() + try: + f = urlopen(url) + except HTTPError: + return False + + f.close() + return True + + def resolve(self): + if self._file: + return self._file.name + + self._file = NamedTemporaryFile(suffix='.py',delete=True) + self._file.write(self.to_str()) + self._file.flush() + + return self._file.name -- 1.8.3.1

1 0

[PATCH] PacketAssert: do not process EOF line
by Jan Tluka 18 Jul '14

18 Jul '14

When the tcpdump capture is empty and there are no grep_filters available the processing of dump file will incorrectly report one captured packet. The fix is to remove empty line that is returned by readline() on EOF from the processing. Signed-off-by: Jan Tluka <jtluka(a)redhat.com> --- test_modules/PacketAssert.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/test_modules/PacketAssert.py b/test_modules/PacketAssert.py index 404f371..b0a7101 100644 --- a/test_modules/PacketAssert.py +++ b/test_modules/PacketAssert.py @@ -125,7 +125,8 @@ class PacketAssert(TestGeneric): break line = line.strip("\n") - self._process_captured_line(line) + if len(line) != 0: + self._process_captured_line(line) tcpdump_file.close() os.remove(tcpdump_file.name) -- 1.8.3.1

1 0

[PATCH 1/2] SlavePool: ignore non-xml files in pools
by Ondrej Lichtner 17 Jul '14

17 Jul '14

From: Ondrej Lichtner <olichtne(a)redhat.com> The controller would crash if there was a non-xml file in a pool directory, for example hidden .swp files created by vim. This patch fixes that by ignoring all non-xml files. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> --- lnst/Controller/SlavePool.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lnst/Controller/SlavePool.py b/lnst/Controller/SlavePool.py index 00ee4b8..0049519 100644 --- a/lnst/Controller/SlavePool.py +++ b/lnst/Controller/SlavePool.py @@ -52,7 +52,9 @@ class SlavePool: res = [] for dirent in dentries: - res.append(self.add_file("%s/%s" % (pool_dir, dirent))) + m_info = self.add_file("%s/%s" % (pool_dir, dirent)) + if m_info != None: + res.append(m_info) if len(res) == 0: logging.warn("No machines found in this directory") -- 1.9.3

1 1

[lnst] remove redundant interrupt code from PacketAssert test module
by Jiří Pírko 16 Jul '14

16 Jul '14

commit 6caa514bf838c4560af3d02506b5d7922dc5afdf Author: Jan Tluka <jtluka(a)redhat.com> Date: Thu Jul 10 18:34:48 2014 +0200 remove redundant interrupt code from PacketAssert test module Because of using os.killpg(), the process group variant of kill, for background process termination the interrupt handler in PacketAssert test module lacks its original purpose. The os.killpg() method sends signal to whole process group including the tcpdump command running as forked process therefore the interrupt handler can be removed as it is redundant to send the signal to the process twice. Signed-off-by: Jan Tluka <jtluka(a)redhat.com> Signed-off-by: Jiri Pirko <jiri(a)resnulli.us> test_modules/PacketAssert.py | 13 ------------- 1 files changed, 0 insertions(+), 13 deletions(-) --- diff --git a/test_modules/PacketAssert.py b/test_modules/PacketAssert.py index 0611179..404f371 100644 --- a/test_modules/PacketAssert.py +++ b/test_modules/PacketAssert.py @@ -36,17 +36,6 @@ class PacketAssert(TestGeneric): _num_recv = 0 - def _set_interrupt_handler(self): - signal.signal(signal.SIGINT, self._interrupt_handler) - signal.signal(signal.SIGTERM, self._interrupt_handler) - - def _interrupt_handler(self, signum, frame): - """ Kill tcpdump when interrupted """ - try: - self._tcpdump.terminate() - except OSError: - raise Exception("Caught exception in interrupt handler") - def _prepare_grep_filters(self): """ Parse `grep_for' test options """ filters = self.get_multi_opt("grep_for") @@ -107,8 +96,6 @@ class PacketAssert(TestGeneric): return False def run(self): - self._set_interrupt_handler() - self._prepare_grep_filters() self._prepare_conditions() self._compose_cmd()

1 0

[PATCH] remove redundant interrupt code from PacketAssert test module
by Jan Tluka 10 Jul '14

10 Jul '14

Because of using os.killpg(), the process group variant of kill, for background process termination the interrupt handler in PacketAssert test module lacks its original purpose. The os.killpg() method sends signal to whole process group including the tcpdump command running as forked process therefore the interrupt handler can be removed as it is redundant to send the signal to the process twice. Signed-off-by: Jan Tluka <jtluka(a)redhat.com> --- test_modules/PacketAssert.py | 13 ------------- 1 file changed, 13 deletions(-) diff --git a/test_modules/PacketAssert.py b/test_modules/PacketAssert.py index 0611179..404f371 100644 --- a/test_modules/PacketAssert.py +++ b/test_modules/PacketAssert.py @@ -36,17 +36,6 @@ class PacketAssert(TestGeneric): _num_recv = 0 - def _set_interrupt_handler(self): - signal.signal(signal.SIGINT, self._interrupt_handler) - signal.signal(signal.SIGTERM, self._interrupt_handler) - - def _interrupt_handler(self, signum, frame): - """ Kill tcpdump when interrupted """ - try: - self._tcpdump.terminate() - except OSError: - raise Exception("Caught exception in interrupt handler") - def _prepare_grep_filters(self): """ Parse `grep_for' test options """ filters = self.get_multi_opt("grep_for") @@ -107,8 +96,6 @@ class PacketAssert(TestGeneric): return False def run(self): - self._set_interrupt_handler() - self._prepare_grep_filters() self._prepare_conditions() self._compose_cmd() -- 1.8.1.4

1 0

[lnst] NetTestSlave: add missing import of ExecCmdFail
by Jiří Pírko 09 Jul '14

09 Jul '14

commit ffa9169c8983dc7f9a24078cb271286fd66b9c25 Author: Jan Tluka <jtluka(a)redhat.com> Date: Wed Jul 9 17:45:56 2014 +0200 NetTestSlave: add missing import of ExecCmdFail Method restore_system_config is using exception ExecCmdFail that is not imported. This results in an exception. The patch adds missing import statement. Signed-off-by: Jan Tluka <jtluka(a)redhat.com> Signed-off-by: Jiri Pirko <jiri(a)resnulli.us> lnst/Slave/NetTestSlave.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) --- diff --git a/lnst/Slave/NetTestSlave.py b/lnst/Slave/NetTestSlave.py index 13b07a2..0b00a68 100644 --- a/lnst/Slave/NetTestSlave.py +++ b/lnst/Slave/NetTestSlave.py @@ -26,7 +26,7 @@ from lnst.Common.PacketCapture import PacketCapture from lnst.Common.Utils import die_when_parent_die from lnst.Common.NetUtils import scan_netdevs, test_tcp_connection from lnst.Common.NetUtils import normalize_hwaddr -from lnst.Common.ExecCmd import exec_cmd +from lnst.Common.ExecCmd import exec_cmd, ExecCmdFail from lnst.Common.ResourceCache import ResourceCache from lnst.Common.NetTestCommand import NetTestCommandContext from lnst.Common.NetTestCommand import CommandException, NetTestCommand

1 0

[PATCH] NetTestSlave: add missing import of ExecCmdFail
by Jan Tluka 09 Jul '14

09 Jul '14

Method restore_system_config is using exception ExecCmdFail that is not imported. This results in an exception. The patch adds missing import statement. Signed-off-by: Jan Tluka <jtluka(a)redhat.com> --- lnst/Slave/NetTestSlave.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lnst/Slave/NetTestSlave.py b/lnst/Slave/NetTestSlave.py index 13b07a2..0b00a68 100644 --- a/lnst/Slave/NetTestSlave.py +++ b/lnst/Slave/NetTestSlave.py @@ -26,7 +26,7 @@ from lnst.Common.PacketCapture import PacketCapture from lnst.Common.Utils import die_when_parent_die from lnst.Common.NetUtils import scan_netdevs, test_tcp_connection from lnst.Common.NetUtils import normalize_hwaddr -from lnst.Common.ExecCmd import exec_cmd +from lnst.Common.ExecCmd import exec_cmd, ExecCmdFail from lnst.Common.ResourceCache import ResourceCache from lnst.Common.NetTestCommand import NetTestCommandContext from lnst.Common.NetTestCommand import CommandException, NetTestCommand -- 1.8.1.4

1 0

[lnst] TaskAPI: Fix crash on missing test tool
by Jiří Pírko 09 Jul '14

09 Jul '14

commit 40d832d9b7a15b0f0947ddb6cf184aafe754b59f Author: Jan Tluka <jtluka(a)redhat.com> Date: Tue Jul 8 17:35:23 2014 +0200 TaskAPI: Fix crash on missing test tool The patch fixes the crash in case of missing test tool on slave machine. Without the patch the controller crashes with an exception because 'cmd' is undefined. Signed-off-by: Jan Tluka <jtluka(a)redhat.com> Signed-off-by: Jiri Pirko <jiri(a)resnulli.us> lnst/Controller/Task.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) --- diff --git a/lnst/Controller/Task.py b/lnst/Controller/Task.py index ad7fb68..e23ad9d 100644 --- a/lnst/Controller/Task.py +++ b/lnst/Controller/Task.py @@ -252,7 +252,7 @@ class HostAPI(object): else: msg = "Module '%s' not found on the controller"\ % mod - raise TaskError(msg, cmd) + raise TaskError(msg) for tool in tools: if tool in res_table['tools']: @@ -260,7 +260,7 @@ class HostAPI(object): else: msg = "Tool '%s' not found on the controller"\ % tool - raise TaskError(msg, cmd) + raise TaskError(msg) self._m.sync_resources(sync_table)

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

LNST-developers July 2014