LNST-developers May 2016

lnst-developers@lists.fedorahosted.org

5 participants
23 discussions

[PATCH v2 1/5] regression_tests: phase3: fix vxlan_test
by Ondrej Lichtner 20 May '16

20 May '16

From: Ondrej Lichtner <olichtne(a)redhat.com> There are no bonds in this test, and the hosts only have a single eth interface with id 'eth'. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> --- recipes/regression_tests/phase3/vxlan_test.py | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-) diff --git a/recipes/regression_tests/phase3/vxlan_test.py b/recipes/regression_tests/phase3/vxlan_test.py index f96fc46..ed14276 100644 --- a/recipes/regression_tests/phase3/vxlan_test.py +++ b/recipes/regression_tests/phase3/vxlan_test.py @@ -56,16 +56,9 @@ if nperf_cpupin: m1.run("service irqbalance stop") m2.run("service irqbalance stop") - m1_phy1 = m1.get_interface("eth1") - m1_phy2 = m1.get_interface("eth2") - dev_list = [(m1, m1_phy1), (m1, m1_phy2)] - - if test_if2.get_type() == "bond": - m2_phy1 = m2.get_interface("eth1") - m2_phy2 = m2.get_interface("eth2") - dev_list.extend([(m2, m2_phy1), (m2, m2_phy2)]) - else: - dev_list.append((m2, test_if2)) + m1_phy1 = m1.get_interface("eth") + m2_phy1 = m2.get_interface("eth") + dev_list = [(m1, m1_phy1), (m2, m2_phy1)] # this will pin devices irqs to cpu #0 for m, d in dev_list: -- 2.8.2

2 5

[PATCH] InterfaceManager, Machine: replace viewitems with items
by Ondrej Lichtner 20 May '16

20 May '16

From: Ondrej Lichtner <olichtne(a)redhat.com> The viewitems dictionary method was introduced in Python2.7. This breaks our setups on RHEL6 where only Python2.6 is present. This fixes the issue by replacing viewitems() with items(). The functionality is the same, the difference is that viewitems() is just a lightweight view at the contents of the dictionary, whereas items() creates an actual list of the contents. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> --- lnst/Controller/Machine.py | 2 +- lnst/Slave/InterfaceManager.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/lnst/Controller/Machine.py b/lnst/Controller/Machine.py index 5a627ec..c5d9191 100644 --- a/lnst/Controller/Machine.py +++ b/lnst/Controller/Machine.py @@ -1222,7 +1222,7 @@ class Device(object): def get_ip_addrs(self, selector={}): return [ip["addr"] for ip in self._ip_addrs - if selector.viewitems() <= ip.viewitems()] + if selector.items() <= ip.items()] @pre_call_decorate def get_ip_addr(self, num, selector={}): diff --git a/lnst/Slave/InterfaceManager.py b/lnst/Slave/InterfaceManager.py index 5a68f2e..5852254 100644 --- a/lnst/Slave/InterfaceManager.py +++ b/lnst/Slave/InterfaceManager.py @@ -454,7 +454,7 @@ class Device(object): def find_addrs(self, addr_spec): ret = [] for addr in self._ip_addrs: - if addr_spec.viewitems() <= addr.viewitems(): + if addr_spec.items() <= addr.items(): ret.append(addr) return ret -- 2.8.2

2 1

[PATCH] SlaveMachineParser: strip authentication type string
by Jiri Prochazka 20 May '16

20 May '16

When SlaveMachine config is created from Pool Wizard, it contains trailing whitespaces and SlaveMachineParser can't deal with them. This patch strips parsed auth_type string so configs created from Pool Wizard can be used straight away. Signed-off-by: Jiri Prochazka <jprochaz(a)redhat.com> --- lnst/Controller/SlaveMachineParser.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lnst/Controller/SlaveMachineParser.py b/lnst/Controller/SlaveMachineParser.py index 3fee680..70af0dc 100644 --- a/lnst/Controller/SlaveMachineParser.py +++ b/lnst/Controller/SlaveMachineParser.py @@ -76,7 +76,7 @@ class SlaveMachineParser(XmlParser): return sec auth_type_tag = sec_tag.find("auth_type") - sec["auth_type"] = auth_type_tag.text + sec["auth_type"] = auth_type_tag.text.strip() auth_passwd_tag = sec_tag.find("auth_password") if auth_passwd_tag is not None: -- 2.4.11

3 2

[PATCH] lnst-slave: change place where PID file is created to /var/run
by Jiri Prochazka 20 May '16

20 May '16

When running lnst-slave as daemon it tried to create PID file in current working directory, which could fail if user was in dir with read only access. This patch sets default location for PID file to be in /var/run where most of .pid files are stored. Fixes issue #169 Signed-off-by: Jiri Prochazka <jprochaz(a)redhat.com> --- lnst-slave | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lnst-slave b/lnst-slave index 93856c2..ebb4cd3 100755 --- a/lnst-slave +++ b/lnst-slave @@ -64,7 +64,7 @@ def main(): debug = False daemon = False - pidfile = "lnst-slave.pid" + pidfile = "/var/run/lnst-slave.pid" port = None coloured_output = not lnst_config.get_option("colours", "disable_colours") for opt, arg in opts: -- 2.4.11

3 2

[PATCH 1/7] regression_tests: fix vxlan_test for phase3
by Ondrej Lichtner 20 May '16

20 May '16

2 8

[PATCH v2 00/26] [RFC] PyRecipes draft
by Jiri Prochazka 17 May '16

17 May '16

Hello everyone, first working draft implementation is completed. Please, read this cover letter, check the patches and let me know what you like, what you don't like, etc. Please, note, that this is not final implementation. NOTE - patches are not rebased to current HEAD, if you want to try it, please, reset your branch to commit 2398f93. What doesn't work ================= multi_match - didn't have time and resources to check physical machines, on virt setup it works, but ends with exception due to no res value returned loopbacks - can't be created netem support - not implemented yet Description of new mechanics ============================ Since we need to run TaskAPI methods to get machine requirements from PyRecipe, methods like provision_machines, prepare_network are run after the python recipe is executed. Methods add_host and add_interface create machine requirements dict in ControllerAPI object. When lnst.match() is called, it runs match algorithm with mreq dict and if match is found, it prepares network and binds Machine and Interface objects with their HostAPI and InterfaceAPI counterparts. When everyting is prepared, execution returns to PyRecipe, where task phase of PyRecipe is executed. Task phase is the same like before, only new method is breakpoint() which will be useful for debugging, as it allows user to pause the execution in whatever part of test. TaskAPI is now used right from lnst module, all callable methods are exported via __init__.py file from lnst/ dir. Folder pyrecipes/ contains few working examples you can try. TODO ==== * multimatch support * config_only mode should be renamed and polished * TaskAPI create_ovs() method * support for loopbacks * NetEm support * polishing of the code Jiri Prochazka (26): __init__.py: draft for PyRecipes lnst-ctl: draft for PyRecipes NetTestController: remove RecipeParser import and related method calls NetTestController: add run_mode attribute NetTestController: remove obsolete methods NetTestController: use mreq dict from PyRecipe NetTestController: split provision_machines() in two methods NetTestController: cleanup _preapre_interface() NetTestController: remove resource sync from _prepare_machine() NetTestController: add multi_match attribute ti NetTestController NetTestController: don't call abs_path on _recipe_path in __init__ NetTestController: rework alias handling NetTestController: add prepare_test_env() NetTestController: rework match_setup(), config_only_recipe(), run_recipe(), _run_python_task() NetTestController: add init_taskapi() NetTestController: use Task name instead of module Task: remove deprecated methods Task: add default param for get_alias() Task: define TaskAPI methods on global level Task: add breakpoint() Task: add add_host() and init_hosts() Task: add interface handling TaskAPI methods Task: HostAPI get_id now returns its generated id Task: add HostAPI methods required by PyRecipes Task: add match() and minor rework of ControllerAPI PyRecipes: add example PyRecipes lnst-ctl | 16 +- lnst/Controller/NetTestController.py | 398 ++++++++++------------------------- lnst/Controller/Task.py | 240 +++++++++++---------- lnst/__init__.py | 1 + pyrecipes/3_vlans.py | 34 +++ pyrecipes/example.py | 33 +++ pyrecipes/ping_flood.py | 48 +++++ 7 files changed, 363 insertions(+), 407 deletions(-) create mode 100644 pyrecipes/3_vlans.py create mode 100644 pyrecipes/example.py create mode 100644 pyrecipes/ping_flood.py -- 2.4.11

4 50

[PATCH v4 1/5] SlavePool: add get method for _pools attribute
by Jiri Prochazka 17 May '16

17 May '16

This will be used in list_pools action for accessing dictionary with available pools. Signed-off-by: Jiri Prochazka <jprochaz(a)redhat.com> --- lnst/Controller/SlavePool.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/lnst/Controller/SlavePool.py b/lnst/Controller/SlavePool.py index e825998..02715ed 100644 --- a/lnst/Controller/SlavePool.py +++ b/lnst/Controller/SlavePool.py @@ -57,6 +57,9 @@ class SlavePool: self._mapper.set_pools(self._pools) logging.info("Finished loading pools.") + def get_pools(self): + return self._pools + def add_dir(self, pool_name, dir_path): logging.info("Processing pool '%s', directory '%s'" % (pool_name, dir_path)) -- 2.4.11

1 4

[PATCH v2 1/5] lnst-ctl: add list_pools action
by Jiri Prochazka 17 May '16

17 May '16

This patch adds new action called list_pools. It has two modes of execution, non-verbose and verbose. Non verbose, executed just with "lnst-ctl list_pools" command, prints out pool names and their paths in pool_name=/path/to/pool format. For verbose mode, option -v or --verbose must be entered. It prints pool names and their paths and in addition, it lists through all slave machine config files in pool directories and prints their contents in more human readable format. Pools are acquired from all the configs lnst-ctl would use in normal run. list_pools action can be combined with --pools argument in both verbose and nonverbose mode so you can filter the output as you wish. list_pools action replaces --dump-pools option changes in v2: * mentioned that list_pools action replaces --dump-pools option * used SlavePool object with new get method for getting pools instead of calling private methods Signed-off-by: Jiri Prochazka <jprochaz(a)redhat.com> --- lnst-ctl | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 56 insertions(+), 3 deletions(-) diff --git a/lnst-ctl b/lnst-ctl index f3c2eb2..4686eb2 100755 --- a/lnst-ctl +++ b/lnst-ctl @@ -24,6 +24,7 @@ from lnst.Common.Utils import mkdir_p from lnst.Controller.NetTestController import NetTestController, NetTestError from lnst.Controller.NetTestController import NoMatchError from lnst.Controller.NetTestResultSerializer import NetTestResultSerializer +from lnst.Controller.SlavePool import SlavePool from lnst.Controller.XmlProcessing import XmlProcessingError RETVAL_PASS = 0 @@ -35,7 +36,8 @@ def usage(retval=0): """ print "Usage: %s [OPTIONS...] ACTION [RECIPES...]" % sys.argv[0] print "" - print "ACTION = [ run | config_only | deconfigure | match_setup ]" + print "ACTION = [ run | config_only | deconfigure | match_setup | " \ + "list_pools]" print "" print "OPTIONS" print " -A, --override-alias name=value define top-level alias that " \ @@ -47,6 +49,8 @@ def usage(retval=0): "configuration files on stdout and exits" print " --dump-pools dumps the available machine " \ "pools and exits" + print " -v, --verbose verbose version of list_pools " \ + "command" print " -h, --help print this message" print " -m, --no-colours disable coloured terminal output" print " -o, --disable-pool-checks don't check the availability of " \ @@ -67,6 +71,46 @@ def usage(retval=0): print " -x, --result=FILE file to write xml_result" sys.exit(retval) +def list_pools(restrict_pools, verbose): + conf_pools = lnst_config.get_pools() + pools = {} + if len(restrict_pools) > 0: + for pool_name in restrict_pools: + if pool_name in conf_pools: + pools[pool_name] = conf_pools[pool_name] + elif len(restrict_pools) == 1 and os.path.isdir(pool_name): + pools = {"cmd_line_pool": pool_name} + else: + raise NetTestError("Pool %s does not exist!" % pool_name) + else: + pools = conf_pools + + sp = SlavePool(pools, pool_checks=False) + + out = "" + # iterate over all pools + sp_pools = sp.get_pools() + for pool_name, pool_content in sp_pools.iteritems(): + out += "%s (%s)\n" % (pool_name, pools[pool_name]) + # verbose output + if verbose: + # iterate over all slave machine cfgs + for filename, pool in pool_content.iteritems(): + # print in human-readable format + out += 3*" " + filename + ".xml\n" + out += 5*" " + "params:\n" + for key in pool['params']: + out += 7*" " + key+" : " + pool['params'][key] + "\n" + if pool['interfaces']: + out += 5*" " + "interfaces:\n" + for key in sorted(pool['interfaces']): + out += 7*" " + "id : " + key + "\tnetwork : " + pool['interfaces'][key]['network'] + "\n" + for param in pool['interfaces'][key]['params']: + out += 9*" " + param+" : " + pool['interfaces'][key]['params'][param] + "\n" + out += "\n" + # print wihout newlines on the end of string + print out[:-2] + def store_alias(alias_def, aliases_dict): try: name, value = alias_def.split("=") @@ -190,7 +234,7 @@ def main(): try: opts, args = getopt.getopt( sys.argv[1:], - "A:a:c:dhmoprs:t:ux:", + "A:a:c:dhmoprs:t:ux:v", [ "override_alias=", "define_alias=", @@ -208,6 +252,7 @@ def main(): "result=", "pools=", "dump-pools" + "verbose" ] ) except getopt.GetoptError as err: @@ -248,6 +293,7 @@ def main(): multi_match = False dump_config = False dump_pools = False + verbose = False pools = [] for opt, arg in opts: if opt in ("-d", "--debug"): @@ -286,6 +332,8 @@ def main(): pools.extend(arg.split(",")) elif opt in ("--dump-pools"): dump_pools = True + elif opt in ("-v", "--verbose"): + verbose= True if xslt_url != None: lnst_config.set_option("environment", "xslt_url", xslt_url) @@ -316,12 +364,17 @@ def main(): action = args[0] recipes = args[1:] - if not action in ['run', 'config_only', 'deconfigure', 'match_setup']: + if not action in ['run', 'config_only', 'deconfigure', 'match_setup', + 'list_pools']: logging.error("Action '%s' not recognised" % action) usage(RETVAL_ERR) if action == 'deconfigure': NetTestController.remove_saved_machine_config() return 0 + elif action == 'list_pools': + logging.disable(logging.CRITICAL) + list_pools(pools, verbose) + return RETVAL_PASS if recipes == []: logging.error("No recipe specified!") -- 2.4.11

2 8

[PATCH v3 1/5] SlavePool: add get method for _pools attribute
by Jiri Prochazka 17 May '16

17 May '16

1 4

[PATCH 1/3] InterfaceManager: rescan devices after reconnecting to netlink
by Ondrej Lichtner 15 May '16

15 May '16

From: Ondrej Lichtner <olichtne(a)redhat.com> When we reconnect to netlink we risk bringing the InterfaceManager into an inconsistent state by discarding broadcast messages from the old socket. Because of this we should call rescan_devices every time a new netlink socket is bound. Calling this method manually in the NetTestSlave after namespace creation is therefore not necessary anymore. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> --- lnst/Slave/InterfaceManager.py | 2 ++ lnst/Slave/NetTestSlave.py | 1 - 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/lnst/Slave/InterfaceManager.py b/lnst/Slave/InterfaceManager.py index 62b2805..9e89f2b 100644 --- a/lnst/Slave/InterfaceManager.py +++ b/lnst/Slave/InterfaceManager.py @@ -82,6 +82,8 @@ class InterfaceManager(object): self._nl_socket = IPRSocket() self._nl_socket.bind() + self.rescan_devices() + def get_nl_socket(self): return self._nl_socket diff --git a/lnst/Slave/NetTestSlave.py b/lnst/Slave/NetTestSlave.py index 1c6f47b..8695d5e 100644 --- a/lnst/Slave/NetTestSlave.py +++ b/lnst/Slave/NetTestSlave.py @@ -643,7 +643,6 @@ class SlaveMethods: self._log_ctl.set_origin_name(netns) self._log_ctl.set_connection(write_pipe) - self._if_manager.rescan_devices() logging.debug("Created network namespace %s" % netns) return True else: -- 2.8.2

2 5

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

LNST-developers May 2016