LNST-developers April 2020

lnst-developers@lists.fedorahosted.org

5 participants
28 discussions

[PATCH v3] Sphinx based documentation
by olichtne＠redhat.com 06 Apr '20

06 Apr '20

From: Ondrej Lichtner <olichtne(a)redhat.com> Creating the docs/ directory where we should work on creating documentation. The standard for Python projects is to use "Sphinx" to generate various formats (e.g. pdf, html or others) of documentation from ReStructuredText (rst) source. The source is either stored as ".rst" standalone files or as docstrings with Python source code. This is the first version that should cover some of the major topics and exported APIs available for the tester. It is by no means complete yet but should be enough to get a feel for how the overall structure should look like. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> --- v2: * add master_doc to conf.py * renamed docs source files for consistency * fixed SlavePoolManager reference * added 'none' to ascii art code-block * simplified wording for sub configuration explanation * swapped sentences for net_perf_tool and cpu_perf_tool parameters * updated docs for describe_test_wide_configuration method v3: * fixed --requirements -> --requirement typo * rebase on current master Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> --- README.md | 25 +- docs/Makefile | 20 ++ docs/source/base_enrt_class.rst | 8 + docs/source/conf.py | 66 +++++ docs/source/config_mixins.rst | 3 + docs/source/controller_api.rst | 5 + docs/source/enrt_recipes.rst | 12 + docs/source/index.rst | 25 ++ docs/source/installation.rst | 133 +++++++++ docs/source/parameters.rst | 5 + docs/source/recipe_api.rst | 5 + docs/source/recipe_packages.rst | 7 + docs/source/recipe_requirements_api.rst | 5 + docs/source/simple_network_recipe.rst | 6 + docs/source/specific_scenarios.rst | 5 + docs/source/tester_api.rst | 10 + lnst/Common/Parameters.py | 23 +- lnst/Controller/Controller.py | 98 ++++--- lnst/Controller/Recipe.py | 71 ++--- lnst/Controller/Requirements.py | 80 +++--- lnst/Controller/__init__.py | 7 + lnst/Recipes/ENRT/BaseEnrtRecipe.py | 308 ++++++++++++++++++++- lnst/Recipes/ENRT/ConfigMixins/README.md | 3 + lnst/Recipes/ENRT/ConfigMixins/__init__.py | 9 + lnst/Recipes/ENRT/README.md | 47 ++++ lnst/Recipes/ENRT/SimpleNetworkRecipe.py | 49 ++++ lnst/Recipes/ENRT/__init__.py | 52 ++++ 27 files changed, 970 insertions(+), 117 deletions(-) create mode 100644 docs/Makefile create mode 100644 docs/source/base_enrt_class.rst create mode 100644 docs/source/conf.py create mode 100644 docs/source/config_mixins.rst create mode 100644 docs/source/controller_api.rst create mode 100644 docs/source/enrt_recipes.rst create mode 100644 docs/source/index.rst create mode 100644 docs/source/installation.rst create mode 100644 docs/source/parameters.rst create mode 100644 docs/source/recipe_api.rst create mode 100644 docs/source/recipe_packages.rst create mode 100644 docs/source/recipe_requirements_api.rst create mode 100644 docs/source/simple_network_recipe.rst create mode 100644 docs/source/specific_scenarios.rst create mode 100644 docs/source/tester_api.rst create mode 100644 lnst/Recipes/ENRT/ConfigMixins/README.md create mode 100644 lnst/Recipes/ENRT/README.md diff --git a/README.md b/README.md index 256f79e..dec9aba 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ backwards compatibility are yet in place. This also means that many of our documentation resources outlining how to write recipes on the wiki are also out of date. We'll soon start working on these but -please be paitent with us. +please be patient with us. If you're interested in helping out we accept code contributions via Patches submitted to our mailing list <lnst-developers(a)lists.fedorahosted.org>. @@ -33,13 +33,13 @@ Internet Resources bellow). ## Install -LNST can be installed using python's distutils. +Installation and a simple Hello world example is available at +[Installation](docs/source/installation.rst) -```bash -su -./setup.py install -``` +## Documentation +Documentation is available in the `docs/` directory, you can build it with +`make html` using *Sphinx*. ## Authors/Contributors @@ -53,18 +53,15 @@ su * Jiri Zupka (not active anymore) * Radek Pazdera (not active anymore) +## How to contact us -## Internet Resources - -* Project Wiki: https://github.com/jpirko/lnst/wiki (currently out of date) -* Documentation: https://github.com/jpirko/lnst/wiki#learn (currently out of date) -* Git Source Tree: https://github.com/jpirko/lnst -* Mailing List: <lnst-developers(a)lists.fedorahosted.org> - +* Git Source Tree: https://github.com/jpirko/lnst +* Mailing List: <lnst-developers(a)lists.fedorahosted.org> +* IRC channel: #lnst @ freenode.net ## License -**Copyright (C) 2011-2019 Red Hat, Inc.** +**Copyright (C) 2011-2020 Red Hat, Inc.** LNST is distributed under GNU General Public License version 2. See the file "COPYING" in the source distribution for information on terms & conditions diff --git a/docs/Makefile b/docs/Makefile new file mode 100644 index 0000000..d0c3cbf --- /dev/null +++ b/docs/Makefile @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build +SOURCEDIR = source +BUILDDIR = build + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/docs/source/base_enrt_class.rst b/docs/source/base_enrt_class.rst new file mode 100644 index 0000000..32dd6fd --- /dev/null +++ b/docs/source/base_enrt_class.rst @@ -0,0 +1,8 @@ +BaseEnrtRecipe class +==================== + +.. autoclass:: lnst.Recipes.ENRT.BaseEnrtRecipe.BaseEnrtRecipe + :members: + :show-inheritance: + +.. autoclass:: lnst.Recipes.ENRT.BaseEnrtRecipe.EnrtConfiguration diff --git a/docs/source/conf.py b/docs/source/conf.py new file mode 100644 index 0000000..2f228ba --- /dev/null +++ b/docs/source/conf.py @@ -0,0 +1,66 @@ +# Configuration file for the Sphinx documentation builder. +# +# This file only contains a selection of the most common options. For a full +# list see the documentation: +# https://www.sphinx-doc.org/en/master/usage/configuration.html + +# -- Path setup -------------------------------------------------------------- + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +# import os +# import sys +# sys.path.insert(0, os.path.abspath('.')) + + +# -- Project information ----------------------------------------------------- + +project = 'lnst' +copyright = '2020, Jiri Pirko' +author = 'Jiri Pirko' + + +# -- General configuration --------------------------------------------------- + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [ + 'sphinx.ext.autodoc', +] + +import sys +sys.path = ['../'] + sys.path + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This pattern also affects html_static_path and html_extra_path. +exclude_patterns = [] + + +# -- Options for HTML output ------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +html_theme = 'classic' +html_theme_options = { + #"body_min_width": "100%", + "body_max_width": "100%", +} + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] + +autodoc_default_options = { + 'member-order': 'bysource', +} + +master_doc = "index" diff --git a/docs/source/config_mixins.rst b/docs/source/config_mixins.rst new file mode 100644 index 0000000..66119b6 --- /dev/null +++ b/docs/source/config_mixins.rst @@ -0,0 +1,3 @@ +.. ENRT Config Mixins + +.. automodule:: lnst.Recipes.ENRT.ConfigMixins diff --git a/docs/source/controller_api.rst b/docs/source/controller_api.rst new file mode 100644 index 0000000..f80d840 --- /dev/null +++ b/docs/source/controller_api.rst @@ -0,0 +1,5 @@ +Controller API +^^^^^^^^^^^^^^ + +.. automodule:: lnst.Controller.Controller + :members: Controller diff --git a/docs/source/enrt_recipes.rst b/docs/source/enrt_recipes.rst new file mode 100644 index 0000000..af8935a --- /dev/null +++ b/docs/source/enrt_recipes.rst @@ -0,0 +1,12 @@ +Early Network Regression Testing (ENRT) +======================================= + +.. automodule:: lnst.Recipes.ENRT + +.. toctree:: + :maxdepth: 2 + :caption: ENRT Components + + base_enrt_class + config_mixins + specific_scenarios diff --git a/docs/source/index.rst b/docs/source/index.rst new file mode 100644 index 0000000..1b1600f --- /dev/null +++ b/docs/source/index.rst @@ -0,0 +1,25 @@ +LNST - Linux Network Stack Test +=============================== + +Linux Network Stack Test is a framework that supports development and execution +of automated and portable network tests which usually involve multiple test +systems. This repository contains the implementation of the library to write +:any:`Recipes<BaseRecipe>`, library functions to implement executable script +files which run your tests as well as the code for the "LNST Slave" +application. + +.. toctree:: + :maxdepth: 2 + :caption: General topics: + + installation + tester_api + recipe_packages + + +Indices and tables +================== + +* :ref:`genindex` +* :ref:`modindex` +* :ref:`search` diff --git a/docs/source/installation.rst b/docs/source/installation.rst new file mode 100644 index 0000000..e7a8156 --- /dev/null +++ b/docs/source/installation.rst @@ -0,0 +1,133 @@ +Install LNST and Hello world +============================ + +LNST is logically split into two separate application use cases: + +* Controller - something that controlls the execution of your :any:`Test + Recipes<BaseRecipe>` +* Slave - a server application running on all hosts available for testing, + executes remote procedure calls from the Controller to either run tests or + configure the test machine + +Codebases for both use cases are developed in this repository and as we +currently don't have a stable release yet, the recommended method of +installation involves the following steps: + +.. code-block:: bash + + git clone https://github.com/jpirko/lnst + cd lnst + pip3 install --requirement requirements.txt + pip3 install . + + +This installs both the Controller and the Slave code, and you'll need to run +this on all the test machines that you want to use as well as the machine which +you want to use as the Controller. Optionally a Controller and a Slave CAN run +on the same machine. + +You can start your Slave application immediatelly by running:: + + lnst-slave + +Because the lnst-slave application takes care of network configuration, it +**requires** to be executed with root privileges. This is **A BIG SECURITY +RISK** so make sure you only run this application on test machines that are not +publicly accessible or don't contain any sensitive data. + +The Controller is a bit more complicated and requires you to: + +* create an executable test script +* create a slave machine pool + +Creating an executable "HelloWorld" test script +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +LNST currently doesn't come with a CLI application for the Controller, instead +we need to create an executable python script ourselves that takes care of +creating an instance of a Controller class and an instance of the Test Recipe +that we want to run and calling the Controller.run() method to execute it. + +A minimal "hello world" example of an executable test script looks like this: + +.. code-block:: python + + from lnst.Controller import Controller, HostReq, DeviceReq, BaseRecipe + + class HelloWorldRecipe(BaseRecipe): + machine1 = HostReq() + machine1.nic1 = DeviceReq(label="net1") + + machine2 = HostReq() + machine2.nic1 = DeviceReq(label="net1") + + def test(self): + self.matched.m1.nic1.ip_add("192.168.1.1/24") + self.matched.m1.nic1.up() + self.matched.m2.nic1.ip_add("192.168.1.2/24") + self.matched.m2.nic1.up() + + self.matched.m1.run("ping 192.168.1.2") + + ctl = Controller() + recipe_instance = HelloWorldRecipe() + ctl.run(recipe_instance) + + +This test requires that you have 2 test machines that are directly connected to +each other. +If you write this into a ``hello_world.py`` file you should now be able to +execute this script by running:: + + python3 hello_world.py + +And you'll end up receiving an error about being unable to find a match in your +configured pools, since we didn't configure any yet, this is quite expected. But +running this script did take care of creating a default configuration file and +directory where we'll now be able to create our machine pool. + +Creating a simple machine pool +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The default location for the Controller config file is ``~/.lnst/lnst-ctl.conf``. +At this point in time, you don't need to change anything inside this file. + +At the same time, the default location for a machine pool is ``~/.lnst/pool/``, +to create a pool you'll need to put XML files that describe your test machines +where the ``lnst-slave`` application is running, and how they're connected. You +need to create one file per test machine, so to satisfy the +**HelloWorldRecipe** requirements, we need to create two files: + +.. code-block:: bash + + touch ~/.lnst/pool/test_machine1.xml + touch ~/.lnst/pool/test_machine2.xml + +For the contents of the files you can use the following template: + +.. code-block:: xml + + <slavemachine> + <params> + <param name="hostname" value="HOSTNAME"/> + <param name="rpc_port" value="9999"/> + </params> + <interfaces> + <eth label="A" id="1"> + <params> + <param name="hwaddr" value="MAC_ADDRESS"/> + </params> + </eth> + </interfaces> + </slavemachine> + +You'll need to edit the template and replace the **HOSTNAME** and +**MAC_ADDRESS** strings with values that correspond to the hostname which the +controller can use to connet to the slave, and the mac address of a network +interface usable for testing. This **MUST** be a different interface than the +one used for the Controller-Slave connection, as it's configuration will change +during test execution, the Controller-Slave connection would break if you used +the same interface. + +After creating your pool, you should now be able to run the ``hello_world.py`` +script successfully and receive back some logs about what happened. diff --git a/docs/source/parameters.rst b/docs/source/parameters.rst new file mode 100644 index 0000000..9646b01 --- /dev/null +++ b/docs/source/parameters.rst @@ -0,0 +1,5 @@ +Parameters +^^^^^^^^^^ + +.. automodule:: lnst.Common.Parameters + :members: diff --git a/docs/source/recipe_api.rst b/docs/source/recipe_api.rst new file mode 100644 index 0000000..32e7b4c --- /dev/null +++ b/docs/source/recipe_api.rst @@ -0,0 +1,5 @@ +Recipe API +^^^^^^^^^^ + +.. automodule:: lnst.Controller.Recipe + :members: BaseRecipe, RecipeRun diff --git a/docs/source/recipe_packages.rst b/docs/source/recipe_packages.rst new file mode 100644 index 0000000..ccd6269 --- /dev/null +++ b/docs/source/recipe_packages.rst @@ -0,0 +1,7 @@ +Supported Recipe packages +========================= + +.. toctree:: + :maxdepth: 2 + + enrt_recipes diff --git a/docs/source/recipe_requirements_api.rst b/docs/source/recipe_requirements_api.rst new file mode 100644 index 0000000..4cf1c59 --- /dev/null +++ b/docs/source/recipe_requirements_api.rst @@ -0,0 +1,5 @@ +Recipe Requirements +^^^^^^^^^^^^^^^^^^^ + +.. automodule:: lnst.Controller.Requirements + :members: HostReq, DeviceReq diff --git a/docs/source/simple_network_recipe.rst b/docs/source/simple_network_recipe.rst new file mode 100644 index 0000000..1f08feb --- /dev/null +++ b/docs/source/simple_network_recipe.rst @@ -0,0 +1,6 @@ +SimpleNetworkRecipe +^^^^^^^^^^^^^^^^^^^ + +.. autoclass:: lnst.Recipes.ENRT.SimpleNetworkRecipe.SimpleNetworkRecipe + :members: + :show-inheritance: diff --git a/docs/source/specific_scenarios.rst b/docs/source/specific_scenarios.rst new file mode 100644 index 0000000..aea3230 --- /dev/null +++ b/docs/source/specific_scenarios.rst @@ -0,0 +1,5 @@ +Specific ENRT scenarios +======================= + +.. toctree:: + simple_network_recipe diff --git a/docs/source/tester_api.rst b/docs/source/tester_api.rst new file mode 100644 index 0000000..e7cf743 --- /dev/null +++ b/docs/source/tester_api.rst @@ -0,0 +1,10 @@ +Test developer API +================== + +.. toctree:: + :maxdepth: 2 + + recipe_requirements_api + recipe_api + controller_api + parameters diff --git a/lnst/Common/Parameters.py b/lnst/Common/Parameters.py index e56c32d..f73fd3a 100644 --- a/lnst/Common/Parameters.py +++ b/lnst/Common/Parameters.py @@ -4,10 +4,6 @@ This module defines the Param class, it's type specific derivatives Param instances. This can be used by a BaseRecipe class to specify optional/mandatory parameters for the entire test, or by HostReq and DeviceReq classes to define specific parameters needed for the matching algorithm. - -Copyright 2017 Red Hat, Inc. -Licensed under the GNU General Public License, version 2 as -published by the Free Software Foundation; see COPYING for details. """ __author__ = """ @@ -24,12 +20,31 @@ class ParamError(LnstError): pass class Param(object): + """Base Parameter class + + Can beused to define your own specific parameter type. Param derived classes + serve as *type checkers* to enable earlier failure of the recipe. + + :param mandatory: if `True`, marks the parameter as mandatory + :type mandatory: bool + + :param default: the default value for the parameter, is also type-checked, + immediately at Param object creation + """ def __init__(self, mandatory=False, **kwargs): self.mandatory = mandatory if "default" in kwargs: self.default = self.type_check(kwargs["default"]) def type_check(self, value): + """The type check method + + Implementation depends on the specific Param derived class. + + :return: the type checked or converted value + + :raises: :any:`ParamError` if the type check or conversion is invalid + """ return value class IntParam(Param): diff --git a/lnst/Controller/Controller.py b/lnst/Controller/Controller.py index ace0caf..160e8b7 100644 --- a/lnst/Controller/Controller.py +++ b/lnst/Controller/Controller.py @@ -2,10 +2,6 @@ This module defines the Controller class that brings together individual implementation parts of an LNST Controller. When instantiated, it allows the tester to configure and run his own recipes with the LNST 'infrastructure'. - -Copyright 2017 Red Hat, Inc. -Licensed under the GNU General Public License, version 2 as -published by the Free Software Foundation; see COPYING for details. """ __author__ = """ @@ -33,34 +29,65 @@ from lnst.Controller.Recipe import BaseRecipe, RecipeRun from lnst.Controller.RecipeControl import RecipeControl class Controller(object): - """The LNST Controller class - - Most importantly allows the tester to run instantiated Recipe tests using - the LNST infrastructure. - - Can be configured with custom implementation of several objects used for - setting up the infrastructure. + """Allows to run LNST Recipe instances + + This is the main mechanism that allows users to create their own executable + test scripts that execute LNST Recipes. + + The Controller class implementation provides the most common default values + for various parameters that can significantly change the way that Recipes + are executed. This includes custom implementations of classes that are used + for setting up the testing infrastructure such as the PoolManager or the + MachineMapper. + + :param poolMgr: + class that implements the + :py:class:`lnst.Controller.SlavePoolManager.SlavePoolManager` interface + will be instantiated by the Controller to provide the mapper with pools + available for matching, also handles the creation of + :py:class:`Machine` objects (internal LNST class used to access the + slave hosts) + :type poolMgr: + :py:class:`lnst.Controller.SlavePoolManager.SlavePoolManager` + (this is also the default class) + + :param mapper: + class that implements the + :py:class:`lnst.Controller.MachineMapper.MachineMapper` interface will + be instantiated by the Controller to match Recipe requirements to the + available pools + :type mapper: :py:class:`lnst.Controller.MachineMapper.MachineMapper` + (this is also the default class) + + :param config: + optional LNST configuration object, if None the Controller will + load it's own configuration from default paths. If not provided, the + Controller init method will create a CtlConfig object instance + automatically and load it with values from default configuration file + paths. + :type config: :py:class:`lnst.Controller.Config.CtlConfig` + + :param pools: + a list of pool names to restrict the used pool directories + :type pools: List[str] (default []) + + :param pool_checks: + if False, will disable checking the online status of Slaves + :type pool_checks: boolean (default True) + + :param debug: + sets the debug level of LNST + :type debug: integer (default 0) + + Example:: + + lnst_controller = Controller() + recipe_instance = MyRecipe(test_parameter=123) + lnst_controller.run(recipe_instance) """ def __init__(self, poolMgr=SlavePoolManager, mapper=MachineMapper, config=None, pools=[], pool_checks=True, debug=0): - """ - Args: - poolMgr -- class that implements the SlavePoolManager interface - will be instantiated by the Controller to provide the mapper - with pools available for matching, also handles the creation - of Machine objects (internal LNST class used to access the - slave hosts) - mapper -- class that implements the MachineMapper interface - will be instantiated by the Controller to match Recipe - requirements to the available pools - config -- optional LNST configuration object, if None the - Controller will load it's own configuration from default paths - pools -- a list of pool names to restrict the used pool directories - pool_checks -- boolean (default True), if False will disable - checking online status of Slaves - debug -- integer (default 0), sets debug level of LNST - """ self._config = self._load_ctl_config(config) config = self._config @@ -96,13 +123,17 @@ class Controller(object): def run(self, recipe, **kwargs): """Execute the provided Recipe - This method takes care of both finding a Slave hosts matching the Recipe - requirements, provisioning them and calling the 'test' method of the + This method takes care of both finding Slave hosts matching the Recipe + requirements, provisioning them and calling the *test* method of the Recipe object with proper references to the mapped Hosts - Args: - recipe -- an instantiated Recipe object (isinstance BaseRecipe) - kwargs -- optional keyword arguments passed to the configured Mapper + :param recipe: + an instantiated Recipe object + :type recipe: :py:class:`lnst.Controller.Recipe.BaseRecipe` + + :param kwargs: + optional keyword arguments passed to the configured Mapper + :type kwargs: Dict[str, Any] """ if not isinstance(recipe, BaseRecipe): raise ControllerError("recipe argument must be a BaseRecipe instance.") @@ -134,7 +165,6 @@ class Controller(object): finally: self._cleanup_slaves() - def _map_match(self, match, requested, recipe): self._machines = {} self._hosts = Hosts() diff --git a/lnst/Controller/Recipe.py b/lnst/Controller/Recipe.py index 42f4e3d..26e0737 100644 --- a/lnst/Controller/Recipe.py +++ b/lnst/Controller/Recipe.py @@ -1,9 +1,5 @@ """ Module implementing the BaseRecipe class. - -Copyright 2017 Red Hat, Inc. -Licensed under the GNU General Public License, version 2 as -published by the Free Software Foundation; see COPYING for details. """ __author__ = """ @@ -23,27 +19,30 @@ class RecipeError(ControllerError): pass class BaseRecipe(object): - """BaseRecipe class + """Base class for LNST Recipe definition. Every LNST Recipe written by testers should be inherited from this class. An LNST Recipe is composed of several parts: + * Requirements definition - you define recipe requirements in a derived - class by defining class attributes of the HostReq type. You can further - specify Ethernet Device requirements by defining DeviceReq attributes - of the HostReq object. - Example: - m1 = HostReq(arch="x86_64") - m1.eth0 = DeviceReq(driver="ixgbe") - * Parameter definition (optional) - you can define paramaters of your Recipe - by defining class attributes of the Param type (or inherited). These - parameters can then be accessed from the test() method to change it's - behaviour. Parameter validity (type) is checked during the - instantiation of the Recipe object by the base __init__ method. - You can define your own __init__ method to implement more complex - Parameter checking if needed, but you MUST call the base __init__ - method first. - Example: - MyRecipe(BaseRecipe): + class by defining class attributes of the HostReq type. You can further + specify Ethernet Device requirements by defining DeviceReq attributes of + the :py:class:`lnst.Controller.Requirements.HostReq` object. Example:: + + class MyRecipe(BaseRecipe): + m1 = HostReq(arch="x86_64") + m1.eth0 = DeviceReq(driver="ixgbe") + + * Parameter definition (optional) - you can define paramaters of your + Recipe by defining class attributes of the :any:`Param` type (or + inherited). These parameters can then be accessed from the test() method + to change it's behaviour. Parameter validity (type) is checked during the + instantiation of the Recipe object by the base __init__ method. You can + define your own __init__ method to implement more complex Parameter + checking if needed, but you MUST call the base __init__ method first. + Example:: + + class MyRecipe(BaseRecipe): int_param = IntParam(mandatory=True) optional_param = IntParam() @@ -55,18 +54,24 @@ class BaseRecipe(object): MyRecipe(int_param = 2, optional_param = 3) * Test definition - this is done by defining the test() method, in this - method the tester has direct access to mapped LNST slave Hosts, can - manipulate them and implement his tests. - - Attributes: - matched -- when running the Recipe the Controller will fill this - attribute with a Hosts object after the Mapper finds suitable slave - hosts. - req -- instantiated Requirements object, you can optionally change the - Recipe requirements through this object during runtime (e.g. - variable number of hosts or devices of a host based on a Parameter) - params -- instantiated Parameters object, can be used to access the - calculated parameters during Recipe initialization/execution + method the tester has direct access to mapped LNST slave Hosts, can + manipulate them and implement his tests. + + :ivar matched: + When running the Recipe the Controller will fill this attribute with a + Hosts object after the Mapper finds suitable slave hosts. + :type matched: :py:class:`lnst.Controller.Host.Hosts` + + :ivar req: + Instantiated Requirements object, you can optionally change the Recipe + requirements through this object during runtime (e.g. variable number + of hosts or devices of a host based on a Parameter) + :type req: :py:class:`lnst.Controller.Requirements._Requirements` + + :ivar params: + Instantiated Parameters object, can be used to access the calculated + parameters during Recipe initialization/execution + :type params: :py:class:`lnst.Common.Parameters.Parameters` """ def __init__(self, **kwargs): """ diff --git a/lnst/Controller/Requirements.py b/lnst/Controller/Requirements.py index 342454f..ed61996 100644 --- a/lnst/Controller/Requirements.py +++ b/lnst/Controller/Requirements.py @@ -1,19 +1,19 @@ """ -This module defines the DeviceReq and HostReq classes, which can be used to +This module defines the :py:class:`lnst.Controller.Requirements.DeviceReq` and +:py:class:`lnst.Controller.Requirements.HostReq` classes, which can be used to create a global description of Requirements for a network test. You can use -these to define class attributes of a BaseRecipe derived class to specify +these to define class attributes of a +:py:class:`lnst.Controller.Recipe.BaseRecipe` derived class to specify "general" requirements for that Recipe, or you can add them to an instance of a Recipe derived class based on it's parameters to define requirements "specific" for that single test run. -The module also specifies a Requirements class which serves as a container for -HostReq objects, while HostReq classes also serve as containers for DeviceReq -objects. The object tree created this way is translated to a dictionary used by -the internal LNST matching algorithm against available machines. - -Copyright 2017 Red Hat, Inc. -Licensed under the GNU General Public License, version 2 as -published by the Free Software Foundation; see COPYING for details. +The module also specifies a +:py:class:`lnst.Controller.Requirements._Requirements` class (currently for +internal use only) which serves as a container for HostReq objects, while +HostReq classes also serve as containers for DeviceReq objects. The object tree +created this way is translated to a dictionary used by the internal LNST +matching algorithm against available machines. """ __author__ = """ @@ -63,15 +63,22 @@ class HostReq(BaseReq): To define a Host requirement you assign a HostReq instance to a class attribute of a BaseRecipe derived class. - Example: - class MyRecipe(BaseRecipe): - m1 = HostReq() - - Args: - kwargs -- any other arguments will be treated as arbitrary string - parameters that will be matched to parameters of Slave machines - which can define their parameter values based on the implementation - of the SlaveMachineParser + + :param kwargs: + any argument will be treated as arbitrary string parameters that will + be matched to parameters of Slave machines which can define their + parameter values based on the implementation of the SlaveMachineParser + + A special case is the use of a + :py:mod:`lnst.Controller.Requirements.RecipeParam` instance as value. + This is used to link to a value provided as a Parameter to the Recipe. + :type kwargs: Dict[str, Any] + + Example:: + + class MyRecipe(BaseRecipe): + m1 = HostReq() + m2 = HostReq(architecture="x86_64") """ def reinit_with_params(self, recipe_params): super(HostReq, self).reinit_with_params(recipe_params) @@ -93,21 +100,32 @@ class HostReq(BaseReq): return res class DeviceReq(BaseReq): - """Specifies an Ethernet Device requirement + """Specifies a static test network Device requirement + + This will be used to find a matching test machine in the configured slave + machine pools, specifically this will be used to match against a test + device on a slave machine that is "statically" present on the machine. In + other words an actual REAL network device connected to a network usable for + testing. To define a Device requirement you assign a DeviceReq instance to a HostReq instance in a BaseRecipe derived class. - Example: - class MyRecipe(BaseRecipe): - m1 = HostReq() - m1.eth0 = DeviceReq(label="net1") - - Args: - label -- string value indicating the network the Device is connected to - kwargs -- any other arguments will be treated as arbitrary string - parameters that will be matched to parameters of Slave machines - which can define their parameter values based on the implementation - of the SlaveMachineParser + + :param label: + string value indicating the network the Device is connected to + :type label: string + + :param kwargs: + any other arguments will be treated as arbitrary string parameters that + will be matched to parameters of Slave machines which can define their + parameter values based on the implementation of the SlaveMachineParser + :type kwargs: Dict[str, Any] + + Example:: + + class MyRecipe(BaseRecipe): + m1 = HostReq() + m1.eth0 = DeviceReq(label="net1") """ def __init__(self, label, **kwargs): self.label = label diff --git a/lnst/Controller/__init__.py b/lnst/Controller/__init__.py index 106bb91..cb0392d 100644 --- a/lnst/Controller/__init__.py +++ b/lnst/Controller/__init__.py @@ -1,3 +1,10 @@ +""" +Controller package +================== + +This package exposes public facing APIs that can be used to create Recipes, as +well as executable test scripts. +""" from lnst.Controller.Controller import Controller from lnst.Controller.Recipe import BaseRecipe from lnst.Controller.Requirements import HostReq, DeviceReq, RecipeParam diff --git a/lnst/Recipes/ENRT/BaseEnrtRecipe.py b/lnst/Recipes/ENRT/BaseEnrtRecipe.py index cc8af57..9a35cd4 100644 --- a/lnst/Recipes/ENRT/BaseEnrtRecipe.py +++ b/lnst/Recipes/ENRT/BaseEnrtRecipe.py @@ -24,10 +24,148 @@ from lnst.RecipeCommon.Perf.Evaluators import NonzeroFlowEvaluator from lnst.RecipeCommon.Ping.Evaluators import RatePingEvaluator class EnrtConfiguration(object): + """Container object for configuration + + Intentionally left empty as it is intended to be used as a container to + store any values relevant to configuration being applied during the lifetime + of the Recipe. + """ pass class BaseEnrtRecipe(BaseSubConfigMixin, PingTestAndEvaluate, PerfRecipe): - #common requirements parameters + """Base Recipe class for the ENRT recipe package + + This class defines the shared *test* method defining the common test + procedure in a very generic way. This common test procedure involves a + single main *test_wide* configuration that is different for every specific + scenario. After the main configuration there is usually a loop of several + minor *sub* configrations types that can take different values to slightly + change the tested use cases. + + Finally, for each combination of a **test_wide** + **sub** configuration we + do a several ping connection test and several performance measurement tests. + + **test_wide** and **sub** configurations are implemented with **context + manager** methods which ensure that if any exceptions are raised (for + example because of a bug in the recipe) that deconfiguration is called. + + Both **test_wide** and **sub** configurations are to be implemented in + different classes, the BaseEnrtRecipe class only defines the common API and + the base versions of the relevant methods. + + Test wide configuration is implemented via the following methods: + + * :any:`test_wide_configuration` + * :any:`test_wide_deconfiguration` + * :any:`generate_test_wide_description` + + Sub configurations are **mixed into** classes defining the specific + scenario that is being tested. Various sub configurations are implemented as + individual Python **Mixin** classes in the + :any:`ConfigMixins<config_mixins>` package. These make use of Pythons + collaborative inheritance by calling the `super` function in a specific way. + The "machinery" for that is defined in the :any:`BaseSubConfigMixin` class. + It is then used in this class from the `test` method loop. + + :param driver: + The driver parameter is used to modify the hw network requirements, + specifically to request Devices using the specified driver. This is + common enough in the Enrt recipes that it can be part of the Base class. + + :type driver: :any:`StrParam` (default "ixgbe") + + :param ip_versions: + Parameter that determines which IP protocol versions will be tested. + :type ip_versions: Tuple[Str] (default ("ipv4", "ipv6")) + + :param ping_parallel: + Parameter used by the :any:`generate_ping_configurations` generator. + Tells the generator method to create :any:`PingConf` objects that will + be run in parallel. + :type ping_parallel: :any:`BoolParam` (default False) + + :param ping_bidirect: + Parameter used by the :any:`generate_ping_configurations` generator. + Tells the generator method to create :any:`PingConf` objects for both + directions between the ping endpoints. + :type ping_bidirect: :any:`BoolParam` (default False) + + :param ping_count: + Parameter used by the :any:`generate_ping_configurations` generator. + Tells the generator how many pings should be sent for each ping test. + :type ping_count: :any:`IntParam` (default 100) + + :param ping_interval: + Parameter used by the :any:`generate_ping_configurations` generator. + Tells the generator how fast should the pings be sent in each ping test. + :type ping_interval: :any:`FloatParam` (default 0.2) + + :param ping_psize: + Parameter used by the :any:`generate_ping_configurations` generator. + Tells the generator how big should the pings packets be in each ping + test. + :type ping_psize: :any:`IntParam` (default None) + + :param perf_tests: + Parameter used by the :any:`generate_flow_combinations` generator. + Tells the generator what types of network flow measurements to generate + perf test configurations for. + :type perf_tests: Tuple[str] (default ("tcp_stream", "udp_stream", + "sctp_stream")) + + :param perf_tool_cpu: + Parameter used by the :any:`generate_flow_combinations` generator. To + indicate that the flow measurement should be pinned to a specific CPU + core. + :type perf_tool_cpu: :any:`IntParam` (optional parameter) + + :param perf_duration: + Parameter used by the :any:`generate_perf_configurations` generator. To + specify the duration of the performance measurements, in seconds. + :type perf_duration: :any:`IntParam` (default 60) + + :param perf_iterations: + Parameter used by the :any:`generate_perf_configurations` generator. To + specify how many times should each performance measurement be repeated + to generate cumulative results which can be statistically analyzed. + :type perf_iterations: :any:`IntParam` (default 5) + + :param perf_parallel_streams: + Parameter used by the :any:`generate_flow_combinations` generator. To + specify how many parallel streams of the same network flow should be + measured at the same time. + :type perf_parallel_streams: :any:`IntParam` (default 1) + + :param perf_msg_sizes: + Parameter used by the :any:`generate_flow_combinations` generator. To + specify what different message sizes (in bytes) used generated for the + network flow should be tested - each message size resulting in a + separate performance measurement. + :type perf_msg_sizes: List[Int] (default [123]) + + :param perf_reverse: + Parameter used by the :any:`generate_flow_combinations` generator. To + specify that both directions between the endpoints of a network flow + should be measured for the same test. + :type perf_reverse: :any:`BoolParam` (default False) + + + :param net_perf_tool: + Parameter used by the :any:`generate_perf_configurations` generator to + create a PerfRecipeConf object. + Specifies a network flow measurement class that accepts :any:`PerfFlow` + objects and can be used to measure those specified flows + :type net_perf_tool: :any:`BaseFlowMeasurement` (default + IperfFlowMeasurement) + + :param cpu_perf_tool: + Parameter used by the :any:`generate_perf_configurations` generator to + create a PerfRecipeConf object. + Specifies a cpu measurement class that can be used to measure CPU + utilization on specified hosts. + :type cpu_perf_tool: :any:`BaseCPUMeasurement` (default StatCPUMeasurement) + """ + driver = StrParam(default="ixgbe") #common test parameters @@ -53,6 +191,20 @@ class BaseEnrtRecipe(BaseSubConfigMixin, PingTestAndEvaluate, PerfRecipe): cpu_perf_tool = Param(default=StatCPUMeasurement) def test(self): + """Main test loop shared by all the Enrt recipes + + The test loop involves a single application of a **test_wide** + configuration, then a loop over multiple **sub** configurations that + involves: + + * creating the combined sub configuration of all available SubConfig + Mixin classes via :any:`generate_sub_configurations` + * applying the generated sub configuration via the :any:`_sub_context` + context manager method + * running tests + * removing the current sub configuration via the :any:`_sub_context` + context manager method + """ with self._test_wide_context() as main_config: for sub_config in self.generate_sub_configurations(main_config): with self._sub_context(sub_config) as recipe_config: @@ -68,19 +220,90 @@ class BaseEnrtRecipe(BaseSubConfigMixin, PingTestAndEvaluate, PerfRecipe): self.test_wide_deconfiguration(config) def test_wide_configuration(self): + """Creates an empty :any:`EnrtConfiguration` object + + This is again used in potential collaborative inheritance design that + may potentially be useful for Enrt recipes. Derived classes will each + individually add their own values to the instance created here. This way + the complete test wide configuration is tracked in a single object. + + :return: returns a config object that tracks the applied configuration + that can be used during testing to inspect the current state and + make test decisions based on it. + :rtype: :any:`EnrtConfiguration` + + Example:: + + class Derived: + def test_wide_configuration(): + config = super().test_wide_configuration() + + # ... configure something + config.something = what_was_configured + + return config + """ return EnrtConfiguration() def test_wide_deconfiguration(self, config): + """Base deconfiguration method. + + In the base class this should maybe only check if there's any leftover + configuration and warn about it. In derived classes this can be + overriden to take care of deconfiguring what was configured in the + respective test_wide_configuration method. + + Example:: + + class Derived: + def test_wide_deconfiguration(config): + # ... deconfigure something + del config.something #cleanup tracking + + return super().test_wide_deconfiguration() + """ #TODO check if anything is still applied and throw exception? return def describe_test_wide_configuration(self, config): + """Describes the current test wide configuration + + Creates a new result object that contains the description of the full + test wide configuration applied by all the + :any:`test_wide_configuration` methods in the class hierarchy. + + The description needs to be generated by the + :any:`generate_test_wide_description` method. Additionally the + description contains the state of all the parameters and their values + passed to the recipe class instance during initialization. + """ description = self.generate_test_wide_description(config) self.add_result(True, "Summary of used Recipe parameters:\n{}".format( pprint.pformat(self.params._to_dict()))) self.add_result(True, "\n".join(description)) def generate_test_wide_description(self, config): + """Generates the test wide configuration description + + Another class inteded to be used with the collaborative version of the + `super` method to cumulatively desribe the full test wide configuration + that was applied through multiple classes. + + The base class version of this method creates the initial list of + strings containing just the header line. Each string added to this list + will later be printed on its own line. + + :return: list of strings, each representing a single line + :rtype: List[str] + + Example:: + + class Derived: + def generate_sub_configuration_description(config): + desc = super().generate_sub_configuration_description(config) + desc.append("Configured something: {}".format(config.something)) + return desc + """ return [ "Testwide configuration for recipe {} description:".format( self.__class__.__name__ @@ -101,20 +324,47 @@ class BaseEnrtRecipe(BaseSubConfigMixin, PingTestAndEvaluate, PerfRecipe): self.add_result(True, "\n".join(description)) def do_tests(self, recipe_config): + """Entry point for actual tests + + The common scenario is to do ping and performance tests, however the + method can be overriden to add more tests if needed. + """ self.do_ping_tests(recipe_config) self.do_perf_tests(recipe_config) def do_ping_tests(self, recipe_config): + """Ping testing loop + + Loops over all various ping configurations generated by the + :any:`generate_ping_configurations` method, then uses the PingRecipe + methods to execute, report and evaluate the results. + """ for ping_configs in self.generate_ping_configurations(recipe_config): result = self.ping_test(ping_configs) self.ping_report_and_evaluate(result) def do_perf_tests(self, recipe_config): + """Performance testing loop + + Loops over all various perf configurations generated by the + :any:`generate_perf_configurations` method, then uses the PerfRecipe + methods to execute, report and evaluate the results. + """ for perf_config in self.generate_perf_configurations(recipe_config): result = self.perf_test(perf_config) self.perf_report_and_evaluate(result) def generate_ping_configurations(self, config): + """Base ping test configuration generator + + The generator loops over all endpoint pairs to test ping between + (generated by the :any:`generate_ping_endpoints` method) then over all + the selected :any:`ip_versions` and finally over all the IP addresses + that fit those criteria. + + :return: list of Ping configurations to test in parallel + :rtype: List[:any:`PingConf`] + """ for endpoints in self.generate_ping_endpoints(config): for ipv in self.params.ip_versions: ip_filter = {} @@ -157,12 +407,33 @@ class BaseEnrtRecipe(BaseSubConfigMixin, PingTestAndEvaluate, PerfRecipe): yield ping_conf_list def generate_ping_endpoints(self, config): + """Generator for ping endpoints + + To be overriden by a derived class. + + :return: list of device pairs + :rtype: List[Tuple[:any:`Device`, :any:`Device`]] + """ return [] def generate_ping_evaluators(self, pconf, endpoints): return [RatePingEvaluator(min_rate=50)] def generate_perf_configurations(self, config): + """Base perf test configuration generator + + The generator loops over all flow combinations to measure performance + for (generated by the :any:`generate_flow_combinations` method). In + addition to that during each flow combination measurement we add CPU + utilization measurement to run on the background. + + Finally for each generated perf test configuration we register + measurement evaluators based on the :any:`cpu_perf_evaluators` and + :any:`net_perf_evaluators` properties. + + :return: list of Perf test configurations + :rtype: List[:any:`PerfRecipeConf`] + """ for flows in self.generate_flow_combinations(config): perf_recipe_conf=dict( recipe_config=config, @@ -199,6 +470,18 @@ class BaseEnrtRecipe(BaseSubConfigMixin, PingTestAndEvaluate, PerfRecipe): yield perf_conf def generate_flow_combinations(self, config): + """Base flow combination generator + + The generator loops over all endpoint pairs to test performance between + (generated by the :any:`generate_perf_endpoints` method) then over all + the selected :any:`ip_versions` and uses the first IP address fitting + these criteria. Then the generator loops over the selected performance + tests as selected via :any:`perf_tests`, then message sizes from + :any:`msg_sizes`. + + :return: list of Flow combinations to measure in parallel + :rtype: List[:any:`PerfFlow`] + """ for client_nic, server_nic in self.generate_perf_endpoints(config): for ipv in self.params.ip_versions: ip_filter = {} @@ -231,14 +514,37 @@ class BaseEnrtRecipe(BaseSubConfigMixin, PingTestAndEvaluate, PerfRecipe): yield [reverse_flow] def generate_perf_endpoints(self, config): + """Generator for perf endpoints + + To be overriden by a derived class. + + :return: list of device pairs + :rtype: List[Tuple[:any:`Device`, :any:`Device`]] + """ return [] @property def cpu_perf_evaluators(self): + """CPU measurement evaluators + + To be overriden by a derived class. Returns the list of evaluators to + use for CPU utilization measurement evaluation. + + :return: a list of cpu evaluator objects + :rtype: List[BaseEvaluator] + """ return [] @property def net_perf_evaluators(self): + """Network flow measurement evaluators + + To be overriden bby a derived class. Returns the list of evaluators to + use for Network flow measurement evaluation. + + :return: a list of flow evaluator objects + :rtype: List[BaseEvaluator] + """ return [NonzeroFlowEvaluator()] def wait_tentative_ips(self, devices): diff --git a/lnst/Recipes/ENRT/ConfigMixins/README.md b/lnst/Recipes/ENRT/ConfigMixins/README.md new file mode 100644 index 0000000..0472be0 --- /dev/null +++ b/lnst/Recipes/ENRT/ConfigMixins/README.md @@ -0,0 +1,3 @@ +# SubConfig Mixins + +TODO diff --git a/lnst/Recipes/ENRT/ConfigMixins/__init__.py b/lnst/Recipes/ENRT/ConfigMixins/__init__.py index e69de29..7ae6450 100644 --- a/lnst/Recipes/ENRT/ConfigMixins/__init__.py +++ b/lnst/Recipes/ENRT/ConfigMixins/__init__.py @@ -0,0 +1,9 @@ +""" +ENRT Config Mixins +================== + +.. autoclass:: lnst.Recipes.ENRT.ConfigMixins.BaseSubConfigMixin.BaseSubConfigMixin + +""" + +from lnst.Recipes.ENRT.ConfigMixins.BaseSubConfigMixin import BaseSubConfigMixin diff --git a/lnst/Recipes/ENRT/README.md b/lnst/Recipes/ENRT/README.md new file mode 100644 index 0000000..3500cdc --- /dev/null +++ b/lnst/Recipes/ENRT/README.md @@ -0,0 +1,47 @@ +# ENRT Recipes + +ENRT stands for Early Network Regression Testing, which was the name of the +project a couple of years back when we started developing this set of tests +when we were still using the Legacy LNST API. + +This package aims to reimplement the same set of tests, using the new LNST-next +APIs, while fully utilizing Python to address the largest problems with the old +implementation: +* large amounts of code duplication +* copy paste errors caused by code duplication +* very hard to maintain the test set and fix bugs +* very hard to add new tests due to limitations of the old LNST Framework + +With that in mind, the main goals for this reimplementation were as follows: +* reduce code duplication as much as possible by utilizing class inheritance +* separate individual types of configurations into smaller Mixin classes that + can be mixed and matched based on what a specific Test scenario requires +* writing new test scenarios should be very quick because it should be possible + to reuse most of the functionality already defined +* it should be possible to very easily extend the recipes with new features such + as more types of parallel or sequential measurements, more types of + Evaluations for different types of measurements, or to be able to switch out + and use different measurement tools + +The resulting design is split into several parts that interact with each other +in a specific way: + +* [BaseEnrtRecipe](BaseEnrtRecipe.py) serves as the base class for all the + specific test scenarios we want to test. Defines the common test loop and the + default implementation for configuration generators used in this common test + loop. +* [ConfigMixins](ConfigMixins/README.md) package contains the various types of + "SubConfig" mixin classes, these are classes that implement some form of + configuration on test machine(s) which can be reused between mutliple recipes, + but can often times be looped over to try different variations of the + configuration. A good example is configuration of hardware offloads, it's + relevant to test it for many different test scenarios, but only some + combinations of offloads make sense based on the scenario and we often time + want to test more than one combination +* Specific test scenario implementation that defines the requirements and the + main configuration for the specific scenario that we want to test. It also + defines the combination of various "SubConfig" configurations that we want to + include by adding them to it's inheritance tree. If required the recipe can + also override any of the default functionality defined by it's parent classes, + a good example could be the generator method for creating various + configurations for flow performance measurement. diff --git a/lnst/Recipes/ENRT/SimpleNetworkRecipe.py b/lnst/Recipes/ENRT/SimpleNetworkRecipe.py index 2f2a63a..508afdc 100644 --- a/lnst/Recipes/ENRT/SimpleNetworkRecipe.py +++ b/lnst/Recipes/ENRT/SimpleNetworkRecipe.py @@ -13,6 +13,25 @@ from lnst.RecipeCommon.Ping.PingEndpoints import PingEndpoints class SimpleNetworkRecipe( CommonHWSubConfigMixin, OffloadSubConfigMixin, BaseEnrtRecipe ): + """ + This recipe implements Enrt testing for a simple network scenario that looks + as follows + + .. code-block:: none + + +--------+ + +------+ switch +-----+ + | +--------+ | + +--+-+ +-+--+ + +-|eth0|-+ +-|eth0|-+ + | +----+ | | +----+ | + | host1 | | host2 | + +--------+ +--------+ + + All sub configurations are included via Mixin classes. + + The actual test machinery is implemented in the :any:`BaseEnrtRecipe` class. + """ host1 = HostReq() host1.eth0 = DeviceReq(label="net1", driver=RecipeParam("driver")) @@ -27,6 +46,14 @@ class SimpleNetworkRecipe( dict(gro="on", gso="on", tso="on", tx="on", rx="off"))) def test_wide_configuration(self): + """ + Test wide configuration for this recipe involves just adding an IPv4 and + IPv6 address to the matched eth0 nics on both hosts. + + host1.eth0 = 192.168.101.1/24 and fc00::1/64 + + host2.eth0 = 192.168.101.2/24 and fc00::2/64 + """ host1, host2 = self.matched.host1, self.matched.host2 configuration = super().test_wide_configuration() configuration.test_wide_devices = [] @@ -42,6 +69,9 @@ class SimpleNetworkRecipe( return configuration def generate_test_wide_description(self, config): + """ + Test wide description is extended with the configured addresses + """ desc = super().generate_test_wide_description(config) desc += [ "Configured {}.{}.ips = {}".format( @@ -52,14 +82,33 @@ class SimpleNetworkRecipe( return desc def test_wide_deconfiguration(self, config): + "" # overriding the parent docstring del config.test_wide_devices super().test_wide_deconfiguration(config) def generate_ping_endpoints(self, config): + """ + The ping endpoints for this recipe are simply the two matched NICs: + + host1.eth0 and host2.eth0 + + Returned as:: + + [PingEndpoints(self.matched.host1.eth0, self.matched.host2.eth0)] + """ return [PingEndpoints(self.matched.host1.eth0, self.matched.host2.eth0)] def generate_perf_endpoints(self, config): + """ + The perf endpoints for this recipe are simply the two matched NICs: + + host1.eth0 and host2.eth0 + + Returned as:: + + [(self.matched.host1.eth0, self.matched.host2.eth0)] + """ return [(self.matched.host1.eth0, self.matched.host2.eth0)] @property diff --git a/lnst/Recipes/ENRT/__init__.py b/lnst/Recipes/ENRT/__init__.py index 7c8be6c..89d6133 100644 --- a/lnst/Recipes/ENRT/__init__.py +++ b/lnst/Recipes/ENRT/__init__.py @@ -1,3 +1,55 @@ +''' +ENRT Recipes +------------ + +ENRT stands for Early Network Regression Testing, which was the name of the +project a couple of years back when we started developing this set of tests +when we were still using the Legacy LNST API. + +This package aims to reimplement the same set of tests, using the new LNST-next +APIs, while fully utilizing Python to address the largest problems with the old +implementation: + +* large amounts of code duplication +* copy paste errors caused by code duplication +* very hard to maintain the test set and fix bugs +* very hard to add new tests due to limitations of the old LNST Framework + +With that in mind, the main goals for this reimplementation were as follows: + +* reduce code duplication as much as possible by utilizing class inheritance +* separate individual types of configurations into smaller Mixin classes that + can be mixed and matched based on what a specific Test scenario requires +* writing new test scenarios should be very quick because it should be possible + to reuse most of the functionality already defined +* it should be possible to very easily extend the recipes with new features + such as more types of parallel or sequential measurements, more types of + Evaluations for different types of measurements, or to be able to switch out + and use different measurement tools + +The resulting design is split into several parts that interact with each other +in a specific way: + +* BaseEnrtRecipe serves as the base class for all the specific test scenarios + we want to test. Defines the common test loop and the default implementation + for configuration generators used in this common test loop. +* ConfigMixins package contains the various types of "SubConfig" mixin classes, + these are classes that implement some form of configuration on test + machine(s) which can be reused between mutliple recipes, but can often times + be looped over to try different variations of the configuration. A good + example is configuration of hardware offloads, it's relevant to test it for + many different test scenarios, but only some combinations of offloads make + sense based on the scenario and we often time want to test more than one + combination +* Specific test scenario implementation that defines the requirements and the + main configuration for the specific scenario that we want to test. It also + defines the combination of various "SubConfig" configurations that we want to + include by adding them to it's inheritance tree. If required the recipe can + also override any of the default functionality defined by it's parent + classes, a good example could be the generator method for creating various + configurations for flow performance measurement. + +''' from lnst.Recipes.ENRT.SimpleNetworkRecipe import SimpleNetworkRecipe from lnst.Recipes.ENRT.BondRecipe import BondRecipe from lnst.Recipes.ENRT.DoubleBondRecipe import DoubleBondRecipe -- 2.26.0

2 1

[PATCH] Recipes.ENRT: move wait_tentative_ips to BaseEnrtRecipe
by Jan Tluka 03 Apr '20

03 Apr '20

This method is same in all of the recipes. There's no need to duplicate the code. Signed-off-by: Jan Tluka <jtluka(a)redhat.com> --- lnst/Recipes/ENRT/BaseEnrtRecipe.py | 8 ++++++++ lnst/Recipes/ENRT/BondRecipe.py | 8 -------- lnst/Recipes/ENRT/DoubleBondRecipe.py | 8 -------- lnst/Recipes/ENRT/DoubleTeamRecipe.py | 8 -------- lnst/Recipes/ENRT/IpsecEspAeadRecipe.py | 8 -------- lnst/Recipes/ENRT/IpsecEspAhCompRecipe.py | 8 -------- lnst/Recipes/ENRT/NoVirtOvsVxlanRecipe.py | 8 -------- lnst/Recipes/ENRT/ShortLivedConnectionsRecipe.py | 8 -------- lnst/Recipes/ENRT/SimpleMacsecRecipe.py | 8 -------- lnst/Recipes/ENRT/SimpleNetworkRecipe.py | 8 -------- lnst/Recipes/ENRT/TeamRecipe.py | 8 -------- lnst/Recipes/ENRT/TeamVsBondRecipe.py | 8 -------- lnst/Recipes/ENRT/VirtOvsVxlanRecipe.py | 8 -------- .../ENRT/VirtualBridgeVlanInGuestMirroredRecipe.py | 8 -------- lnst/Recipes/ENRT/VirtualBridgeVlanInGuestRecipe.py | 8 -------- .../Recipes/ENRT/VirtualBridgeVlanInHostMirroredRecipe.py | 8 -------- lnst/Recipes/ENRT/VirtualBridgeVlanInHostRecipe.py | 8 -------- lnst/Recipes/ENRT/VirtualBridgeVlansOverBondRecipe.py | 8 -------- .../ENRT/VirtualOvsBridgeVlanInGuestMirroredRecipe.py | 8 -------- lnst/Recipes/ENRT/VirtualOvsBridgeVlanInGuestRecipe.py | 8 -------- .../ENRT/VirtualOvsBridgeVlanInHostMirroredRecipe.py | 8 -------- lnst/Recipes/ENRT/VirtualOvsBridgeVlanInHostRecipe.py | 8 -------- lnst/Recipes/ENRT/VirtualOvsBridgeVlansOverBondRecipe.py | 8 -------- lnst/Recipes/ENRT/VlansOverBondRecipe.py | 8 -------- lnst/Recipes/ENRT/VlansOverTeamRecipe.py | 8 -------- lnst/Recipes/ENRT/VlansRecipe.py | 8 -------- lnst/Recipes/ENRT/VxlanMulticastRecipe.py | 8 -------- lnst/Recipes/ENRT/VxlanRemoteRecipe.py | 8 -------- 28 files changed, 8 insertions(+), 216 deletions(-) diff --git a/lnst/Recipes/ENRT/BaseEnrtRecipe.py b/lnst/Recipes/ENRT/BaseEnrtRecipe.py index bd26cbe..cc8af57 100644 --- a/lnst/Recipes/ENRT/BaseEnrtRecipe.py +++ b/lnst/Recipes/ENRT/BaseEnrtRecipe.py @@ -241,6 +241,14 @@ class BaseEnrtRecipe(BaseSubConfigMixin, PingTestAndEvaluate, PerfRecipe): def net_perf_evaluators(self): return [NonzeroFlowEvaluator()] + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + def _create_reverse_flow(self, flow): rev_flow = PerfFlow( type = flow.type, diff --git a/lnst/Recipes/ENRT/BondRecipe.py b/lnst/Recipes/ENRT/BondRecipe.py index 466684c..6ef40db 100644 --- a/lnst/Recipes/ENRT/BondRecipe.py +++ b/lnst/Recipes/ENRT/BondRecipe.py @@ -89,14 +89,6 @@ class BondRecipe(CommonHWSubConfigMixin, OffloadSubConfigMixin, def generate_perf_endpoints(self, config): return [(self.matched.host1.bond0, self.matched.host2.eth0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): return [self.matched.host1.bond0, self.matched.host2.eth0] diff --git a/lnst/Recipes/ENRT/DoubleBondRecipe.py b/lnst/Recipes/ENRT/DoubleBondRecipe.py index 25f10ac..73a7222 100644 --- a/lnst/Recipes/ENRT/DoubleBondRecipe.py +++ b/lnst/Recipes/ENRT/DoubleBondRecipe.py @@ -96,14 +96,6 @@ class DoubleBondRecipe(CommonHWSubConfigMixin, OffloadSubConfigMixin, def generate_perf_endpoints(self, config): return [(self.matched.host1.bond0, self.matched.host2.bond0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): return [self.matched.host1.bond0, self.matched.host2.bond0] diff --git a/lnst/Recipes/ENRT/DoubleTeamRecipe.py b/lnst/Recipes/ENRT/DoubleTeamRecipe.py index 82762af..12f56fc 100644 --- a/lnst/Recipes/ENRT/DoubleTeamRecipe.py +++ b/lnst/Recipes/ENRT/DoubleTeamRecipe.py @@ -97,14 +97,6 @@ class DoubleTeamRecipe(CommonHWSubConfigMixin, OffloadSubConfigMixin, return [(self.matched.host1.team0, self.matched.host2.team0), (self.matched.host2.team0, self.matched.host1.team0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): return [self.matched.host1.team0, self.matched.host2.team0] diff --git a/lnst/Recipes/ENRT/IpsecEspAeadRecipe.py b/lnst/Recipes/ENRT/IpsecEspAeadRecipe.py index 0daf30f..845796c 100644 --- a/lnst/Recipes/ENRT/IpsecEspAeadRecipe.py +++ b/lnst/Recipes/ENRT/IpsecEspAeadRecipe.py @@ -79,14 +79,6 @@ class IpsecEspAeadRecipe(CommonHWSubConfigMixin, BaseEnrtRecipe, super().test_wide_deconfiguration(config) - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - def generate_sub_configurations(self, config): ipsec_mode = self.params.ipsec_mode spi_values = self.spi_values diff --git a/lnst/Recipes/ENRT/IpsecEspAhCompRecipe.py b/lnst/Recipes/ENRT/IpsecEspAhCompRecipe.py index 60dc2b9..a788143 100644 --- a/lnst/Recipes/ENRT/IpsecEspAhCompRecipe.py +++ b/lnst/Recipes/ENRT/IpsecEspAhCompRecipe.py @@ -80,14 +80,6 @@ class IpsecEspAhCompRecipe(CommonHWSubConfigMixin, BaseEnrtRecipe, super().test_wide_deconfiguration(config) - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - def generate_sub_configurations(self, config): ipsec_mode = self.params.ipsec_mode spi_values = self.spi_values diff --git a/lnst/Recipes/ENRT/NoVirtOvsVxlanRecipe.py b/lnst/Recipes/ENRT/NoVirtOvsVxlanRecipe.py index 3fbb384..92dc3a1 100644 --- a/lnst/Recipes/ENRT/NoVirtOvsVxlanRecipe.py +++ b/lnst/Recipes/ENRT/NoVirtOvsVxlanRecipe.py @@ -98,14 +98,6 @@ class NoVirtOvsVxlanRecipe(CommonHWSubConfigMixin, BaseEnrtRecipe): def generate_perf_endpoints(self, config): return [(self.matched.host1.int0, self.matched.host2.int0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def mtu_hw_config_dev_list(self): return [self.matched.host1.int0, self.matched.host2.int0] diff --git a/lnst/Recipes/ENRT/ShortLivedConnectionsRecipe.py b/lnst/Recipes/ENRT/ShortLivedConnectionsRecipe.py index 0bc4b51..85c768a 100644 --- a/lnst/Recipes/ENRT/ShortLivedConnectionsRecipe.py +++ b/lnst/Recipes/ENRT/ShortLivedConnectionsRecipe.py @@ -54,14 +54,6 @@ class ShortLivedConnectionsRecipe(CommonHWSubConfigMixin, BaseEnrtRecipe): def generate_perf_endpoints(self, config): return [(self.matched.host1.eth0, self.matched.host2.eth0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def mtu_hw_config_dev_list(self): return [self.matched.host1.eth0, self.matched.host2.eth0] diff --git a/lnst/Recipes/ENRT/SimpleMacsecRecipe.py b/lnst/Recipes/ENRT/SimpleMacsecRecipe.py index 131a62a..92926b0 100644 --- a/lnst/Recipes/ENRT/SimpleMacsecRecipe.py +++ b/lnst/Recipes/ENRT/SimpleMacsecRecipe.py @@ -70,14 +70,6 @@ class SimpleMacsecRecipe(CommonHWSubConfigMixin, BaseEnrtRecipe): super().test_wide_deconfiguration(config) - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - def generate_sub_configurations(self, config): for subconf in ConfMixin.generate_sub_configurations(self, config): for encryption in self.macsec_settings: diff --git a/lnst/Recipes/ENRT/SimpleNetworkRecipe.py b/lnst/Recipes/ENRT/SimpleNetworkRecipe.py index 06893f6..2f2a63a 100644 --- a/lnst/Recipes/ENRT/SimpleNetworkRecipe.py +++ b/lnst/Recipes/ENRT/SimpleNetworkRecipe.py @@ -62,14 +62,6 @@ class SimpleNetworkRecipe( def generate_perf_endpoints(self, config): return [(self.matched.host1.eth0, self.matched.host2.eth0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def no_pause_frames_dev_list(self): return [self.matched.host1.eth0, self.matched.host2.eth0] diff --git a/lnst/Recipes/ENRT/TeamRecipe.py b/lnst/Recipes/ENRT/TeamRecipe.py index 3a30b01..e083b90 100644 --- a/lnst/Recipes/ENRT/TeamRecipe.py +++ b/lnst/Recipes/ENRT/TeamRecipe.py @@ -92,14 +92,6 @@ class TeamRecipe(CommonHWSubConfigMixin, OffloadSubConfigMixin, return [(self.matched.host1.team0, self.matched.host2.eth0), (self.matched.host2.eth0, self.matched.host1.team0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): return [self.matched.host1.team0, self.matched.host2.eth0] diff --git a/lnst/Recipes/ENRT/TeamVsBondRecipe.py b/lnst/Recipes/ENRT/TeamVsBondRecipe.py index 4b66199..8971bc2 100644 --- a/lnst/Recipes/ENRT/TeamVsBondRecipe.py +++ b/lnst/Recipes/ENRT/TeamVsBondRecipe.py @@ -114,14 +114,6 @@ class TeamVsBondRecipe(CommonHWSubConfigMixin, OffloadSubConfigMixin, return [(self.matched.host1.team0, self.matched.host2.bond0), (self.matched.host2.bond0, self.matched.host1.team0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): return [self.matched.host1.team0, self.matched.host2.bond0] diff --git a/lnst/Recipes/ENRT/VirtOvsVxlanRecipe.py b/lnst/Recipes/ENRT/VirtOvsVxlanRecipe.py index 44391b7..9e9c0c5 100644 --- a/lnst/Recipes/ENRT/VirtOvsVxlanRecipe.py +++ b/lnst/Recipes/ENRT/VirtOvsVxlanRecipe.py @@ -141,14 +141,6 @@ class VirtOvsVxlanRecipe(VlanPingEvaluatorMixin, def generate_perf_endpoints(self, config): return [(self.matched.guest1.eth0, self.matched.guest3.eth0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def mtu_hw_config_dev_list(self): return [self.matched.guest1.eth0, self.matched.guest2.eth0, diff --git a/lnst/Recipes/ENRT/VirtualBridgeVlanInGuestMirroredRecipe.py b/lnst/Recipes/ENRT/VirtualBridgeVlanInGuestMirroredRecipe.py index 13eacd0..494e79d 100644 --- a/lnst/Recipes/ENRT/VirtualBridgeVlanInGuestMirroredRecipe.py +++ b/lnst/Recipes/ENRT/VirtualBridgeVlanInGuestMirroredRecipe.py @@ -127,14 +127,6 @@ class VirtualBridgeVlanInGuestMirroredRecipe(CommonHWSubConfigMixin, def generate_perf_endpoints(self, config): return [(self.matched.guest1.vlan0, self.matched.guest2.vlan0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): return [self.matched.host1.eth0, self.matched.host2.eth0, diff --git a/lnst/Recipes/ENRT/VirtualBridgeVlanInGuestRecipe.py b/lnst/Recipes/ENRT/VirtualBridgeVlanInGuestRecipe.py index d557dd5..65732bb 100644 --- a/lnst/Recipes/ENRT/VirtualBridgeVlanInGuestRecipe.py +++ b/lnst/Recipes/ENRT/VirtualBridgeVlanInGuestRecipe.py @@ -112,14 +112,6 @@ class VirtualBridgeVlanInGuestRecipe(CommonHWSubConfigMixin, def generate_perf_endpoints(self, config): return [(self.matched.guest1.vlan0, self.matched.host2.vlan0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): return [self.matched.host1.eth0, self.matched.host2.eth0, diff --git a/lnst/Recipes/ENRT/VirtualBridgeVlanInHostMirroredRecipe.py b/lnst/Recipes/ENRT/VirtualBridgeVlanInHostMirroredRecipe.py index 52d88de..f17228b 100644 --- a/lnst/Recipes/ENRT/VirtualBridgeVlanInHostMirroredRecipe.py +++ b/lnst/Recipes/ENRT/VirtualBridgeVlanInHostMirroredRecipe.py @@ -126,14 +126,6 @@ class VirtualBridgeVlanInHostMirroredRecipe(CommonHWSubConfigMixin, def generate_perf_endpoints(self, config): return [(self.matched.guest1.eth0, self.matched.guest2.eth0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): return [self.matched.host1.eth0, self.matched.host2.eth0, diff --git a/lnst/Recipes/ENRT/VirtualBridgeVlanInHostRecipe.py b/lnst/Recipes/ENRT/VirtualBridgeVlanInHostRecipe.py index 728cf06..d94ed70 100644 --- a/lnst/Recipes/ENRT/VirtualBridgeVlanInHostRecipe.py +++ b/lnst/Recipes/ENRT/VirtualBridgeVlanInHostRecipe.py @@ -110,14 +110,6 @@ class VirtualBridgeVlanInHostRecipe(CommonHWSubConfigMixin, def generate_perf_endpoints(self, config): return [(self.matched.guest1.eth0, self.matched.host2.vlan0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): return [self.matched.host1.eth0, self.matched.host2.eth0, diff --git a/lnst/Recipes/ENRT/VirtualBridgeVlansOverBondRecipe.py b/lnst/Recipes/ENRT/VirtualBridgeVlansOverBondRecipe.py index 21b3781..eb29fdc 100644 --- a/lnst/Recipes/ENRT/VirtualBridgeVlansOverBondRecipe.py +++ b/lnst/Recipes/ENRT/VirtualBridgeVlansOverBondRecipe.py @@ -188,14 +188,6 @@ class VirtualBridgeVlansOverBondRecipe(VlanPingEvaluatorMixin, def generate_perf_endpoints(self, config): return [(self.matched.guest1.eth0, self.matched.guest3.eth0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): host1, host2, guest1, guest2, guest3, guest4 = (self.matched.host1, diff --git a/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInGuestMirroredRecipe.py b/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInGuestMirroredRecipe.py index 05245c0..daba9f4 100644 --- a/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInGuestMirroredRecipe.py +++ b/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInGuestMirroredRecipe.py @@ -121,14 +121,6 @@ class VirtualOvsBridgeVlanInGuestMirroredRecipe(CommonHWSubConfigMixin, def generate_perf_endpoints(self, config): return [(self.matched.guest1.vlan0, self.matched.guest2.vlan0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): return [self.matched.host1.eth0, self.matched.host2.eth0, diff --git a/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInGuestRecipe.py b/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInGuestRecipe.py index 5ca44f8..7137600 100644 --- a/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInGuestRecipe.py +++ b/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInGuestRecipe.py @@ -105,14 +105,6 @@ class VirtualOvsBridgeVlanInGuestRecipe(CommonHWSubConfigMixin, def generate_perf_endpoints(self, config): return [(self.matched.host2.vlan0, self.matched.guest1.vlan0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): return [self.matched.host1.eth0, self.matched.host2.eth0, diff --git a/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInHostMirroredRecipe.py b/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInHostMirroredRecipe.py index b85350a..5e68862 100644 --- a/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInHostMirroredRecipe.py +++ b/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInHostMirroredRecipe.py @@ -103,14 +103,6 @@ class VirtualOvsBridgeVlanInHostMirroredRecipe(CommonHWSubConfigMixin, def generate_perf_endpoints(self, config): return [(self.matched.guest1.eth0, self.matched.guest2.eth0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): return [self.matched.host1.eth0, self.matched.host2.eth0, diff --git a/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInHostRecipe.py b/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInHostRecipe.py index 93f2c28..3355cdb 100644 --- a/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInHostRecipe.py +++ b/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInHostRecipe.py @@ -96,14 +96,6 @@ class VirtualOvsBridgeVlanInHostRecipe(CommonHWSubConfigMixin, def generate_perf_endpoints(self, config): return [(self.matched.guest1.eth0, self.matched.host2.vlan0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): return [self.matched.host1.eth0, self.matched.host2.eth0, diff --git a/lnst/Recipes/ENRT/VirtualOvsBridgeVlansOverBondRecipe.py b/lnst/Recipes/ENRT/VirtualOvsBridgeVlansOverBondRecipe.py index d6a7d35..b1ace76 100644 --- a/lnst/Recipes/ENRT/VirtualOvsBridgeVlansOverBondRecipe.py +++ b/lnst/Recipes/ENRT/VirtualOvsBridgeVlansOverBondRecipe.py @@ -157,14 +157,6 @@ class VirtualOvsBridgeVlansOverBondRecipe(VlanPingEvaluatorMixin, def generate_perf_endpoints(self, config): return [(self.matched.guest1.eth0, self.matched.guest3.eth0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): host1, host2, guest1, guest2, guest3, guest4 = (self.matched.host1, diff --git a/lnst/Recipes/ENRT/VlansOverBondRecipe.py b/lnst/Recipes/ENRT/VlansOverBondRecipe.py index dc59432..45c3e18 100644 --- a/lnst/Recipes/ENRT/VlansOverBondRecipe.py +++ b/lnst/Recipes/ENRT/VlansOverBondRecipe.py @@ -139,14 +139,6 @@ class VlansOverBondRecipe(VlanPingEvaluatorMixin, def generate_perf_endpoints(self, config): return [(self.matched.host1.vlan0, self.matched.host2.vlan0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): host1, host2 = self.matched.host1, self.matched.host2 diff --git a/lnst/Recipes/ENRT/VlansOverTeamRecipe.py b/lnst/Recipes/ENRT/VlansOverTeamRecipe.py index 1c44266..5ef430b 100644 --- a/lnst/Recipes/ENRT/VlansOverTeamRecipe.py +++ b/lnst/Recipes/ENRT/VlansOverTeamRecipe.py @@ -137,14 +137,6 @@ class VlansOverTeamRecipe(VlanPingEvaluatorMixin, def generate_perf_endpoints(self, config): return [(self.matched.host1.vlan0, self.matched.host2.vlan0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): host1, host2 = self.matched.host1, self.matched.host2 diff --git a/lnst/Recipes/ENRT/VlansRecipe.py b/lnst/Recipes/ENRT/VlansRecipe.py index b18ec15..e6cf599 100644 --- a/lnst/Recipes/ENRT/VlansRecipe.py +++ b/lnst/Recipes/ENRT/VlansRecipe.py @@ -111,14 +111,6 @@ class VlansRecipe(VlanPingEvaluatorMixin, def generate_perf_endpoints(self, config): return [(self.matched.host1.vlan0, self.matched.host2.vlan0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def offload_nics(self): return [self.matched.host1.eth0, self.matched.host2.eth0] diff --git a/lnst/Recipes/ENRT/VxlanMulticastRecipe.py b/lnst/Recipes/ENRT/VxlanMulticastRecipe.py index 235e64e..f6eec48 100644 --- a/lnst/Recipes/ENRT/VxlanMulticastRecipe.py +++ b/lnst/Recipes/ENRT/VxlanMulticastRecipe.py @@ -110,14 +110,6 @@ class VxlanMulticastRecipe(CommonHWSubConfigMixin, BaseEnrtRecipe): self.matched.guest1) return [(self.matched.host1.vxlan0, self.matched.host2.vxlan0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def mtu_hw_config_dev_list(self): host1, host2, guest1 = (self.matched.host1, self.matched.host2, diff --git a/lnst/Recipes/ENRT/VxlanRemoteRecipe.py b/lnst/Recipes/ENRT/VxlanRemoteRecipe.py index 4978b6d..465fdc1 100644 --- a/lnst/Recipes/ENRT/VxlanRemoteRecipe.py +++ b/lnst/Recipes/ENRT/VxlanRemoteRecipe.py @@ -90,14 +90,6 @@ class VxlanRemoteRecipe(CommonHWSubConfigMixin, BaseEnrtRecipe): def generate_perf_endpoints(self, config): return [(self.matched.host1.vxlan0, self.matched.host2.vxlan0)] - def wait_tentative_ips(self, devices): - def condition(): - return all( - [not ip.is_tentative for dev in devices for ip in dev.ips] - ) - - self.ctl.wait_for_condition(condition, timeout=5) - @property def mtu_hw_config_dev_list(self): return [self.matched.host1.vxlan0, self.matched.host2.vxlan0] -- 2.21.1

2 2

[PATCH] Recipes.ENRT.BaseEnrtRecipe: filter out linklocal IPv6 addresses for Perf configs
by Jan Tluka 03 Apr '20

03 Apr '20

In IPv6 scenarios the devices contain also automatically configured link-local addresses. Currently the selection of addresses chosen for the Perf flows is not deterministic and may select the link local address. This is fixed by adding the same filter that we have for ping configurations. Signed-off-by: Jan Tluka <jtluka(a)redhat.com> --- lnst/Recipes/ENRT/BaseEnrtRecipe.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/lnst/Recipes/ENRT/BaseEnrtRecipe.py b/lnst/Recipes/ENRT/BaseEnrtRecipe.py index 51b9348..bd26cbe 100644 --- a/lnst/Recipes/ENRT/BaseEnrtRecipe.py +++ b/lnst/Recipes/ENRT/BaseEnrtRecipe.py @@ -201,13 +201,15 @@ class BaseEnrtRecipe(BaseSubConfigMixin, PingTestAndEvaluate, PerfRecipe): def generate_flow_combinations(self, config): for client_nic, server_nic in self.generate_perf_endpoints(config): for ipv in self.params.ip_versions: + ip_filter = {} if ipv == "ipv4": - family = AF_INET + ip_filter.update(family = AF_INET) elif ipv == "ipv6": - family = AF_INET6 + ip_filter.update(family = AF_INET6) + ip_filter.update(is_link_local = False) - client_bind = client_nic.ips_filter(family=family)[0] - server_bind = server_nic.ips_filter(family=family)[0] + client_bind = client_nic.ips_filter(**ip_filter)[0] + server_bind = server_nic.ips_filter(**ip_filter)[0] for perf_test in self.params.perf_tests: for size in self.params.perf_msg_sizes: -- 2.21.1

2 2

[PATCH] Recipes.ENRT: include additional recipes in ENRT package
by Jan Tluka 03 Apr '20

03 Apr '20

Signed-off-by: Jan Tluka <jtluka(a)redhat.com> --- lnst/Recipes/ENRT/__init__.py | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/lnst/Recipes/ENRT/__init__.py b/lnst/Recipes/ENRT/__init__.py index ea5aaf01..7c8be6c1 100644 --- a/lnst/Recipes/ENRT/__init__.py +++ b/lnst/Recipes/ENRT/__init__.py @@ -2,10 +2,16 @@ from lnst.Recipes.ENRT.SimpleNetworkRecipe import SimpleNetworkRecipe from lnst.Recipes.ENRT.BondRecipe import BondRecipe from lnst.Recipes.ENRT.DoubleBondRecipe import DoubleBondRecipe from lnst.Recipes.ENRT.DoubleTeamRecipe import DoubleTeamRecipe +from lnst.Recipes.ENRT.IpsecEspAeadRecipe import IpsecEspAeadRecipe +from lnst.Recipes.ENRT.IpsecEspAhCompRecipe import IpsecEspAhCompRecipe +from lnst.Recipes.ENRT.NoVirtOvsVxlanRecipe import NoVirtOvsVxlanRecipe from lnst.Recipes.ENRT.OvS_DPDK_PvP import OvSDPDKPvPRecipe from lnst.Recipes.ENRT.PingFloodRecipe import PingFloodRecipe +from lnst.Recipes.ENRT.SimpleMacsecRecipe import SimpleMacsecRecipe +from lnst.Recipes.ENRT.ShortLivedConnectionsRecipe import ShortLivedConnectionsRecipe from lnst.Recipes.ENRT.TeamRecipe import TeamRecipe from lnst.Recipes.ENRT.TeamVsBondRecipe import TeamVsBondRecipe +from lnst.Recipes.ENRT.VirtOvsVxlanRecipe import VirtOvsVxlanRecipe from lnst.Recipes.ENRT.VirtualBridgeVlanInGuestMirroredRecipe import VirtualBridgeVlanInGuestMirroredRecipe from lnst.Recipes.ENRT.VirtualBridgeVlanInGuestRecipe import VirtualBridgeVlanInGuestRecipe from lnst.Recipes.ENRT.VirtualBridgeVlanInHostMirroredRecipe import VirtualBridgeVlanInHostMirroredRecipe @@ -19,5 +25,7 @@ from lnst.Recipes.ENRT.VirtualOvsBridgeVlansOverBondRecipe import VirtualOvsBrid from lnst.Recipes.ENRT.VlansOverBondRecipe import VlansOverBondRecipe from lnst.Recipes.ENRT.VlansOverTeamRecipe import VlansOverTeamRecipe from lnst.Recipes.ENRT.VlansRecipe import VlansRecipe +from lnst.Recipes.ENRT.VxlanMulticastRecipe import VxlanMulticastRecipe +from lnst.Recipes.ENRT.VxlanRemoteRecipe import VxlanRemoteRecipe from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe -- 2.21.1

2 3

[PATCH v2 1/4] Devices.OvsBridgeDevice: add _dict_to_keyvalues method
by Jan Tluka 03 Apr '20

03 Apr '20

This method transforms dictionary items into key=value pairs that are used in ovs syntax. Signed-off-by: Jan Tluka <jtluka(a)redhat.com> --- lnst/Devices/OvsBridgeDevice.py | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lnst/Devices/OvsBridgeDevice.py b/lnst/Devices/OvsBridgeDevice.py index 7e5bf0c4..044b5411 100644 --- a/lnst/Devices/OvsBridgeDevice.py +++ b/lnst/Devices/OvsBridgeDevice.py @@ -35,6 +35,13 @@ class OvsBridgeDevice(SoftDevice): def destroy(self): exec_cmd("ovs-vsctl del-br %s" % self.name) + def _dict_to_keyvalues(self, options): + opts = "" + for opt_name, opt_value in options.items(): + opts += " %s=%s" % (opt_name, opt_value) + + return opts + def port_add(self, dev, **kwargs): options = "" for opt_name, opt_value in kwargs.items(): -- 2.21.1

2 5

[PATCH v3 00/17] Refactor RecipeCommon.Ping
by Jan Tluka 03 Apr '20

03 Apr '20

This is a series of patches that attempts to remove the duplicated code in Recipes/ENRT in case of cross-vlan ping runs. The changes include: * split of ping command execution and evaluation of results * introducing two basic Ping evaluators * replacing the duplicated code in Recipes/ENRT with the new reusable code v2: * fixed a whitespace error * moved Perf BaseEvaluator to BaseResultEvaluator * used plural for ping_confing argument in ping_test() * instead of overriding the do_ping_tests base class method I added few mixin classes that do the same thing in even more reusable way * be aware of additional patches compared to previous v1 v3: * renamed get_ping_evaluators to generate_ping_evaluators * added PingEndpoints class that holds the information whether the endpoints are reachable * removed PingConditionMixin because PingEndpoints provides the needed information about endpoint reachability Jan Tluka (17): RecipeCommon.Ping: rename running_ping to ping RecipeCommon.Ping: move prepare_job to ping_init RecipeCommon.Ping: remove ping_config from ping_evaluate_and_report Tests.Ping: update failure logic Tests.Ping: do not check rtt data if no packets were received RecipeCommon.Ping: rename ping_evaluate_and_report method lnst.RecipeCommon: move Ping Recipe to separate directory lnst.RecipeCommon: rename Perf BaseEvaluator to BaseResultEvaluator lnst.RecipeCommon.Ping: add Ping evaluators RecipeCommon.Ping.Recipe: add evaluators to PingConf object lnst.RecipeCommon.Ping: add PingEndpoints class Recipes.ENRT.BaseEnrtRecipe: use PingEndpoints in generate_ping_configurations Recipes.ENRT.BaseEnrtRecipe: register default PingEvaluator RecipeCommon.Ping.Recipe: split report and evaluate lnst.Recipes.ENRT: add PingMixins lnst.Recipes.ENRT: use PingMixins to override Ping evaluators lnst.RecipeCommon.Ping.Recipe: use plural for ping_config in ping_test ...aseEvaluator.py => BaseResultEvaluator.py} | 2 +- .../Perf/Evaluators/BaselineEvaluator.py | 4 +- .../Perf/Evaluators/NonzeroFlowEvaluator.py | 4 +- .../Ping/Evaluators/RatePingEvaluator.py | 59 +++++++++++++ .../Ping/Evaluators/ZeroPassPingEvaluator.py | 21 +++++ lnst/RecipeCommon/Ping/Evaluators/__init__.py | 2 + lnst/RecipeCommon/Ping/PingEndpoints.py | 20 +++++ lnst/RecipeCommon/{Ping.py => Ping/Recipe.py} | 55 +++++++----- lnst/RecipeCommon/Ping/__init__.py | 0 lnst/Recipes/ENRT/BaseEnrtRecipe.py | 19 ++-- lnst/Recipes/ENRT/BasePvPRecipe.py | 2 +- lnst/Recipes/ENRT/BondRecipe.py | 3 +- lnst/Recipes/ENRT/DoubleBondRecipe.py | 3 +- lnst/Recipes/ENRT/DoubleTeamRecipe.py | 7 +- lnst/Recipes/ENRT/IpsecEspAeadRecipe.py | 16 ++-- lnst/Recipes/ENRT/IpsecEspAhCompRecipe.py | 26 +++--- lnst/Recipes/ENRT/NoVirtOvsVxlanRecipe.py | 3 +- lnst/Recipes/ENRT/OvS_DPDK_PvP.py | 2 +- lnst/Recipes/ENRT/PingFloodRecipe.py | 4 +- .../ENRT/PingMixins/PingEvaluatorMixins.py | 13 +++ lnst/Recipes/ENRT/PingMixins/__init__.py | 3 + lnst/Recipes/ENRT/SimpleNetworkRecipe.py | 3 +- lnst/Recipes/ENRT/TeamRecipe.py | 7 +- lnst/Recipes/ENRT/TeamVsBondRecipe.py | 7 +- lnst/Recipes/ENRT/VirtOvsVxlanRecipe.py | 77 +++-------------- .../VirtualBridgeVlanInGuestMirroredRecipe.py | 3 +- .../ENRT/VirtualBridgeVlanInGuestRecipe.py | 3 +- .../VirtualBridgeVlanInHostMirroredRecipe.py | 3 +- .../ENRT/VirtualBridgeVlanInHostRecipe.py | 3 +- .../ENRT/VirtualBridgeVlansOverBondRecipe.py | 86 ++++--------------- ...rtualOvsBridgeVlanInGuestMirroredRecipe.py | 3 +- .../ENRT/VirtualOvsBridgeVlanInGuestRecipe.py | 3 +- ...irtualOvsBridgeVlanInHostMirroredRecipe.py | 3 +- .../ENRT/VirtualOvsBridgeVlanInHostRecipe.py | 3 +- .../VirtualOvsBridgeVlansOverBondRecipe.py | 86 ++++--------------- lnst/Recipes/ENRT/VlansOverBondRecipe.py | 63 ++------------ lnst/Recipes/ENRT/VlansOverTeamRecipe.py | 64 ++------------ lnst/Recipes/ENRT/VlansRecipe.py | 63 ++------------ lnst/Recipes/ENRT/VxlanMulticastRecipe.py | 5 +- lnst/Recipes/ENRT/VxlanRemoteRecipe.py | 3 +- lnst/Tests/Ping.py | 20 ++--- 41 files changed, 317 insertions(+), 459 deletions(-) rename lnst/RecipeCommon/{Perf/Evaluators/BaseEvaluator.py => BaseResultEvaluator.py} (70%) create mode 100644 lnst/RecipeCommon/Ping/Evaluators/RatePingEvaluator.py create mode 100644 lnst/RecipeCommon/Ping/Evaluators/ZeroPassPingEvaluator.py create mode 100644 lnst/RecipeCommon/Ping/Evaluators/__init__.py create mode 100644 lnst/RecipeCommon/Ping/PingEndpoints.py rename lnst/RecipeCommon/{Ping.py => Ping/Recipe.py} (63%) create mode 100644 lnst/RecipeCommon/Ping/__init__.py create mode 100644 lnst/Recipes/ENRT/PingMixins/PingEvaluatorMixins.py create mode 100644 lnst/Recipes/ENRT/PingMixins/__init__.py -- 2.21.1

2 25

[PATCH 1/4] Devices.OvsBridgeDevice: add _dict_to_keyvalues method
by Jan Tluka 02 Apr '20

02 Apr '20

2 6

[PATCH] lnst.Recipes.ENRT.BaseEnrtRecipe: add missing FloatParam import
by olichtne＠redhat.com 01 Apr '20

01 Apr '20

From: Ondrej Lichtner <olichtne(a)redhat.com> Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> --- lnst/Recipes/ENRT/BaseEnrtRecipe.py | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/lnst/Recipes/ENRT/BaseEnrtRecipe.py b/lnst/Recipes/ENRT/BaseEnrtRecipe.py index a90c837..54e38bc 100644 --- a/lnst/Recipes/ENRT/BaseEnrtRecipe.py +++ b/lnst/Recipes/ENRT/BaseEnrtRecipe.py @@ -2,7 +2,14 @@ import pprint from contextlib import contextmanager from lnst.Common.LnstError import LnstError -from lnst.Common.Parameters import Param, IntParam, StrParam, BoolParam, ListParam +from lnst.Common.Parameters import ( + Param, + IntParam, + StrParam, + BoolParam, + ListParam, + FloatParam, +) from lnst.Common.IpAddress import AF_INET, AF_INET6 from lnst.Recipes.ENRT.ConfigMixins.BaseSubConfigMixin import BaseSubConfigMixin -- 2.25.2

3 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

LNST-developers April 2020