Currently the TRex Test has a hardcoded set of streams.
In order to make it more generic and extendable, this RFC first refactors the client and server code into a library that does not depend on other LNST code. That way, a small cli appliation can be implemented to inject traffic using the same code as LNST would. This is useful for early prototyping.
An example of such as tool is introduced: test_tools/tperf. It basically runs the client and server as LNST would and report the aggregated throughput.
Finally, the stream can be modularized into TRex compatible modules so that: - New stream generators can be easily implemented - TRex tools (e.g: stl-sym) can be used for stream generator development
The current stream generation is modularized as an example: udp_simple
I'm only a sporadic contributor to LNST so sending this RFC to get some feedback.
Adrian Moreno (3): lnst.Tests.TRex Refactor TRex client and server test_tools: Add tperf lnst.TRex Use stl compatible modules to generate streams
lnst/TRex/TRex.py | 205 ++++++++++++++++++++++++++++++++++++++++ lnst/TRex/__init__.py | 0 lnst/TRex/udp_simple.py | 38 ++++++++ lnst/Tests/TRex.py | 137 ++++----------------------- test_tools/tperf/tperf | 192 +++++++++++++++++++++++++++++++++++++ 5 files changed, 452 insertions(+), 120 deletions(-) create mode 100644 lnst/TRex/TRex.py create mode 100644 lnst/TRex/__init__.py create mode 100644 lnst/TRex/udp_simple.py create mode 100755 test_tools/tperf/tperf
Move client and server code to a new directory under lnst.TRex.
lnst.TRex is left with some wrapper code to adapt TRex client and server to the LNST framework
Signed-off-by: Adrian Moreno amorenoz@redhat.com --- lnst/TRex/TRex.py | 213 ++++++++++++++++++++++++++++++++++++++++++ lnst/TRex/__init__.py | 0 lnst/Tests/TRex.py | 137 ++++----------------------- 3 files changed, 230 insertions(+), 120 deletions(-) create mode 100644 lnst/TRex/TRex.py create mode 100644 lnst/TRex/__init__.py
diff --git a/lnst/TRex/TRex.py b/lnst/TRex/TRex.py new file mode 100644 index 0000000..e1b11dc --- /dev/null +++ b/lnst/TRex/TRex.py @@ -0,0 +1,213 @@ +import os +import sys +import time +import logging +import subprocess +import tempfile +import signal +import yaml + +TREX_CLI_DEFAULT_PARAMS = { + "warmup_time": 5, + "server_hostname": "localhost", + "trex_stl_path": 'trex_client/interactive', + "msg_size": 64 + } + +class TRexCli: + """ + TRex client. + In its constructor, it accepts any object with the following attributes + - trex_dir (str): Path to the trex directory + - ports (list): List of integer values ranging 0 to len(flows) + - flows (list): A list of tuples of dictionaries each containing the following keys: + mac_addr: Source MAC address of the flow + pci_addr: PCI address of the interface to use + ip_addr: Source IP address of the flow + - duration (int): Integer value of the duration of the test + - warmup_time (int): Time to wait before starting to take measurements. Default: 5 + - server_hostname (str): Host where the server is running. + - msg_size (int): Message size + """ + trex_stl_path = 'trex_client/interactive' + + def __init__(self, params): + self.params = params + for key in TREX_CLI_DEFAULT_PARAMS: + if key not in params: + self.params[key] = TREX_CLI_DEFAULT_PARAMS[key] + + def run(self): + sys.path.insert(0, os.path.join(self.params.trex_dir, + self.trex_stl_path)) + + from trex.stl import api as trex_api + + try: + return self._run(trex_api) + except trex_api.TRexError as e: + raise TRexError(str(e)) + + def _run(self, trex_api): + client = trex_api.STLClient(server=self.params.server_hostname) + client.connect() + + self._res_data = {} + + try: + client.acquire(ports=self.params.ports, force=True) + except: + self._res_data["msg"] = "Failed to acquire ports" + return False + + try: + client.reset(ports=self.params.ports) + except: + client.release(ports=self.params.ports) + self._res_data["msg"] = "Failed to reset ports" + return False + + for i, (src, dst) in enumerate(self.params.flows): + L2 = trex_api.Ether( + src=str(src["mac_addr"]), + dst=str(dst["mac_addr"])) + L3 = trex_api.IP( + src=str(src["ip_addr"]), + dst=str(dst["ip_addr"])) + L4 = trex_api.UDP() + base_pkt = L2/L3/L4 + + pad = max(0, self.params.msg_size - len(base_pkt)) * 'x' + packet = base_pkt/pad + + trex_packet = trex_api.STLPktBuilder(pkt=packet) + + trex_stream = trex_api.STLStream( + packet=trex_packet, + mode=trex_api.STLTXCont(percentage=100)) + + port = self.params.ports[i] + client.add_streams(trex_stream, ports=[port]) + + client.set_port_attr(ports=self.params.ports, promiscuous=True) + + + measurements = [] + + client.start(ports=self.params.ports) + + time.sleep(self.params.warmup_time) + + client.clear_stats(ports=self.params.ports) + self._res_data["start_time"] = time.time() + + for i in range(self.params.duration): + time.sleep(1) + measurements.append(dict(timestamp=time.time(), + measurement=client.get_stats( + ports=self.params.ports, + sync_now=True))) + + client.stop(ports=self.params.ports) + client.release(ports=self.params.ports) + + self._res_data["data"] = measurements + return True + +class TRexSrv: + """ + TRex server. This class runs TRex in server mode and waits for it to be killed + + In its constructor, it accepts any object with the following attributes + - trex_dir (str): Path to the trex directory + - flows (list): A list of tuples of dictionaries each containing the following keys: + mac_addr: Source MAC address of the flow + pci_addr: PCI address of the interface to use + ip_addr: Source IP address of the flow + - cores (list): List of CPU cores to use + """ + def __init__(self, params): + self.params = params + + def run(self): + trex_server_conf = [{'port_limit': len(self.params.flows), + 'version': 2, + 'interfaces': [], + 'platform': { + 'dual_if': [{ + 'socket': 0, + 'threads': self.params.cores}], + 'latency_thread_id': 0, + 'master_thread_id': 1}, + 'port_info': []}] + + for src, dst in self.params.flows: + short_pci_addr = src["pci_addr"].partition(':')[2] + trex_server_conf[0]['interfaces'].append(short_pci_addr) + trex_server_conf[0]['port_info'].append( + {'src_mac': str(src["mac_addr"]), + 'dest_mac': str(dst["mac_addr"])}) + + with tempfile.NamedTemporaryFile(mode="w+") as cfg_file: + yaml.dump(trex_server_conf, cfg_file) + cfg_file.flush() + os.fsync(cfg_file.file.fileno()) + + os.chdir(self.params.trex_dir) + server = subprocess.Popen( + [os.path.join(self.params.trex_dir, "t-rex-64"), + "--cfg", cfg_file.name, "-i"], + stdin=open('/dev/null'), stdout=open('/dev/null','w'), + stderr=subprocess.PIPE, close_fds=True) + + self._wait_for_interrupt() + + server.send_signal(signal.SIGINT) + out, err = server.communicate() + if err: + logging.error(err) + return False + return True + + def _wait_for_interrupt(self): + class InterruptException(Exception): + pass + + def handler(signum, frame): + raise InterruptException + + try: + old_handler = signal.signal(signal.SIGINT, handler) + signal.pause() + except InterruptException: + pass + finally: + signal.signal(signal.SIGINT, old_handler) + +class TRexError(Exception): + pass + +class TRexParams: + """ + TRexParams is a simple class that encapsulates a dictionary as attributes + """ + def __init__(self, **kwargs): + for key in kwargs: + setattr(self, key, kwargs[key]) + + def __str__(self): + string = "" + for key, val in list(self.__dict__.items()): + string += "%s: %s\n" % (key, str(val)) + return string + + def __iter__(self): + for attr, val in list(self.__dict__.items()): + yield (attr, val) + + def __setitem__(self, name, val): + setattr(self, name, val) + + def __getitem__(self, name, val): + getattr(self, name, val) + diff --git a/lnst/TRex/__init__.py b/lnst/TRex/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/lnst/Tests/TRex.py b/lnst/Tests/TRex.py index e09be00..000e719 100644 --- a/lnst/Tests/TRex.py +++ b/lnst/Tests/TRex.py @@ -1,20 +1,13 @@ -import os -import sys -import yaml -import time -import logging -import subprocess -import tempfile -import signal from lnst.Common.Parameters import Param, StrParam, IntParam, FloatParam from lnst.Common.Parameters import IpParam, DeviceOrIpParam from lnst.Tests.BaseTestModule import BaseTestModule, TestModuleError +from lnst.TRex.TRex import TRexCli, TRexSrv, TRexError +
class TRexCommon(BaseTestModule): trex_dir = StrParam(mandatory=True)
class TRexClient(TRexCommon): - #make Int List ports = Param(mandatory=True)
flows = Param(mandatory=True) @@ -27,6 +20,10 @@ class TRexClient(TRexCommon): server_hostname = StrParam(default="localhost") trex_stl_path = 'trex_client/interactive'
+ def __init__(self, **kwargs): + super(TRexClient, self).__init__(kwargs) + self.impl = TRexCli(self.params) + def runtime_estimate(self): _duration_overhead = 5 return (self.params.duration + @@ -34,82 +31,12 @@ class TRexClient(TRexCommon): _duration_overhead)
def run(self): - sys.path.insert(0, os.path.join(self.params.trex_dir, - self.trex_stl_path)) - - from trex.stl import api as trex_api - try: - return self._run(trex_api) - except trex_api.TRexError as e: + return self.impl.run() + except TRexError as e: #TRex errors aren't picklable so we wrap them like this raise TestModuleError(str(e))
- def _run(self, trex_api): - client = trex_api.STLClient(server=self.params.server_hostname) - client.connect() - - self._res_data = {} - - try: - client.acquire(ports=self.params.ports, force=True) - except: - self._res_data["msg"] = "Failed to acquire ports" - return False - - try: - client.reset(ports=self.params.ports) - except: - client.release(ports=self.params.ports) - self._res_data["msg"] = "Failed to reset ports" - return False - - for i, (src, dst) in enumerate(self.params.flows): - L2 = trex_api.Ether( - src=str(src["mac_addr"]), - dst=str(dst["mac_addr"])) - L3 = trex_api.IP( - src=str(src["ip_addr"]), - dst=str(dst["ip_addr"])) - L4 = trex_api.UDP() - base_pkt = L2/L3/L4 - - pad = max(0, self.params.msg_size - len(base_pkt)) * 'x' - packet = base_pkt/pad - - trex_packet = trex_api.STLPktBuilder(pkt=packet) - - trex_stream = trex_api.STLStream( - packet=trex_packet, - mode=trex_api.STLTXCont(percentage=100)) - - port = self.params.ports[i] - client.add_streams(trex_stream, ports=[port]) - - client.set_port_attr(ports=self.params.ports, promiscuous=True) - - - measurements = [] - - client.start(ports=self.params.ports) - - time.sleep(self.params.warmup_time) - - client.clear_stats(ports=self.params.ports) - self._res_data["start_time"] = time.time() - - for i in range(self.params.duration): - time.sleep(1) - measurements.append(dict(timestamp=time.time(), - measurement=client.get_stats( - ports=self.params.ports, - sync_now=True))) - - client.stop(ports=self.params.ports) - client.release(ports=self.params.ports) - - self._res_data["data"] = measurements - return True
class TRexServer(TRexCommon): #TODO make ListParam @@ -117,43 +44,13 @@ class TRexServer(TRexCommon):
cores = Param(mandatory=True)
- def run(self): - trex_server_conf = [{'port_limit': len(self.params.flows), - 'version': 2, - 'interfaces': [], - 'platform': { - 'dual_if': [{ - 'socket': 0, - 'threads': self.params.cores}], - 'latency_thread_id': 0, - 'master_thread_id': 1}, - 'port_info': []}] - - for src, dst in self.params.flows: - short_pci_addr = src["pci_addr"].partition(':')[2] - trex_server_conf[0]['interfaces'].append(short_pci_addr) - trex_server_conf[0]['port_info'].append( - {'src_mac': str(src["mac_addr"]), - 'dest_mac': str(dst["mac_addr"])}) + def __init__(self, **kwargs): + super(TRexServer, self).__init__(kwargs) + self.impl = TRexSrv(self.params)
- with tempfile.NamedTemporaryFile(mode="w+") as cfg_file: - yaml.dump(trex_server_conf, cfg_file) - cfg_file.flush() - os.fsync(cfg_file.file.fileno()) - - os.chdir(self.params.trex_dir) - server = subprocess.Popen( - [os.path.join(self.params.trex_dir, "t-rex-64"), - "--cfg", cfg_file.name, "-i"], - stdin=open('/dev/null'), stdout=open('/dev/null','w'), - stderr=subprocess.PIPE, close_fds=True) - - self.wait_for_interrupt() - - server.send_signal(signal.SIGINT) - out, err = server.communicate() - if err: - logging.error(err) - return False - - return True + def run(self): + try: + return self.impl.run() + except TRexError as e: + #TRex errors aren't picklable so we wrap them like this + raise TestModuleError(str(e))
Add a TRex Performance Tool
User the same Trex client and server code as LNST to enable quick test prototyping
Signed-off-by: Adrian Moreno amorenoz@redhat.com --- test_tools/tperf/tperf | 192 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 192 insertions(+) create mode 100755 test_tools/tperf/tperf
diff --git a/test_tools/tperf/tperf b/test_tools/tperf/tperf new file mode 100755 index 0000000..7959ad9 --- /dev/null +++ b/test_tools/tperf/tperf @@ -0,0 +1,192 @@ +#! /usr/bin/env python3 + +""" +TPerf is a TRex-based performance tool aimed to quickly inject traffic and measure +performace. + +Usage: + tperf --trex /path/to/trex-dir --server 0000:01:01.{0,1} + (in another terminal) + tperf --trex /path/to/trex-dir --client 0000:01:01.{0,1} + +TODO: + Remove hardcoded cores +""" + +import argparse +import json +import statistics +import os +import sys +import inspect + +currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe()))) +root_path = os.path.join(currentdir,"..", "..") +sys.path.insert(0, root_path) +from lnst.TRex.TRex import TRexCli, TRexSrv, TRexParams + + +def main(): + parser = argparse.ArgumentParser() + + parser.add_argument('iface', nargs=2, help='interfaces to use') + parser.add_argument('--trex', metavar='T', type=str, nargs=1, help='trex directory', required=True) + parser.add_argument('--raw', metavar='R', type=str, nargs=1, help='raw output file. If set, file where the raw output will be stored (in json format)') + parser.add_argument('--server', dest='server', action='store_const', const=True, default=False, help='Run server (default: runs client side)') + args = parser.parse_args() + + flows = get_flows(args.iface) + trex_dir=args.trex[0] + + if args.server: + trex_srv_params = TRexParams ( + trex_dir=trex_dir, + flows=flows, + cores=[6, 8], + ) + + server = TRexSrv(trex_srv_params) + server.run() + else: + trex_cli_params = TRexParams ( + trex_dir=trex_dir, + ports=list(range(len(flows))), + flows=flows, + duration=20, + ) + client = TRexCli(trex_cli_params) + client.run() + + if args.raw: + with open(args.raw[0], 'w') as f: + json.dump(client._res_data, f) + + print_stats(stats(digest(client._res_data))) + + +def get_flows(ifaces): + flow_src1= { + "mac_addr":"01:af:bf:cf:df:01", + "ip_addr":"192.168.1.1", + "pci_addr":ifaces[0] + } + flow_dest1 = { + "mac_addr":"01:af:bf:cf:df:02", + "ip_addr":"192.168.1.2", + "pci_addr":"" + } + + flow_src2= { + "mac_addr":"02:af:bf:cf:df:01", + "ip_addr":"192.168.2.1", + "pci_addr":ifaces[1] + } + flow_dest2 = { + "mac_addr":"02:af:bf:cf:df:02", + "ip_addr":"192.168.2.2", + "pci_addr":"" + } + + + return [(flow_src1, flow_dest1), (flow_src2, flow_dest2)] + +""" +Print stats +""" +def print_stats(stats): + print("----------- Test Results -----------") + print("Number of samples: %d" % stats["nsamples"]) + for port in stats["result"]: + print("Port %s:" % port) + print(" TX: %.3f Kpps" % stats["result"][port]["TX"]) + print(" RX: %.3f Kpps" % stats["result"][port]["RX"]) + + +def stats(digest): + """ + Given a digested result, calculate mean tx/rx kpps + Args: Digested samples + Returns: a dictionary with the following format + { + "nsamples": 52 + "result": { + 0: { + "TX": 2352.238 + "RX": 4581.312 + }, + 1: ... + } + } + """ + result= {} + for port in digest[0].get("packets"): + result[port]= { + "TX": statistics.mean( + [sample["packets"][port]["tx_delta"]/ + sample["time_delta"] for sample in digest]) / 1000, + "RX": statistics.mean( + [sample["packets"][0]["rx_delta"]/ + sample["time_delta"] for sample in digest]) / 1000 + } + + return { + "nsamples": len(digest), + "result": result + } + +def digest(result): + """ + Chew the results a bit and show a nice summary + Args: raw trex results + Returns: A list of samples with the following format: + [ + { + "time_delta": 0.1 + "packets" + [ + "port0": { + "tx_delta": 12345 + "rx_delta": 12334 + }, + + "port0": { + "tx_delta": 12345 + "rx_delta": 12334 + } + } + ] + """ + prev_time = result["start_time"] + prev_tx_val = {} + prev_rx_val = {} + digested_results=[] + for res in result["data"]: + sample={} + time_delta = res["timestamp"] - prev_time + sample["time_delta"]=time_delta + packets={} + + for port in res["measurement"]: + if port == "global" or port == "total" or port == "flow_stats" or port == "latency": + continue + + tx_delta = res["measurement"][port]["opackets"] - (prev_tx_val.get(port) or 0) + rx_delta = res["measurement"][port]["ipackets"] - (prev_rx_val.get(port) or 0) + + packets[port] = { + "tx_delta": tx_delta, + "rx_delta": rx_delta + } + + prev_tx_val[port] = res["measurement"][port]["opackets"] + prev_rx_val[port] = res["measurement"][port]["ipackets"] + + sample["packets"]=packets + digested_results.append(sample) + + prev_time = res["timestamp"] + + return digested_results + +if __name__ == "__main__": + main()
Use external, dynamically loaded modules to generate the streams that get added to the client.
The module API is compatible with TRex's STL [1] to ease prototyping.
[1] https://trex-tgn.cisco.com/trex/doc/trex_stateless.html
Signed-off-by: Adrian Moreno amorenoz@redhat.com --- lnst/TRex/TRex.py | 32 ++++++++++++-------------------- lnst/TRex/udp_simple.py | 38 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 50 insertions(+), 20 deletions(-) create mode 100644 lnst/TRex/udp_simple.py
diff --git a/lnst/TRex/TRex.py b/lnst/TRex/TRex.py index e1b11dc..fa41828 100644 --- a/lnst/TRex/TRex.py +++ b/lnst/TRex/TRex.py @@ -1,3 +1,4 @@ +import importlib import os import sys import time @@ -11,7 +12,8 @@ TREX_CLI_DEFAULT_PARAMS = { "warmup_time": 5, "server_hostname": "localhost", "trex_stl_path": 'trex_client/interactive', - "msg_size": 64 + "msg_size": 64, + "mod": "udp_simple" }
class TRexCli: @@ -28,6 +30,7 @@ class TRexCli: - warmup_time (int): Time to wait before starting to take measurements. Default: 5 - server_hostname (str): Host where the server is running. - msg_size (int): Message size + - mod(str): The python module to call for stream creation. Default (udp_simple) """ trex_stl_path = 'trex_client/interactive'
@@ -67,31 +70,20 @@ class TRexCli: self._res_data["msg"] = "Failed to reset ports" return False
- for i, (src, dst) in enumerate(self.params.flows): - L2 = trex_api.Ether( - src=str(src["mac_addr"]), - dst=str(dst["mac_addr"])) - L3 = trex_api.IP( - src=str(src["ip_addr"]), - dst=str(dst["ip_addr"])) - L4 = trex_api.UDP() - base_pkt = L2/L3/L4 + module = importlib.import_module('.'.join(["lnst", "TRex", self.params.mod])) + stream_generator = module.register()
- pad = max(0, self.params.msg_size - len(base_pkt)) * 'x' - packet = base_pkt/pad + for port in self.params.ports: + modkwargs = {} + modkwargs["port_id"] = port + modkwargs["msg_size"] = self.params.msg_size
- trex_packet = trex_api.STLPktBuilder(pkt=packet) + trex_streams = stream_generator.get_streams(direction=(port%2), **modkwargs)
- trex_stream = trex_api.STLStream( - packet=trex_packet, - mode=trex_api.STLTXCont(percentage=100)) - - port = self.params.ports[i] - client.add_streams(trex_stream, ports=[port]) + client.add_streams(trex_streams, ports=[port])
client.set_port_attr(ports=self.params.ports, promiscuous=True)
- measurements = []
client.start(ports=self.params.ports) diff --git a/lnst/TRex/udp_simple.py b/lnst/TRex/udp_simple.py new file mode 100644 index 0000000..3d59852 --- /dev/null +++ b/lnst/TRex/udp_simple.py @@ -0,0 +1,38 @@ +from trex_stl_lib.api import * + + +class SimpleUdp(object): + """ + Generate a simple continuous UDP stream + Port MAC and IP addresses are used + Extra arguments in kwargs: + msg_size: the size of the packet to use (default 64) + port_id: The port the stream will be added to + """ + + def create_stream (self, **kwargs): + # Use port's configured mac and ip addresses + L2 = Ether() + L3 = IP() + L4 = UDP() + + size = kwargs.get("msg_size", 64) + + base_pkt = L2/L3/L4 + + pad = max(0, size - len(base_pkt)) * 'x' + packet = base_pkt/pad + trex_packet = STLPktBuilder(pkt=packet) + + return STLStream( + packet=trex_packet, + mode=STLTXCont(percentage=100)) + + def get_streams (self, direction = 0, **kwargs): + # create 1 stream + return [ self.create_stream(**kwargs) ] + +# dynamic load - used for trex console or simulator +def register(): + return SimpleUdp() +
On Mon, Jun 15, 2020 at 06:22:53PM +0200, Adrian Moreno wrote:
Currently the TRex Test has a hardcoded set of streams.
In order to make it more generic and extendable, this RFC first refactors the client and server code into a library that does not depend on other LNST code.
I'm OK with refactoring the LNST TRex test module code into a more generic wrapper layer used by both the test module as well as the tperf tool.
I think we need to be a little careful about how to approach that so that this doesn't become "too big" at that point it probably makes more sense to completely split that into a separate project that wraps TRex (because TRex is a pain) and also provides a tperf cli for ease of use and then implementing an LNST Test module that further builds on top of this wrapper library. I think this has a lot of issues - such as supporting another project, or the fact that this really just means that TRex itself should be made better.
Also, at this point the lnst.Tests.TRex module doesn't have a very big dependency on LNST - it uses a couple of parameters for type checking and defines a module interface API, which is just that there's a "run" method, but you still kept that in the standalone library. I guess this is your main point for possible future extensions?
That way, a small cli appliation can be implemented to inject traffic using the same code as LNST would. This is useful for early prototyping.
An example of such as tool is introduced: test_tools/tperf. It basically runs the client and server as LNST would and report the aggregated throughput.
I also like the idea of this small "tperf" tool to help with prototyping, that definitelly sounds interesting since TRex can be a lot of pain... probably not worth investigating a generic solution at this point as everything else is much simpler IMO.
Finally, the stream can be modularized into TRex compatible modules so that:
- New stream generators can be easily implemented
- TRex tools (e.g: stl-sym) can be used for stream generator development
This took me a bit to understand... I was confused by the module implementation with a "register" function... I now see that it's another one of TRex weirdnes, I would personally prefer a class based approach but I guess this is fine since it follows some "standard"
The current stream generation is modularized as an example: udp_simple
So does this patchset require any other changes to the recipes that currently use TRex? or is it drop in? Haven't tested this since I don't have a pre-setup dpdk environment...
I'm only a sporadic contributor to LNST so sending this RFC to get some feedback.
I think the only thing I would complain about is the placement of the "lnst.TRex" module... I'm not sure I want it at this top level, I think it also contributed to my slower understanding of the RFC. Not exactly sure where to put it though. I'll keep thinking about this
-Ondrej
Adrian Moreno (3): lnst.Tests.TRex Refactor TRex client and server test_tools: Add tperf lnst.TRex Use stl compatible modules to generate streams
lnst/TRex/TRex.py | 205 ++++++++++++++++++++++++++++++++++++++++ lnst/TRex/__init__.py | 0 lnst/TRex/udp_simple.py | 38 ++++++++ lnst/Tests/TRex.py | 137 ++++----------------------- test_tools/tperf/tperf | 192 +++++++++++++++++++++++++++++++++++++ 5 files changed, 452 insertions(+), 120 deletions(-) create mode 100644 lnst/TRex/TRex.py create mode 100644 lnst/TRex/__init__.py create mode 100644 lnst/TRex/udp_simple.py create mode 100755 test_tools/tperf/tperf
-- , 2.26.2
LNST-developers mailing list -- lnst-developers@lists.fedorahosted.org To unsubscribe send an email to lnst-developers-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/lnst-developers@lists.fedorahos...
Thanks for your feedback Ondrej.
Comments below.
On 6/18/20 12:21 PM, Ondrej Lichtner wrote:
On Mon, Jun 15, 2020 at 06:22:53PM +0200, Adrian Moreno wrote:
Currently the TRex Test has a hardcoded set of streams.
In order to make it more generic and extendable, this RFC first refactors the client and server code into a library that does not depend on other LNST code.
I'm OK with refactoring the LNST TRex test module code into a more generic wrapper layer used by both the test module as well as the tperf tool.
I think we need to be a little careful about how to approach that so that this doesn't become "too big" at that point it probably makes more sense to completely split that into a separate project that wraps TRex (because TRex is a pain) and also provides a tperf cli for ease of use and then implementing an LNST Test module that further builds on top of this wrapper library. I think this has a lot of issues - such as supporting another project, or the fact that this really just means that TRex itself should be made better.
For now, I would not like to pay the maintenance price of another project to be honest :)
Also, at this point the lnst.Tests.TRex module doesn't have a very big dependency on LNST - it uses a couple of parameters for type checking and defines a module interface API, which is just that there's a "run" method, but you still kept that in the standalone library. I guess this is your main point for possible future extensions?
Right, the more that can be added to the standalone library, the better because the development is much easier. However, I must admit it's my lack of expertise in LNST talking here.
Could we implement tperf using the existing "Tests/Trex.py" without having to run a full blown lnst-slave? If it's possible without doing too much LNST-specific things, that would save us the refactoring, the weird "placement" and reduce the temptations of creating another project. What do you think?
That way, a small cli appliation can be implemented to inject traffic using the same code as LNST would. This is useful for early prototyping.
An example of such as tool is introduced: test_tools/tperf. It basically runs the client and server as LNST would and report the aggregated throughput.
I also like the idea of this small "tperf" tool to help with prototyping, that definitelly sounds interesting since TRex can be a lot of pain... probably not worth investigating a generic solution at this point as everything else is much simpler IMO.
Finally, the stream can be modularized into TRex compatible modules so that:
- New stream generators can be easily implemented
- TRex tools (e.g: stl-sym) can be used for stream generator development
This took me a bit to understand... I was confused by the module implementation with a "register" function... I now see that it's another one of TRex weirdnes, I would personally prefer a class based approach but I guess this is fine since it follows some "standard"
Right. It's a trex thing. We can have another class-based API for LNST but keep the trex stl one for prototyping. But we might break the latter without knowing.
The current stream generation is modularized as an example: udp_simple
So does this patchset require any other changes to the recipes that currently use TRex? or is it drop in? Haven't tested this since I don't have a pre-setup dpdk environment...
Right now, no change is needed. The default module implements the current functionality so we're good.
When we add more modules, we can have an optional parameter that can be set by new tests pointing to the new modules.
I'm only a sporadic contributor to LNST so sending this RFC to get some feedback.
I think the only thing I would complain about is the placement of the "lnst.TRex" module... I'm not sure I want it at this top level, I think it also contributed to my slower understanding of the RFC. Not exactly sure where to put it though. I'll keep thinking about this
I agree. I dislike the placement as well.
If you agree with the approach described above (tperf depend on Tests/Trex.py directly) we could have:
lnst/Tests/Trex.py lnst/Tests/TRexModules/udp_simple.py lnst/Tests/TRexModules/...(otherModules).py test-tools/tperf/tperf
What do you think?
-Ondrej
Adrian Moreno (3): lnst.Tests.TRex Refactor TRex client and server test_tools: Add tperf lnst.TRex Use stl compatible modules to generate streams
lnst/TRex/TRex.py | 205 ++++++++++++++++++++++++++++++++++++++++ lnst/TRex/__init__.py | 0 lnst/TRex/udp_simple.py | 38 ++++++++ lnst/Tests/TRex.py | 137 ++++----------------------- test_tools/tperf/tperf | 192 +++++++++++++++++++++++++++++++++++++ 5 files changed, 452 insertions(+), 120 deletions(-) create mode 100644 lnst/TRex/TRex.py create mode 100644 lnst/TRex/__init__.py create mode 100644 lnst/TRex/udp_simple.py create mode 100755 test_tools/tperf/tperf
-- , 2.26.2
LNST-developers mailing list -- lnst-developers@lists.fedorahosted.org To unsubscribe send an email to lnst-developers-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/lnst-developers@lists.fedorahos...
On Thu, Jun 18, 2020 at 05:15:53PM +0200, Adrian Moreno wrote:
Thanks for your feedback Ondrej.
Comments below.
On 6/18/20 12:21 PM, Ondrej Lichtner wrote:
On Mon, Jun 15, 2020 at 06:22:53PM +0200, Adrian Moreno wrote:
For now, I would not like to pay the maintenance price of another project to be honest :)
Same here, hence why I said it's an "issue" :).
Also, at this point the lnst.Tests.TRex module doesn't have a very big dependency on LNST - it uses a couple of parameters for type checking and defines a module interface API, which is just that there's a "run" method, but you still kept that in the standalone library. I guess this is your main point for possible future extensions?
Right, the more that can be added to the standalone library, the better because the development is much easier. However, I must admit it's my lack of expertise in LNST talking here.
Could we implement tperf using the existing "Tests/Trex.py" without having to run a full blown lnst-slave? If it's possible without doing too much LNST-specific things, that would save us the refactoring, the weird "placement" and reduce the temptations of creating another project. What do you think?
So... without context I think the _current_ implementation of tperf could be done without the refactor, the lnst.Test.TRex really doesn't require a running lnst-slave, it's actually a "reverse dependency" - if you want to run trex on a slave then you need the lnst.Test.TRex module - the slave depends on the module, not the other way around.
BUT, with context I think a refactor and split into a "generic" lib is better for future development.
Finally, the stream can be modularized into TRex compatible modules so that:
- New stream generators can be easily implemented
- TRex tools (e.g: stl-sym) can be used for stream generator development
This took me a bit to understand... I was confused by the module implementation with a "register" function... I now see that it's another one of TRex weirdnes, I would personally prefer a class based approach but I guess this is fine since it follows some "standard"
Right. It's a trex thing. We can have another class-based API for LNST but keep the trex stl one for prototyping. But we might break the latter without knowing.
Nah, I think this is good, reimplementing the lnst stream creation via the simple TRex module is IMO better as long as it integrates well (and it looks like it does) with how we want to run if from actual recipes.
And refactoring the lnst.trex module so that it supports the generic trex modules is good as it just expands the possibilities for lnst.
My original implementation is just flawed because I had a hard time getting into TRex and this was the easiest way _for me at the time_ to get it running...
The current stream generation is modularized as an example: udp_simple
So does this patchset require any other changes to the recipes that currently use TRex? or is it drop in? Haven't tested this since I don't have a pre-setup dpdk environment...
Right now, no change is needed. The default module implements the current functionality so we're good.
When we add more modules, we can have an optional parameter that can be set by new tests pointing to the new modules.
cool
I'm only a sporadic contributor to LNST so sending this RFC to get some feedback.
I think the only thing I would complain about is the placement of the "lnst.TRex" module... I'm not sure I want it at this top level, I think it also contributed to my slower understanding of the RFC. Not exactly sure where to put it though. I'll keep thinking about this
I agree. I dislike the placement as well.
If you agree with the approach described above (tperf depend on Tests/Trex.py directly) we could have:
lnst/Tests/Trex.py lnst/Tests/TRexModules/udp_simple.py lnst/Tests/TRexModules/...(otherModules).py test-tools/tperf/tperf
What do you think?
So if keeping the refactor I'm thinking something like:
lnst/Tests/TRex.py for the test module lnst/Tests/TRexWrap/*.py for the refactored trex wrapper code lnst/Tests/TRexModules/udp_simple.py sounds good for additional stream generation modules
Another option could be:
lnst/Tests/TRex/TestModule.py for the test module lnst/Tests/TRex/*.py for the wrapped stuff, can be multiple modules or submodules
lnst/Tests/TRex/__init__.py here you can just export the Client/Server test modules and whatever else is important - I think that will still work with the current imports in Recipes
test-tools/tperf/tperf looks ok in both situations I think.
-Ondrej
On 6/19/20 12:02 PM, Ondrej Lichtner wrote:
On Thu, Jun 18, 2020 at 05:15:53PM +0200, Adrian Moreno wrote:
Thanks for your feedback Ondrej.
Comments below.
On 6/18/20 12:21 PM, Ondrej Lichtner wrote:
On Mon, Jun 15, 2020 at 06:22:53PM +0200, Adrian Moreno wrote:
For now, I would not like to pay the maintenance price of another project to be honest :)
Same here, hence why I said it's an "issue" :).
Also, at this point the lnst.Tests.TRex module doesn't have a very big dependency on LNST - it uses a couple of parameters for type checking and defines a module interface API, which is just that there's a "run" method, but you still kept that in the standalone library. I guess this is your main point for possible future extensions?
Right, the more that can be added to the standalone library, the better because the development is much easier. However, I must admit it's my lack of expertise in LNST talking here.
Could we implement tperf using the existing "Tests/Trex.py" without having to run a full blown lnst-slave? If it's possible without doing too much LNST-specific things, that would save us the refactoring, the weird "placement" and reduce the temptations of creating another project. What do you think?
So... without context I think the _current_ implementation of tperf could be done without the refactor, the lnst.Test.TRex really doesn't require a running lnst-slave, it's actually a "reverse dependency" - if you want to run trex on a slave then you need the lnst.Test.TRex module
- the slave depends on the module, not the other way around.
BUT, with context I think a refactor and split into a "generic" lib is better for future development.
Finally, the stream can be modularized into TRex compatible modules so that:
- New stream generators can be easily implemented
- TRex tools (e.g: stl-sym) can be used for stream generator development
This took me a bit to understand... I was confused by the module implementation with a "register" function... I now see that it's another one of TRex weirdnes, I would personally prefer a class based approach but I guess this is fine since it follows some "standard"
Right. It's a trex thing. We can have another class-based API for LNST but keep the trex stl one for prototyping. But we might break the latter without knowing.
Nah, I think this is good, reimplementing the lnst stream creation via the simple TRex module is IMO better as long as it integrates well (and it looks like it does) with how we want to run if from actual recipes.
And refactoring the lnst.trex module so that it supports the generic trex modules is good as it just expands the possibilities for lnst.
My original implementation is just flawed because I had a hard time getting into TRex and this was the easiest way _for me at the time_ to get it running...
The current stream generation is modularized as an example: udp_simple
So does this patchset require any other changes to the recipes that currently use TRex? or is it drop in? Haven't tested this since I don't have a pre-setup dpdk environment...
Right now, no change is needed. The default module implements the current functionality so we're good.
When we add more modules, we can have an optional parameter that can be set by new tests pointing to the new modules.
cool
I'm only a sporadic contributor to LNST so sending this RFC to get some feedback.
I think the only thing I would complain about is the placement of the "lnst.TRex" module... I'm not sure I want it at this top level, I think it also contributed to my slower understanding of the RFC. Not exactly sure where to put it though. I'll keep thinking about this
I agree. I dislike the placement as well.
If you agree with the approach described above (tperf depend on Tests/Trex.py directly) we could have:
lnst/Tests/Trex.py lnst/Tests/TRexModules/udp_simple.py lnst/Tests/TRexModules/...(otherModules).py test-tools/tperf/tperf
What do you think?
So if keeping the refactor I'm thinking something like:
lnst/Tests/TRex.py for the test module lnst/Tests/TRexWrap/*.py for the refactored trex wrapper code lnst/Tests/TRexModules/udp_simple.py sounds good for additional stream generation modules
Another option could be:
lnst/Tests/TRex/TestModule.py for the test module lnst/Tests/TRex/*.py for the wrapped stuff, can be multiple modules or submodules
lnst/Tests/TRex/__init__.py here you can just export the Client/Server test modules and whatever else is important - I think that will still work with the current imports in Recipes
test-tools/tperf/tperf looks ok in both situations I think.
For me, the second option seems cleaner. I'll prepare v2.
Thanks for the feedback.
-Ondrej
lnst-developers@lists.fedorahosted.org
-
Adrian Moreno -
Ondrej Lichtner