From: Ondrej Lichtner olichtne@redhat.com
Comparison to baseline will now also report a FAIL in case the measurement is higher than the baseline which could indicate a problem with the current baseline (should be adjusted) or a problem with the setup (e.g. the pvp path network wasn't configured correctly).
Signed-off-by: Ondrej Lichtner olichtne@redhat.com --- recipes/regression_tests/phase3/ovs-dpdk-pvp.py | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/recipes/regression_tests/phase3/ovs-dpdk-pvp.py b/recipes/regression_tests/phase3/ovs-dpdk-pvp.py index 1b1f51f..5a5854e 100644 --- a/recipes/regression_tests/phase3/ovs-dpdk-pvp.py +++ b/recipes/regression_tests/phase3/ovs-dpdk-pvp.py @@ -38,13 +38,16 @@ def compare_results(host, result, baseline, max_dev): interval2 = (avg2 - dev2, avg2 + dev2)
if interval1[1] < interval2[0]: - desc = ("Measured rate {:.2f} +-{:.2f} pps is lower " - "than threshold {:.2f} +-{:.2f} pps". + desc = ("Measured rate {:.2f} +-{:.2f} pps is lower than threshold {:.2f} +-{:.2f} pps". + format(avg1, dev1, avg2, dev2)) + custom_mod = ctl.get_module("Custom", options = {"fail": True}) + elif interval1[0] > interval2[1]: + desc = ("Measured rate {:.2f} +-{:.2f} pps is much higher than threshold {:.2f} +-{:.2f} pps\n" + "This could indicate a low baseline or a problem with the setup". format(avg1, dev1, avg2, dev2)) custom_mod = ctl.get_module("Custom", options = {"fail": True}) else: - desc = ("Measured rate {:.2f} +-{:.2f} pps is higher " - "than threshold {:.2f} +-{:.2f} pps". + desc = ("Measured rate {:.2f} +-{:.2f} pps matches the threshold {:.2f} +-{:.2f} pps". format(avg1, dev1, avg2, dev2)) custom_mod = ctl.get_module("Custom")
On Thu, Mar 29, 2018 at 04:08:40PM +0200, olichtne@redhat.com wrote:
From: Ondrej Lichtner olichtne@redhat.com
Comparison to baseline will now also report a FAIL in case the measurement is higher than the baseline which could indicate a problem with the current baseline (should be adjusted) or a problem with the setup (e.g. the pvp path network wasn't configured correctly).
Signed-off-by: Ondrej Lichtner olichtne@redhat.com
recipes/regression_tests/phase3/ovs-dpdk-pvp.py | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/recipes/regression_tests/phase3/ovs-dpdk-pvp.py b/recipes/regression_tests/phase3/ovs-dpdk-pvp.py index 1b1f51f..5a5854e 100644 --- a/recipes/regression_tests/phase3/ovs-dpdk-pvp.py +++ b/recipes/regression_tests/phase3/ovs-dpdk-pvp.py @@ -38,13 +38,16 @@ def compare_results(host, result, baseline, max_dev): interval2 = (avg2 - dev2, avg2 + dev2)
if interval1[1] < interval2[0]:
desc = ("Measured rate {:.2f} +-{:.2f} pps is lower "
"than threshold {:.2f} +-{:.2f} pps".
desc = ("Measured rate {:.2f} +-{:.2f} pps is lower than threshold {:.2f} +-{:.2f} pps".
format(avg1, dev1, avg2, dev2))
custom_mod = ctl.get_module("Custom", options = {"fail": True})
- elif interval1[0] > interval2[1]:
desc = ("Measured rate {:.2f} +-{:.2f} pps is much higher than threshold {:.2f} +-{:.2f} pps\n"
else:"This could indicate a low baseline or a problem with the setup". format(avg1, dev1, avg2, dev2)) custom_mod = ctl.get_module("Custom", options = {"fail": True})
desc = ("Measured rate {:.2f} +-{:.2f} pps is higher "
"than threshold {:.2f} +-{:.2f} pps".
desc = ("Measured rate {:.2f} +-{:.2f} pps matches the threshold {:.2f} +-{:.2f} pps". format(avg1, dev1, avg2, dev2)) custom_mod = ctl.get_module("Custom")
-- 2.16.1
pushed to master, tested internally as working
-Ondrej
lnst-developers@lists.fedorahosted.org