Hi all,
since we've moved to python3 that is actively developed and versions move between various long term/short term support cycles, we should also adapt LNST to this cycle of updating which minimal version of python LNST requires.
TL;DR: The main questions I'm asking are: * how do we _implement_ a python version requirement? * how do we _upgrade_/_migrate_ in the future? * how do we _document_ a python version requirement? * which version do we want to use _now_?
More context:
I think at the moment we have a "soft" requirement for python3.6. Soft because we: * probably haven't tested on anything older * it's not explicitly configured/documented anywhere
At the same time, there are now at least two reasons to start thinking about moving to python3.8:
* I remember Perry asking about the f-string feature introduced in 3.8 * while working with Adrian on the TRex refactoring I started thinking about a feature for the lnst.Tests package that I've had in mind for a while, which requires a 3.7 feature * python3.8 is the current version on Fedora32, is available in RHEL8 (via dnf install python38), and python3.7 was skipped
The lnst.Tests feature I'm thinking of is "lazy" and "dynamic" loading of BaseTestModule derived modules - for example at the moment, if a Recipe imports any module from lnst.Tests (e.g. lnst.Tests.Ping), the entire package is parsed and "loaded", which means that the python environment will also parse and load lnst.Tests.TRex. This means that a basic hello world recipe that simply calls Ping, will in some way require load time dependencies of TRex.
The "lazy" and "dynamic" loading of test modules would ensure that when a recipe calls:
from lnst.Tests import Ping
Only the Ping module will be parsed, loaded and imported, and nothing else. And the dynamicity here could mean that we could be able to extend test modules exported by the lnst.Tests package via the lnst-ctl config file, for example for user/tester implemented test modules that are not tracked in the main lnst repository.
I wrote a rough patch to experiment with this:
--- diff --git a/lnst/Tests/__init__.py b/lnst/Tests/__init__.py index f7c6c90..a39b6f4 100644 --- a/lnst/Tests/__init__.py +++ b/lnst/Tests/__init__.py @@ -12,8 +12,26 @@ olichtne@redhat.com (Ondrej Lichtner) """
-from lnst.Tests.Ping import Ping -from lnst.Tests.PacketAssert import PacketAssert -from lnst.Tests.Iperf import IperfClient, IperfServer +# from lnst.Tests.Ping import Ping +# from lnst.Tests.PacketAssert import PacketAssert +# from lnst.Tests.Iperf import IperfClient, IperfServer +import importlib + +lazy_load_modules = { + "Ping": "lnst.Tests.Ping", + "PacketAssert": "lnst.Tests.PacketAssert", + "IperfClient": "lnst.Tests.Iperf", + "IperfServer": "lnst.Tests.Iperf", +} + + +def __getattr__(name): + if name not in lazy_load_modules: + raise ImportError("Cannot import {}".format(name)) + mod = importlib.import_module(lazy_load_modules[name]) + globals()[name] = getattr(mod, name) + return globals()[name] + + +# #TODO add support for test classes from lnst-ctl.conf
-#TODO add support for test classes from lnst-ctl.conf ---
However this requires the ability to define __getattr__ for a module, which is introduced as a python3.7 feature via PEP562 [0].
-Ondrej
Hey I agree we should standardize on a version of python. I would think 3.8 would be the best.
Just one thing to note, F-strings are actually a python 3.6 feature, so that shouldn't be something we need to worry about here.
--Perry
On Fri, Jul 3, 2020 at 4:39 AM Ondrej Lichtner olichtne@redhat.com wrote:
Hi all,
since we've moved to python3 that is actively developed and versions move between various long term/short term support cycles, we should also adapt LNST to this cycle of updating which minimal version of python LNST requires.
TL;DR: The main questions I'm asking are:
- how do we _implement_ a python version requirement?
- how do we _upgrade_/_migrate_ in the future?
- how do we _document_ a python version requirement?
- which version do we want to use _now_?
More context:
I think at the moment we have a "soft" requirement for python3.6. Soft because we:
- probably haven't tested on anything older
- it's not explicitly configured/documented anywhere
At the same time, there are now at least two reasons to start thinking about moving to python3.8:
- I remember Perry asking about the f-string feature introduced in 3.8
- while working with Adrian on the TRex refactoring I started thinking about a feature for the lnst.Tests package that I've had in mind for a while, which requires a 3.7 feature
- python3.8 is the current version on Fedora32, is available in RHEL8 (via dnf install python38), and python3.7 was skipped
The lnst.Tests feature I'm thinking of is "lazy" and "dynamic" loading of BaseTestModule derived modules - for example at the moment, if a Recipe imports any module from lnst.Tests (e.g. lnst.Tests.Ping), the entire package is parsed and "loaded", which means that the python environment will also parse and load lnst.Tests.TRex. This means that a basic hello world recipe that simply calls Ping, will in some way require load time dependencies of TRex.
The "lazy" and "dynamic" loading of test modules would ensure that when a recipe calls:
from lnst.Tests import PingOnly the Ping module will be parsed, loaded and imported, and nothing else. And the dynamicity here could mean that we could be able to extend test modules exported by the lnst.Tests package via the lnst-ctl config file, for example for user/tester implemented test modules that are not tracked in the main lnst repository.
I wrote a rough patch to experiment with this:
diff --git a/lnst/Tests/__init__.py b/lnst/Tests/__init__.py index f7c6c90..a39b6f4 100644 --- a/lnst/Tests/__init__.py +++ b/lnst/Tests/__init__.py @@ -12,8 +12,26 @@ olichtne@redhat.com (Ondrej Lichtner) """
-from lnst.Tests.Ping import Ping -from lnst.Tests.PacketAssert import PacketAssert -from lnst.Tests.Iperf import IperfClient, IperfServer +# from lnst.Tests.Ping import Ping +# from lnst.Tests.PacketAssert import PacketAssert +# from lnst.Tests.Iperf import IperfClient, IperfServer +import importlib
+lazy_load_modules = {
- "Ping": "lnst.Tests.Ping",
- "PacketAssert": "lnst.Tests.PacketAssert",
- "IperfClient": "lnst.Tests.Iperf",
- "IperfServer": "lnst.Tests.Iperf",
+}
+def __getattr__(name):
- if name not in lazy_load_modules:
raise ImportError("Cannot import {}".format(name))- mod = importlib.import_module(lazy_load_modules[name])
- globals()[name] = getattr(mod, name)
- return globals()[name]
+# #TODO add support for test classes from lnst-ctl.conf
-#TODO add support for test classes from lnst-ctl.conf
However this requires the ability to define __getattr__ for a module, which is introduced as a python3.7 feature via PEP562 [0].
-Ondrej
[0] https://www.python.org/dev/peps/pep-0562/ _______________________________________________ LNST-developers mailing list -- lnst-developers@lists.fedorahosted.org To unsubscribe send an email to lnst-developers-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/lnst-developers@lists.fedorahos...
On Mon, Jul 06, 2020 at 03:29:02PM -0400, Perry Gagne wrote:
Hey I agree we should standardize on a version of python. I would think 3.8 would be the best.
Just one thing to note, F-strings are actually a python 3.6 feature, so that shouldn't be something we need to worry about here.
Hm, I do remember you asking about some feature that wasn't in 3.6 though, didn't I? Maybe I'm just confusing f-strings for something else...
--Perry
On Fri, Jul 3, 2020 at 4:39 AM Ondrej Lichtner olichtne@redhat.com wrote:
Hi all,
since we've moved to python3 that is actively developed and versions move between various long term/short term support cycles, we should also adapt LNST to this cycle of updating which minimal version of python LNST requires.
TL;DR: The main questions I'm asking are:
- how do we _implement_ a python version requirement?
- how do we _upgrade_/_migrate_ in the future?
- how do we _document_ a python version requirement?
- which version do we want to use _now_?
More context:
I think at the moment we have a "soft" requirement for python3.6. Soft because we:
- probably haven't tested on anything older
- it's not explicitly configured/documented anywhere
At the same time, there are now at least two reasons to start thinking about moving to python3.8:
- I remember Perry asking about the f-string feature introduced in 3.8
- while working with Adrian on the TRex refactoring I started thinking about a feature for the lnst.Tests package that I've had in mind for a while, which requires a 3.7 feature
- python3.8 is the current version on Fedora32, is available in RHEL8 (via dnf install python38), and python3.7 was skipped
The lnst.Tests feature I'm thinking of is "lazy" and "dynamic" loading of BaseTestModule derived modules - for example at the moment, if a Recipe imports any module from lnst.Tests (e.g. lnst.Tests.Ping), the entire package is parsed and "loaded", which means that the python environment will also parse and load lnst.Tests.TRex. This means that a basic hello world recipe that simply calls Ping, will in some way require load time dependencies of TRex.
The "lazy" and "dynamic" loading of test modules would ensure that when a recipe calls:
from lnst.Tests import PingOnly the Ping module will be parsed, loaded and imported, and nothing else. And the dynamicity here could mean that we could be able to extend test modules exported by the lnst.Tests package via the lnst-ctl config file, for example for user/tester implemented test modules that are not tracked in the main lnst repository.
I wrote a rough patch to experiment with this:
diff --git a/lnst/Tests/__init__.py b/lnst/Tests/__init__.py index f7c6c90..a39b6f4 100644 --- a/lnst/Tests/__init__.py +++ b/lnst/Tests/__init__.py @@ -12,8 +12,26 @@ olichtne@redhat.com (Ondrej Lichtner) """
-from lnst.Tests.Ping import Ping -from lnst.Tests.PacketAssert import PacketAssert -from lnst.Tests.Iperf import IperfClient, IperfServer +# from lnst.Tests.Ping import Ping +# from lnst.Tests.PacketAssert import PacketAssert +# from lnst.Tests.Iperf import IperfClient, IperfServer +import importlib
+lazy_load_modules = {
- "Ping": "lnst.Tests.Ping",
- "PacketAssert": "lnst.Tests.PacketAssert",
- "IperfClient": "lnst.Tests.Iperf",
- "IperfServer": "lnst.Tests.Iperf",
+}
+def __getattr__(name):
- if name not in lazy_load_modules:
raise ImportError("Cannot import {}".format(name))- mod = importlib.import_module(lazy_load_modules[name])
- globals()[name] = getattr(mod, name)
- return globals()[name]
+# #TODO add support for test classes from lnst-ctl.conf
-#TODO add support for test classes from lnst-ctl.conf
However this requires the ability to define __getattr__ for a module, which is introduced as a python3.7 feature via PEP562 [0].
-Ondrej
[0] https://www.python.org/dev/peps/pep-0562/ _______________________________________________ LNST-developers mailing list -- lnst-developers@lists.fedorahosted.org To unsubscribe send an email to lnst-developers-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/lnst-developers@lists.fedorahos...
Just wanted to reply for posterity sake since it's come up a couple times.
For reference, LNST is really tied to whatever version of python is available for RHEL 8. The default for rhel 8 is python 3.6, python 3.8 is available as well.
However, I did some quick testing with python 3.8 in rhel 8 and quickly ran into dependency issues, specifically with the python3-libvirt library.
Although python 3.8 is shipped with rhel 8, most of the system related bindings (like libvirt-python) that come with RHEL are only available for 3.6.
See https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/htm...
Perhaps there is a way to get this stuff to work, but IMHO I think I'm okay with just sticking to 3.6 for now, since most of the new things are nice to have, and a lot of them I have found can be somewhat easily backported if need be.
On Tue, Jul 7, 2020 at 2:20 AM Ondrej Lichtner olichtne@redhat.com wrote:
On Mon, Jul 06, 2020 at 03:29:02PM -0400, Perry Gagne wrote:
Hey I agree we should standardize on a version of python. I would think
3.8
would be the best.
Just one thing to note, F-strings are actually a python 3.6 feature, so that shouldn't be something we need to worry about here.
Hm, I do remember you asking about some feature that wasn't in 3.6 though, didn't I? Maybe I'm just confusing f-strings for something else...
--Perry
On Fri, Jul 3, 2020 at 4:39 AM Ondrej Lichtner olichtne@redhat.com
wrote:
Hi all,
since we've moved to python3 that is actively developed and versions move between various long term/short term support cycles, we should
also
adapt LNST to this cycle of updating which minimal version of python LNST requires.
TL;DR: The main questions I'm asking are:
- how do we _implement_ a python version requirement?
- how do we _upgrade_/_migrate_ in the future?
- how do we _document_ a python version requirement?
- which version do we want to use _now_?
More context:
I think at the moment we have a "soft" requirement for python3.6. Soft because we:
- probably haven't tested on anything older
- it's not explicitly configured/documented anywhere
At the same time, there are now at least two reasons to start thinking about moving to python3.8:
- I remember Perry asking about the f-string feature introduced in 3.8
- while working with Adrian on the TRex refactoring I started thinking about a feature for the lnst.Tests package that I've had in mind for
a
while, which requires a 3.7 feature
- python3.8 is the current version on Fedora32, is available in RHEL8 (via dnf install python38), and python3.7 was skipped
The lnst.Tests feature I'm thinking of is "lazy" and "dynamic" loading of BaseTestModule derived modules - for example at the moment, if a Recipe imports any module from lnst.Tests (e.g. lnst.Tests.Ping), the entire package is parsed and "loaded", which means that the python environment will also parse and load lnst.Tests.TRex. This means that a basic hello world recipe that simply calls Ping, will in some way require load time dependencies of TRex.
The "lazy" and "dynamic" loading of test modules would ensure that when a recipe calls:
from lnst.Tests import PingOnly the Ping module will be parsed, loaded and imported, and nothing else. And the dynamicity here could mean that we could be able to
extend
test modules exported by the lnst.Tests package via the lnst-ctl config file, for example for user/tester implemented test modules that are not tracked in the main lnst repository.
I wrote a rough patch to experiment with this:
diff --git a/lnst/Tests/__init__.py b/lnst/Tests/__init__.py index f7c6c90..a39b6f4 100644 --- a/lnst/Tests/__init__.py +++ b/lnst/Tests/__init__.py @@ -12,8 +12,26 @@ olichtne@redhat.com (Ondrej Lichtner) """
-from lnst.Tests.Ping import Ping -from lnst.Tests.PacketAssert import PacketAssert -from lnst.Tests.Iperf import IperfClient, IperfServer +# from lnst.Tests.Ping import Ping +# from lnst.Tests.PacketAssert import PacketAssert +# from lnst.Tests.Iperf import IperfClient, IperfServer +import importlib
+lazy_load_modules = {
- "Ping": "lnst.Tests.Ping",
- "PacketAssert": "lnst.Tests.PacketAssert",
- "IperfClient": "lnst.Tests.Iperf",
- "IperfServer": "lnst.Tests.Iperf",
+}
+def __getattr__(name):
- if name not in lazy_load_modules:
raise ImportError("Cannot import {}".format(name))- mod = importlib.import_module(lazy_load_modules[name])
- globals()[name] = getattr(mod, name)
- return globals()[name]
+# #TODO add support for test classes from lnst-ctl.conf
-#TODO add support for test classes from lnst-ctl.conf
However this requires the ability to define __getattr__ for a module, which is introduced as a python3.7 feature via PEP562 [0].
-Ondrej
[0] https://www.python.org/dev/peps/pep-0562/ _______________________________________________ LNST-developers mailing list -- lnst-developers@lists.fedorahosted.org To unsubscribe send an email to lnst-developers-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/lnst-developers@lists.fedorahos...
lnst-developers@lists.fedorahosted.org
-
Ondrej Lichtner -
Perry Gagne