Hello, this mail is status update on PyRecipes. I'm sorry there is no visible progress yet, but it's because I was considering several different formats and tried to satisfy both parties, which won't be probably possible.
Current use cases of LNST (RedHat vs Mellanox) ============================================== Currently, we have both sides, each with their own use case scenarios. Red Hat is using LNST for performance regression testing, along with PerfRepo. Each test uses same tools (ethtool, netperf, ping). The setups are very similar, as for hosts and eth ifaces, what is changing is soft interface setup (bond, team, VLAN, ovs, bridge). Setup of soft interfaces is currently done via XML and in task is only execution of test tools and additional setup (ethtool, MTU). Currently, the code is duplicated in every task (perfrepo methods, ethtool setups, mtu setups, netperf inits and calls) so a new layer of abstraction would be welcomed in order to simplify the code and maintainability.
As for Mellanox side, I can only speak about what I can see in switchdev recipes. Their approach is to define only hardware interfaces in XML and do all soft interface setting in the task. In order to create an abstraction layer a TestLib was created, which looks really appealing to me.
What do RH/Mellanox expect from PyRecipes ========================================= We had a video call in January about PyRecipes with olichtne (RH) and jpirko (Mellanox). Both sides had different view of PyRecipes.
Red Hat POW ----------- 1. Wants to be able to define soft and hard interfaces together in setup 2. Wants to be able to combine network setup with different tasks 3. Task should be understood as a function, with network as argument 4. Task should be generic
Mellanox POW ------------ 1. Wants to get rid of ID's (hosts and interfaces) 2. Wants soft iface definition only in task, not in setup 3. Task should be specific 4. Wrappers for generic stuff 5. 1 task == 1 test == 1 file, do not combine it
Proposed approach #1 (Mlx like) =============================== Description: Setup and task is in one file, no IDs are used, soft interface definition is part of task
Example: import lnst
m1 = lnst.add_host() m2 = lnst.add_host()
m1_eth1 = m1.add_interface(label="tnet") m1_eth2 = m1.add_interface(label="tnet")
m2_eth1 = m2.add_interface(label="tnet")
while match(match=lnst.SingleMatch): m1_team = m1.create_team([m1_eth1, m1_eth2], ip="1.2.3.4/24") m2_eth1.reset(["1.2.3.5/24"])
ping_mod = ... m1_team.run(ping_mod)
Proposed approach #2 (RH like) ============================== Description: Soft interfaces can be defined in both setup() and task() methods. IDs muset used due to different scopes of variables. Setup can be in separate file and can be imported in multiple tasks. In one file, multiple tasks can be called.
Example: import lnst
def setup(): m1 = lnst.add_host("m1") m2 = lnst.add_host("m2")
m1_eth1 = m1.add_interface(id="eth1", label="tnet") m1_eth2 = m1.add_interface(id="eth2", label="tnet")
m2_eth1 = m2.add_interface(id="eth1", label="tnet", ip="1.1.1.1/24")
m1.create_team(id="team1", slaves=[m1_eth1, m1_eth2], ip="1.1.1.2/24")
def task(): m1 = lnst.get_host("m1") m2 = lnst.get_host("m2")
m1_team = m1.get_interface("team1")
m2_eth1 = m2.get_interface("eth1")
ping_mod = ... m1_team.run(ping_mod)
lnst.run(match=lnst.SingleMatch, setup, task)
Proposed approach #3 (RH like) ============================== Description: Task method is portable, it uses machine and interface objects as args so no IDs are required. Soft interfaces can be created in both do_task and in setup phase. In one file, multiple tasks can be called.
Example: import lnst
def do_task(m1, if1, if2): ping_mod = ...src=if1, dst=if2... m1.run(ping_mod)
m1 = lnst.add_machine() m2 = lnst.add_machine()
m1_eth1 = m1.add_interface(label="tnet") m1_eth2 = m1.add_interface(label="tnet") m1_team = m1.create_team(slaves=[m1_eth1, m1_eth2], ip="1.1.1.1/24")
m2_eth1 = m2.add_interface(label="tnet", ip="1.1.1.2/24")
while lnst.match(match=lnst.SingleMatch): do_task(m1, m1_team, m2_eth1)
Summary ======= Drafts above are not meant to be final, it sure can be improved and modified to satisfy our needs. But we need to come to conclusion for both Mlx and RH side, so I can start working on it.
Some important questions regarding PyRecipes: --------------------------------------------- I. Soft interfaces - in setup phase and task phase or only in task phase? II. Portability - one task == one recipe, or allow combinations of networks with different tasks? III. Should task be generic or specific? IV. Do we have to get rid of IDs?
My opinion on the matter ======================== Favourite approach - #3
Answers to questions: --------------------- I. Soft interfaces - only in task phase - to follow 1 task == 1 recipe mentality, will bring easier maintaining II. Portability - one task == one recipe - even our use case shows, that tests don't allow so much combination (1 task is being used in avg. 1-2 recipes in phase1, 2-3 recipes in phase2), so I don't thinks its so important to use to preserve it III. Specific task - generic stuff can be defined by a new layer of abstraction, like TestLib in switchdev tests IV. I do not think IDs are such evil that we should get rid of it, altough I agree object oriented approach (which we want to follow, thus PyRecipes became a thing in the first place) should be only using instances of objects in both task and setup.
Summary ======= Please, devs from RH and Mlx, take a look on these drafts and send an email with your opinion on the matter. Ideally, next week starting Wed I would like to have it decided so I can start implementing it.
In the end, probably we will have to compromise, but hopefully, both parties will end up satisfied and all this work will lead to improving of the quality of the whole LNST.
Thanks for reading, Jiri Prochazka
Thu, Feb 25, 2016 at 12:46:51PM IST, jprochaz@redhat.com wrote:
Hello, this mail is status update on PyRecipes. I'm sorry there is no visible progress yet, but it's because I was considering several different formats and tried to satisfy both parties, which won't be probably possible.
Current use cases of LNST (RedHat vs Mellanox)
s/vs/&/ :)
============================================== Currently, we have both sides, each with their own use case scenarios. Red Hat is using LNST for performance regression testing, along with PerfRepo. Each test uses same tools (ethtool, netperf, ping). The setups are very similar, as for hosts and eth ifaces, what is changing is soft interface setup (bond, team, VLAN, ovs, bridge). Setup of soft interfaces is currently done via XML and in task is only execution of test tools and additional setup (ethtool, MTU). Currently, the code is duplicated in every task (perfrepo methods, ethtool setups, mtu setups, netperf inits and calls) so a new layer of abstraction would be welcomed in order to simplify the code and maintainability.
As for Mellanox side, I can only speak about what I can see in switchdev recipes. Their approach is to define only hardware interfaces in XML and do all soft interface setting in the task. In order to create an abstraction layer a TestLib was created, which looks really appealing to me.
All our recipes are in LNST git, so your view should be complete.
What do RH/Mellanox expect from PyRecipes
We had a video call in January about PyRecipes with olichtne (RH) and jpirko (Mellanox). Both sides had different view of PyRecipes.
Red Hat POW
Prisoner of War? ;)
- Wants to be able to define soft and hard interfaces together in setup
- Wants to be able to combine network setup with different tasks
- Task should be understood as a function, with network as argument
- Task should be generic
Mellanox POW
- Wants to get rid of ID's (hosts and interfaces)
- Wants soft iface definition only in task, not in setup
- Task should be specific
- Wrappers for generic stuff
- 1 task == 1 test == 1 file, do not combine it
Proposed approach #1 (Mlx like)
Description: Setup and task is in one file, no IDs are used, soft interface definition is part of task
Example: import lnst
m1 = lnst.add_host() m2 = lnst.add_host()
m1_eth1 = m1.add_interface(label="tnet") m1_eth2 = m1.add_interface(label="tnet")
m2_eth1 = m2.add_interface(label="tnet")
while match(match=lnst.SingleMatch): m1_team = m1.create_team([m1_eth1, m1_eth2], ip="1.2.3.4/24") m2_eth1.reset(["1.2.3.5/24"])
ping_mod = ... m1_team.run(ping_mod)
Shouldn't this be m1.run(ping_mod)?
Proposed approach #2 (RH like)
Description: Soft interfaces can be defined in both setup() and task()
From the description above of how you use LNST in RH it seems like soft interfaces should only be created in setup(), no?
methods. IDs muset used due to different scopes of variables. Setup can be in separate file and can be imported in multiple tasks. In one file, multiple tasks can be called.
Example: import lnst
def setup(): m1 = lnst.add_host("m1") m2 = lnst.add_host("m2")
m1_eth1 = m1.add_interface(id="eth1", label="tnet") m1_eth2 = m1.add_interface(id="eth2", label="tnet")
m2_eth1 = m2.add_interface(id="eth1", label="tnet", ip="1.1.1.1/24")
m1.create_team(id="team1", slaves=[m1_eth1, m1_eth2], ip="1.1.1.2/24")
def task(): m1 = lnst.get_host("m1") m2 = lnst.get_host("m2")
m1_team = m1.get_interface("team1")
m2_eth1 = m2.get_interface("eth1")
ping_mod = ... m1_team.run(ping_mod)
lnst.run(match=lnst.SingleMatch, setup, task)
Proposed approach #3 (RH like)
Description: Task method is portable, it uses machine and interface objects as args so no IDs are required. Soft interfaces can be created in both do_task and in setup phase. In one file, multiple tasks can be called.
Example: import lnst
def do_task(m1, if1, if2): ping_mod = ...src=if1, dst=if2... m1.run(ping_mod)
m1 = lnst.add_machine() m2 = lnst.add_machine()
m1_eth1 = m1.add_interface(label="tnet") m1_eth2 = m1.add_interface(label="tnet") m1_team = m1.create_team(slaves=[m1_eth1, m1_eth2], ip="1.1.1.1/24")
m2_eth1 = m2.add_interface(label="tnet", ip="1.1.1.2/24")
while lnst.match(match=lnst.SingleMatch): do_task(m1, m1_team, m2_eth1)
What's the main difference from approach #1? If I can create the soft interfaces in do_task(), then it looks pretty much the same to me.
Summary
Drafts above are not meant to be final, it sure can be improved and modified to satisfy our needs. But we need to come to conclusion for both Mlx and RH side, so I can start working on it.
Some important questions regarding PyRecipes:
I. Soft interfaces - in setup phase and task phase or only in task phase? II. Portability - one task == one recipe, or allow combinations of networks with different tasks? III. Should task be generic or specific? IV. Do we have to get rid of IDs?
My opinion on the matter
Favourite approach - #3
Answers to questions:
I. Soft interfaces - only in task phase - to follow 1 task == 1 recipe mentality, will bring easier maintaining II. Portability - one task == one recipe - even our use case shows, that tests don't allow so much combination (1 task is being used in avg. 1-2 recipes in phase1, 2-3 recipes in phase2), so I don't thinks its so important to use to preserve it III. Specific task - generic stuff can be defined by a new layer of abstraction, like TestLib in switchdev tests IV. I do not think IDs are such evil that we should get rid of it, altough I agree object oriented approach (which we want to follow, thus PyRecipes became a thing in the first place) should be only using instances of objects in both task and setup.
Summary
Please, devs from RH and Mlx, take a look on these drafts and send an email with your opinion on the matter. Ideally, next week starting Wed I would like to have it decided so I can start implementing it.
In the end, probably we will have to compromise, but hopefully, both parties will end up satisfied and all this work will lead to improving of the quality of the whole LNST.
Thanks for reading, Jiri Prochazka
Thanks for the detailed summary!
Thu, Feb 25, 2016 at 11:46:51AM CET, jprochaz@redhat.com wrote:
Hello, this mail is status update on PyRecipes. I'm sorry there is no visible progress yet, but it's because I was considering several different formats and tried to satisfy both parties, which won't be probably possible.
Current use cases of LNST (RedHat vs Mellanox)
Currently, we have both sides, each with their own use case scenarios. Red Hat is using LNST for performance regression testing, along with PerfRepo. Each test uses same tools (ethtool, netperf, ping). The setups are very similar, as for hosts and eth ifaces, what is changing is soft interface setup (bond, team, VLAN, ovs, bridge). Setup of soft interfaces is currently done via XML and in task is only execution of test tools and additional setup (ethtool, MTU). Currently, the code is duplicated in every task (perfrepo methods, ethtool setups, mtu setups, netperf inits and calls) so a new layer of abstraction would be welcomed in order to simplify the code and maintainability.
As for Mellanox side, I can only speak about what I can see in switchdev recipes. Their approach is to define only hardware interfaces in XML and do all soft interface setting in the task. In order to create an abstraction layer a TestLib was created, which looks really appealing to me.
All our (Mellanox) recipes are in lnst git. So you can see everything.
What do RH/Mellanox expect from PyRecipes
We had a video call in January about PyRecipes with olichtne (RH) and jpirko (Mellanox). Both sides had different view of PyRecipes.
Red Hat POW
- Wants to be able to define soft and hard interfaces together in setup
- Wants to be able to combine network setup with different tasks
- Task should be understood as a function, with network as argument
- Task should be generic
Mellanox POW
- Wants to get rid of ID's (hosts and interfaces)
- Wants soft iface definition only in task, not in setup
- Task should be specific
- Wrappers for generic stuff
- 1 task == 1 test == 1 file, do not combine it
Proposed approach #1 (Mlx like)
Description: Setup and task is in one file, no IDs are used, soft interface definition is part of task
Example: import lnst
m1 = lnst.add_host() m2 = lnst.add_host()
m1_eth1 = m1.add_interface(label="tnet") m1_eth2 = m1.add_interface(label="tnet")
m2_eth1 = m2.add_interface(label="tnet")
while match(match=lnst.SingleMatch): m1_team = m1.create_team([m1_eth1, m1_eth2], ip="1.2.3.4/24") m2_eth1.reset(["1.2.3.5/24"])
ping_mod = ... m1_team.run(ping_mod)
Proposed approach #2 (RH like)
Description: Soft interfaces can be defined in both setup() and task() methods. IDs muset used due to different scopes of variables. Setup can be in separate file and can be imported in multiple tasks. In one file, multiple tasks can be called.
Example: import lnst
def setup(): m1 = lnst.add_host("m1") m2 = lnst.add_host("m2")
m1_eth1 = m1.add_interface(id="eth1", label="tnet") m1_eth2 = m1.add_interface(id="eth2", label="tnet")
m2_eth1 = m2.add_interface(id="eth1", label="tnet", ip="1.1.1.1/24")
m1.create_team(id="team1", slaves=[m1_eth1, m1_eth2], ip="1.1.1.2/24")
def task(): m1 = lnst.get_host("m1") m2 = lnst.get_host("m2")
m1_team = m1.get_interface("team1")
m2_eth1 = m2.get_interface("eth1")
ping_mod = ... m1_team.run(ping_mod)
lnst.run(match=lnst.SingleMatch, setup, task)
Proposed approach #3 (RH like)
Description: Task method is portable, it uses machine and interface objects as args so no IDs are required. Soft interfaces can be created in both do_task and in setup phase. In one file, multiple tasks can be called.
Example: import lnst
def do_task(m1, if1, if2): ping_mod = ...src=if1, dst=if2... m1.run(ping_mod)
m1 = lnst.add_machine() m2 = lnst.add_machine()
m1_eth1 = m1.add_interface(label="tnet") m1_eth2 = m1.add_interface(label="tnet") m1_team = m1.create_team(slaves=[m1_eth1, m1_eth2], ip="1.1.1.1/24")
m2_eth1 = m2.add_interface(label="tnet", ip="1.1.1.2/24")
while lnst.match(match=lnst.SingleMatch): do_task(m1, m1_team, m2_eth1)
This is the same as #1, only with possibility to create soft devices in setup phase. I personally don't mind much is the api would be the same for soft device manipulation in setup phase and task phase.
But, I believe that it would be very complicated to implement. Also, some runtime methods of the soft device object would be possible to call only in tasks and in setup phase the soft device does not exist yet.
Confusing as hell. And what is the gain? Nothing. You can easily change soft device creation to tasks. Some helper function to do if before you call your actual function that runs tests. Lets not complicate things. With approach:
1 define hw ifaces 2 do match 3 run task - create soft ifaces, manipulate configuration, run tests
you can easily achieve whatever you need. I don't understand why you still persist possibility to create soft devices in phase 1. That simply does not make sense.
Summary
Drafts above are not meant to be final, it sure can be improved and modified to satisfy our needs. But we need to come to conclusion for both Mlx and RH side, so I can start working on it.
Some important questions regarding PyRecipes:
I. Soft interfaces - in setup phase and task phase or only in task phase?
As I wrote above, only in task. It is completely pointless to do it in setup phase.
II. Portability - one task == one recipe, or allow combinations of networks with different tasks?
One task=one recipe. You can easily use some lib that will do the common things for you. Exactly as we do it in switchdev recipes. It's a python, the task here is just an entry point. You can do whatever from that.
III. Should task be generic or specific?
As I wrote above, you can easily split this in: 1 specific wrapper 2 lib methods - called by specific wrapper
IV. Do we have to get rid of IDs?
YES! No reason for them what so ever.
My opinion on the matter
Favourite approach - #3
Answers to questions:
I. Soft interfaces - only in task phase - to follow 1 task == 1 recipe mentality, will bring easier maintaining II. Portability - one task == one recipe - even our use case shows, that tests don't allow so much combination (1 task is being used in avg. 1-2 recipes in phase1, 2-3 recipes in phase2), so I don't thinks its so important to use to preserve it III. Specific task - generic stuff can be defined by a new layer of abstraction, like TestLib in switchdev tests IV. I do not think IDs are such evil that we should get rid of it, altough I agree object oriented approach (which we want to follow, thus PyRecipes became a thing in the first place) should be only using instances of objects in both task and setup.
So I think that according to your answers, it looks like you like approach #1 the most. It's exactly that. With one expection that that is you would preserve the ids. I don't understand why. Nodoby will care about them. Inside the lnst implementation, they can stay, but they should not get into python api.
Summary
Please, devs from RH and Mlx, take a look on these drafts and send an email with your opinion on the matter. Ideally, next week starting Wed I would like to have it decided so I can start implementing it.
In the end, probably we will have to compromise, but hopefully, both parties will end up satisfied and all this work will lead to improving of the quality of the whole LNST.
I think that we are in sync now. Let's have a quick call next week to make things solid.
Thanks for reading,
Thanks for taking care of this! The gain will be huge! Looking forward to it.
Jiri Prochazka
Sat, Feb 27, 2016 at 12:59:26PM IST, jiri@resnulli.us wrote:
Thu, Feb 25, 2016 at 11:46:51AM CET, jprochaz@redhat.com wrote:
Hello, this mail is status update on PyRecipes. I'm sorry there is no visible progress yet, but it's because I was considering several different formats and tried to satisfy both parties, which won't be probably possible.
Current use cases of LNST (RedHat vs Mellanox)
Currently, we have both sides, each with their own use case scenarios. Red Hat is using LNST for performance regression testing, along with PerfRepo. Each test uses same tools (ethtool, netperf, ping). The setups are very similar, as for hosts and eth ifaces, what is changing is soft interface setup (bond, team, VLAN, ovs, bridge). Setup of soft interfaces is currently done via XML and in task is only execution of test tools and additional setup (ethtool, MTU). Currently, the code is duplicated in every task (perfrepo methods, ethtool setups, mtu setups, netperf inits and calls) so a new layer of abstraction would be welcomed in order to simplify the code and maintainability.
As for Mellanox side, I can only speak about what I can see in switchdev recipes. Their approach is to define only hardware interfaces in XML and do all soft interface setting in the task. In order to create an abstraction layer a TestLib was created, which looks really appealing to me.
All our (Mellanox) recipes are in lnst git. So you can see everything.
What do RH/Mellanox expect from PyRecipes
We had a video call in January about PyRecipes with olichtne (RH) and jpirko (Mellanox). Both sides had different view of PyRecipes.
Red Hat POW
- Wants to be able to define soft and hard interfaces together in setup
- Wants to be able to combine network setup with different tasks
- Task should be understood as a function, with network as argument
- Task should be generic
Mellanox POW
- Wants to get rid of ID's (hosts and interfaces)
- Wants soft iface definition only in task, not in setup
- Task should be specific
- Wrappers for generic stuff
- 1 task == 1 test == 1 file, do not combine it
Proposed approach #1 (Mlx like)
Description: Setup and task is in one file, no IDs are used, soft interface definition is part of task
Example: import lnst
m1 = lnst.add_host() m2 = lnst.add_host()
m1_eth1 = m1.add_interface(label="tnet") m1_eth2 = m1.add_interface(label="tnet")
m2_eth1 = m2.add_interface(label="tnet")
while match(match=lnst.SingleMatch): m1_team = m1.create_team([m1_eth1, m1_eth2], ip="1.2.3.4/24") m2_eth1.reset(["1.2.3.5/24"])
ping_mod = ... m1_team.run(ping_mod)
Proposed approach #2 (RH like)
Description: Soft interfaces can be defined in both setup() and task() methods. IDs muset used due to different scopes of variables. Setup can be in separate file and can be imported in multiple tasks. In one file, multiple tasks can be called.
Example: import lnst
def setup(): m1 = lnst.add_host("m1") m2 = lnst.add_host("m2")
m1_eth1 = m1.add_interface(id="eth1", label="tnet") m1_eth2 = m1.add_interface(id="eth2", label="tnet")
m2_eth1 = m2.add_interface(id="eth1", label="tnet", ip="1.1.1.1/24")
m1.create_team(id="team1", slaves=[m1_eth1, m1_eth2], ip="1.1.1.2/24")
def task(): m1 = lnst.get_host("m1") m2 = lnst.get_host("m2")
m1_team = m1.get_interface("team1")
m2_eth1 = m2.get_interface("eth1")
ping_mod = ... m1_team.run(ping_mod)
lnst.run(match=lnst.SingleMatch, setup, task)
Proposed approach #3 (RH like)
Description: Task method is portable, it uses machine and interface objects as args so no IDs are required. Soft interfaces can be created in both do_task and in setup phase. In one file, multiple tasks can be called.
Example: import lnst
def do_task(m1, if1, if2): ping_mod = ...src=if1, dst=if2... m1.run(ping_mod)
m1 = lnst.add_machine() m2 = lnst.add_machine()
m1_eth1 = m1.add_interface(label="tnet") m1_eth2 = m1.add_interface(label="tnet") m1_team = m1.create_team(slaves=[m1_eth1, m1_eth2], ip="1.1.1.1/24")
m2_eth1 = m2.add_interface(label="tnet", ip="1.1.1.2/24")
while lnst.match(match=lnst.SingleMatch): do_task(m1, m1_team, m2_eth1)
This is the same as #1, only with possibility to create soft devices in setup phase. I personally don't mind much is the api would be the same for soft device manipulation in setup phase and task phase.
OK, so it seems like #1 would be the most flexible and can accommodate everyone's needs.
But, I believe that it would be very complicated to implement. Also, some runtime methods of the soft device object would be possible to call only in tasks and in setup phase the soft device does not exist yet.
Confusing as hell. And what is the gain? Nothing. You can easily change soft device creation to tasks. Some helper function to do if before you call your actual function that runs tests. Lets not complicate things. With approach:
1 define hw ifaces 2 do match 3 run task - create soft ifaces, manipulate configuration, run tests
you can easily achieve whatever you need. I don't understand why you still persist possibility to create soft devices in phase 1. That simply does not make sense.
Summary
Drafts above are not meant to be final, it sure can be improved and modified to satisfy our needs. But we need to come to conclusion for both Mlx and RH side, so I can start working on it.
Some important questions regarding PyRecipes:
I. Soft interfaces - in setup phase and task phase or only in task phase?
As I wrote above, only in task. It is completely pointless to do it in setup phase.
II. Portability - one task == one recipe, or allow combinations of networks with different tasks?
One task=one recipe. You can easily use some lib that will do the common things for you. Exactly as we do it in switchdev recipes. It's a python, the task here is just an entry point. You can do whatever from that.
III. Should task be generic or specific?
As I wrote above, you can easily split this in: 1 specific wrapper 2 lib methods - called by specific wrapper
IV. Do we have to get rid of IDs?
YES! No reason for them what so ever.
+1
Sat, Feb 27, 2016 at 11:59:26AM CET, jiri@resnulli.us wrote:
Thu, Feb 25, 2016 at 11:46:51AM CET, jprochaz@redhat.com wrote:
Hello, this mail is status update on PyRecipes. I'm sorry there is no visible progress yet, but it's because I was considering several different formats and tried to satisfy both parties, which won't be probably possible.
Current use cases of LNST (RedHat vs Mellanox)
Currently, we have both sides, each with their own use case scenarios. Red Hat is using LNST for performance regression testing, along with PerfRepo. Each test uses same tools (ethtool, netperf, ping). The setups are very similar, as for hosts and eth ifaces, what is changing is soft interface setup (bond, team, VLAN, ovs, bridge). Setup of soft interfaces is currently done via XML and in task is only execution of test tools and additional setup (ethtool, MTU). Currently, the code is duplicated in every task (perfrepo methods, ethtool setups, mtu setups, netperf inits and calls) so a new layer of abstraction would be welcomed in order to simplify the code and maintainability.
As for Mellanox side, I can only speak about what I can see in switchdev recipes. Their approach is to define only hardware interfaces in XML and do all soft interface setting in the task. In order to create an abstraction layer a TestLib was created, which looks really appealing to me.
All our (Mellanox) recipes are in lnst git. So you can see everything.
What do RH/Mellanox expect from PyRecipes
We had a video call in January about PyRecipes with olichtne (RH) and jpirko (Mellanox). Both sides had different view of PyRecipes.
Red Hat POW
- Wants to be able to define soft and hard interfaces together in setup
- Wants to be able to combine network setup with different tasks
- Task should be understood as a function, with network as argument
- Task should be generic
Mellanox POW
- Wants to get rid of ID's (hosts and interfaces)
- Wants soft iface definition only in task, not in setup
- Task should be specific
- Wrappers for generic stuff
- 1 task == 1 test == 1 file, do not combine it
Proposed approach #1 (Mlx like)
Description: Setup and task is in one file, no IDs are used, soft interface definition is part of task
Example: import lnst
m1 = lnst.add_host() m2 = lnst.add_host()
m1_eth1 = m1.add_interface(label="tnet") m1_eth2 = m1.add_interface(label="tnet")
m2_eth1 = m2.add_interface(label="tnet")
while match(match=lnst.SingleMatch): m1_team = m1.create_team([m1_eth1, m1_eth2], ip="1.2.3.4/24") m2_eth1.reset(["1.2.3.5/24"])
ping_mod = ... m1_team.run(ping_mod)
Proposed approach #2 (RH like)
Description: Soft interfaces can be defined in both setup() and task() methods. IDs muset used due to different scopes of variables. Setup can be in separate file and can be imported in multiple tasks. In one file, multiple tasks can be called.
Example: import lnst
def setup(): m1 = lnst.add_host("m1") m2 = lnst.add_host("m2")
m1_eth1 = m1.add_interface(id="eth1", label="tnet") m1_eth2 = m1.add_interface(id="eth2", label="tnet")
m2_eth1 = m2.add_interface(id="eth1", label="tnet", ip="1.1.1.1/24")
m1.create_team(id="team1", slaves=[m1_eth1, m1_eth2], ip="1.1.1.2/24")
def task(): m1 = lnst.get_host("m1") m2 = lnst.get_host("m2")
m1_team = m1.get_interface("team1")
m2_eth1 = m2.get_interface("eth1")
ping_mod = ... m1_team.run(ping_mod)
lnst.run(match=lnst.SingleMatch, setup, task)
Proposed approach #3 (RH like)
Description: Task method is portable, it uses machine and interface objects as args so no IDs are required. Soft interfaces can be created in both do_task and in setup phase. In one file, multiple tasks can be called.
Example: import lnst
def do_task(m1, if1, if2): ping_mod = ...src=if1, dst=if2... m1.run(ping_mod)
m1 = lnst.add_machine() m2 = lnst.add_machine()
m1_eth1 = m1.add_interface(label="tnet") m1_eth2 = m1.add_interface(label="tnet") m1_team = m1.create_team(slaves=[m1_eth1, m1_eth2], ip="1.1.1.1/24")
m2_eth1 = m2.add_interface(label="tnet", ip="1.1.1.2/24")
while lnst.match(match=lnst.SingleMatch): do_task(m1, m1_team, m2_eth1)
This is the same as #1, only with possibility to create soft devices in setup phase. I personally don't mind much is the api would be the same for soft device manipulation in setup phase and task phase.
But, I believe that it would be very complicated to implement. Also, some runtime methods of the soft device object would be possible to call only in tasks and in setup phase the soft device does not exist yet.
Confusing as hell. And what is the gain? Nothing. You can easily change soft device creation to tasks. Some helper function to do if before you call your actual function that runs tests. Lets not complicate things. With approach:
1 define hw ifaces 2 do match 3 run task - create soft ifaces, manipulate configuration, run tests
you can easily achieve whatever you need. I don't understand why you still persist possibility to create soft devices in phase 1. That simply does not make sense.
Summary
Drafts above are not meant to be final, it sure can be improved and modified to satisfy our needs. But we need to come to conclusion for both Mlx and RH side, so I can start working on it.
Some important questions regarding PyRecipes:
I. Soft interfaces - in setup phase and task phase or only in task phase?
As I wrote above, only in task. It is completely pointless to do it in setup phase.
II. Portability - one task == one recipe, or allow combinations of networks with different tasks?
One task=one recipe. You can easily use some lib that will do the common things for you. Exactly as we do it in switchdev recipes. It's a python, the task here is just an entry point. You can do whatever from that.
III. Should task be generic or specific?
As I wrote above, you can easily split this in: 1 specific wrapper 2 lib methods - called by specific wrapper
IV. Do we have to get rid of IDs?
YES! No reason for them what so ever.
How about logging? How are you going to identify the machines in log? Well, you can use the variable name, I'm sure that's possible. In the code I want to keep host names as short as possible but in log I'd like to see something more human readable.
My opinion on the matter
Favourite approach - #3
Answers to questions:
I. Soft interfaces - only in task phase - to follow 1 task == 1 recipe mentality, will bring easier maintaining II. Portability - one task == one recipe - even our use case shows, that tests don't allow so much combination (1 task is being used in avg. 1-2 recipes in phase1, 2-3 recipes in phase2), so I don't thinks its so important to use to preserve it III. Specific task - generic stuff can be defined by a new layer of abstraction, like TestLib in switchdev tests IV. I do not think IDs are such evil that we should get rid of it, altough I agree object oriented approach (which we want to follow, thus PyRecipes became a thing in the first place) should be only using instances of objects in both task and setup.
So I think that according to your answers, it looks like you like approach #1 the most. It's exactly that. With one expection that that is you would preserve the ids. I don't understand why. Nodoby will care about them. Inside the lnst implementation, they can stay, but they should not get into python api.
Summary
Please, devs from RH and Mlx, take a look on these drafts and send an email with your opinion on the matter. Ideally, next week starting Wed I would like to have it decided so I can start implementing it.
In the end, probably we will have to compromise, but hopefully, both parties will end up satisfied and all this work will lead to improving of the quality of the whole LNST.
I think that we are in sync now. Let's have a quick call next week to make things solid.
Thanks for reading,
Thanks for taking care of this! The gain will be huge! Looking forward to it.
Jiri Prochazka
On Sat, Feb 27, 2016 at 11:59:26AM +0100, Jiri Pirko wrote:
Thu, Feb 25, 2016 at 11:46:51AM CET, jprochaz@redhat.com wrote:
Summary
Drafts above are not meant to be final, it sure can be improved and modified to satisfy our needs. But we need to come to conclusion for both Mlx and RH side, so I can start working on it.
Some important questions regarding PyRecipes:
I. Soft interfaces - in setup phase and task phase or only in task phase?
As I wrote above, only in task. It is completely pointless to do it in setup phase.
I want the option of both... I disagree that it would be hard to implement and that it would be confusing... you're already going to have functions that work only in one half of the recipe and don't make sense in the other part - running commands before match. These checks are basicaly the same thing.
HOWEVER, I want this to finally get going and arguing about what is/isn't confusing is pointless so... I will accept the "just in task" option, for now... if I will need/want soft ifaces before the match I'll just add this functionality myself...
II. Portability - one task == one recipe, or allow combinations of networks with different tasks?
One task=one recipe. You can easily use some lib that will do the common things for you. Exactly as we do it in switchdev recipes. It's a python, the task here is just an entry point. You can do whatever from that.
As long as you're purely in python you should be able to do as much portability as you want, it just depends on the workflow you choose... Which is why I've been saying that the user should have as much options as possible...
III. Should task be generic or specific?
As I wrote above, you can easily split this in: 1 specific wrapper 2 lib methods - called by specific wrapper
IV. Do we have to get rid of IDs?
YES! No reason for them what so ever.
yes, no IDs PLEASE... we have objects, let's use them... The issue of logging can be solved through variable names and informing the user of the mapping at the start of the recipe - we already do this so no change there...
My opinion on the matter
Favourite approach - #3
Answers to questions:
I. Soft interfaces - only in task phase - to follow 1 task == 1 recipe mentality, will bring easier maintaining II. Portability - one task == one recipe - even our use case shows, that tests don't allow so much combination (1 task is being used in avg. 1-2 recipes in phase1, 2-3 recipes in phase2), so I don't thinks its so important to use to preserve it III. Specific task - generic stuff can be defined by a new layer of abstraction, like TestLib in switchdev tests IV. I do not think IDs are such evil that we should get rid of it, altough I agree object oriented approach (which we want to follow, thus PyRecipes became a thing in the first place) should be only using instances of objects in both task and setup.
So I think that according to your answers, it looks like you like approach #1 the most. It's exactly that. With one expection that that is you would preserve the ids. I don't understand why. Nodoby will care about them. Inside the lnst implementation, they can stay, but they should not get into python api.
In the implementation they'll have to stay, for now, because of how the matching algorithm is implemented... But I believe we'll be able to get rid of them completely eventually...
Overall, nice summary. In the end, I think the end goal is to have as generic an approach as possible (with a recommended way of doing things). But for now... let's just do the recommended way (jpirko's approach) of doing things and work on the rest later.
-Ondrej
2016-03-02 11:28 GMT+01:00 Ondrej Lichtner olichtne@redhat.com:
On Sat, Feb 27, 2016 at 11:59:26AM +0100, Jiri Pirko wrote:
Thu, Feb 25, 2016 at 11:46:51AM CET, jprochaz@redhat.com wrote:
Summary
Drafts above are not meant to be final, it sure can be improved and modified to satisfy our needs. But we need to come to conclusion for
both
Mlx and RH side, so I can start working on it.
Some important questions regarding PyRecipes:
I. Soft interfaces - in setup phase and task phase or only in task
phase?
As I wrote above, only in task. It is completely pointless to do it in setup phase.
I want the option of both... I disagree that it would be hard to implement and that it would be confusing... you're already going to have functions that work only in one half of the recipe and don't make sense in the other part - running commands before match. These checks are basicaly the same thing.
HOWEVER, I want this to finally get going and arguing about what is/isn't confusing is pointless so... I will accept the "just in task" option, for now... if I will need/want soft ifaces before the match I'll just add this functionality myself...
Acknowledged.
II. Portability - one task == one recipe, or allow combinations of
networks
with different tasks?
One task=one recipe. You can easily use some lib that will do the common things for you. Exactly as we do it in switchdev recipes. It's a python, the task here is just an entry point. You can do whatever from that.
As long as you're purely in python you should be able to do as much portability as you want, it just depends on the workflow you choose... Which is why I've been saying that the user should have as much options as possible...
III. Should task be generic or specific?
As I wrote above, you can easily split this in: 1 specific wrapper 2 lib methods - called by specific wrapper
IV. Do we have to get rid of IDs?
YES! No reason for them what so ever.
yes, no IDs PLEASE... we have objects, let's use them... The issue of logging can be solved through variable names and informing the user of the mapping at the start of the recipe - we already do this so no change there...
We could
keep IDs as optional param for host, the ID then will be used in logging. If no ID is entered, use variable name.
My opinion on the matter
Favourite approach - #3
Answers to questions:
I. Soft interfaces - only in task phase - to follow 1 task == 1 recipe mentality, will bring easier maintaining II. Portability - one task == one recipe - even our use case shows, that tests don't allow so much combination (1 task is being used in avg. 1-2 recipes in phase1, 2-3 recipes in phase2), so I don't thinks its so important to use to preserve it III. Specific task - generic stuff can be defined by a new layer of abstraction, like TestLib in switchdev tests IV. I do not think IDs are such evil that we should get rid of it,
altough
I agree object oriented approach (which we want to follow, thus
PyRecipes
became a thing in the first place) should be only using instances of objects in both task and setup.
So I think that according to your answers, it looks like you like
approach #1
the most. It's exactly that. With one expection that that is you would preserve the ids. I don't understand why. Nodoby will care about them. Inside the lnst implementation, they can stay, but they should not get into python api.
In the implementation they'll have to stay, for now, because of how the matching algorithm is implemented... But I believe we'll be able to get rid of them completely eventually...
Overall, nice summary. In the end, I think the end goal is to have as generic an approach as possible (with a recommended way of doing things). But for now... let's just do the recommended way (jpirko's approach) of doing things and work on the rest later.
-Ondrej
Thanks everyone for their replies and comments. Looks like #1 approach it is, which may be extended in the future based on our needs.
I'll start working on the implementation right away and once I'll know, I'll let you know when the patches will be ready for testing.
Thanks again everyone, Jiri Prochazka
Wed, Mar 02, 2016 at 12:50:07PM CET, jprochaz@redhat.com wrote:
2016-03-02 11:28 GMT+01:00 Ondrej Lichtner olichtne@redhat.com:
On Sat, Feb 27, 2016 at 11:59:26AM +0100, Jiri Pirko wrote:
Thu, Feb 25, 2016 at 11:46:51AM CET, jprochaz@redhat.com wrote:
Summary
Drafts above are not meant to be final, it sure can be improved and modified to satisfy our needs. But we need to come to conclusion for
both
Mlx and RH side, so I can start working on it.
Some important questions regarding PyRecipes:
I. Soft interfaces - in setup phase and task phase or only in task
phase?
As I wrote above, only in task. It is completely pointless to do it in setup phase.
I want the option of both... I disagree that it would be hard to implement and that it would be confusing... you're already going to have functions that work only in one half of the recipe and don't make sense in the other part - running commands before match. These checks are basicaly the same thing.
HOWEVER, I want this to finally get going and arguing about what is/isn't confusing is pointless so... I will accept the "just in task" option, for now... if I will need/want soft ifaces before the match I'll just add this functionality myself...
Acknowledged.
II. Portability - one task == one recipe, or allow combinations of
networks
with different tasks?
One task=one recipe. You can easily use some lib that will do the common things for you. Exactly as we do it in switchdev recipes. It's a python, the task here is just an entry point. You can do whatever from that.
As long as you're purely in python you should be able to do as much portability as you want, it just depends on the workflow you choose... Which is why I've been saying that the user should have as much options as possible...
III. Should task be generic or specific?
As I wrote above, you can easily split this in: 1 specific wrapper 2 lib methods - called by specific wrapper
IV. Do we have to get rid of IDs?
YES! No reason for them what so ever.
yes, no IDs PLEASE... we have objects, let's use them... The issue of logging can be solved through variable names and informing the user of the mapping at the start of the recipe - we already do this so no change there...
We could
keep IDs as optional param for host, the ID then will be used in logging. If no ID is entered, use variable name.
no need. IDs have 0 sense in python
My opinion on the matter
Favourite approach - #3
Answers to questions:
I. Soft interfaces - only in task phase - to follow 1 task == 1 recipe mentality, will bring easier maintaining II. Portability - one task == one recipe - even our use case shows, that tests don't allow so much combination (1 task is being used in avg. 1-2 recipes in phase1, 2-3 recipes in phase2), so I don't thinks its so important to use to preserve it III. Specific task - generic stuff can be defined by a new layer of abstraction, like TestLib in switchdev tests IV. I do not think IDs are such evil that we should get rid of it,
altough
I agree object oriented approach (which we want to follow, thus
PyRecipes
became a thing in the first place) should be only using instances of objects in both task and setup.
So I think that according to your answers, it looks like you like
approach #1
the most. It's exactly that. With one expection that that is you would preserve the ids. I don't understand why. Nodoby will care about them. Inside the lnst implementation, they can stay, but they should not get into python api.
In the implementation they'll have to stay, for now, because of how the matching algorithm is implemented... But I believe we'll be able to get rid of them completely eventually...
Overall, nice summary. In the end, I think the end goal is to have as generic an approach as possible (with a recommended way of doing things). But for now... let's just do the recommended way (jpirko's approach) of doing things and work on the rest later.
-Ondrej
Thanks everyone for their replies and comments. Looks like #1 approach it is, which may be extended in the future based on our needs.
I'll start working on the implementation right away and once I'll know, I'll let you know when the patches will be ready for testing.
Thanks!
And please start to use some sane email client... :)
lnst-developers@lists.fedorahosted.org
-
Ido Schimmel -
Jan Tluka -
Jiri Pirko -
Jiri Prochazka -
Ondrej Lichtner