LNST-developers August 2013

lnst-developers@lists.fedorahosted.org

6 participants
103 discussions

[lnst] variables.conf: value change from true/false to pass/fail
by Jiří Pírko 16 Aug '13

16 Aug '13

commit 812faa2b939375b4bfb4421a40e67e3ede09b44c Author: Ondrej Lichtner <olichtne(a)redhat.com> Date: Fri Aug 16 10:02:02 2013 +0200 variables.conf: value change from true/false to pass/fail The recent change of the xml format changed the accepted values of the expect attribute from true/false to pass/fail. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> Signed-off-by: Jiri Pirko <jiri(a)resnulli.us> recipes/smoke/lib/variables.conf | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) --- diff --git a/recipes/smoke/lib/variables.conf b/recipes/smoke/lib/variables.conf index a0ad9f9..0c9b066 100644 --- a/recipes/smoke/lib/variables.conf +++ b/recipes/smoke/lib/variables.conf @@ -1,14 +1,14 @@ [defaults] -icmp_result = true +icmp_result=pass [bond-vlan] -icmp_result=false +icmp_result=fail [team-vlan] -icmp_result=false +icmp_result=fail [eth-vlan] -icmp_result=false +icmp_result=fail [vlan-bond] -icmp_result=false +icmp_result=fail [vlan-team] -icmp_result=false +icmp_result=fail [vlan-eth] -icmp_result=false +icmp_result=fail

1 0

[lnst] smoke tests: fix expect values
by Jiří Pírko 16 Aug '13

16 Aug '13

commit 83ddef769168d6373646c99ad736a0b2e09735cb Author: Ondrej Lichtner <olichtne(a)redhat.com> Date: Fri Aug 16 10:02:01 2013 +0200 smoke tests: fix expect values Values of the expect attribute are generated based on values stored in the variables.conf file. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> Signed-off-by: Jiri Pirko <jiri(a)resnulli.us> recipes/smoke/lib/task-bg.xml | 4 ++-- recipes/smoke/lib/task-ping.xml | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) --- diff --git a/recipes/smoke/lib/task-bg.xml b/recipes/smoke/lib/task-bg.xml index eea6ef3..b9388d2 100644 --- a/recipes/smoke/lib/task-bg.xml +++ b/recipes/smoke/lib/task-bg.xml @@ -8,5 +8,5 @@ </options> </run> <ctl_wait seconds="5"/> - <intr bg_id="1" expect="fail" machine="1" timeout="30"/> -</task> \ No newline at end of file + <intr bg_id="1" expect="#icmp_result#" machine="1" timeout="30"/> +</task> diff --git a/recipes/smoke/lib/task-ping.xml b/recipes/smoke/lib/task-ping.xml index 8e84d7e..1c275f7 100644 --- a/recipes/smoke/lib/task-ping.xml +++ b/recipes/smoke/lib/task-ping.xml @@ -1,5 +1,5 @@ <task> - <run expect="fail" machine="1" module="IcmpPing" timeout="30"> + <run expect="#icmp_result#" machine="1" module="IcmpPing" timeout="30"> <options> <option name="addr" value="{ip(2,testiface)}"/> <option name="count" value="40"/> @@ -7,4 +7,4 @@ <option name="limit_rate" value="95"/> </options> </run> -</task> \ No newline at end of file +</task>

1 0

[lnst] NmConfigDevice: fix vlan is_nm_managed method
by Jiří Pírko 16 Aug '13

16 Aug '13

commit 68affe960288f5e7fcacf11dee96384e74ab4de4 Author: Ondrej Lichtner <olichtne(a)redhat.com> Date: Fri Aug 16 10:02:00 2013 +0200 NmConfigDevice: fix vlan is_nm_managed method When accessing the vlan slave interface I used the index 1 instead of 0. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> Signed-off-by: Jiri Pirko <jiri(a)resnulli.us> lnst/Slave/NmConfigDevice.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) --- diff --git a/lnst/Slave/NmConfigDevice.py b/lnst/Slave/NmConfigDevice.py index 7a236f9..3d12c55 100644 --- a/lnst/Slave/NmConfigDevice.py +++ b/lnst/Slave/NmConfigDevice.py @@ -581,7 +581,7 @@ class NmConfigDeviceVlan(NmConfigDeviceGeneric): managed = super(NmConfigDeviceBond, cls).is_nm_managed(netdev, config) else: - slave_id = get_slaves(netdev)[1] + slave_id = get_slaves(netdev)[0] netdev = config[slave_id] managed = is_nm_managed(netdev, config)

1 0

[lnst] NetUtils: rewrite scan_netdevs function
by Jiří Pírko 16 Aug '13

16 Aug '13

commit bce6e4902320eb10b3d01093b1070db90115a77c Author: Ondrej Lichtner <olichtne(a)redhat.com> Date: Fri Aug 16 09:54:45 2013 +0200 NetUtils: rewrite scan_netdevs function The scan_netdev function was starting to be a bit unreliable sometimes causing crashes. It also wasn't a very pretty solution of the problem. The new scan_netdev function connects to a netlink socket and requests a dump of all interfaces on the system, and parses the result to return a similar data structure to the previous implementation of the function. Again it contains the addditional kernel index of the interface that will be used in the future. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> Signed-off-by: Jiri Pirko <jiri(a)resnulli.us> lnst/Common/NetUtils.py | 57 ++++++++++++++++++++++++++++++++++++---------- 1 files changed, 44 insertions(+), 13 deletions(-) --- diff --git a/lnst/Common/NetUtils.py b/lnst/Common/NetUtils.py index fb39a41..efa14f5 100644 --- a/lnst/Common/NetUtils.py +++ b/lnst/Common/NetUtils.py @@ -15,26 +15,57 @@ import os import re import socket import subprocess +from resource import getpagesize +from pyroute2.netlink import NetlinkSocket +from pyroute2.netlink import NLM_F_REQUEST +from pyroute2.netlink import NLM_F_DUMP +from pyroute2.netlink import NLMSG_DONE +from pyroute2.netlink import NLMSG_ERROR +from pyroute2.netlink.generic import NETLINK_ROUTE +from pyroute2.netlink.iproute import MarshalRtnl +from pyroute2.netlink.iproute import RTM_GETLINK +from pyroute2.netlink.iproute import RTM_NEWLINK +from pyroute2.netlink.rtnl.ifinfmsg import ifinfmsg def normalize_hwaddr(hwaddr): return hwaddr.upper().rstrip("\n") def scan_netdevs(): - sys_dir = "/sys/class/net" scan = [] - for root, dirs, files in os.walk(sys_dir): - if "lo" in dirs: - dirs.remove("lo") - for d in dirs: - dev_path = os.path.join(sys_dir, d) - addr_path = os.path.join(dev_path, "address") - if not os.path.isfile(addr_path): + nl_socket = NetlinkSocket(family=NETLINK_ROUTE) + msg = ifinfmsg() + msg["family"] = socket.AF_UNSPEC + msg["header"]["type"] = RTM_GETLINK + msg["header"]["flags"] = NLM_F_REQUEST | NLM_F_DUMP + msg["header"]["pid"] = os.getpid() + msg["header"]["sequence_number"] = 1 + msg.encode() + + nl_socket.sendto(msg.buf.getvalue(), (0,0)) + + finished = False + marshal = MarshalRtnl() + while not finished: + response = nl_socket.recv(getpagesize()) + parts = marshal.parse(response) + for part in parts: + if part["header"]["type"] in [NLMSG_DONE, NLMSG_ERROR]: + finished = True continue - handle = open(addr_path, "rb") - addr = handle.read() - handle.close() - addr = normalize_hwaddr(addr) - scan.append({"name": d, "hwaddr": addr}) + if part["header"]["sequence_number"] != 1: + continue + + if part["header"]["type"] == RTM_NEWLINK: + new_link = {} + for name, value in part["attrs"]: + if name == "IFLA_IFNAME": + new_link["name"] = value + elif name == "IFLA_ADDRESS": + new_link["hwaddr"] = normalize_hwaddr(value) + new_link["index"] = part["index"] + scan.append(new_link) + + nl_socket.close() return scan def test_tcp_connection(host, port):

1 0

[lnst] Machine: fix minor bugs
by Jiří Pírko 16 Aug '13

16 Aug '13

commit feb59055c8102614c0dcdbe9006220e845782e1f Author: Ondrej Lichtner <olichtne(a)redhat.com> Date: Fri Aug 16 09:54:44 2013 +0200 Machine: fix minor bugs This commit fixes several Exception messages that were formatted with variables, but there were no formatting escape sequences in them which caused crashes. Also the initialize method of class Interface uses the result of the get_devices_by_hwaddr slave method to properly set the interface devname. Finally I had to add a sleep call after the interface deconfiguration call because sometimes dynamically added devices (through libvirt) would change their MAC address based on their master bond device, and wouldn't have enough time to change back before deleting them. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> Signed-off-by: Jiri Pirko <jiri(a)resnulli.us> lnst/Controller/Machine.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) --- diff --git a/lnst/Controller/Machine.py b/lnst/Controller/Machine.py index 88a4c21..f773bdb 100644 --- a/lnst/Controller/Machine.py +++ b/lnst/Controller/Machine.py @@ -351,7 +351,7 @@ class Interface(object): def get_hwaddr(self): if not self._hwaddr: - msg = "Hardware address is not available for interface ''" \ + msg = "Hardware address is not available for interface '%s'" \ % self.get_id() raise MachineError(msg) return self._hwaddr @@ -361,7 +361,7 @@ class Interface(object): def get_devname(self): if not self._devname: - msg = "Device name is not available for interface ''" \ + msg = "Device name is not available for interface '%s'" \ % self.get_id() raise MachineError(msg) return self._devname @@ -371,7 +371,7 @@ class Interface(object): def get_network(self): if not self._network: - msg = "Network segment is not available for interface ''" \ + msg = "Network segment is not available for interface '%s'" \ % self.get_id() raise MachineError(msg) return self._network @@ -409,7 +409,7 @@ class Interface(object): phys_devs = self._machine._rpc_call("get_devices_by_hwaddr", self._hwaddr) if len(phys_devs) == 1: - pass + self.set_devname(phys_devs[0]["name"]) elif len(phys_devs) < 1: msg = "Device %s not found on machine %s" \ % (self.get_id(), self._machine.get_id()) @@ -451,6 +451,7 @@ class Interface(object): self._machine._rpc_call("deconfigure_interface", self.get_id()) self._configured = False + sleep(1) class StaticInterface(Interface): """ Static interface

1 0

[lnst] Controller: add handling of update messages
by Jiří Pírko 16 Aug '13

16 Aug '13

commit ae19fc5bdfef0c987f23a49a17b1686fc8b30c64 Author: Ondrej Lichtner <olichtne(a)redhat.com> Date: Fri Aug 16 09:54:43 2013 +0200 Controller: add handling of update messages This commit extends the message handling on the controller, to recieve update messages from slave machines. A new method was added to the Machine objects- interface_update that will be called when an update message is recieved. At the moment it only looks up the corresponding interface (by devname) and updates its hw address. In the future this could be used to update ip addresses, and devnames them selves (when the kernel indices are stored). This change also required the addition of a dictionary to the message dispatcher object that maps machine id to the machine object. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> Signed-off-by: Jiri Pirko <jiri(a)resnulli.us> lnst/Controller/Machine.py | 10 +++++++++- lnst/Controller/NetTestController.py | 9 ++++++++- 2 files changed, 17 insertions(+), 2 deletions(-) --- diff --git a/lnst/Controller/Machine.py b/lnst/Controller/Machine.py index 96a0faa..88a4c21 100644 --- a/lnst/Controller/Machine.py +++ b/lnst/Controller/Machine.py @@ -70,6 +70,14 @@ class Machine(object): self._interfaces.append(iface) return iface + def interface_update(self, if_data): + for iface in self._interfaces: + try: + if iface.get_devname() == if_data["devname"]: + iface.set_hwaddr(if_data["hwaddr"]) + except: + pass + # # Factory methods for constructing interfaces on this machine. The # types of interfaces are explained with the classes below. @@ -116,7 +124,7 @@ class Machine(object): logging.info("Connecting to RPC on machine %s (%s)", m_id, hostname) connection = socket.create_connection((hostname, port)) - self._msg_dispatcher.add_slave(self._id, connection) + self._msg_dispatcher.add_slave(self, connection) hello = self._rpc_call("hello", recipe_name) if hello != "hello": diff --git a/lnst/Controller/NetTestController.py b/lnst/Controller/NetTestController.py index fee06ed..b8ffd26 100644 --- a/lnst/Controller/NetTestController.py +++ b/lnst/Controller/NetTestController.py @@ -598,8 +598,11 @@ class MessageDispatcher(ConnectionHandler): def __init__(self, log_ctl): super(MessageDispatcher, self).__init__() self._log_ctl = log_ctl + self._machines = dict() - def add_slave(self, machine_id, connection): + def add_slave(self, machine, connection): + machine_id = machine.get_id() + self._machines[machine_id] = machine self.add_connection(machine_id, connection) def send_message(self, machine_id, data): @@ -641,6 +644,9 @@ class MessageDispatcher(ConnectionHandler): elif message[1]["type"] == "result": msg = "Recieved result message from different slave %s" % message[0] logging.debug(msg) + elif message[1]["type"] == "if_update": + machine = self._machines[message[0]] + machine.interface_update(message[1]) elif message[1]["type"] == "exception": msg = "Recieved an exception from slave: %s" % message[0] raise CommandException(msg) @@ -655,3 +661,4 @@ class MessageDispatcher(ConnectionHandler): def disconnect_slave(self, machine_id): soc = self.get_connection(machine_id) self.remove_connection(soc) + del self._machines[machine_id]

1 0

[lnst] ConnectionHandler: handle NetlinkSockets
by Jiří Pírko 16 Aug '13

16 Aug '13

commit 74020f34811a0434a442fc1f22acb044cf853baa Author: Ondrej Lichtner <olichtne(a)redhat.com> Date: Fri Aug 16 09:54:42 2013 +0200 ConnectionHandler: handle NetlinkSockets This commit adds the ability to recieve data from NetlinkSockets. The data is also parsed and decoded by a NETLINK_ROUTE marshaller, since that is the only type of netlink socket we will be using at the moment. The order of the if/elif statements is important and the NetlinkSocket must come before the socket.SocketType check. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> Signed-off-by: Jiri Pirko <jiri(a)resnulli.us> lnst/Common/ConnectionHandler.py | 9 ++++++++- 1 files changed, 8 insertions(+), 1 deletions(-) --- diff --git a/lnst/Common/ConnectionHandler.py b/lnst/Common/ConnectionHandler.py index 80a9538..b44f159 100644 --- a/lnst/Common/ConnectionHandler.py +++ b/lnst/Common/ConnectionHandler.py @@ -15,6 +15,8 @@ import select import cPickle import socket from _multiprocessing import Connection +from pyroute2.netlink import NetlinkSocket +from pyroute2.netlink.iproute import MarshalRtnl def send_data(s, data): try: @@ -33,7 +35,12 @@ def send_data(s, data): return True def recv_data(s): - if isinstance(s, socket.SocketType): + if isinstance(s, NetlinkSocket): + marshaller = MarshalRtnl() + msg = s.recv(16384) + decoded_msg = marshaller.parse(msg) + data = {"type": "netlink", "data": decoded_msg} + elif isinstance(s, socket.SocketType): length = "" while True: c = s.recv(1)

1 0

[lnst] NetTestSlave: listen for netlink notifications
by Jiří Pírko 16 Aug '13

16 Aug '13

commit 59a6d0615fbc1ef7676dc4050852ab43097404b0 Author: Ondrej Lichtner <olichtne(a)redhat.com> Date: Fri Aug 16 09:54:41 2013 +0200 NetTestSlave: listen for netlink notifications The slave server now listens to NETLINK_ROUTE notifications. At the moment only RTM_NEWLINK messages are interesting to us, so we construct update messages from these and send them to the controller. The update messages contain the name of the interface, it's hw address, and the index under which it is recognized in the kernel. The index is useless right now but it will be useful later when we will have an internal interface database. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> Signed-off-by: Jiri Pirko <jiri(a)resnulli.us> lnst/Slave/NetTestSlave.py | 37 +++++++++++++++++++++++++++++++++++++ 1 files changed, 37 insertions(+), 0 deletions(-) --- diff --git a/lnst/Slave/NetTestSlave.py b/lnst/Slave/NetTestSlave.py index 0f17839..641b83b 100644 --- a/lnst/Slave/NetTestSlave.py +++ b/lnst/Slave/NetTestSlave.py @@ -38,6 +38,26 @@ from lnst.Common.ConnectionHandler import ConnectionHandler from lnst.Common.Config import lnst_config from lnst.Common.NetTestCommand import NetTestCommandConfig +#TODO this is temporary, until python-pyroute2 package is updated +from pyroute2.netlink import NetlinkSocket +from pyroute2.netlink.generic import NETLINK_ROUTE + +RTNLGRP_LINK = 0x1 +RTNLGRP_NEIGH = 0x4 +RTNLGRP_TC = 0x8 +RTNLGRP_IPV4_IFADDR = 0x10 +RTNLGRP_IPV4_ROUTE = 0x40 +RTNLGRP_IPV6_IFADDR = 0x100 +RTNLGRP_IPV6_ROUTE = 0x400 + +RTNL_GROUPS = RTNLGRP_IPV4_IFADDR |\ + RTNLGRP_IPV6_IFADDR |\ + RTNLGRP_IPV4_ROUTE |\ + RTNLGRP_IPV6_ROUTE |\ + RTNLGRP_NEIGH |\ + RTNLGRP_LINK |\ + RTNLGRP_TC + DefaultRPCPort = 9999 class SlaveMethods: @@ -433,6 +453,10 @@ class NetTestSlave: self._start = True + self._nl_socket = NetlinkSocket(family=NETLINK_ROUTE) + self._nl_socket.bind(RTNL_GROUPS) + self._server_handler.add_connection('netlink', self._nl_socket) + def run(self): while not self._finished: if self._server_handler.get_ctl_sock() == None: @@ -511,6 +535,19 @@ class NetTestSlave: self._cmd_context.del_cmd(cmd) else: cmd.set_result(msg["result"]) + elif msg["type"] == "netlink": + for sub_msg in msg["data"]: + if sub_msg["event"] == "RTM_NEWLINK": + response = dict() + response["type"] = "if_update" + response["if_index"] = sub_msg["index"] + msg_attrs = sub_msg["attrs"] + for name, value in msg_attrs: + if name == "IFLA_IFNAME": + response["devname"] = value + elif name == "IFLA_ADDRESS": + response["hwaddr"] = value + self._server_handler.send_data_to_ctl(response) else: raise Exception("Recieved unknown command")

1 0

[lnst] Fix slave options handling
by Jiří Pírko 16 Aug '13

16 Aug '13

commit 5361453833401120f2103e9942fb1d46ad84d5ec Author: Jan Tluka <jtluka(a)redhat.com> Date: Wed Aug 14 17:18:02 2013 +0200 Fix slave options handling The patch fixes crash of lnst-slave when options are specified for an enslaved device. Signed-off-by: Jan Tluka <jtluka(a)redhat.com> Signed-off-by: Jiri Pirko <jiri(a)resnulli.us> lnst/Controller/Machine.py | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) --- diff --git a/lnst/Controller/Machine.py b/lnst/Controller/Machine.py index 5876347..96a0faa 100644 --- a/lnst/Controller/Machine.py +++ b/lnst/Controller/Machine.py @@ -375,7 +375,9 @@ class Interface(object): self._slaves[iface.get_id()] = iface def set_slave_option(self, slave_id, name, value): - self._slave_options[slave_id] = (name, value) + if slave_id not in self._slave_options: + self._slave_options[slave_id] = [] + self._slave_options[slave_id].append((name, value)) def add_address(self, addr): self._addresses.append(addr)

1 0

[lnst] Remove NetConfigDeviceAllCleanup from machine cleanup
by Jiří Pírko 16 Aug '13

16 Aug '13

commit 6a676159d58178bf114049784b5cdf33f673b9e4 Author: Jan Tluka <jtluka(a)redhat.com> Date: Wed Aug 14 14:48:02 2013 +0200 Remove NetConfigDeviceAllCleanup from machine cleanup NetConfigDeviceAllCleanup would clean the system from netdevice modules even if they had not been configured at all. This can break a setup with virt guest running on a host system both acting as lnst slaves and using bridge netdevice driver. Signed-off-by: Jan Tluka <jtluka(a)redhat.com> Signed-off-by: Jiri Pirko <jiri(a)resnulli.us> lnst/Slave/NetTestSlave.py | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-) --- diff --git a/lnst/Slave/NetTestSlave.py b/lnst/Slave/NetTestSlave.py index 75fa7ab..0f17839 100644 --- a/lnst/Slave/NetTestSlave.py +++ b/lnst/Slave/NetTestSlave.py @@ -31,7 +31,6 @@ from lnst.Common.ResourceCache import ResourceCache from lnst.Common.NetTestCommand import NetTestCommandContext from lnst.Common.NetTestCommand import CommandException, NetTestCommand from lnst.Slave.NetConfig import NetConfig -from lnst.Slave.NetConfigDevice import NetConfigDeviceAllCleanup from lnst.Slave.NmConfigDevice import is_nm_managed_by_name from lnst.Common.Utils import check_process_running from lnst.Common.ConnectionHandler import recv_data, send_data @@ -271,7 +270,6 @@ class SlaveMethods: self._command_context.cleanup() self._netconfig.deconfigure_all() self._netconfig.cleanup() - NetConfigDeviceAllCleanup() self._cache.del_old_entries() self.restore_system_config() return True

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

LNST-developers August 2013