Am Donnerstag, den 23.01.2014, 12:34 -0300 schrieb Francisco Pérez:
Hi. I have configured an ovirt-engine to manage an ovirt node succesfully. I have created VM and all runs well but im trying to do some escenario testing and i have found an issue.
If somehow the ovirt node is rebooted(Power failure, crash, manual reboot, etc) when its comes up it does not start the ovirtmgmt network. Even if in the ovirt-engine you save network configuration or if you execute persist on /etc/sysconfig/network-scripts/ifcfg-ovirtmgmt. It tells you it's already been added to the configuration.
Is this normal behavior after ovirt node's is rebooted? Am i doing something wrong?
Hey Francisco,
that sounds like a bug. VDSM Devs, how can we debug why the network is not brought up?
- fabian
On 01/23/2014 06:12 PM, Fabian Deutsch wrote:
Am Donnerstag, den 23.01.2014, 12:34 -0300 schrieb Francisco Pérez:
Hi. I have configured an ovirt-engine to manage an ovirt node succesfully. I have created VM and all runs well but im trying to do some escenario testing and i have found an issue.
If somehow the ovirt node is rebooted(Power failure, crash, manual reboot, etc) when its comes up it does not start the ovirtmgmt network. Even if in the ovirt-engine you save network configuration or if you execute persist on /etc/sysconfig/network-scripts/ifcfg-ovirtmgmt. It tells you it's already been added to the configuration.
Is this normal behavior after ovirt node's is rebooted? Am i doing something wrong?
Hey Francisco,
that sounds like a bug. VDSM Devs, how can we debug why the network is not brought up?
- fabian
is there an open bug tracking this?
----- Original Message -----
From: "Fabian Deutsch" fabiand@redhat.com To: "Francisco Pérez" fperez.x@gmail.com, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Cc: users@ovirt.org Sent: Thursday, January 23, 2014 5:12:17 PM Subject: Re: [Users] ovirtmgmt vanishes after reboot
Am Donnerstag, den 23.01.2014, 12:34 -0300 schrieb Francisco Pérez:
Hi. I have configured an ovirt-engine to manage an ovirt node succesfully. I have created VM and all runs well but im trying to do some escenario testing and i have found an issue.
If somehow the ovirt node is rebooted(Power failure, crash, manual reboot, etc) when its comes up it does not start the ovirtmgmt network. Even if in the ovirt-engine you save network configuration or if you execute persist on /etc/sysconfig/network-scripts/ifcfg-ovirtmgmt. It tells you it's already been added to the configuration.
Is this normal behavior after ovirt node's is rebooted? Am i doing something wrong?
Hey Francisco,
that sounds like a bug. VDSM Devs, how can we debug why the network is not brought up?
Well, we should check if after the reboot /etc/sysconfig/network-scripts/ifcfg-ovirtmgmt and /etc/sysconfig/network-scripts/ifcfg-emX (or whatever the name of the interface ovirtmgmt sits on is) are present. If vdsmd is started and properly listening and if that is the case. virsh -r net-list has vdsm-ovirtmgmt
- fabian
Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On Thu, Jan 23, 2014 at 05:12:17PM +0100, Fabian Deutsch wrote:
Am Donnerstag, den 23.01.2014, 12:34 -0300 schrieb Francisco Pérez:
Hi. I have configured an ovirt-engine to manage an ovirt node succesfully. I have created VM and all runs well but im trying to do some escenario testing and i have found an issue.
If somehow the ovirt node is rebooted(Power failure, crash, manual reboot, etc) when its comes up it does not start the ovirtmgmt network. Even if in the ovirt-engine you save network configuration or if you execute persist on /etc/sysconfig/network-scripts/ifcfg-ovirtmgmt. It tells you it's already been added to the configuration.
Is this normal behavior after ovirt node's is rebooted? Am i doing something wrong?
Hey Francisco,
that sounds like a bug. VDSM Devs, how can we debug why the network is not brought up?
Could you share vdsm.log and supervdsm.log? When you set up ovirtmgmt, as setupNetworks command should be visible there. Later, on "save", a call to setSafeNetworkConfig is sent, which is expected to persist your network configuration.
Is it visible? Is it successful?
What do you see in the logs post boot? Can you tell whether the vdsm-restore-net-config.init server was run? (its job is to revert unsaved network config changes on non-ovirt-nodes).
Im setting up two networks. ovirtmgmt and a LAN. Both of them are gone after reboot on the node, its starts only with eth0 initial configuration.
Antoni, here are the outputs you requested:
[root@ovirth1 ~]# /etc/init.d/vdsmd status VDS daemon server is running [root@ovirth1 ~]# service vdsmd status VDS daemon server is running [root@ovirth1 ~]# virsh -r net-list setlocale: No such file or directory Name State Autostart Persistent -------------------------------------------------- ;vdsmdummy; active no no
[root@ovirth1 ~]# ls /etc/sysconfig/network-scripts/ ifcfg-eth0 ifdown-eth ifdown-post ifdown-tunnel ifup-eth ifup-plip ifup-routes init.ipv6-global route-LAN ifcfg-lo ifdown-ippp ifdown-ppp ifup ifup-ippp ifup-plusb ifup-sit net.hotplug route-ovirtmgmt ifdown ifdown-ipv6 ifdown-routes ifup-aliases ifup-ipv6 ifup-post ifup-tunnel network-functions rule-LAN ifdown-bnep ifdown-isdn ifdown-sit ifup-bnep ifup-isdn ifup-ppp ifup-wireless network-functions-ipv6 rule-ovirtmgmt
Dan those command are excecuted where? on the manager or the node? Im new at this and the logs are a little overwhelming so if you help me to interpret them i'll apreciate them. I have attached the whole log directory /var/log/vdsm/
Regards
On Thu, Jan 23, 2014 at 4:32 PM, Dan Kenigsberg danken@redhat.com wrote:
On Thu, Jan 23, 2014 at 05:12:17PM +0100, Fabian Deutsch wrote:
Am Donnerstag, den 23.01.2014, 12:34 -0300 schrieb Francisco Pérez:
Hi. I have configured an ovirt-engine to manage an ovirt node succesfully. I have created VM and all runs well but im trying to do some escenario testing and i have found an issue.
If somehow the ovirt node is rebooted(Power failure, crash, manual reboot, etc) when its comes up it does not start the ovirtmgmt network. Even if in the ovirt-engine you save network configuration or if you execute persist on /etc/sysconfig/network-scripts/ifcfg-ovirtmgmt. It tells you it's already been added to the configuration.
Is this normal behavior after ovirt node's is rebooted? Am i doing something wrong?
Hey Francisco,
that sounds like a bug. VDSM Devs, how can we debug why the network is not brought up?
Could you share vdsm.log and supervdsm.log? When you set up ovirtmgmt, as setupNetworks command should be visible there. Later, on "save", a call to setSafeNetworkConfig is sent, which is expected to persist your network configuration.
Is it visible? Is it successful?
What do you see in the logs post boot? Can you tell whether the vdsm-restore-net-config.init server was run? (its job is to revert unsaved network config changes on non-ovirt-nodes).
On Tue, Jan 28, 2014 at 11:27:21AM -0300, Francisco Pérez wrote:
Im setting up two networks. ovirtmgmt and a LAN. Both of them are gone after reboot on the node, its starts only with eth0 initial configuration.
Antoni, here are the outputs you requested:
[root@ovirth1 ~]# /etc/init.d/vdsmd status VDS daemon server is running [root@ovirth1 ~]# service vdsmd status VDS daemon server is running [root@ovirth1 ~]# virsh -r net-list setlocale: No such file or directory Name State Autostart Persistent
;vdsmdummy; active no no
[root@ovirth1 ~]# ls /etc/sysconfig/network-scripts/ ifcfg-eth0 ifdown-eth ifdown-post ifdown-tunnel ifup-eth ifup-plip ifup-routes init.ipv6-global route-LAN ifcfg-lo ifdown-ippp ifdown-ppp ifup ifup-ippp ifup-plusb ifup-sit net.hotplug route-ovirtmgmt ifdown ifdown-ipv6 ifdown-routes ifup-aliases ifup-ipv6 ifup-post ifup-tunnel network-functions rule-LAN ifdown-bnep ifdown-isdn ifdown-sit ifup-bnep ifup-isdn ifup-ppp ifup-wireless network-functions-ipv6 rule-ovirtmgmt
Dan those command are excecuted where? on the manager or the node? Im new at this and the logs are a little overwhelming so if you help me to interpret them i'll apreciate them. I have attached the whole log directory /var/log/vdsm/
Could you share vdsm.log and supervdsm.log? When you set up ovirtmgmt, as setupNetworks command should be visible there. Later, on "save", a call to setSafeNetworkConfig is sent, which is expected to persist your network configuration.
Is it visible? Is it successful?
Your supervdsmd has
MainProcess|Thread-26::DEBUG::2014-01-27 15:51:48,866::configNetwork::561::Storage.Misc.excCmd::(setSafeNetworkConfig) '/usr/share/vdsm/vdsm-store-net-config' (cwd None)
which reports success, too.
What do you see in the logs post boot? Can you tell whether the vdsm-restore-net-config.init server was run? (its job is to revert unsaved network config changes on non-ovirt-nodes).
Maybe you can debug the issue: Setup up a network but do not save it yet. Log into your ovirt-node.
You should see /etc/sysconfig/network-scripts/ifcfg-<YOURNET> and /var/lib/vdsm/netconfback/ifcfg-<YOURNET>, the latter meaning that it's not yet persisted.
Now run locally vdsClient -s 0 setSafeNetworkConfig
You should see /config/etc/sysconfig/network-scripts/ifcfg-<YOURNET> created, and /var/lib/vdsm/netconfback/ifcfg-<YOURNET> gone.
Now please run /usr/share/vdsm/vdsm-restore-net-config which should do absolutely nothing.
If the process above breaks, it's a Vdsm bug. If not, it most probably something in ovirt-node.
----- Original Message -----
From: "Dan Kenigsberg" danken@redhat.com To: "Francisco Pérez" fperez.x@gmail.com Cc: "Fabian Deutsch" fabiand@redhat.com, amuller@redhat.com, asegurap@redhat.com, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org, users@ovirt.org Sent: Tuesday, January 28, 2014 10:47:17 PM Subject: Re: [vdsm] [Users] ovirtmgmt vanishes after reboot
On Tue, Jan 28, 2014 at 11:27:21AM -0300, Francisco Pérez wrote:
Im setting up two networks. ovirtmgmt and a LAN. Both of them are gone after reboot on the node, its starts only with eth0 initial configuration.
Antoni, here are the outputs you requested:
[root@ovirth1 ~]# /etc/init.d/vdsmd status VDS daemon server is running [root@ovirth1 ~]# service vdsmd status VDS daemon server is running [root@ovirth1 ~]# virsh -r net-list setlocale: No such file or directory Name State Autostart Persistent
;vdsmdummy; active no no
[root@ovirth1 ~]# ls /etc/sysconfig/network-scripts/ ifcfg-eth0 ifdown-eth ifdown-post ifdown-tunnel ifup-eth ifup-plip ifup-routes init.ipv6-global route-LAN ifcfg-lo ifdown-ippp ifdown-ppp ifup ifup-ippp ifup-plusb ifup-sit net.hotplug route-ovirtmgmt ifdown ifdown-ipv6 ifdown-routes ifup-aliases ifup-ipv6 ifup-post ifup-tunnel network-functions rule-LAN ifdown-bnep ifdown-isdn ifdown-sit ifup-bnep ifup-isdn ifup-ppp ifup-wireless network-functions-ipv6 rule-ovirtmgmt
Dan those command are excecuted where? on the manager or the node? Im new at this and the logs are a little overwhelming so if you help me to interpret them i'll apreciate them. I have attached the whole log directory /var/log/vdsm/
Could you share vdsm.log and supervdsm.log? When you set up ovirtmgmt, as setupNetworks command should be visible there. Later, on "save", a call to setSafeNetworkConfig is sent, which is expected to persist your network configuration.
Is it visible? Is it successful?
Your supervdsmd has
MainProcess|Thread-26::DEBUG::2014-01-27 15:51:48,866::configNetwork::561::Storage.Misc.excCmd::(setSafeNetworkConfig) '/usr/share/vdsm/vdsm-store-net-config' (cwd None)which reports success, too.
What do you see in the logs post boot? Can you tell whether the vdsm-restore-net-config.init server was run? (its job is to revert unsaved network config changes on non-ovirt-nodes).
Maybe you can debug the issue: Setup up a network but do not save it yet. Log into your ovirt-node.
You should see /etc/sysconfig/network-scripts/ifcfg-<YOURNET> and /var/lib/vdsm/netconfback/ifcfg-<YOURNET>, the latter meaning that it's not yet persisted.
Now run locally vdsClient -s 0 setSafeNetworkConfig
You should see /config/etc/sysconfig/network-scripts/ifcfg-<YOURNET> created, and /var/lib/vdsm/netconfback/ifcfg-<YOURNET> gone.
/var/lib/vdsm/netconfback/ifcfg-<YOURNET> is not removed on oVirt Node as /var/lib/vdsm/netconfback is sitting on a tmpfs so there was no need to call it for the reboot case.
We made some more tests with Francisco on #ovirt today and we saw that for some unknown reason, when rebooting the machine the ifcfg-<YOURNET> files disappear while the rule-<YOURNET> and route-<YOURNET> stay. That happens even when the both ifcfg, rule and route have the correct entry in /config/files and are bound to /config/etc/sysconfig/
After reboot it returns to eth0 having the connectivity as it was defined with the TUI.
For libvirt networks a bit of the same as for ifcfg files. Before reboot /etc/libvirt/qemu/networks is present in /config/files and findmnt reports that /etc/libvirt/qemu/networks is a bound mount of /config/etc/libvirt/qemu/networks. After reboot it is not the case anymore.
@Fabian: Do you think there is something on the reboot that restores the network conf to the TUI settings?
Now please run /usr/share/vdsm/vdsm-restore-net-config which should do absolutely nothing.
If the process above breaks, it's a Vdsm bug. If not, it most probably something in ovirt-node.
Am Dienstag, den 28.01.2014, 17:42 -0500 schrieb Antoni Segura Puimedon:
We made some more tests with Francisco on #ovirt today and we saw that for some unknown reason, when rebooting the machine the ifcfg-<YOURNET> files disappear while the rule-<YOURNET> and route-<YOURNET> stay. That happens even when the both ifcfg, rule and route have the correct entry in /config/files and are bound to /config/etc/sysconfig/
After reboot it returns to eth0 having the connectivity as it was defined with the TUI.
For libvirt networks a bit of the same as for ifcfg files. Before reboot /etc/libvirt/qemu/networks is present in /config/files and findmnt reports that /etc/libvirt/qemu/networks is a bound mount of /config/etc/libvirt/qemu/networks. After reboot it is not the case anymore.
@Fabian: Do you think there is something on the reboot that restores the network conf to the TUI settings?
Hey Antoni,
I vaguely remember that we had a problem with Node removing some ifcfg-* files. But this should have been solved in the TestDay iso.
What ISO did you use to reproduce this problem?
- fabian
----- Original Message -----
From: "Fabian Deutsch" fabiand@redhat.com To: "Antoni Segura Puimedon" asegurap@redhat.com Cc: "Dan Kenigsberg" danken@redhat.com, "Francisco Pérez" fperez.x@gmail.com, amuller@redhat.com, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org, users@ovirt.org Sent: Wednesday, January 29, 2014 10:59:43 AM Subject: Re: [vdsm] [Users] ovirtmgmt vanishes after reboot
Am Dienstag, den 28.01.2014, 17:42 -0500 schrieb Antoni Segura Puimedon:
We made some more tests with Francisco on #ovirt today and we saw that for some unknown reason, when rebooting the machine the ifcfg-<YOURNET> files disappear while the rule-<YOURNET> and route-<YOURNET> stay. That happens even when the both ifcfg, rule and route have the correct entry in /config/files and are bound to /config/etc/sysconfig/
After reboot it returns to eth0 having the connectivity as it was defined with the TUI.
For libvirt networks a bit of the same as for ifcfg files. Before reboot /etc/libvirt/qemu/networks is present in /config/files and findmnt reports that /etc/libvirt/qemu/networks is a bound mount of /config/etc/libvirt/qemu/networks. After reboot it is not the case anymore.
@Fabian: Do you think there is something on the reboot that restores the network conf to the TUI settings?
Hey Antoni,
I vaguely remember that we had a problem with Node removing some ifcfg-* files. But this should have been solved in the TestDay iso.
What ISO did you use to reproduce this problem?
We reproduced on Francisco's setup, I think it was oVirt node 3.0.1
- fabian
Am Mittwoch, den 29.01.2014, 05:02 -0500 schrieb Antoni Segura Puimedon:
----- Original Message -----
From: "Fabian Deutsch" fabiand@redhat.com To: "Antoni Segura Puimedon" asegurap@redhat.com Cc: "Dan Kenigsberg" danken@redhat.com, "Francisco Pérez"
fperez.x@gmail.com, amuller@redhat.com, "VDSM Project
Development" vdsm-devel@lists.fedorahosted.org, users@ovirt.org Sent: Wednesday, January 29, 2014 10:59:43 AM Subject: Re: [vdsm] [Users] ovirtmgmt vanishes after reboot
Am Dienstag, den 28.01.2014, 17:42 -0500 schrieb Antoni Segura
Puimedon:
We made some more tests with Francisco on #ovirt today and we saw
that
for some unknown reason, when rebooting the machine the ifcfg-<YOURNET> files disappear while the rule-<YOURNET> and route-<YOURNET> stay. That happens even when the both ifcfg, rule and route have the correct entry in /config/files and are bound to /config/etc/sysconfig/
After reboot it returns to eth0 having the connectivity as it was defined with the TUI.
For libvirt networks a bit of the same as for ifcfg files. Before reboot /etc/libvirt/qemu/networks is present in /config/files and findmnt reports that /etc/libvirt/qemu/networks is a bound mount of /config/etc/libvirt/qemu/networks. After reboot it is not the case anymore.
@Fabian: Do you think there is something on the reboot that
restores
the network conf to the TUI settings?
Hey Antoni,
I vaguely remember that we had a problem with Node removing some
ifcfg-*
files. But this should have been solved in the TestDay iso.
What ISO did you use to reproduce this problem?
We reproduced on Francisco's setup, I think it was oVirt node 3.0.1
Right. I believe the bug you are seeing is: http://gerrit.ovirt.org/#/c/20068/
This has been merged into out stable branch, but the release of a new oVirt Node ISO for 3.3 is pending because of the vdsm-python-cpopen dependency problem.
We plan to do a build early next week.
- fabian
OK. thanks all of you for your help. I'll wait until the new release.
On Wed, Jan 29, 2014 at 7:07 AM, Fabian Deutsch fabiand@redhat.com wrote:
Am Mittwoch, den 29.01.2014, 05:02 -0500 schrieb Antoni Segura Puimedon:
----- Original Message -----
From: "Fabian Deutsch" fabiand@redhat.com To: "Antoni Segura Puimedon" asegurap@redhat.com Cc: "Dan Kenigsberg" danken@redhat.com, "Francisco Pérez"
fperez.x@gmail.com, amuller@redhat.com, "VDSM Project
Development" vdsm-devel@lists.fedorahosted.org, users@ovirt.org Sent: Wednesday, January 29, 2014 10:59:43 AM Subject: Re: [vdsm] [Users] ovirtmgmt vanishes after reboot
Am Dienstag, den 28.01.2014, 17:42 -0500 schrieb Antoni Segura
Puimedon:
We made some more tests with Francisco on #ovirt today and we saw
that
for some unknown reason, when rebooting the machine the ifcfg-<YOURNET> files disappear while the rule-<YOURNET> and route-<YOURNET> stay. That happens even when the both ifcfg, rule and route have the correct entry in /config/files and are bound to /config/etc/sysconfig/
After reboot it returns to eth0 having the connectivity as it was defined with the TUI.
For libvirt networks a bit of the same as for ifcfg files. Before reboot /etc/libvirt/qemu/networks is present in /config/files and findmnt reports that /etc/libvirt/qemu/networks is a bound mount of /config/etc/libvirt/qemu/networks. After reboot it is not the case anymore.
@Fabian: Do you think there is something on the reboot that
restores
the network conf to the TUI settings?
Hey Antoni,
I vaguely remember that we had a problem with Node removing some
ifcfg-*
files. But this should have been solved in the TestDay iso.
What ISO did you use to reproduce this problem?
We reproduced on Francisco's setup, I think it was oVirt node 3.0.1
Right. I believe the bug you are seeing is: http://gerrit.ovirt.org/#/c/20068/
This has been merged into out stable branch, but the release of a new oVirt Node ISO for 3.3 is pending because of the vdsm-python-cpopen dependency problem.
We plan to do a build early next week.
- fabian
On Wed, Jan 29, 2014 at 11:07:08AM +0100, Fabian Deutsch wrote:
Am Mittwoch, den 29.01.2014, 05:02 -0500 schrieb Antoni Segura Puimedon:
Right.
I believe the bug you are seeing is: http://gerrit.ovirt.org/#/c/20068/
This has been merged into out stable branch, but the release of a new oVirt Node ISO for 3.3 is pending because of the vdsm-python-cpopen dependency problem.
Would you agree to patch `yum` on your build machine? I do not fully understand the issue, but with the following patch, the dependency problem is resolved for me:
diff --git a/yum/depsolve.py b/yum/depsolve.py index 95c21bc..57cf379 100644 --- a/yum/depsolve.py +++ b/yum/depsolve.py @@ -720,7 +720,7 @@ class Depsolve(object): else: self.verbose_logger.debug(_('TSINFO: Marking %s as install for %s'), best, requiringPo) - reqtuple = misc.string_to_prco_tuple(needname + str(needflags) + needversion) + reqtuple = misc.string_to_prco_tuple(requirement) txmbrs = self.install(best, provides_for=reqtuple) for txmbr in txmbrs: txmbr.setAsDep(po=requiringPo)
vdsm-devel@lists.stg.fedorahosted.org
-
asegurap＠redhat.com -
Dan Kenigsberg -
fabiand＠redhat.com -
Francisco José Pérez González -
iheim＠redhat.com