On Mon, Jun 22, 2020 at 04:45:00AM +0000, Vishal Deep Ajmera wrote:
Hi NM Team,
The problem we are facing is that initially when the OVS bridge and interface is not present, the Network Manager is able to properly create them and configures the IP address on the interface. After this, when we reboot the machine we see that the interface is not configured by NetworkManager. It fails with error "dependency-failed". Unfortunately no other error is seen in the journalctl output. Also attached is the .nmconnection file generated by nmstate cli.
Hi Vishal Deep Ajmera,
I failed to reproduce in my VM.
Can you create `/etc/NetworkManager/conf.d/debug.conf` with these contents:
[logging] level=TRACE domains=ALL
And then provide output of `sudo journalctl -t NetworkManager -b0`?
As a workaround we tried by removing the line "autoconnect-slaves=1" from the .nmconnection files and tried the reboot again.
Have you seen similar issue before? Any idea on how this is connected to "autoconnect-slaves" flag?
Hi Beniamino,
This sounds like the old problem of NetworkManager: https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_request...
Is it?
Also, we were not able to find any parameter which can be set in the input yaml file which can disable this flag.
Is there something available or the default is always "1"?
Nmstate currently does not support disable this option yet.
And I am afraid we cannot add the support, cause: * For complex nested bridge/vlan/bond setup, we need to trust NetworkManager do the slave activation in the correct order.
* The slave activation order will impact the MAC address choosing, if nmstate do the slave activation manually, the MAC address of master(bridge/bond) might be different after OS reboot.
But I will work with you to find out the root cause and get it fixed in NetworkManager.
The next actions would be:
* Vishal collects NetworkManager trace/debug logs. * Beniamino review this email and/or the logs, then feedback.
Best regards.
On Mon, Jun 22, 2020 at 01:55:35PM +0800, Gris Ge wrote:
On Mon, Jun 22, 2020 at 04:45:00AM +0000, Vishal Deep Ajmera wrote:
Hi NM Team,
The problem we are facing is that initially when the OVS bridge and interface is not present, the Network Manager is able to properly create them and configures the IP address on the interface. After this, when we reboot the machine we see that the interface is not configured by NetworkManager. It fails with error "dependency-failed". Unfortunately no other error is seen in the journalctl output. Also attached is the .nmconnection file generated by nmstate cli.
Hi Vishal Deep Ajmera,
I failed to reproduce in my VM.
Can you create `/etc/NetworkManager/conf.d/debug.conf` with these contents:
[logging] level=TRACE domains=ALL
And then provide output of `sudo journalctl -t NetworkManager -b0`?
As a workaround we tried by removing the line "autoconnect-slaves=1" from the .nmconnection files and tried the reboot again.
Have you seen similar issue before? Any idea on how this is connected to "autoconnect-slaves" flag?
Hi Beniamino,
This sounds like the old problem of NetworkManager: https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_request...
Is it?
Yes, there is a race condition if you set autoconnect-slaves=1 on the master and also set the slaves to autoconnect. I'm working on fixing it; the merge request you linked had to be reverted because it introduced regressions.
To workaround this bug, nmstate could avoid setting autoconnect-slaves, since the slaves are already configured to autoconnect.
Beniamino
Also, we were not able to find any parameter which can be set in the input yaml file which can disable this flag.
Is there something available or the default is always "1"?
Nmstate currently does not support disable this option yet.
And I am afraid we cannot add the support, cause: * For complex nested bridge/vlan/bond setup, we need to trust NetworkManager do the slave activation in the correct order.
* The slave activation order will impact the MAC address choosing, if nmstate do the slave activation manually, the MAC address of master(bridge/bond) might be different after OS reboot.
But I will work with you to find out the root cause and get it fixed in NetworkManager.
The next actions would be:
* Vishal collects NetworkManager trace/debug logs. * Beniamino review this email and/or the logs, then feedback.
Best regards.
-- Gris Ge
On Mon, Jun 22, 2020 at 09:09:20AM +0200, Beniamino Galvani wrote:
To workaround this bug, nmstate could avoid setting autoconnect-slaves, since the slaves are already configured to autoconnect.
Beniamino
Hi Beniamino,
We are blocking the autoconnect when we create/modify the connection/profile, so nmstate have to manually activate the slaves with the order which might be different from NetworkManager configured.
I will wait the NetworkManager patch for this.
Thanks for the clarification.
Best regards.
Hi,
As requested please find attached logs collected after enabling tracing in network-manager. I have created one OVS bridge 'bridge0' and 3 interfaces. I see that randomly only one interface gets the IP and others don't after every reboot of the system.
Warm Regards, Vishal Ajmera
-----Original Message----- From: Gris Ge fge@redhat.com Sent: Monday, June 22, 2020 1:13 PM To: Beniamino Galvani bgalvani@redhat.com Cc: Vishal Deep Ajmera vishal.deep.ajmera@ericsson.com; nmstate- devel@lists.fedorahosted.org; Ganesh Vasudevan ganesh.vasudevan@ericsson.com; Prakash Padmanabhan prakash.padmanabhan@ericsson.com; Anurag Agarwal anurag.agarwal@ericsson.com; ehaas@redhat.com Subject: Re: [nmstate-devel] Reg. nmstate issue with OVS interfaces.
On Mon, Jun 22, 2020 at 09:09:20AM +0200, Beniamino Galvani wrote:
To workaround this bug, nmstate could avoid setting autoconnect-slaves, since the slaves are already configured to autoconnect.
Beniamino
Hi Beniamino,
We are blocking the autoconnect when we create/modify the connection/profile, so nmstate have to manually activate the slaves with the order which might be different from NetworkManager configured.
I will wait the NetworkManager patch for this.
Thanks for the clarification.
Best regards.
-- Gris Ge
On Mon, Jun 22, 2020 at 09:09:20AM +0200, Beniamino Galvani wrote:
On Mon, Jun 22, 2020 at 01:55:35PM +0800, Gris Ge wrote:
On Mon, Jun 22, 2020 at 04:45:00AM +0000, Vishal Deep Ajmera wrote:
Hi NM Team,
The problem we are facing is that initially when the OVS bridge and interface is not present, the Network Manager is able to properly create them and configures the IP address on the interface. After this, when we reboot the machine we see that the interface is not configured by NetworkManager. It fails with error "dependency-failed". Unfortunately no other error is seen in the journalctl output. Also attached is the .nmconnection file generated by nmstate cli.
Hi Vishal,
The fix has been included in NetworkManager 1.26 branch:
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/commit/0911d2...
It's __not__ in 1.26.0 release, but in the nm-1-26 git branch.
I can compile a SUSE rpm for you to test on. Can you provide which SLES version you are using?
Sorry for the late response.
Best regards.
Hi Vishal,
The fix has been included in NetworkManager 1.26 branch:
https://protect2.fireeye.com/v1/url?k=7b99a9c1-25293459-7b99e95a- 86d8a30ca42b-6bcec662344c9f63&q=1&e=01d8243f-3f91-44c6-9f40- 765c5ce5a0af&u=https%3A%2F%2Fgitlab.freedesktop.org%2FNetworkManag er%2FNetworkManager%2F- %2Fcommit%2F0911d2a4ee1c317f455c01cbad048b994f69844d
It's __not__ in 1.26.0 release, but in the nm-1-26 git branch.
I can compile a SUSE rpm for you to test on. Can you provide which SLES version you are using?
Thanks Gris for your support. We are using SLES 15 SP1.
Warm Regards, Vishal Ajmera
nmstate-devel@lists.fedorahosted.org