#14: Investigate systemd-networkd ------------------------------+------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Keywords: ------------------------------+------------------------------- * What: new service to do very basic network config * Where: remove the sysv network scripts, add config for this * Why: remove very kludgey shell scripts; try the new hotness * When: Before F21 alpha * Who: Janez Nemanic
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by janeznemanic):
Just to let everybody know I am still working on this ticket. Slowly I am making progress.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by mattdm):
Cool, thanks. Any particular blockers? We can try it in rawhide when you're ready.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by janeznemanic):
Report on my work.
First I had to learn a little bit about systemd. What is all about, how does it work etc. Then I focused on networking subsystem in Fedora and tried to understand how it works. After setting my little lab environment I started with hands on approach.
My test environment: - host system: 32 bit Fedora 20 - hypervisor: VirtualBox - guest system: 32 bit Fedora 20 - systemd release: rawhide
My experiment and current findings
I disabled Gnome's NetworkManager, NetworkManager-dispatcher and NetworkManager-wait-online and dropped network configuration file into /usr/lib/systemd/network. The content of this file is:
[Match] Name=p*
[Network] DHCP=yes
Systemd names network interface p2p1 on my guest Fedora. So that's why I used Name=p*. That's the most simple and basic network configuration with systemd-networkd. I think that DHCP is also default configuration in desktop Fedora 20. I restarted systemd-networkd service and I could ping www.google.com and do DNS look up with dig. Ping and dig work as you would expect on my guest and on my host system.
I did some research on scripts in /etc/sysconfig/network-scripts. All those scripts depend on ifcfg-* files in the same directory. But systemd- networkd doesn't use ifcfg-* files. I removed ifcfg-* files and systemd- netword worked fine. I was also able to ping google. But if I tried to use for example ifup or ifdown script then that script failed. Because it couldn't source ifcfg-*. I think so, I might be wrong. Since ifup and ifdown are links to /usr/sbin/{ifup,ifdown} it might be possible to remove /usr/sbin/{ifup,ifdown}. But I haven't tested that so I'm not sure. I think we also don't need to use /etc/rc.d/init.d/network. I removed that file and networking worked fine with systemd-networkd.
So to sum up. If systemd-networkd service is used it is possible in my opinion to remove scripts in /etc/sysconfig/network-scripts and /etc/rc.d/init.d/network script. Besides that it is possible to remove NetworkManager.
'''Thoughts? Suggestions? Ideas?'''
One more thing that's been bothering me for a few days now. How does systemd name network interfaces when Fedora is run on top of different hypervisors? If xen or kvm is used how does systemd name network interfaces?
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by johannbg):
For the first always use /etc/systemd/system never /usr/lib/systemd <-- this is for packages and will get overwritten flushed out on updates secondly we will be migrating everything to network units when the time is right as well as splitting out and removing the remaining parts of the legacy sysv initscript if that work wont be halted by traditionalists ( initscript contain for example the chkconfig/service command ) within and outside Red Hat as well as RHEL 8 since there seems to be somekind of internal RH dispute going on determine how the future of network should look like in products which is being kept on need to know bases.
Anyway split probably will end up somewhere along these lines
initscript
Requires changes in the FPG
initscripts-doc initscripts-man initscripts-locale initscripts-network
Split into it's own package outside initscript
netconsole netreport ppp systemd-fedora
These changes along with other necessary once that might never be allowed to be implemented in Fedora due to RHEL but should be, will land in a core/baseOS copr repo which should emerge as soon as copr starts supporting groups in FAS ( we dont want to tie this to a single individual ) which the cloud community could use for testing and what not and at that same time it will also probably emerge a test repository which will contain the latest systemd release for GA releases as well.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by gospo):
Replying to [comment:3 janeznemanic]:
Report on my work.
[...]
I did some research on scripts in /etc/sysconfig/network-scripts. All
those scripts depend on ifcfg-* files in the same directory. But systemd- networkd doesn't use ifcfg-* files. I removed ifcfg-* files and systemd- netword worked fine. I was also able to ping google. But if I tried to use for example ifup or ifdown script then that script failed. Because it couldn't source ifcfg-*. I think so, I might be wrong. Since ifup and ifdown are links to /usr/sbin/{ifup,ifdown} it might be possible to remove /usr/sbin/{ifup,ifdown}. But I haven't tested that so I'm not sure. I think we also don't need to use /etc/rc.d/init.d/network. I removed that file and networking worked fine with systemd-networkd.
So to sum up. If systemd-networkd service is used it is possible in my
opinion to remove scripts in /etc/sysconfig/network-scripts and /etc/rc.d/init.d/network script. Besides that it is possible to remove NetworkManager.
I took a quick look at the files from initscripts that are likely networking related. It seems possible to remote these from initscripts core and (as Johannes suggested above) possibly break initscripts out into a new package. If there was a desire to add the missing functionality to networkd that is currently in initscripts then the new initscripts package could be slowly whittled away to nothing when ppp and netconsole support are added to networkd (that appears to be the only missing functionality).
/etc/NetworkManager /etc/NetworkManager/dispatcher.d /etc/NetworkManager/dispatcher.d/00-netreport /etc/networks /etc/ppp /etc/ppp/ip-down /etc/ppp/ip-down.ipv6to4 /etc/ppp/ip-up /etc/ppp/ip-up.ipv6to4 /etc/ppp/ipv6-down /etc/ppp/ipv6-up /etc/ppp/peers /etc/rc.d/init.d/netconsole /etc/rc.d/init.d/network /etc/sysconfig/netconsole /etc/sysconfig/network-scripts /etc/sysconfig/network-scripts/* /usr/bin/ipcalc /usr/lib/udev/rename_device /usr/lib/udev/rules.d/60-net.rules /usr/libexec/initscripts /usr/libexec/initscripts/legacy-actions /usr/sbin/ifdown /usr/sbin/ifup /usr/sbin/netreport /usr/sbin/ppp-watch /usr/sbin/usernetctl /usr/share/doc/initscripts/changes.ipv6 /usr/share/doc/initscripts/ipv6-6to4.howto /usr/share/doc/initscripts/ipv6-tunnel.howto /usr/share/doc/initscripts/static-routes-ipv6 /usr/share/man/man1/genhostid.1.gz /usr/share/man/man1/ipcalc.1.gz /usr/share/man/man1/netreport.1.gz /usr/share/man/man8/ifdown.8.gz /usr/share/man/man8/ifup.8.gz /usr/share/man/man8/ppp-watch.8.gz}}}
'''Thoughts? Suggestions? Ideas?'''
One more thing that's been bothering me for a few days now. How does
systemd name network interfaces when Fedora is run on top of different hypervisors? If xen or kvm is used how does systemd name network interfaces?
Systemd will name the device based on the bus/device/function of the interfaces, so there could be some differences in the name chosen depending on the hypervisor used (since each hypervisor may chose a difference bus/dev/function for a given emulated interface.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by gospo):
Replying to [comment:3 janeznemanic]:
But if I tried to use for example ifup or ifdown script then that script
failed. Because it couldn't source ifcfg-*. I think so, I might be wrong. Since ifup and ifdown are links to /usr/sbin/{ifup,ifdown} it might be possible to remove /usr/sbin/{ifup,ifdown}. But I haven't tested that so I'm not sure. I think we also don't need to use /etc/rc.d/init.d/network. I removed that file and networking worked fine with systemd-networkd.
Sorry, I forgot to address this in my last post. As was mentioned above there are some who will want to preserve some of the ability to use ifup/ifdown in some manner. I know that controlling the network interfaces is a current TODO for networkd, but it would seem reasonable that the networkd unit can be stopped/started or restarted if one wanted to reset the network config, correct?
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by mattdm):
Replying to [comment:5 gospo]:
One more thing that's been bothering me for a few days now. How does
systemd name network interfaces when Fedora is run on top of different hypervisors? If xen or kvm is used how does systemd name network interfaces?
Systemd will name the device based on the bus/device/function of the
interfaces, so there could be some differences in the name chosen depending on the hypervisor used (since each hypervisor may chose a difference bus/dev/function for a given emulated interface.
It's important to note that the Fedora Cloud image, we disable this feature (ln -s /dev/null /etc/udev/rules.d/80-net-name-slot.rules), because systemd's priority is consistency of naming when there are multiple interfaces on the same system. By and large, our concern is with consistency of a *single* device name whereever the image boots, so the network interface is traditional eth0 (or eth* if more than one).
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by mattdm):
Also, note that we are not using NetworkManager -- we're using the traditional network initscripts. It would be nice to eventually move away from these, and I don't have a strong view over which one should win (once NetworkManager gains the ability to configure a dhcp interface and then stop running).
From my (cloud SIG) point of view, NetworkManager has the disadvantage of being dependency heavy (and the run-once mode is not yet done -- see https://bugzilla.redhat.com/show_bug.cgi?id=863515), but the significant advantage of parsing the existing config files.
If there were a generator for systemd-networkd which could read the basic values from the legacy files and create network units on the fly (akin to systemd-fstab-generator), that would level that advantage. (No need to implement the full syntax; just basic static and dhcp, plus log when something isn't understood and perhaps recommend NetworkManager for this cases.) And ifup/ifdown compatibility scripts would be nice (although I don't think we generally care for the cloud case)
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by gospo):
Replying to [comment:7 mattdm]:
It's important to note that the Fedora Cloud image, we disable this
feature (ln -s /dev/null /etc/udev/rules.d/80-net-name-slot.rules), because systemd's priority is consistency of naming when there are multiple interfaces on the same system. By and large, our concern is with consistency of a *single* device name whereever the image boots, so the network interface is traditional eth0 (or eth* if more than one).
I was thinking about the fact that the slot-naming-rules could be modified so that the first NIC found was named em1, second em2, etc, but disabling the rules completely seems fine.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by johannbg):
As Matt points out we lack any kind of ifcfg-generator we also lack tunnelling support etc so I suggest people *wait* until the networkd code in systemd stabilizes before proceeding any further with this
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by janeznemanic):
Replying to [comment:6 gospo]:
Replying to [comment:3 janeznemanic]:
But if I tried to use for example ifup or ifdown script then that
script failed. Because it couldn't source ifcfg-*. I think so, I might be wrong. Since ifup and ifdown are links to /usr/sbin/{ifup,ifdown} it might be possible to remove /usr/sbin/{ifup,ifdown}. But I haven't tested that so I'm not sure. I think we also don't need to use /etc/rc.d/init.d/network. I removed that file and networking worked fine with systemd-networkd.
Sorry, I forgot to address this in my last post. As was mentioned above
there are some who will want to preserve some of the ability to use ifup/ifdown in some manner. I know that controlling the network interfaces is a current TODO for networkd, but it would seem reasonable that the networkd unit can be stopped/started or restarted if one wanted to reset the network config, correct?
You can do ''systemctl {start,stop,restart} systemd-networkd.service'' to control networking. I'm not fedora networking expert, but I think it is possible to use ip command instead of ifup/ifdown to bring interface up or down.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by janeznemanic):
Replying to [comment:7 mattdm]:
Replying to [comment:5 gospo]:
One more thing that's been bothering me for a few days now. How does
systemd name network interfaces when Fedora is run on top of different hypervisors? If xen or kvm is used how does systemd name network interfaces?
Systemd will name the device based on the bus/device/function of the
interfaces, so there could be some differences in the name chosen depending on the hypervisor used (since each hypervisor may chose a difference bus/dev/function for a given emulated interface.
It's important to note that the Fedora Cloud image, we disable this
feature (ln -s /dev/null /etc/udev/rules.d/80-net-name-slot.rules), because systemd's priority is consistency of naming when there are multiple interfaces on the same system. By and large, our concern is with consistency of a *single* device name whereever the image boots, so the network interface is traditional eth0 (or eth* if more than one).
That's a good news. It makes things a lot less complicated.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by mattdm):
Replying to [comment:10 johannbg]:
As Matt points out we lack any kind of ifcfg-generator we also lack
tunnelling support etc so I suggest people *wait* until the networkd code in systemd stabilizes before proceeding any further with this
Let's start collecting a list of basic requirements. Just to brainstorm a bit
* Single eth0 interface with DHCP * Possible other interfaces, also with DHCP (common in OpenNebula) * Static ipv4 configuration * ipv6? * Nice-to-have: ifcfg compatibility (basic config files, ifup/ifdown commands)
Some use cases for basic use and for more advanced functionality like tunneling would help make a prioritized list.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by janeznemanic):
Replying to [comment:8 mattdm]:
Also, note that we are not using NetworkManager -- we're using the
traditional network initscripts. It would be nice to eventually move away from these, and I don't have a strong view over which one should win (once NetworkManager gains the ability to configure a dhcp interface and then stop running).
From my (cloud SIG) point of view, NetworkManager has the disadvantage
of being dependency heavy (and the run-once mode is not yet done -- see https://bugzilla.redhat.com/show_bug.cgi?id=863515), but the significant advantage of parsing the existing config files.
If there were a generator for systemd-networkd which could read the
basic values from the legacy files and create network units on the fly (akin to systemd-fstab-generator), that would level that advantage. (No need to implement the full syntax; just basic static and dhcp, plus log when something isn't understood and perhaps recommend NetworkManager for this cases.) And ifup/ifdown compatibility scripts would be nice (although I don't think we generally care for the cloud case)
Found out today on mailing list that cloud image is not using NetworkManager?. To me the idea of having both NetworkManager? and systemd-networkd in the cloud image is a little bit strange since both are network managers. I assume that NetworkManager? is superior to systemd- networkd, but I think systemd-networkd does it's job good in virtualized environment. Anyway I'm not an expert so that's just my opinion.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by janeznemanic):
Replying to [comment:9 gospo]:
Replying to [comment:7 mattdm]:
It's important to note that the Fedora Cloud image, we disable this
feature (ln -s /dev/null /etc/udev/rules.d/80-net-name-slot.rules), because systemd's priority is consistency of naming when there are multiple interfaces on the same system. By and large, our concern is with consistency of a *single* device name whereever the image boots, so the network interface is traditional eth0 (or eth* if more than one).
I was thinking about the fact that the slot-naming-rules could be
modified so that the first NIC found was named em1, second em2, etc, but disabling the rules completely seems fine.
I had same idea.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by janeznemanic):
Replying to [comment:13 mattdm]:
Replying to [comment:10 johannbg]:
As Matt points out we lack any kind of ifcfg-generator we also lack
tunnelling support etc so I suggest people *wait* until the networkd code in systemd stabilizes before proceeding any further with this
Let's start collecting a list of basic requirements. Just to brainstorm
a bit
- Single eth0 interface with DHCP
- Possible other interfaces, also with DHCP (common in OpenNebula)
- Static ipv4 configuration
- ipv6?
- Nice-to-have: ifcfg compatibility (basic config files, ifup/ifdown
commands)
Some use cases for basic use and for more advanced functionality like
tunneling would help make a prioritized list.
Which other interfaces do you have in mind? Bridge?
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by janeznemanic):
So what's the plan here? How do we proceed? What shall I do?
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by mattdm):
Replying to [comment:17 janeznemanic]:
So what's the plan here? How do we proceed? What shall I do?
It might be useful to put together a variant of the kickstart file used to produce the current cloud image which uses systemd-networkd for eth0 instead of initscripts. See https://git.fedorahosted.org/cgit/spin- kickstarts.git/tree/fedora-cloud-base.ks and basically search for "network" for parts that might need to be changed.
Then, you should be able to feed that kickstart file into ImageFactory (or appliance-creator) and get a qcow2 image we can test.
I suppose that someone should file an RFE for systemd to have a "generator" (that's a typical systemd construct) for handling legacy ifcfg files (as above), and to create the ifup/ifdown compatibility scripts. Most ideally, someone will contribute that as patches to systemd upstream, because code always speaks louder than wishes :) but, just explaining what we need is a good start.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by janeznemanic):
Ok, I will try to put together kickstart file and create test image. Systemd generator is a good idea. We should keep ifup/ifdown scripts. I think they simplify things. It is possible to do the same with ip utility and related tools, but that's more complicated. I was wondering which process is currently responsible for creating and modifying ifcfg-* files? I tried to find out that by myself but I can't find the answer. Can someone explain to me what are scripts in /etc/ppp/ used for? Are scripts in /etc/sysconfig/network-scripts/ the same in desktop Fedora and in cloud Fedora?
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by janeznemanic):
Here is the kickstart file. I added few lines at the end of file. Have a look at it and let me know if something is wrong.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by red):
Can't we just -initscripts (maybe you also need to list -dhclient, not sure which one is pulling in which) in %packages? Also ^services --enabled should not list network anymore, but systemd-networkd instead (I don't see where you active it and it seems to be disabled by default). Furthermore, there's a few things in %post that can now be removed completely ("Network fixes", writing if ifcfg-eth0).
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by janeznemanic):
Please be patient guys, I'm doing this for the first time. I published kickstart file here to get feedback, to discover what I'm doing wrong and how to improve that. I'll fix what's wrong.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by janeznemanic):
Replying to [comment:21 red]:
Can't we just -initscripts (maybe you also need to list -dhclient, not
sure which one is pulling in which) in %packages? Also ^services --enabled should not list network anymore, but systemd-networkd instead (I don't see where you active it and it seems to be disabled by default). Furthermore, there's a few things in %post that can now be removed completely ("Network fixes", writing if ifcfg-eth0).
I'm not sure if we can simply remove initscripts. There are files in initscripts used by udev, systemd and some scripts other than network- scripts. I left lines that deal with writing of ifcfg-* in the %post section, because as I understand we are planning to keep those files. Since we are planning to keep /etc/sysconfig/network-scripts and current network scripts depend on ifcfg-* files. But if not needed those lines can be easily removed.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by mattdm):
Replying to [comment:23 janeznemanic]:
I'm not sure if we can simply remove initscripts. There are files in
initscripts used by udev, systemd and some scripts other than network- scripts.
Let's try it and see what breaks. On my test cloud instance, dhclient is the only thing that currently requires initscripts. If anything else requires it but doesn't say so, those packages should be fixed to either not have the requirement or have an explicit dependency.
I haven't looked into it myself, but it's my understanding that systemd has or will have its _own_ dhcp client, which raises some concerns about compatibility testing but is also interesting on its own because on our current cloud image, dhclient is by far the largest single memory consumer. If systemd has a more lightweight dhcp client, that's a valuable plus in itself.
I left lines that deal with writing of ifcfg-* in the %post section,
because as I understand we are planning to keep those files. Since we are planning to keep /etc/sysconfig/network-scripts and current network scripts depend on ifcfg-* files. But if not needed those lines can be easily removed.
Yeah, I think we do want that. These files are used by a lot of software, including, for example, Amazon's ec2-net-utils. And they're the common language between initscripts and NetworkManager (and hopefully systemd)
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by red):
Replying to [comment:24 mattdm]:
I haven't looked into it myself, but it's my understanding that systemd
has or will have its _own_ dhcp client, which raises some concerns about compatibility testing but is also interesting on its own because on our current cloud image, dhclient is by far the largest single memory consumer. If systemd has a more lightweight dhcp client, that's a valuable plus in itself.
Before I've written comment:22, I removed both initscripts and dhclient from a F20 image that I upgraded to Rawhide, and it worked just fine with systemd-networkd (in a non-cloud VM, i.e. there also was no cloud-init).
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by janeznemanic):
Replying to [comment:25 red]:
Replying to [comment:24 mattdm]:
I haven't looked into it myself, but it's my understanding that
systemd has or will have its _own_ dhcp client, which raises some concerns about compatibility testing but is also interesting on its own because on our current cloud image, dhclient is by far the largest single memory consumer. If systemd has a more lightweight dhcp client, that's a valuable plus in itself.
Before I've written comment:22, I removed both initscripts and dhclient
from a F20 image that I upgraded to Rawhide, and it worked just fine with systemd-networkd (in a non-cloud VM, i.e. there also was no cloud-init).
Let us try to remove initscripts and dhclient. Then we'll see what happens.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by janeznemanic):
Systemd-networkd enabled, initscripts and dhclient removed. Why do we use rsyslog and not systemd's journald? Let me try and build image with koji.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by mattdm):
Replying to [comment:27 janeznemanic]:
Systemd-networkd enabled, initscripts and dhclient removed. Why do we
use rsyslog and not systemd's journald?
Because cloud-init pulls it in. See https://fedorahosted.org/cloud/ticket/48 (I think this is another "help wanted" situation.)
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by mattdm):
RFE for the generator: https://bugzilla.redhat.com/show_bug.cgi?id=1090090
I didn't yet file one for ifup/ifdown scripts. That's _probably_ best done as a new "ifupdown-scripts" package with an explicit conflicts with initscripts, and I'm really not _quite_ clear on how it would work exactly (since networkd isn't really designed around fine control over individual interfaces).
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by gospo):
Replying to [comment:8 mattdm]:
If there were a generator for systemd-networkd which could read the
basic values from the legacy files and create network units on the fly (akin to systemd-fstab-generator), that would level that advantage. (No need to implement the full syntax; just basic static and dhcp, plus log when something isn't understood and perhaps recommend NetworkManager for this cases.) And ifup/ifdown compatibility scripts would be nice (although I don't think we generally care for the cloud case)
Sorry, but I miss why we would want to use the legacy files used by initscipts for networkd? If we are concerned about the ability to migrate from networkd to NM we should probably consider a plug-in for NM to read networkd files.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by mattdm):
Replying to [comment:30 gospo]:
Sorry, but I miss why we would want to use the legacy files used by
initscipts for networkd? If we are concerned about the ability to migrate from networkd to NM we should probably consider a plug-in for NM to read networkd files.
Many people are used to these files being *the* Fedora (and Red Hat distribution family) way to configure network interfaces. Their Puppet modules and Chef recipies work this way. In fact, the upstream code includes modules that understand this format natively. Tools like ec2-net- utils do too. All of our existing documentation talks about these files. And, it's already a common language between initscripts and Network Manager.
I don't think we would be doing anyone any favors by making Fedora a special case. In the future, if systemd-networkd becomes common and native tool support is widespread, it'll be easy to transition people.
In the future
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by gospo):
Replying to [comment:31 mattdm]:
Replying to [comment:30 gospo]:
Sorry, but I miss why we would want to use the legacy files used by
initscipts for networkd? If we are concerned about the ability to migrate from networkd to NM we should probably consider a plug-in for NM to read networkd files.
Many people are used to these files being *the* Fedora (and Red Hat
distribution family) way to configure network interfaces. Their Puppet modules and Chef recipies work this way. In fact, the upstream code includes modules that understand this format natively. Tools like ec2-net- utils do too. All of our existing documentation talks about these files. And, it's already a common language between initscripts and Network Manager.
I don't think we would be doing anyone any favors by making Fedora a
special case. In the future, if systemd-networkd becomes common and native tool support is widespread, it'll be easy to transition people.
Ah yes, I agree, then. Thanks for the explanation.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by janeznemanic):
Help needed with koji. I'd like to build qcow2 cloud image. I think I found the right command, but that's where I get really stuck. So the command is:
koji image-build [options] <name> <version> <target> <install-tree-url> <arch> [<arch>...]
1. options
The way I see it I need to use these options: - format=qcow2 - kickstart=path to ks file - no-wait
2. What are the values of these arguments:
<name> = ??? (perhaps Fedora-cloud) <version> = ??? (probably F21) <target> = ??? (no idea here) <install-tree-url> = ??? (no idea at all) <arch> = ??? (probably i386 or x86_64)
Could someone give an example how to use koji image-build and a short explanation? So that I can build the above image.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by mattdm):
janeznemanic - you need special permissions to build images in koji. We can check with release engineering about getting you permission to do at least scratch builds in this way.
Also, FESCo wants me to let you know that there is some skepticism with doing this change for f21, and that ifcfg compatibility is probably a must-have.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by janeznemanic):
@mattdm - So I guess there is nothing else to do at the moment then to wait for generator.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by janeznemanic):
This has been on my mind for some time now. I have been wondering if all the scripts currently in /etc/sysconfig/network-scripts are needed in cloud environment? For example do we need if{up,down}-{isdn,bnep} scripts in cloud image?
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+-------------------------------- Changes (by immanetize):
* cc: immanetize@… (added)
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: janeznemanic Type: task | Status: new Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+-------------------------------- Changes (by mattdm):
* cc: kushaldas@… (added)
Comment:
So, the version of NetworkManager in Fedora 22 has some exciting improvements -- it can do a "configure and exit" mode, is much more modular (no mandatory bluetooth or model support), and uses the same lightweight dhcp client from systemd. Since that currently _already_ understands ifcfg files, that's kind of my first inclination for F22.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: kushal Type: task | Status: accepted Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: meeting | ------------------------------+-------------------------------- Changes (by kushal):
* keywords: => meeting * status: new => accepted * owner: janeznemanic => kushal
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: kushal Type: task | Status: accepted Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: meeting | ------------------------------+--------------------------------
Comment (by kushal):
I have tested this on a running instance, will submit the F23 change page tomorrow. Just wondering if this should be a contained change or the system wide change?
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: kushal Type: task | Status: accepted Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: meeting | ------------------------------+--------------------------------
Comment (by dustymabe):
Replying to [comment:40 kushal]:
I have tested this on a running instance, will submit the F23 change
page tomorrow. Just wondering if this should be a contained change or the system wide change?
If we are talking about moving exclusively to systemd-networkd I don't think it makes sense as a systemwide change. There are a lot of benefits that NetworkManager gives for other use cases that make sense. systemd- networkd makes sense for cloud I think. The one use case that we had for systemd-networkd not behaving was cases where people plop down their own network files in the image (in /etc/sysconfig/network-scripts). Ideally we'd like to have systemd-networkd handle this and there was discussion of doing this (see https://bugzilla.redhat.com/show_bug.cgi?id=1194623).
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: kushal Type: task | Status: accepted Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: meeting | ------------------------------+--------------------------------
Comment (by gholms):
While networkd is certainly suited to simple network setups like those of most clouds, we need to keep the consequences of using it within the broader context of Fedora in mind. Every other Fedora build, including Atomic, uses !NetworkManager to address the issues this feature aims to address, so choosing to use something else incurs costs that we should acknowledge:
1. Network configuration will not be compatible with Fedora Workstation. As that is Fedora's development environment, this conflicts with the [https://fedoraproject.org/wiki/Cloud/Cloud_PRD Cloud PRD]'s goal of avoiding compatibility issues or surprises from differences in the environment. Perhaps we could make this change in concert with Workstation. 1. Network configuration will not be compatible with Fedora Server. The cloudtoserver script could handle migration from networkd to NM, or the Server SIG could simply acknowledge that some fraction of Fedora Server users will wind up using networkd. 1. Network configuration will not be compatible with Fedora Atomic. This may not actually be a big deal, but shipping two cloud products that use incompatible means of configuring networking is somewhat surprising. 1. This adds another tree of potential errors and breakage. Using the same technology across multiple products benefits all of Fedora because they can share the testing burden. This is unfamiliar code; we will need additional test cases to cover it and it will take some time and effort for people to spin up on it so they can debug it and provide feedback more useful than, "It doesn't work." 1. This adds another tree of documentation since networkd can't use NM configuration files.
We can address nearly all of these issues with a systemd [https://bugzilla.redhat.com/show_bug.cgi?id=1090090 generator] that reads the ifcfg-* files that NM uses. People won't be able to use nmcli, but ifcfg files are still well-understood and what we configure NM to emit anyway.
As for the additional testing burden, we probably just need to get QA buy- in. Has anyone from QA chimed in on this feature yet?
To be clear, I'm not against using networkd here, but the feature page is quite light on details like these that ought to be acknowledged.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: kushal Type: task | Status: accepted Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: meeting | ------------------------------+--------------------------------
Comment (by mattdm):
NetworkManager has come a long way since I filed this, with a lot of work on paring down for minimal use cases.
Maybe a crazy idea, but perhaps we could do a systemd-networkd vs NetworkManager showdown for the F23 Alpha, and then decide on one for F23 Beta based on results.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: kushal Type: task | Status: accepted Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: meeting | ------------------------------+--------------------------------
Comment (by dustymabe):
In discussions about this we often bring up the need to have a set of requirements on network that the cloud images (and cloud environments) have. I have sent an email to the list [1] to solicit input and roshi has created a wiki page [2] for us to add information to.
[1] - https://lists.fedoraproject.org/pipermail/cloud/2015-August/005670.html [2] - https://fedoraproject.org/wiki/Cloud/Network-Requirements
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: kushal Type: task | Status: accepted Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: meeting | ------------------------------+--------------------------------
Comment (by mhayden):
Kushal and I talked about this proposal in detail at Flock for the cloud images. While it's great for server devices that aren't going to be moving around often, it's not ideal for laptops or desktops that change their physical location frequently. NetworkManager makes a lot more sense there.
I spoke with zbyszek about improving the documentation around systemd- networkd as well. Much of the documentation is very focused and doesn't speak to full use cases where you're creating multiple .netdev and .network (and perhaps .link) files.
With that said, I've been using systemd-networkd instead of NetworkManager on 10-20 production cloud instances, over 250 hypervisors, and several other physical systems since Fedora 21. Other than the bonding shenanigans (fixed in systemd 216), I haven't seen any issues.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: kushal Type: task | Status: accepted Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+-------------------------------- Changes (by roshi):
* keywords: meeting =>
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: kushal Type: task | Status: accepted Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by maxamillion):
As a side note, just to add the mhayden's update. He's written a lot of great blog posts about using systemd-networkd in a number of use cases.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: kushal Type: task | Status: accepted Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by maxamillion):
Also, action item from 2015-09-23 Cloud SIG meeting is to request mhayden to do a formal write up of the comparison between NetworkManager and systemd-nspawn with different use cases and/or performance metrics (where applicable) since he appears to be the most experienced in the topic space.
https://twitter.com/TheMaxamillion/status/646737193839423488
https://meetbot.fedoraproject.org/fedora- meeting-1/2015-09-23/cloud_wg.2015-09-23-17.00.html http://meetbot.fedoraproject.org/fedora- meeting-1/2015-09-23/cloud_wg.2015-09-23-17.00.txt
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: kushal Type: task | Status: accepted Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by mhayden):
I'm happy to do this, but I'd like to understand exactly what I need to convey.
So we're talking about systemd-networkd vs. NetworkManager for containers and virtual machine guests and how they compare in:
* Ease of use * Resource usage * Managing network interfaces once they're up/configured
(Feel free to add more bullets)
Should this be a wiki post? Blog post? Fedora docs?
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: kushal Type: task | Status: accepted Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by mhayden):
Tried to add a bunch of notes:
* https://fedoraproject.org/wiki/Cloud/Network-Requirements
Feedback?
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: kushal Type: task | Status: accepted Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by mhayden):
FWIW, I'm putting together a talk proposal for RH Summit 2016 to teach/demo systemd-networkd. If anyone has any feedback on that, I'm all ears. ;)
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: kushal Type: task | Status: accepted Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords: | ------------------------------+--------------------------------
Comment (by kushal):
During the Cloud WG meeting on 2015-01-06 it was decided that we come back to this ticket during Fedora 25 time.
#14: Investigate systemd-networkd ------------------------------+-------------------------------- Reporter: mattdm | Owner: kushal Type: task | Status: closed Priority: normal | Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: fixed Keywords: | ------------------------------+-------------------------------- Changes (by kushal):
* status: accepted => closed * resolution: => fixed
Comment:
Closing this ticket as part of trac clean up process. If you want to reopen, please reopen it after we move to pagure.io as atomic-wg.
cloud@lists.stg.fedoraproject.org
-
Fedora Cloud Trac Tickets -
Nobody