Hi,
Let's decide what to do about https://bugzilla.redhat.com/show_bug.cgi?id=1136905
Long story short: timedated no longer supports NTP clients other than timesyncd (the new systemd NTP client). Oversimplifying, the chrony developers say that chrony is better than timesyncd, and the systemd developers say that timesyncd is better than chrony.
We need to remove chrony from the default install and figure out what to do for users upgrading to F21 (if we do not uninstall chrony, the switch will be broken), or patch out the change to timedated, or build systemd without timedated and provide our own implementation of it, or patch gnome-control-center.
None of these seem like particularly good options. Removing chrony seems like an easy solution, but I'm not sure how we could handle upgrades without hurting users who don't want chrony to be removed.
Michael
On 09/29/2014 03:32 PM, Michael Catanzaro wrote:
Hi,
Let's decide what to do about https://bugzilla.redhat.com/show_bug.cgi?id=1136905
Long story short: timedated no longer supports NTP clients other than timesyncd (the new systemd NTP client). Oversimplifying, the chrony developers say that chrony is better than timesyncd, and the systemd developers say that timesyncd is better than chrony.
We need to remove chrony from the default install and figure out what to do for users upgrading to F21 (if we do not uninstall chrony, the switch will be broken), or patch out the change to timedated, or build systemd without timedated and provide our own implementation of it, or patch gnome-control-center.
As I understand it, it's a systemd 216 change and as such only affects rawhide. F21 has systemd 215. No need to panic, not just yet :)
Also, rtcm and I both tried systemd timesyncd NTP support a month ago on F21 and couldn't get it to actually sync or do anything useful while it worked fine with chrony. It looked like the newer rawhide systemd 216 had a bunch of fixes in that area though. Some of those might have been backported to F21 in the mean time, but I'd still prefer playing the safe card here and sticking with chrony for F21 -- unless we get a systemd rebase to 216 and are forced to change this, of course.
On Mon, Sep 29, 2014 at 4:00 PM, Kalev Lember kalevlember@gmail.com wrote:
Also, rtcm and I both tried systemd timesyncd NTP support a month ago on F21 and couldn't get it to actually sync or do anything useful while it worked fine with chrony. It looked like the newer rawhide systemd 216 had a bunch of fixes in that area though. Some of those might have been backported to F21 in the mean time, but I'd still prefer playing the safe card here and sticking with chrony for F21 -- unless we get a systemd rebase to 216 and are forced to change this, of course.
timesyncd didn't do anything if networkd wasn't running until
http://cgit.freedesktop.org/systemd/systemd/commit/?id=cedc8c44d43c8b6689ae5...
which is only included in systemd 216.
Rui
On Mon, 2014-09-29 at 16:00 +0200, Kalev Lember wrote:
As I understand it, it's a systemd 216 change and as such only affects rawhide. F21 has systemd 215. No need to panic, not just yet :)
Looks like F21 is about to get a git snapshot of systemd, but with the NTP change patched out. While there's no urgency for F21, there's also no particular reason to leave this broken in rawhide for ages, so I propose we drop chronyd beginning with F22. I'm not opposed to keeping chrony around if we can integrate it with timedated, but the path of least resistance is certainly to drop it. Any objections?
On Wed, Oct 08, 2014 at 11:04:20AM -0500, Michael Catanzaro wrote:
Looks like F21 is about to get a git snapshot of systemd, but with the NTP change patched out. While there's no urgency for F21, there's also no particular reason to leave this broken in rawhide for ages, so I propose we drop chronyd beginning with F22. I'm not opposed to keeping chrony around if we can integrate it with timedated, but the path of least resistance is certainly to drop it. Any objections?
I understand that maintainers don't like keeping Fedora specific patches in their packages, but fixing timedated to control chronyd instead of timesyncd in the current code is trivial. It doesn't even require a patch, it can be done with a sed command.
sed -i 's|systemd-timesyncd.service|chronyd.service|' src/timedate/timedated.c
As for switching to timesyncd as default NTP client, I'm familiar with the code. It's a minimal SNTP client based on the ConnMan SNTP implementation. I've tested it, found and fixed some bugs, and also made some small improvements. In its current state, however, I'm not sure it would be a good default client in a distribution like Fedora.
On a LAN with trusted NTP servers it should be fine, but when using public NTP servers on internet, which is what we do by default, or not connected permanently I wouldn't recommend it just yet. There were some good reasons when we switched from ntpd to chronyd in Fedora 16 [1] and from what I can tell it has worked very well so far. With timesyncd I think we would be worse off than we started.
For those who didn't see the thread about making timedated configurable for other NTP services on the systemd list [2], some of the issues with timesyncd are:
- reliability, timesyncd is an SNTP client and not a full NTP client, it doesn't compare data from multiple serverss, so it's not able to detect broken servers and will blindly follow almost any reply - no guarantees on monotonicity of the clock, time jumping back and forth with network connections suffering from bufferbloat - no integration with NetworkManager, unaware of NTP servers from DHCP - dependency on systemd-networkd, which is not enabled by default
[1] http://fedoraproject.org/wiki/Features/ChronyDefaultNTP [2] http://lists.freedesktop.org/archives/systemd-devel/2014-August/022367.html
On 10/09/2014 01:45 PM, Miroslav Lichvar wrote:
As for switching to timesyncd as default NTP client, I'm familiar with the code. It's a minimal SNTP client based on the ConnMan SNTP implementation. I've tested it, found and fixed some bugs, and also made some small improvements. In its current state, however, I'm not sure it would be a good default client in a distribution like Fedora.
From gnome-control-center point of view, we'd like to keep on using the org.freedesktop.timedate1 dbus interface that systemd provides. It's a really nice abstraction and lets us configure NTP without having to deal with editing configuration files for different NTP services that a distro might have chosen to use.
Beyond that, whether it's systemd's internal code or chronyd or anything else running underneath, gnome-control-center doesn't really care, As long as it works well and we can use the timedate1 interface to configure it. :)
On Thu, 2014-10-09 at 14:19 +0200, Kalev Lember wrote:
Beyond that, whether it's systemd's internal code or chronyd or anything else running underneath, gnome-control-center doesn't really care, As long as it works well and we can use the timedate1 interface to configure it. :)
Miroslav, I think if the Fedora systemd maintainers are not willing to change this, then I think our options are to (a) escalate to FESCo, or (b) drop chrony. Like Kalev says, we're completely fine with keeping chrony so long as it can be controlled by timedated.
Michael
On Thu, Oct 09, 2014 at 08:07:41AM -0500, Michael Catanzaro wrote:
On Thu, 2014-10-09 at 14:19 +0200, Kalev Lember wrote:
Beyond that, whether it's systemd's internal code or chronyd or anything else running underneath, gnome-control-center doesn't really care, As long as it works well and we can use the timedate1 interface to configure it. :)
Miroslav, I think if the Fedora systemd maintainers are not willing to change this, then I think our options are to (a) escalate to FESCo, or (b) drop chrony. Like Kalev says, we're completely fine with keeping chrony so long as it can be controlled by timedated.
After the systemd rebase in f21, there is a patch reverting the upstream change, but it wasn't included in rawhide yet. I've asked three of the Fedora systemd (co)maintainers if they could look into it. Unsurprisingly, nobody seems to be interested in maintaining the patch in rawhide indefinitely and I suspect trying to force it through FESCo would do more harm than good.
Maybe the best option here really is an alternative timedate implementation. I gave it a shot and here is the result:
https://github.com/mlichvar/timedatex
The controlled NTP unit is selected from the ntp-units.d directory, similarly to what systemd-timedated did before v216. There are also some other minor improvements.
If it passes the initial round of testing and bug fixing, I think it will be a low maintenance package. The timedate API doesn't seem to be changing very frequently (so far only in v198 and v209). Maybe other distributions will find it useful and help with the maintenance.
If anyone would like to help with the testing or review, here is the review request with an srpm package:
https://bugzilla.redhat.com/show_bug.cgi?id=1165679
Thanks,
On Thu, 09.10.14 13:45, Miroslav Lichvar (mlichvar@redhat.com) wrote:
- reliability, timesyncd is an SNTP client and not a full NTP client, it doesn't compare data from multiple serverss, so it's not able to detect broken servers and will blindly follow almost any reply
Well, while it is certainly nice to be resiliant to such things, I am not convinced though that for a normal client this is really a necessity. The commonly used NTP servers are good enough for most cases and are used in SNTP mode by a multitude of devices and operating systems, and if people really care they can install a full NTP implementation easily. But somehow I have the suspicion that people who need this extra resiliance would probably run PTP or something like that anyway.
And if this is about security: neither NTP nor SNTP really can guarantee anything there.
- no guarantees on monotonicity of the clock, time jumping back and forth with network connections suffering from bufferbloat
Well, that's not precisely true. There's a threshold in place, to use clock_settime() instead of the PLL if the time difference is too large. But as long as the network is not completely borked we should easily stay below that threshold, especially since the measurements are actually filtered for spikes before we pass them into the PLL. Only if a continuous stream of bogus measurements is collected the PLL might be trashed, but even that will not cause time to go backwards...
Also note that timesyncd saves and restores the clock on disk, to improve behaviour on RTC-less devices, and keep the clock monotonic even for them. Since it may run during early boot (in contrast to chrony) it will actually apply this during early boot, so that normal daemons will never see an uninitialized clock.
- no integration with NetworkManager, unaware of NTP servers from DHCP
Well it is not hooked up with NM but it *is* hooked up with networkd via its native API, and hence even picks up manually configured per-connection NTP servers.
I wasn't aware though that chrony and NM were integrated like that? What interface is used there?
- dependency on systemd-networkd, which is not enabled by default
There is no hard dependency. If you run both in conjunction, then timesyncd will pick up the DHCP supplied NTP servers, and the ones explicitly configured in the network configuration. If you don't run them in conjunction both will work fine independently.
In general: we try to make timesyncd a good *client* for NTP, and focus on that. Unlike ntpd or chrony it will not accept NTP requests, it will not contain drivers for hardware clocks. timesyncd is supposed to be a generic daemon for everything the 99% of devices that just need correct time, and nothing more.
In this light our plans are actually to add a minimal PTP client as well, that is supposed to just work, if PTP is supported on a LAN. Also, in contrast to ntpd we really want to make sure to optimize timesyncd for power management, and reduce wakeups. In fact, we are looking to syncing about the NTP syncs to wakeups of the network hw, so that we never end up waking up hardware for the clock.
So, anyway, I am pretty sure that timesyncd at least in the middle term is the way to go for all clients.
Lennart
On Thu, Oct 9, 2014 at 6:46 PM, Lennart Poettering mzerqung@0pointer.de wrote:
- no integration with NetworkManager, unaware of NTP servers from DHCP
Well it is not hooked up with NM but it *is* hooked up with networkd via its native API, and hence even picks up manually configured per-connection NTP servers.
I really want to see networkd in Fedora by default. What are the plans/timeframe for that? How does it play together with NM?
However, for Fedora 21 we probably won't see networkd by default, which means we need a solution for the meantime. I understand that getting NTP servers from DHCP is an important feature to many people. Since timesyncd can't do that without networkd, can we make timedatectl (and the relevant interfaces) use chrony for now, and revise that when we have networkd by default? (ie. switch to timesyncd when we switch to networkd).
Overall, all the features you mentioned of timesyncd sound like really good things we really need to have by default :)
On Thu, 09.10.14 18:59, Elad Alfassa (elad@fedoraproject.org) wrote:
On Thu, Oct 9, 2014 at 6:46 PM, Lennart Poettering mzerqung@0pointer.de wrote:
- no integration with NetworkManager, unaware of NTP servers from DHCP
Well it is not hooked up with NM but it *is* hooked up with networkd via its native API, and hence even picks up manually configured per-connection NTP servers.
I really want to see networkd in Fedora by default. What are the plans/timeframe for that? How does it play together with NM?
networkd doesn't do WLAN really (well, you can run it together with wpa_supplicant, and configure both individually, but that's not particularly nice. networkd currently also lacks a bus interface (since it runs in early boot + initrd, where dbus-daemon is not available currently), and that stalls on kdbus.
networkd as it is right now is great for servers, or the initrd, to run inside containers or for a lot of the embedded usecases. It's not suitable for the desktop usecase.
Lennart
On Thu, Oct 9, 2014 at 7:22 PM, Lennart Poettering mzerqung@0pointer.de wrote:
On Thu, 09.10.14 18:59, Elad Alfassa (elad@fedoraproject.org) wrote:
On Thu, Oct 9, 2014 at 6:46 PM, Lennart Poettering <mzerqung@0pointer.de
wrote:
- no integration with NetworkManager, unaware of NTP servers from DHCP
Well it is not hooked up with NM but it *is* hooked up with networkd via its native API, and hence even picks up manually configured per-connection NTP servers.
I really want to see networkd in Fedora by default. What are the plans/timeframe for that? How does it play together with NM?
networkd doesn't do WLAN really (well, you can run it together with wpa_supplicant, and configure both individually, but that's not particularly nice. networkd currently also lacks a bus interface (since it runs in early boot + initrd, where dbus-daemon is not available currently), and that stalls on kdbus.
networkd as it is right now is great for servers, or the initrd, to run inside containers or for a lot of the embedded usecases. It's not suitable for the desktop usecase.
Lennart
-- Lennart Poettering, Red Hat -- desktop mailing list desktop@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/desktop
Okay, got it. Speaking of the far future, will networkd ever be suitable for desktop usecases? Will NM interface with it in some way to provide the missing functionality? If networkd is not good (yet?) for desktop, and timesyncd needs it for a functionality that is important for many people, then I guess timesyncd will need to learn to work with NetworkManager if you want to have it in Workstation (or convince people that DHCP-provided NTP servers are not something they want to support).
For the F21 timeframe I think it would be useful to patch timedatectl to use chrony. Do you agree?
On Thu, Oct 9, 2014 at 6:28 PM, Elad Alfassa elad@fedoraproject.org wrote:
For the F21 timeframe I think it would be useful to patch timedatectl to use chrony. Do you agree?
As mentioned earlier in the thread, the systemd version shipped in F21 does not contain the NTP changes, so everything discussed here is post-F21 timeframe - for F21, everything should just work without any further actions (famous last words)
On Thu, Oct 9, 2014 at 7:50 PM, Florian Müllner fmuellner@gnome.org wrote:
On Thu, Oct 9, 2014 at 6:28 PM, Elad Alfassa elad@fedoraproject.org wrote:
For the F21 timeframe I think it would be useful to patch timedatectl to
use
chrony. Do you agree?
As mentioned earlier in the thread, the systemd version shipped in F21 does not contain the NTP changes, so everything discussed here is post-F21 timeframe - for F21, everything should just work without any further actions (famous last words) -- desktop mailing list desktop@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/desktop
Oh, I misread. In that case, it's not as urgent as I thought.
Regardless, I'd really like to see timesyncd by default in Workstation in the future, so I really hope it will learn to work with NM.
On Thu, 09.10.14 19:28, Elad Alfassa (elad@fedoraproject.org) wrote:
Okay, got it.
Speaking of the far future, will networkd ever be suitable for desktop usecases? Will NM interface with it in some way to provide the missing functionality?
Nobody in the systemd team is working on WLAN support. That doesn't mean though that other people might, and we'd certainly consider merging good patches if we get some. People have certainly asked for it, so I wouldn't be surprised if some of the embedded folks one day will provide us with a patch and we'd merge it.
If networkd is not good (yet?) for desktop, and timesyncd needs it for a functionality that is important for many people, then I guess timesyncd will need to learn to work with NetworkManager if you want to have it in Workstation (or convince people that DHCP-provided NTP servers are not something they want to support).
Well, it is news to me that chrony and NM area integrated regarding the DHCP bits. If they are I'd be curious about the interface they use.
Lennart
On Thu, 2014-10-09 at 19:28 +0200, Lennart Poettering wrote:
On Thu, 09.10.14 19:28, Elad Alfassa (elad@fedoraproject.org) wrote:
Okay, got it.
Speaking of the far future, will networkd ever be suitable for desktop usecases? Will NM interface with it in some way to provide the missing functionality?
Nobody in the systemd team is working on WLAN support. That doesn't mean though that other people might, and we'd certainly consider merging good patches if we get some. People have certainly asked for it, so I wouldn't be surprised if some of the embedded folks one day will provide us with a patch and we'd merge it.
If networkd is not good (yet?) for desktop, and timesyncd needs it for a functionality that is important for many people, then I guess timesyncd will need to learn to work with NetworkManager if you want to have it in Workstation (or convince people that DHCP-provided NTP servers are not something they want to support).
Well, it is news to me that chrony and NM area integrated regarding the DHCP bits. If they are I'd be curious about the interface they use.
NM dispatcher scripts. NM puts network information (including DHCP options) into the environment of the scripts, and the script can do whatever it wants to do with that information.
Dan
On Thu, 2014-10-09 at 19:28 +0300, Elad Alfassa wrote:
On Thu, Oct 9, 2014 at 7:22 PM, Lennart Poettering mzerqung@0pointer.de wrote:
On Thu, 09.10.14 18:59, Elad Alfassa (elad@fedoraproject.org) wrote:
On Thu, Oct 9, 2014 at 6:46 PM, Lennart Poettering <mzerqung@0pointer.de
wrote:
- no integration with NetworkManager, unaware of NTP servers from DHCP
Well it is not hooked up with NM but it *is* hooked up with networkd via its native API, and hence even picks up manually configured per-connection NTP servers.
I really want to see networkd in Fedora by default. What are the plans/timeframe for that? How does it play together with NM?
networkd doesn't do WLAN really (well, you can run it together with wpa_supplicant, and configure both individually, but that's not particularly nice. networkd currently also lacks a bus interface (since it runs in early boot + initrd, where dbus-daemon is not available currently), and that stalls on kdbus.
networkd as it is right now is great for servers, or the initrd, to run inside containers or for a lot of the embedded usecases. It's not suitable for the desktop usecase.
Lennart
-- Lennart Poettering, Red Hat -- desktop mailing list desktop@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/desktop
Okay, got it. Speaking of the far future, will networkd ever be suitable for desktop usecases? Will NM interface with it in some way to provide the missing functionality? If networkd is not good (yet?) for desktop, and timesyncd needs it for a functionality that is important for many people, then I guess timesyncd will need to learn to work with NetworkManager if you want to have it in Workstation (or convince people that DHCP-provided NTP servers are not something they want to support).
I'd love to have timesyncd and resolved from systemd play well with NetworkManager, but unfortunately upstream systemd is not moving very quickly on these services grabbin/accepting information from sources other than systemd-networkd.
Dan
On Fri, 10.10.14 12:20, Dan Williams (dcbw@redhat.com) wrote:
I'd love to have timesyncd and resolved from systemd play well with NetworkManager, but unfortunately upstream systemd is not moving very quickly on these services grabbin/accepting information from sources other than systemd-networkd.
We have been discussing this quite a bit LPC, but we didn't really manage to come up with a nice, low-level, synchronous and simple way how we could attach this information to network interfaces with good lifecycle behaviour, wthout defining a completely new API with multiple implementors and whatnot.
The best idea we had was to use xattrs on /sys/class/net/$NETIF. That way, defining new fields of information to pass around would be trivial to add, easy to explore from userspace. We'd get notifications, and their life-cycle would be bound inherently to the one of the kernel objects. It's really close to what we want. There's just one problem: only xattrs in the "trusted." namespace are supported on /sys (for a good reason, as they are stored in kernel memory and thus unpriviliged clients should not be able to attach arbitrary numbers of these things to kernel objects), and those may not be read by unpriviliged clients. As it turns out though all of systemd-networkd, systemd-resolved, systemd-timesyncd run unpriviliged as non-root users with no or minimal capabilities on top. Having to add CAP_SYS_ADMIN so that they can read the variables added by other stacks sounds much less than ideal.
We could make them work for us by either:
a) introducing a new "semi-trusted." xattr namespace in the kernel that can only be written to by CAP_SYS_ADMIN, but read by unpriviliged clients
or
b) introducing a new CAP_XATTR_TRUSTED capability or so that allows writing/reading the trusted xattrs.
But I am not sure who wants to sit down and actually implement that...
Anyway, other ideas?
I am really not keen though to do this via dbus, nor via some dir in /run (like networkd does) due to the awkward ownership and lifecycle semantics.
Lennart
On Tue, 2014-10-21 at 20:37 +0200, Lennart Poettering wrote:
On Fri, 10.10.14 12:20, Dan Williams (dcbw@redhat.com) wrote:
I'd love to have timesyncd and resolved from systemd play well with NetworkManager, but unfortunately upstream systemd is not moving very quickly on these services grabbin/accepting information from sources other than systemd-networkd.
We have been discussing this quite a bit LPC, but we didn't really manage to come up with a nice, low-level, synchronous and simple way how we could attach this information to network interfaces with good lifecycle behaviour, wthout defining a completely new API with multiple implementors and whatnot.
Thomas said he talked to Tom about it at LPC but it was mostly brainstorming. So it's getting discussed more now, which is great!
The best idea we had was to use xattrs on /sys/class/net/$NETIF. That way, defining new fields of information to pass around would be trivial to add, easy to explore from userspace. We'd get notifications, and their life-cycle would be bound inherently to the one of the kernel objects. It's really close to what we want. There's just one problem: only xattrs in the "trusted." namespace are supported on /sys (for a good reason, as they are stored in kernel memory and thus unpriviliged clients should not be able to attach arbitrary numbers of these things to kernel objects), and those may not be read by unpriviliged clients. As it turns out though all of systemd-networkd, systemd-resolved, systemd-timesyncd run unpriviliged as non-root users with no or minimal capabilities on top. Having to add CAP_SYS_ADMIN so that they can read the variables added by other stacks sounds much less than ideal.
We could make them work for us by either:
a) introducing a new "semi-trusted." xattr namespace in the kernel that can only be written to by CAP_SYS_ADMIN, but read by unpriviliged clients
or
b) introducing a new CAP_XATTR_TRUSTED capability or so that allows writing/reading the trusted xattrs.
But I am not sure who wants to sit down and actually implement that...
Anyway, other ideas?
I am really not keen though to do this via dbus, nor via some dir in /run (like networkd does) due to the awkward ownership and lifecycle semantics.
That was my major issue with resolvconf too, the lifecycle issues. But that could be solved via D-Bus connections, where if the client disconnects, resolved removes the information that client pushed down.
Yes, disconnects happen for various reasons, and two in particular are worth considering:
1) the DNS source gets kicked off the bus because it sent a bad/malformed message (eg, non-UTF8-string). This is bad coding on the part of the DNS source and should not happen, so I don't consider it a problem. Perhaps kdbus is more forgiving here anyway?
2) the DNS source restarts (due to being upgraded, or restarted on segfault, etc), which ideally should not disrupt networking at all. This is harder, but since all we're talking about is DNS information and not connectivity itself, this probably isn't a huge deal either.
For ownership, I don't think that's solved by XATTRs. Unless you namespace them or otherwise tie them to the DNS source, all resolved can do is read the "dns1" attribute on eth0. But that attribute could be written to by anything with privilege, be that NetworkManager or networkd or whatever. If this isn't what you mean by ownership, could you elaborate?
Dan
On Tue, 21.10.14 14:43, Dan Williams (dcbw@redhat.com) wrote:
I am really not keen though to do this via dbus, nor via some dir in /run (like networkd does) due to the awkward ownership and lifecycle semantics.
That was my major issue with resolvconf too, the lifecycle issues. But that could be solved via D-Bus connections, where if the client disconnects, resolved removes the information that client pushed down.
Well, we thought about that too. But what we didn't like about this was that it actually binds the life-cycle of things to the wrong object: it should be attached to the interface, not the daemon installing it. I think it would be a really good idea to support a scheme where some program figures out something, attaches it to the iface and dies, but the data isn't lost.
For ownership, I don't think that's solved by XATTRs. Unless you namespace them or otherwise tie them to the DNS source, all resolved can do is read the "dns1" attribute on eth0. But that attribute could be written to by anything with privilege, be that NetworkManager or networkd or whatever. If this isn't what you mean by ownership, could you elaborate?
well, by ownership I just mean that the lifecycle is bound to some other suitable object. xattrs allow us to do this naturally to network interfaces.
Lennart
On Thu, 2014-10-09 at 17:46 +0200, Lennart Poettering wrote:
In general: we try to make timesyncd a good *client* for NTP, and focus on that. Unlike ntpd or chrony it will not accept NTP requests, it will not contain drivers for hardware clocks. timesyncd is supposed to be a generic daemon for everything the 99% of devices that just need correct time, and nothing more.
In this light our plans are actually to add a minimal PTP client as well, that is supposed to just work, if PTP is supported on a LAN. Also, in contrast to ntpd we really want to make sure to optimize timesyncd for power management, and reduce wakeups. In fact, we are looking to syncing about the NTP syncs to wakeups of the network hw, so that we never end up waking up hardware for the clock.
So, anyway, I am pretty sure that timesyncd at least in the middle term is the way to go for all clients.
The functionality you described sounds 'good enough' for the workstation use case to me. The one thing that makes me tend towards 'stick with chrony' for now is the (lack of) dhcp configuration under nm - which is what we'll have to use for networking on the desktop for the forseeable future...
On Thu, 09.10.14 12:44, Matthias Clasen (mclasen@redhat.com) wrote:
On Thu, 2014-10-09 at 17:46 +0200, Lennart Poettering wrote:
In general: we try to make timesyncd a good *client* for NTP, and focus on that. Unlike ntpd or chrony it will not accept NTP requests, it will not contain drivers for hardware clocks. timesyncd is supposed to be a generic daemon for everything the 99% of devices that just need correct time, and nothing more.
In this light our plans are actually to add a minimal PTP client as well, that is supposed to just work, if PTP is supported on a LAN. Also, in contrast to ntpd we really want to make sure to optimize timesyncd for power management, and reduce wakeups. In fact, we are looking to syncing about the NTP syncs to wakeups of the network hw, so that we never end up waking up hardware for the clock.
So, anyway, I am pretty sure that timesyncd at least in the middle term is the way to go for all clients.
The functionality you described sounds 'good enough' for the workstation use case to me. The one thing that makes me tend towards 'stick with chrony' for now is the (lack of) dhcp configuration under nm - which is what we'll have to use for networking on the desktop for the forseeable future...
NTP servers supplied via DHCP is a nice feature (and as mentioned works fine between networkd and timesyncd), but then again, it's actually not very common that DHCP servers supply that information.
Lennart
On Thu, Oct 09, 2014 at 05:46:19PM +0200, Lennart Poettering wrote:
On Thu, 09.10.14 13:45, Miroslav Lichvar (mlichvar@redhat.com) wrote:
- reliability, timesyncd is an SNTP client and not a full NTP client, it doesn't compare data from multiple serverss, so it's not able to detect broken servers and will blindly follow almost any reply
Well, while it is certainly nice to be resiliant to such things, I am not convinced though that for a normal client this is really a necessity. The commonly used NTP servers are good enough for most cases and are used in SNTP mode by a multitude of devices and operating systems
Which major operating systems do use by default a SNTP client with pool.ntp.org? Microsoft and Apple use SNTP, but they have their own trusted NTP servers. In the Linux world, I think the most popular choice is the reference NTP implementation and on smaller devices it's the busybox NTP client.
and if people really care they can install a full NTP implementation easily.
Most people don't understand how NTP works and what it needs to do to work reliably. Most people also don't know much about filesystems, so for default installations we choose one that has journaling, has features we need, is reliable and performs well, not the simplest one, even if it can be good enough in some situations.
But somehow I have the suspicion that people who need this extra resiliance would probably run PTP or something like that anyway.
PTP is actually worse than NTP in this respect as there is only one master, which is selected by a fixed algorithm (BMC) and slaves don't have a choice. The main advantage of PTP over NTP is hardware support in network cards, which eliminates most of the jitter and allows submicrosecond accuracy.
And if this is about security: neither NTP nor SNTP really can guarantee anything there.
NTP has authentication mechanisms for that. With pool.ntp.org that would not be very useful, however, as anyone could join it and serve false timestamps.
- no guarantees on monotonicity of the clock, time jumping back and forth with network connections suffering from bufferbloat
Well, that's not precisely true. There's a threshold in place, to use clock_settime() instead of the PLL if the time difference is too large.
When the network is congested, the offset can easily reach the threshold (0.4 second) and timesyncd will step the clock by the adjtimex(ADJ_SETOFFSET) call. When the network is good again, timesyncd will measure an opposite offset and step the clock back to where it was before.
People with slow ADSL often complain about this with ntpd. There is an option to handle this better (the huff-n'-puff filter), but it's not enabled by default as it makes things worse in normal conditions. chronyd doesn't use the PLL, so it can simply drop the measurements and drift through. Also, it controls the frequency of the clock directly and can correct any offset by slewing.
Also note that timesyncd saves and restores the clock on disk, to improve behaviour on RTC-less devices, and keep the clock monotonic even for them.
chronyd can do that too and even more, it can compensate for the RTC drift.
Since it may run during early boot (in contrast to chrony) it will actually apply this during early boot, so that normal daemons will never see an uninitialized clock.
The clock is initialized from the RTC or the save file, but it's not set from NTP yet, so the daemons started later can still observe a step backwards. With chronyd, you can use the chrony-wait service or block start of the chronyd service by using the initstepslew directive in chrony.conf to guarantee services started later won't see any steps.
I wasn't aware though that chrony and NM were integrated like that? What interface is used there?
There is an NM dispatcher script which calls the chrony dhclient script, which uses chronyc to configure chronyd with the server from DHCP. For ntpd there is a similar script. Not very elegant, but it works.
In this light our plans are actually to add a minimal PTP client as well, that is supposed to just work, if PTP is supported on a LAN.
Ok, I'm interested in how it will compare to ptp4l from linuxptp.
Also, in contrast to ntpd we really want to make sure to optimize timesyncd for power management, and reduce wakeups. In fact, we are looking to syncing about the NTP syncs to wakeups of the network hw, so that we never end up waking up hardware for the clock.
That sounds interesting. I'm not sure if this was already fixed, but the last time I was testing timesyncd I noticed it was woken up due to updates from systemd-networkd every few seconds, even when there were no interesting changes in the network configuration.
So, anyway, I am pretty sure that timesyncd at least in the middle term is the way to go for all clients.
If you can show timesyncd is better than chronyd in the typical use cases, I will be ok with that. Switching to something worse probably wouldn't make much sense to me.
On Thu, Oct 09, 2014 at 07:53:24PM +0200, Miroslav Lichvar wrote:
On Thu, Oct 09, 2014 at 05:46:19PM +0200, Lennart Poettering wrote:
Well, while it is certainly nice to be resiliant to such things, I am not convinced though that for a normal client this is really a necessity. The commonly used NTP servers are good enough for most cases and are used in SNTP mode by a multitude of devices and operating systems
Which major operating systems do use by default a SNTP client with pool.ntp.org? Microsoft and Apple use SNTP, but they have their own trusted NTP servers. In the Linux world, I think the most popular choice is the reference NTP implementation and on smaller devices it's the busybox NTP client.
It seems I was wrong here. Mac OS X apparently uses the reference NTP implementation modified to work in combination with another daemon called pacemaker, which adjusts the clock [1] and the w32time service used in Windows does implement the NTP algorithms [2].
It looks like pretty much everyone has a full NTP client running on their system, I'm sure we don't want Fedora to be the exception.
[1] https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPa... [2] http://technet.microsoft.com/en-us/library/cc773013%28v=ws.10%29.aspx
An active MITM attackr could change the system time by intercepting NTP packets. Such an attack could be used, for example, to bypass HSTS, as described in [1], which specifically criticizes Fedora's default configuration as insecure on page three (though I imagine chrony must have some limits as to how far the system clock can be changed at a time?). It looks like ntpd can be configured to prevent such if the NTP server signs its messages and the client is configured with its public key. It seems safe to assume timesyncd cannot handle this; is that correct? What's the status with chrony?
[1] https://www.blackhat.com/docs/eu-14/materials/eu-14-Selvi-Bypassing-HTTP-Str...
On Sat, Oct 25, 2014 at 11:41:52AM -0500, Michael Catanzaro wrote:
An active MITM attackr could change the system time by intercepting NTP packets. Such an attack could be used, for example, to bypass HSTS, as described in [1], which specifically criticizes Fedora's default configuration as insecure on page three (though I imagine chrony must have some limits as to how far the system clock can be changed at a time?).
It's an interesting paper, but I'm wondering if they have actually tried the attack with a Fedora machine. In our default configuration chronyd is allowed to step the clock only in the first three updates, i.e. in the first ~1 minute after that start. After that, chronyd will not step the clock no matter how large is the measured offset. When the initial updates are missed, the worst thing an MITM attacker can do is change the frequency of the clock by 10%.
Also, the polling interval is normally increased up to the maximum of 1024 seconds (similarly to ntpd), it doesn't stay at 1 minute, and I'm not sure how is the rtcsync option or the update of the RTC relevant to the attack.
To summarize the stepping behavior in the default configurations for chronyd, ntpd and timesyncd:
- chronyd steps the clock to any value, but only on start - ntpd steps to any value on start, after that each individual step has to be smaller than 1000 seconds, otherwise ntpd exits. A step is not done immediately, ntpd waits at least 15 minutes for another measurement (stepout interval). - timesyncd steps to any value later than the compiled-in time of the systemd release and it does that at any time
If chronyd was configured to track the RTC by default, we could theoretically disable even the initial step from NTP. The initial offset would be small unless the system was down/offline for weeks, another application or operating system was messing with the RTC, an attacker is spoofing NTP packets, etc. For larger offsets, a mechanism to inform the desktop user and confirm the correction might be useful here.
It looks like ntpd can be configured to prevent such if the NTP server signs its messages and the client is configured with its public key. It seems safe to assume timesyncd cannot handle this; is that correct? What's the status with chrony?
timesyncd doesn't support any authentication. chronyd implements the authentication using symmetric cryptography, but not the public key cryptography (RFC 5906).
It would be a great feature to have NTP authentication by default, but I'm wondering if it's technically even possible to deploy it on such a large scale as pool.ntp.org. I've not seen any discussion on that so far.
The authentication doesn't prevent all attacks, however. If the attacker just needs to adjust the victim's clock by a small number of seconds, it's possible to trick the NTP client to reduce its polling interval and throw its clock off just by delaying the authenticated requests or replies. The NTP traffic is then cut off and the client will drift away. That's why it's important to monitor the NTP reachability in critical applications.
[1] https://www.blackhat.com/docs/eu-14/materials/eu-14-Selvi-Bypassing-HTTP-Str...
desktop@lists.stg.fedoraproject.org
-
Dan Williams -
Elad Alfassa -
Florian Müllner -
Kalev Lember -
Lennart Poettering -
Matthias Clasen -
Michael Catanzaro -
Miroslav Lichvar -
Rui Tiago Cação Matos