Reading https://lwn.net/Articles/616241/, getting myself in happily over my level of actual knowledge about the state of things.
From that article, it seems like it might be worth trying
net.core.default_qdisc = fq_codel by default. Just tossing this out there for the consideration of someone who really knows what they're talking about in this area.
On Wed, Oct 15, 2014 at 21:02:07 -0400, Matthew Miller mattdm@fedoraproject.org wrote:
Reading https://lwn.net/Articles/616241/, getting myself in happily over my level of actual knowledge about the state of things.
From that article, it seems like it might be worth trying net.core.default_qdisc = fq_codel by default. Just tossing this out there for the consideration of someone who really knows what they're talking about in this area.
I don't think that will help much for typical Fedora use. I think you need to be controlling the bottleneck for that to help much. And I doubt Fedora is being used as a gateway router very often. I think using byte queue limits is supposed to be more helpful for end nodes. (Though you could do both.)
I bought a couple of refurbished wndr3800s a month ago for my birthday to replace my home router (and to have a backup) using cerowrt. The default config doesn't fit my service (it assumes normal residential service) so I need to do some significant changes to how they assign IP addresses to interfaces and haven't gotten around to it yet, but am planning to work on it this weekend. I'm hoping ssh will work better while I am updating my local Fedora mirror or when my wife is using netflix.
On Wed, Oct 15, 2014 at 9:02 PM, Matthew Miller mattdm@fedoraproject.org wrote:
Reading https://lwn.net/Articles/616241/, getting myself in happily over my level of actual knowledge about the state of things.
From that article, it seems like it might be worth trying net.core.default_qdisc = fq_codel by default. Just tossing this out there for the consideration of someone who really knows what they're talking about in this area.
I think the answer, as usual, is: "It depends." It's not a silver bullet for all problems, but it would probably help in some cases. I would actually think Server could use this more than Workstation, as the article points out that wireless networking has a number of other issues to consider. I've added Neil on CC, who's probably much more equipped to weigh in on this than either of us.
In any case, if this change is done it should be done in the initscripts package. It's a runtime tunable and we don't need to be patching the kernel to change the default.
josh
Dne 16.10.2014 v 16:19 Josh Boyer napsal(a):
On Wed, Oct 15, 2014 at 9:02 PM, Matthew Miller mattdm@fedoraproject.org wrote:
Reading https://lwn.net/Articles/616241/, getting myself in happily over my level of actual knowledge about the state of things.
From that article, it seems like it might be worth trying net.core.default_qdisc = fq_codel by default. Just tossing this out there for the consideration of someone who really knows what they're talking about in this area.
I think the answer, as usual, is: "It depends." It's not a silver bullet for all problems, but it would probably help in some cases. I would actually think Server could use this more than Workstation, as the article points out that wireless networking has a number of other issues to consider. I've added Neil on CC, who's probably much more equipped to weigh in on this than either of us.
In any case, if this change is done it should be done in the initscripts package. It's a runtime tunable and we don't need to be patching the kernel to change the default.
Josh, we're considering adding "net.core.default_qdisc = fq_codel" to the default sysctl config shipped with systemd.
Assuming we do that, would you change CONFIG_NET_SCH_FQ_CODEL in the Fedora config to make it built-in rather than a module?
As pointed out by TomH in https://lwn.net/Articles/616505/, having it as a module may require loosening the SELinux policy. Also it seems silly to have to load a module for a distro's default.
Thanks, Michal
On Fri, Oct 17, 2014 at 05:01:34PM +0200, Michal Schmidt wrote:
we're considering adding "net.core.default_qdisc = fq_codel" to the default sysctl config shipped with systemd.
That's kind of surprising to me. What's the rationale behind setting these kind of defaults at the systemd level?
On Fri, Oct 17, 2014 at 11:43 AM, Matthew Miller mattdm@fedoraproject.org wrote:
On Fri, Oct 17, 2014 at 05:01:34PM +0200, Michal Schmidt wrote:
we're considering adding "net.core.default_qdisc = fq_codel" to the default sysctl config shipped with systemd.
That's kind of surprising to me. What's the rationale behind setting these kind of defaults at the systemd level?
I'd be curious to hear that as well, but for a location it's kind of irrelevant to Fedora. Either the default sysctl config in systemd or the one provided in initscripts results in the same net effect.
josh
On Fri, Oct 17, 2014 at 11:45:32AM -0400, Josh Boyer wrote:
On Fri, Oct 17, 2014 at 11:43 AM, Matthew Miller mattdm@fedoraproject.org wrote:
On Fri, Oct 17, 2014 at 05:01:34PM +0200, Michal Schmidt wrote:
we're considering adding "net.core.default_qdisc = fq_codel" to the default sysctl config shipped with systemd.
That's kind of surprising to me. What's the rationale behind setting these kind of defaults at the systemd level?
We provide the following defaults in systemd.rpm:
# System Request functionality of the kernel (SYNC) kernel.sysrq = 16 ( a safe subset of sysrq )
# Source route verification net.ipv4.conf.default.rp_filter = 1 (various safety features) net.ipv4.conf.all.rp_filter = 1
# Do not accept source routing net.ipv4.conf.default.accept_source_route = 0 net.ipv4.conf.all.accept_source_route = 0
# Enable hard and soft link protection fs.protected_hardlinks = 1 fs.protected_symlinks = 1
# Promote secondary addresses when the primary address is removed net.ipv4.conf.default.promote_secondaries = 1 ( avoid nasty surprises when adding net.ipv4.conf.all.promote_secondaries = 1 dropping addresses dynamicall )
# Disable netfilter on bridges. net.bridge.bridge-nf-call-ip6tables = 0 ( this allows normal libvirt setups net.bridge.bridge-nf-call-iptables = 0 to work out of the box) net.bridge.bridge-nf-call-arptables = 0
(+ some coredump related stuff which is actually overriden by abrt.)
In a way, adding those settings through sysctl files is nicer for the end user because it is very transparent. It is also trivial to override by copying the file to /etc and changing some settings.
It seems that net.core.default_qdisc also fits here, although changing the default in the kernel might be better (slightly more efficient, etc).
I'd be curious to hear that as well, but for a location it's kind of irrelevant to Fedora. Either the default sysctl config in systemd or the one provided in initscripts results in the same net effect.
The location is not terribly important, but we've migrated sysctl overrides from initscripts to systemd, because initscripts is nowadays mostly about scripts to bring up the network and optional, and since systemd actually writes the sysctl settings, it seems appropriate to keep those settings there instead of spread out over two packages.
Zbyszek
Hi folks:
I would have thought that it should be the system program which brings up the interface which should set the queuing behaviour. Neither the kernel nor systems know the intent of the use of the interface, and thus if CoDel, FIFO, or other AQM is appropriate.
Consider an ethernet interface, all using the same physical hardware:
- enterprise LAN interface -- CoDel
- ADSL link with PPP over ethernet -- ppp0 should run Codel, eth0 should run a short prioritised FIFO.
- data centre LAN interface -- hierarchical queuing with a high priority, random drop queue for control plane traffic (LACP, spanning tree, etc) and CoDel for other traffic.
Intent is vital in setting the correct AQM for each interface. The program which configures interfaces (NetworkManager, systemd-networkd, ...) is where intent is best known.
-glen
On 10/28/2014 02:38 AM, Glen Turner wrote:
I would have thought that it should be the system program which brings up the interface which should set the queuing behaviour. Neither the kernel nor systems know the intent of the use of the interface, and thus if CoDel, FIFO, or other AQM is appropriate.
Consider an ethernet interface, all using the same physical hardware:
enterprise LAN interface -- CoDel
ADSL link with PPP over ethernet -- ppp0 should run Codel, eth0
should run a short prioritised FIFO.
- data centre LAN interface -- hierarchical queuing with a high
priority, random drop queue for control plane traffic (LACP, spanning tree, etc) and CoDel for other traffic.
Intent is vital in setting the correct AQM for each interface. The program which configures interfaces (NetworkManager, systemd-networkd, ...) is where intent is best known.
Sure. NM, networkd, etc. may want to configure qdiscs per-interface. Setting default_qdisc does not conflict with that.
Michal
On Wed, Oct 29, 2014 at 12:15:32PM +0100, Michal Schmidt wrote:
- data centre LAN interface -- hierarchical queuing with a high
priority, random drop queue for control plane traffic (LACP, spanning tree, etc) and CoDel for other traffic.
Random drop for Spanning Tree and LACP? Really? That is a disaster waiting to happen. Critical control plane traffic should be highly prioritized. Most enterprise/datacenter network gear puts control plane traffic in a NC queue with a small amount of bandwidth reservation, typically 5%. Dropping STP creates network loops. Dropping LACP breaks bonded links. These are things you don't want to happen in your datacenter.
Matthew Miller (mattdm@fedoraproject.org) said:
On Fri, Oct 17, 2014 at 05:01:34PM +0200, Michal Schmidt wrote:
we're considering adding "net.core.default_qdisc = fq_codel" to the default sysctl config shipped with systemd.
That's kind of surprising to me. What's the rationale behind setting these kind of defaults at the systemd level?
AIUI, it's the same rationale as to why Fedora would set it in systemd or initscripts - if systemd thinks the default should be X, but the upstream kernel won't change it from Y, then systemd sets it in the defaults there. It's just a little more unusual to do so in the upstream project, but it's already done for things like source routing, a default sysrq value, etc.
Bill
On Fri, Oct 17, 2014 at 11:01 AM, Michal Schmidt mschmidt@redhat.com wrote:
Dne 16.10.2014 v 16:19 Josh Boyer napsal(a):
On Wed, Oct 15, 2014 at 9:02 PM, Matthew Miller mattdm@fedoraproject.org wrote:
Reading https://lwn.net/Articles/616241/, getting myself in happily over my level of actual knowledge about the state of things.
From that article, it seems like it might be worth trying net.core.default_qdisc = fq_codel by default. Just tossing this out there for the consideration of someone who really knows what they're talking about in this area.
I think the answer, as usual, is: "It depends." It's not a silver bullet for all problems, but it would probably help in some cases. I would actually think Server could use this more than Workstation, as the article points out that wireless networking has a number of other issues to consider. I've added Neil on CC, who's probably much more equipped to weigh in on this than either of us.
In any case, if this change is done it should be done in the initscripts package. It's a runtime tunable and we don't need to be patching the kernel to change the default.
Josh, we're considering adding "net.core.default_qdisc = fq_codel" to the default sysctl config shipped with systemd.
Assuming we do that, would you change CONFIG_NET_SCH_FQ_CODEL in the Fedora config to make it built-in rather than a module?
I've made this change in 3.17.1-301.fc21. It doesn't change the default by itself so it seemed safe to do either way and avoids the SELinux issue.
josh
On Fri, Oct 17, 2014 at 01:33:12PM -0400, Josh Boyer wrote:
On Fri, Oct 17, 2014 at 11:01 AM, Michal Schmidt mschmidt@redhat.com wrote:
Dne 16.10.2014 v 16:19 Josh Boyer napsal(a):
Josh, we're considering adding "net.core.default_qdisc = fq_codel" to the default sysctl config shipped with systemd.
Assuming we do that, would you change CONFIG_NET_SCH_FQ_CODEL in the Fedora config to make it built-in rather than a module?
I've made this change in 3.17.1-301.fc21. It doesn't change the default by itself so it seemed safe to do either way and avoids the SELinux issue.
Great.
BTW, I filed https://bugzilla.redhat.com/show_bug.cgi?id=1154148 for the selinux issue. I think we should allow overriding sysctl settings independently of this issue.
Zbyszek
On Thu, 2014-10-16 at 10:19 -0400, Josh Boyer wrote:
On Wed, Oct 15, 2014 at 9:02 PM, Matthew Miller mattdm@fedoraproject.org wrote:
Reading https://lwn.net/Articles/616241/, getting myself in happily over my level of actual knowledge about the state of things.
From that article, it seems like it might be worth trying net.core.default_qdisc = fq_codel by default. Just tossing this out there for the consideration of someone who really knows what they're talking about in this area.
I think the answer, as usual, is: "It depends." It's not a silver bullet for all problems, but it would probably help in some cases. I would actually think Server could use this more than Workstation, as the article points out that wireless networking has a number of other issues to consider. I've added Neil on CC, who's probably much more equipped to weigh in on this than either of us.
I'll weigh-in here -- while fq_codel is not an ideal solution for wireless (particularly .11n and .11ac) networks, IMHO it will be better than the existing defaults for most users. While there might be some limited effects on throughput, the improvements in latency will be generally worthwhile. YMMV, of course.
In any case, if this change is done it should be done in the initscripts package. It's a runtime tunable and we don't need to be patching the kernel to change the default.
Agreed, of course. The runtime tunability also provides an escape hatch for anyone that is negatively impacted by this change of the default qdisc.
I support the propose change to use the fq_codel default.
John
kernel@lists.fedoraproject.org
-
Bill Nottingham -
Bruno Wolff III -
Chuck Anderson -
Glen Turner -
John W. Linville -
Josh Boyer -
Matthew Miller -
Michal Schmidt -
Zbigniew Jędrzejewski-Szmek