Hello, I'm experimenting with VMs and firewalld while trying to get comfortable with CentOS 8 and am hitting a bit of a wall, hopefully someone here can point me in the right direction.
[What I'd like to do]
I have a LAN (192.168.2.0/24) with my CentOS 8 VM host (192.168.2.3). I also have a file server with a samba share at 192.168.2.2. On my VM host, I'm trying to run a Windows VM so I can use a USB device that has Windows only drivers, and then write some files to my samba share. However, I don't trust Windows, so I want to block all network traffic associated with this VM *except* the samba connection to my file server. I'm using libvirt to run this VM, and thought I'd like to have a dedicated libvirt virt network that's "disconnected" in this way; it has no connection ability apart from that which I whitelist. This network is called "libvirt_discon", and it uses libvirt NAT forwarding and DHCP. Its subnet is 192.168.100.0/24.
[What I've done]
I created a new firewalld zone called "libvirt_discon" and assigned the libvirt network's dev "virbr1" to the zone (this is done via the libvirt network xml). I added dhcp to the list of the zone's services, and can confirm this allows the machine to get an ip address. I set the zone's target to "%%REJECT%%" since I want this to be the default behavior of the zone. Then I tried to whitelist samba with the following rich rule:
firewall-cmd --zone=libvirt_discon --add-rich-rule='rule family=ipv4 destination address=192.168.2.2 service name=samba accept'
This does not work; I'm testing with a fedora machine on the libvirt-discon network and trying to list the samba shares, my connection is immediately rejected:
$ smbclient --user=samba --list=192.168.2.2 do_connect: Connection to 192.168.2.2 failed (Error NT_STATUS_HOST_UNREACHABLE)
[Questions]
I suspect I'm stumbling because I'm using libvirt NAT instead of a bridged device (which admitedly I don't fully understand). Dumping the nft ruleset, it looks like my zone settings strictly affect the zone's input chain. At some point my packets need to traverse the NAT from virbr1 (zone libvirt_discon) to my eth NIC (zone public), and back again. I'm not sure how this works, could someone shed some light on that?
Do I need to enable masquerade on one of the zones here, and if so which one or both? When exactly do I need to enable masquerade?
Are these considered FORWARDed packets, and therefore the INPUT chain rules I've actually written with my rich rule not apply? (They demonstrably are not logging..)
Thank you very much for your time, hopefully someone can point me in the right direction based on what I'm trying to accomplish.
GN
On 2020-07-15 19:55, Gunnar Niels wrote:
Hello, I'm experimenting with VMs and firewalld while trying to get comfortable with CentOS 8 and am hitting a bit of a wall, hopefully someone here can point me in the right direction.
[What I'd like to do]
I have a LAN (192.168.2.0/24) with my CentOS 8 VM host (192.168.2.3). I also have a file server with a samba share at 192.168.2.2. On my VM host, I'm trying to run a Windows VM so I can use a USB device that has Windows only drivers, and then write some files to my samba share. However, I don't trust Windows, so I want to block all network traffic associated with this VM *except* the samba connection to my file server. I'm using libvirt to run this VM, and thought I'd like to have a dedicated libvirt virt network that's "disconnected" in this way; it has no connection ability apart from that which I whitelist. This network is called "libvirt_discon", and it uses libvirt NAT forwarding and DHCP. Its subnet is 192.168.100.0/24.
[What I've done]
I created a new firewalld zone called "libvirt_discon" and assigned the libvirt network's dev "virbr1" to the zone (this is done via the libvirt network xml). I added dhcp to the list of the zone's services, and can confirm this allows the machine to get an ip address. I set the zone's target to "%%REJECT%%" since I want this to be the default behavior of the zone. Then I tried to whitelist samba with the following rich rule:
firewall-cmd --zone=libvirt_discon --add-rich-rule='rule family=ipv4 destination address=192.168.2.2 service name=samba accept'
This does not work; I'm testing with a fedora machine on the libvirt-discon network and trying to list the samba shares, my connection is immediately rejected:
$ smbclient --user=samba --list=192.168.2.2 do_connect: Connection to 192.168.2.2 failed (Error NT_STATUS_HOST_UNREACHABLE)
[Questions]
I suspect I'm stumbling because I'm using libvirt NAT instead of a bridged device (which admitedly I don't fully understand). Dumping the nft ruleset, it looks like my zone settings strictly affect the zone's input chain. At some point my packets need to traverse the NAT from virbr1 (zone libvirt_discon) to my eth NIC (zone public), and back again. I'm not sure how this works, could someone shed some light on that?
Do I need to enable masquerade on one of the zones here, and if so which one or both? When exactly do I need to enable masquerade?
Are these considered FORWARDed packets, and therefore the INPUT chain rules I've actually written with my rich rule not apply? (They demonstrably are not logging..)
Thank you very much for your time, hopefully someone can point me in the right direction based on what I'm trying to accomplish.
I believe you're having the same types of issues I had earlier in June. You may want to read through the archive and the thread "How to troubleshoot".
I was unaware that libvirt inserts its own set of FW rules.
Basically, enlightenment came for me when I was directed to
https://wiki.libvirt.org/page/Networking#Forwarding_Incoming_Connections
The passage which addressed the issue was....
"By default, guests that are connected via a virtual network with <forward mode='nat'/> can make any outgoing network connection they like. Incoming connections are allowed from the host, and from other guests connected to the same libvirt network, but all other incoming connections are blocked by iptables rules."
I changed the mode to "route" and I get the behavior I need for all the VM's and IPv4.
On Wed, Jul 15, 2020 at 07:55:47AM -0400, Gunnar Niels wrote: [..]
[Questions]
I suspect I'm stumbling because I'm using libvirt NAT instead of a bridged device (which admitedly I don't fully understand). Dumping the nft ruleset, it looks like my zone settings strictly affect the zone's input chain.
Right. Firewalld does not yet support forward filtering. It's in the works [1], but not functional yet.
[1]: https://github.com/firewalld/firewalld/pull/639
At some point my packets need to traverse the NAT from virbr1 (zone libvirt_discon) to my eth NIC (zone public), and back again. I'm not sure how this works, could someone shed some light on that?
firewalld allows all output by default. If the ingress zone (i.e. incoming interface is virbr1) uses --set-target=ACCEPT, then firewalld will allow all forwarded packets as well.
The return path (world --> host) will be allowed if the connection originated from the host (host --> world). This is done via a top level, generic accept, "ct state established,untracked accept".
Do I need to enable masquerade on one of the zones here, and if so which one or both? When exactly do I need to enable masquerade?
No. libvirt should be setting up masquerade itself.
Are these considered FORWARDed packets, and therefore the INPUT chain rules I've actually written with my rich rule not apply? (They demonstrably are not logging..)
Correct.
I suspect I'm stumbling because I'm using libvirt NAT instead of a bridged device (which admitedly I don't fully understand). Dumping the nft ruleset, it looks
like my
zone settings strictly affect the zone's input chain.
Right. Firewalld does not yet support forward filtering. It's in the works [1], but not functional yet.
...
Are these considered FORWARDed packets, and therefore the INPUT chain
rules I've actually written with my rich rule not apply? (They demonstrably are not logging..)
Correct.
So what would be the recommended way to block traffic out of the vm but whitelist it's connection with another machine on the LAN? It sounds like I need to be writing rules that belong to the forward chain, but there isn't a way to do that with firewalld yet. Is this when a direct rule would be appropriate? And to which zone should it apply?
On Thu, Jul 16, 2020 at 07:53:19AM -0400, Gunnar Niels wrote:
I suspect I'm stumbling because I'm using libvirt NAT instead of a bridged device (which admitedly I don't fully understand). Dumping the nft ruleset, it looks
like my
zone settings strictly affect the zone's input chain.
Right. Firewalld does not yet support forward filtering. It's in the works [1], but not functional yet.
...
Are these considered FORWARDed packets, and therefore the INPUT chain
rules I've actually written with my rich rule not apply? (They demonstrably are not logging..)
Correct.
So what would be the recommended way to block traffic out of the vm but whitelist it's connection with another machine on the LAN? It sounds like I need to be writing rules that belong to the forward chain, but there isn't a way to do that with firewalld yet. Is this when a direct rule would be appropriate? And to which zone should it apply?
You must use a direct rule. Direct rules are "global" in the sense that they aren't applied to a zone. Often they occur _before_ all zone rules.
So what would be the recommended way to block traffic out of the vm but whitelist it's connection with another machine on the LAN? It sounds like I need to be writing rules that belong to the forward chain, but there isn't a way to do that with firewalld yet. Is this when a direct rule would be appropriate? And to which zone should it apply?
You must use a direct rule. Direct rules are "global" in the sense that they aren't applied to a zone. Often they occur _before_ all zone rules.
Has anything changed about the direct rule syntax now that nftables underlies the system in Centos8 instead of IP tables? What's the recommended reference for direct rule syntax currently?
On Thu, Jul 16, 2020 at 10:48:11AM -0400, Gunnari Niels wrote:
So what would be the recommended way to block traffic out of the vm but whitelist it's connection with another machine on the LAN? It sounds like I need to be writing rules that belong to the forward chain, but there isn't a way to do that with firewalld yet. Is this when a direct rule would be appropriate? And to which zone should it apply?
You must use a direct rule. Direct rules are "global" in the sense that they aren't applied to a zone. Often they occur _before_ all zone rules.
Has anything changed about the direct rule syntax now that nftables underlies the system in Centos8 instead of IP tables?
No. It's still the iptables syntax. The direct rules are still applied to the iptables rule set.
What's the recommended reference for direct rule syntax currently?
The firewalld man page gives this:
[--permanent] --direct --add-rule { ipv4 | ipv6 | eb } table chain priority args Add a rule with the arguments args to chain chain in table table with priority priority.
The priority is used to order rules. Priority 0 means add rule on top of the chain, with a higher priority the rule will be added further down. Rules with the same priority are on the same level and the order of these rules is not fixed and may change. If you want to make sure that a rule will be added after another one, use a low priority for the first and a higher for the following.
where "args" are verbatim iptables arguments. For those see the iptables man pages.
On 7/16/20 12:36 PM, Eric Garver wrote:
On Thu, Jul 16, 2020 at 10:48:11AM -0400, Gunnari Niels wrote:
So what would be the recommended way to block traffic out of the vm but whitelist it's connection with another machine on the LAN? It sounds like I need to be writing rules that belong to the forward chain, but there isn't a way to do that with firewalld yet. Is this when a direct rule would be appropriate? And to which zone should it apply?
You must use a direct rule. Direct rules are "global" in the sense that they aren't applied to a zone. Often they occur _before_ all zone rules.
Thanks to your advice and some fighting with libvirt I was able to achieve the samba whitelist with these direct rules:
# Whitelist samba to file server sudo firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -p udp -s 192.168.100.0/24 -d 192.168.2.2 --dport 137 -j ACCEPT sudo firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -p udp -s 192.168.100.0/24 -d 192.168.2.2 --dport 138 -j ACCEPT sudo firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -p tcp -s 192.168.100.0/24 -d 192.168.2.2 --dport 139 -j ACCEPT sudo firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -p tcp -s 192.168.100.0/24 -d 192.168.2.2 --dport 445 -j ACCEPT
# Reject all other traffic sudo firewall-cmd --direct --add-rule ipv4 filter FORWARD 1 -s 192.168.100.0/24 -j REJECT
These runtime rules will *insert* the rules at the top of the chain, so they short-circuit the LIBVIRT_{FWX,FWI,FWO} chains.
My problem now is that if I add them as permanent to persist them and reload, my direct rules end up *appended* to the bottom of the FORWARD table despite my priority setting. I gather the priority settings is *relative to eachother*, and doesn't necessarily guarantee anything about non-firewalld rules like the LIBVIRT chains. What ends up happening is the LIBVIRT_X chains take precedence above mine. At the bottom of LIBVIRT_FWO, its configured to accept traffic from the net that I reject with my last direct rule...so my rule gets ignored.
What's the right way to ensure that my rules get loaded with precedence over the LIBVIRT ones? Is it expected behavior that a runtime applied direct rule is inserted, but a persistent rule gets appended to the bottom of the chain?
- GN
On Fri, Jul 17, 2020 at 09:45:20PM -0400, Gunnar Niels wrote:
On 7/16/20 12:36 PM, Eric Garver wrote:
On Thu, Jul 16, 2020 at 10:48:11AM -0400, Gunnari Niels wrote:
So what would be the recommended way to block traffic out of the vm but whitelist it's connection with another machine on the LAN? It sounds like I need to be writing rules that belong to the forward chain, but there isn't a way to do that with firewalld yet. Is this when a direct rule would be appropriate? And to which zone should it apply?
You must use a direct rule. Direct rules are "global" in the sense that they aren't applied to a zone. Often they occur _before_ all zone rules.
Thanks to your advice and some fighting with libvirt I was able to achieve the samba whitelist with these direct rules:
# Whitelist samba to file server sudo firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -p udp -s 192.168.100.0/24 -d 192.168.2.2 --dport 137 -j ACCEPT sudo firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -p udp -s 192.168.100.0/24 -d 192.168.2.2 --dport 138 -j ACCEPT sudo firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -p tcp -s 192.168.100.0/24 -d 192.168.2.2 --dport 139 -j ACCEPT sudo firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -p tcp -s 192.168.100.0/24 -d 192.168.2.2 --dport 445 -j ACCEPT
# Reject all other traffic sudo firewall-cmd --direct --add-rule ipv4 filter FORWARD 1 -s 192.168.100.0/24 -j REJECT
These runtime rules will *insert* the rules at the top of the chain, so they short-circuit the LIBVIRT_{FWX,FWI,FWO} chains.
My problem now is that if I add them as permanent to persist them and reload, my direct rules end up *appended* to the bottom of the FORWARD table despite my priority setting. I gather the priority settings is *relative to eachother*, and doesn't necessarily guarantee anything about non-firewalld rules like the LIBVIRT chains.
That's accurate. Firewalld doesn't know anything about the rules added out-of-band of firewalld.
What ends up happening is the LIBVIRT_X chains take precedence above mine. At the bottom of LIBVIRT_FWO, its configured to accept traffic from the net that I reject with my last direct rule...so my rule gets ignored.
What's the right way to ensure that my rules get loaded with precedence over the LIBVIRT ones? Is it expected behavior that a runtime applied direct rule is inserted, but a persistent rule gets appended to the bottom of the chain?
You'll have to add the rules _after_ libvirt starts. Use "-I" instead of "-A" to insert the rules at the head of the chain.
firewalld-users@lists.fedorahosted.org
-
Ed Greshko -
Eric Garver -
Gunnar Niels -
Gunnari Niels