Hi, all,
Since upgrading to Fedora 33, I am having some problems accessing files on NFS shares. I should say first that, immediately after the upgrade, the shares would not mount at all. I was getting the error:
mount.nfs4: Operation not permitted when mounting NFS device
even when attempting to mount manually. Eventually, I found some info online that suggested mounting manually as root, which created some symlink or other. After that, the shares would mount automatically at boot.
However, now, as I said, I have problems accessing files on said shares. The directory ~/files/ is an NFS mount from a server running CentOS 7. My LyX user directory is at ~/files/config/lyx/ (symlinked from ~/.lyx/). LyX refuses to start, blocking at a call to lockf(fd, F_LOCK, 0) at support/filetools.cpp, during startup. (I believe that fd here points at a file on the NFS share.) I can successfully start LyX via:
lyx -userdir ~/tmp/lyx
say. I can then restart LyX with the same command and all is well. I can also start it with
lyx -userdir ~/files/lyxnew/
where that is a *non-existent* directory on the NFS share. In that case, LyX goes through its configuration process and starts normally. However, if I then attempt to run that same command again, LyX does not start. Also, if ~/files/lyxnew/ does exist, then LyX will not start, even if it is empty.
I am seeing a similar problem with Libre Office. I can starti fine, but if I then attempt to open a file from ~/files/, Libre Office freezes. It does not matter when it is ODT or ODS, say. I have not seen the problem in other programs, but I only upgraded yesterday.
I am guessing that there is something wrong with the NFS configuration. But I did not change anything myself.
Riki
On Nov 23, 2020, at 19:05, Richard Kimberly Heck rikiheck@lyx.org wrote:
LyX refuses to start, blocking at a call to lockf(fd, F_LOCK, 0) at support/filetools.cpp, during startup. (I believe that fd here points at a file on the NFS share.)
Did you end up using NFSv4? It sounds like a locking problem but the implementation differs significantly between v3 and v4.
-- Jonathan Billings billings@negate.org
On 11/23/20 9:17 PM, Jonathan Billings wrote:
On Nov 23, 2020, at 19:05, Richard Kimberly Heck rikiheck@lyx.org wrote:
LyX refuses to start, blocking at a call to lockf(fd, F_LOCK, 0) at support/filetools.cpp, during startup. (I believe that fd here points at a file on the NFS share.)
Did you end up using NFSv4? It sounds like a locking problem but the implementation differs significantly between v3 and v4.
I believe it is v3 on the server. If I specify nfs4 as the filesystem type, then I get:
mount.nfs4: mounting 192.168.1.2://home/photos/ failed, reason given by server: No such file or directory
The directory exists, so I guess it must not be exported that way.
Here's the relevant portion of /etc/fstab on the client:
192.168.1.2://home/rikiheck/files /home/rikiheck/files nfs auto,nouser,rw,dev,nosuid,exec,_netdev 0 0 192.168.1.2://multi/ /mnt/mail/multi nfs auto,nouser,rw,dev,nosuid,noexec,_netdev 0 0 192.168.1.2://home/photos /mnt/mail/photos nfs auto,nouser,rw,dev,nosuid,noexec,_netdev 0 0
Riki
On 24/11/20 1:55 pm, Richard Kimberly Heck wrote:
On 11/23/20 9:17 PM, Jonathan Billings wrote:
On Nov 23, 2020, at 19:05, Richard Kimberly Heck rikiheck@lyx.org wrote:
LyX refuses to start, blocking at a call to lockf(fd, F_LOCK, 0) at support/filetools.cpp, during startup. (I believe that fd here points at a file on the NFS share.)
Did you end up using NFSv4? It sounds like a locking problem but the implementation differs significantly between v3 and v4.
I believe it is v3 on the server. If I specify nfs4 as the filesystem type, then I get:
mount.nfs4: mounting 192.168.1.2://home/photos/ failed, reason given by server: No such file or directory
The directory exists, so I guess it must not be exported that way.
Here's the relevant portion of /etc/fstab on the client:
192.168.1.2://home/rikiheck/files /home/rikiheck/files nfs auto,nouser,rw,dev,nosuid,exec,_netdev 0 0 192.168.1.2://multi/ /mnt/mail/multi nfs auto,nouser,rw,dev,nosuid,noexec,_netdev 0 0 192.168.1.2://home/photos /mnt/mail/photos nfs auto,nouser,rw,dev,nosuid,noexec,_netdev 0 0
Riki
I think Fedora uses nfs version 4 by default. As one of the options I think you can specify nfsver=3 which should cause Fedora to use version 3 of nfs, which might resolve your issue. I have a nas disk which uses nfs version 1 and I have had to specify nfsver=3 to mount it.
regards, Steve
users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
On 11/24/20 2:18 AM, Stephen Morris wrote:
On 24/11/20 1:55 pm, Richard Kimberly Heck wrote:
On 11/23/20 9:17 PM, Jonathan Billings wrote:
On Nov 23, 2020, at 19:05, Richard Kimberly Heck rikiheck@lyx.org wrote:
LyX refuses to start, blocking at a call to lockf(fd, F_LOCK, 0) at support/filetools.cpp, during startup. (I believe that fd here points at a file on the NFS share.)
Did you end up using NFSv4? It sounds like a locking problem but the implementation differs significantly between v3 and v4.
I believe it is v3 on the server. If I specify nfs4 as the filesystem type, then I get:
mount.nfs4: mounting 192.168.1.2://home/photos/ failed, reason given by server: No such file or directory
The directory exists, so I guess it must not be exported that way.
Here's the relevant portion of /etc/fstab on the client:
192.168.1.2://home/rikiheck/files /home/rikiheck/files nfs auto,nouser,rw,dev,nosuid,exec,_netdev 0 0 192.168.1.2://multi/ /mnt/mail/multi nfs auto,nouser,rw,dev,nosuid,noexec,_netdev 0 0 192.168.1.2://home/photos /mnt/mail/photos nfs auto,nouser,rw,dev,nosuid,noexec,_netdev 0 0
Riki
I think Fedora uses nfs version 4 by default. As one of the options I think you can specify nfsver=3 which should cause Fedora to use version 3 of nfs, which might resolve your issue. I have a nas disk which uses nfs version 1 and I have had to specify nfsver=3 to mount it.
Thanks for the suggestion. Unfortunately, while it didn't hurt, it didn't help, either.
Riki
On Nov 23, 2020, at 22:08, Richard Kimberly Heck rikiheck@lyx.org wrote:
I believe it is v3 on the server.
If so, you need to make sure that the rpc.lockd can communicate on both sides. That is probably why locking fails, and any program that is hanging is because locks are hanging.
-- Jonathan Billings
On 11/24/20 7:55 AM, Jonathan Billings wrote:
On Nov 23, 2020, at 22:08, Richard Kimberly Heck rikiheck@lyx.org wrote:
I believe it is v3 on the server.
If so, you need to make sure that the rpc.lockd can communicate on both sides. That is probably why locking fails, and any program that is hanging is because locks are hanging.
This looks like the issue:
Nov 22 23:10:03 rkhstudy kernel: lockd: server 192.168.1.2 not responding, still trying
This was working fine in Fedora 32, though, so I'm puzzled what's wrong now. There's no firewall on the client, and nothing has changed on the server, and I do not see any error messages there. Is lockd perhaps using a different port, which is blocked on the server? If so, how do I find out what that is?
Riki
On Tue, Nov 24, 2020 at 02:06:46PM -0500, Richard Kimberly Heck wrote:
This looks like the issue:
Nov 22 23:10:03 rkhstudy kernel: lockd: server 192.168.1.2 not responding, still trying
This was working fine in Fedora 32, though, so I'm puzzled what's wrong now. There's no firewall on the client, and nothing has changed on the server, and I do not see any error messages there. Is lockd perhaps using a different port, which is blocked on the server? If so, how do I find out what that is?
'rpcinfo -p' should tell you port numbers, although it's usually something in the firewall that causes the issue.
On 11/24/20 2:12 PM, Jonathan Billings wrote:
On Tue, Nov 24, 2020 at 02:06:46PM -0500, Richard Kimberly Heck wrote:
This looks like the issue:
Nov 22 23:10:03 rkhstudy kernel: lockd: server 192.168.1.2 not responding, still trying
This was working fine in Fedora 32, though, so I'm puzzled what's wrong now. There's no firewall on the client, and nothing has changed on the server, and I do not see any error messages there. Is lockd perhaps using a different port, which is blocked on the server? If so, how do I find out what that is?
'rpcinfo -p' should tell you port numbers, although it's usually something in the firewall that causes the issue.
Yes, thank you, that is definitely the problem. Disabling the firewall on the server solves it. (Disabling the firewall on the client has no effect.) But of course that's not a long-term solution, and I'm still puzzled why this is failing after the upgrade but was working before.... I found a few relevant pages on the web, but they tell me to enable services in the firewall that are already enabled.
Here's the firewall on the server:
# firewall-cmd --list-all public (active) target: default icmp-block-inversion: no interfaces: enp2s0 sources: services: dhcpv6-client mountd nfs rpc-bind ssh ports: 9000/tcp 9000/udp 3483/tcp 3483/udp 9090/tcp 9090/udp 1900/tcp 1900/udp protocols: masquerade: no forward-ports: source-ports: icmp-blocks: rich rules:
I restarted it in case something was wonky. No effect.
Riki
On 25/11/2020 05:43, Richard Kimberly Heck wrote:
On 11/24/20 2:12 PM, Jonathan Billings wrote:
On Tue, Nov 24, 2020 at 02:06:46PM -0500, Richard Kimberly Heck wrote:
This looks like the issue:
Nov 22 23:10:03 rkhstudy kernel: lockd: server 192.168.1.2 not responding, still trying
This was working fine in Fedora 32, though, so I'm puzzled what's wrong now. There's no firewall on the client, and nothing has changed on the server, and I do not see any error messages there. Is lockd perhaps using a different port, which is blocked on the server? If so, how do I find out what that is?
'rpcinfo -p' should tell you port numbers, although it's usually something in the firewall that causes the issue.
Yes, thank you, that is definitely the problem. Disabling the firewall on the server solves it. (Disabling the firewall on the client has no effect.) But of course that's not a long-term solution, and I'm still puzzled why this is failing after the upgrade but was working before.... I found a few relevant pages on the web, but they tell me to enable services in the firewall that are already enabled.
Here's the firewall on the server:
# firewall-cmd --list-all public (active) target: default icmp-block-inversion: no interfaces: enp2s0 sources: services: dhcpv6-client mountd nfs rpc-bind ssh ports: 9000/tcp 9000/udp 3483/tcp 3483/udp 9090/tcp 9090/udp 1900/tcp 1900/udp protocols: masquerade: no forward-ports: source-ports: icmp-blocks: rich rules:
I restarted it in case something was wonky. No effect.
If you are using nfs-V3 you need to enable the nfs3 service in the active zone.
I haven't used nfs-V3 in a long time. However, I seem to remember that if you allow nfs3 in the services there is no need to define open ports as that is taken care of.
--- The key to getting good answers is to ask good questions.
On 11/24/20 5:53 PM, Ed Greshko wrote:
On 25/11/2020 05:43, Richard Kimberly Heck wrote:
On 11/24/20 2:12 PM, Jonathan Billings wrote:
On Tue, Nov 24, 2020 at 02:06:46PM -0500, Richard Kimberly Heck wrote:
This looks like the issue:
Nov 22 23:10:03 rkhstudy kernel: lockd: server 192.168.1.2 not responding, still trying
This was working fine in Fedora 32, though, so I'm puzzled what's wrong now. There's no firewall on the client, and nothing has changed on the server, and I do not see any error messages there. Is lockd perhaps using a different port, which is blocked on the server? If so, how do I find out what that is?
'rpcinfo -p' should tell you port numbers, although it's usually something in the firewall that causes the issue.
Yes, thank you, that is definitely the problem. Disabling the firewall on the server solves it. (Disabling the firewall on the client has no effect.) But of course that's not a long-term solution, and I'm still puzzled why this is failing after the upgrade but was working before.... I found a few relevant pages on the web, but they tell me to enable services in the firewall that are already enabled.
Here's the firewall on the server:
# firewall-cmd --list-all public (active) target: default icmp-block-inversion: no interfaces: enp2s0 sources: services: dhcpv6-client mountd nfs rpc-bind ssh ports: 9000/tcp 9000/udp 3483/tcp 3483/udp 9090/tcp 9090/udp 1900/tcp 1900/udp protocols: masquerade: no forward-ports: source-ports: icmp-blocks: rich rules:
I restarted it in case something was wonky. No effect.
If you are using nfs-V3 you need to enable the nfs3 service in the active zone.
I haven't used nfs-V3 in a long time. However, I seem to remember that if you allow nfs3 in the services there is no need to define open ports as that is taken care of.
Thanks for the suggestion. Unfortunately, it does not seem to help. It still seems weird to me that it worked with F32 and now not with F33. What could have changed on that end?
I will check later with a machine still running F32 that it does still work there.
Riki
On 25/11/2020 07:59, Richard Kimberly Heck wrote:
On 11/24/20 5:53 PM, Ed Greshko wrote:
On 25/11/2020 05:43, Richard Kimberly Heck wrote:
On 11/24/20 2:12 PM, Jonathan Billings wrote:
On Tue, Nov 24, 2020 at 02:06:46PM -0500, Richard Kimberly Heck wrote:
This looks like the issue:
Nov 22 23:10:03 rkhstudy kernel: lockd: server 192.168.1.2 not responding, still trying
This was working fine in Fedora 32, though, so I'm puzzled what's wrong now. There's no firewall on the client, and nothing has changed on the server, and I do not see any error messages there. Is lockd perhaps using a different port, which is blocked on the server? If so, how do I find out what that is?
'rpcinfo -p' should tell you port numbers, although it's usually something in the firewall that causes the issue.
Yes, thank you, that is definitely the problem. Disabling the firewall on the server solves it. (Disabling the firewall on the client has no effect.) But of course that's not a long-term solution, and I'm still puzzled why this is failing after the upgrade but was working before.... I found a few relevant pages on the web, but they tell me to enable services in the firewall that are already enabled.
Here's the firewall on the server:
# firewall-cmd --list-all public (active) target: default icmp-block-inversion: no interfaces: enp2s0 sources: services: dhcpv6-client mountd nfs rpc-bind ssh ports: 9000/tcp 9000/udp 3483/tcp 3483/udp 9090/tcp 9090/udp 1900/tcp 1900/udp protocols: masquerade: no forward-ports: source-ports: icmp-blocks: rich rules:
I restarted it in case something was wonky. No effect.
If you are using nfs-V3 you need to enable the nfs3 service in the active zone.
I haven't used nfs-V3 in a long time. However, I seem to remember that if you allow nfs3 in the services there is no need to define open ports as that is taken care of.
Thanks for the suggestion. Unfortunately, it does not seem to help. It still seems weird to me that it worked with F32 and now not with F33. What could have changed on that end?
I will check later with a machine still running F32 that it does still work there.
Can you confirm that the NFS server is on a Centos7 system?
Is NFS-v3 required, or can you switch to NFS-v4?
--- The key to getting good answers is to ask good questions.
On 25/11/2020 09:40, Ed Greshko wrote:
Can you confirm that the NFS server is on a Centos7 system?
Is NFS-v3 required, or can you switch to NFS-v4?
FWIW, I installed a Centos7 VM. And enabled nfs-server.service. The server has the following:
[root@cos7 etc]# firewall-cmd --get-active-zones public interfaces: eth0 [root@cos7 etc]# firewall-cmd --info-zone=public public (active) target: default icmp-block-inversion: no interfaces: eth0 sources: services: dhcpv6-client mountd nfs nfs3 rpc-bind ssh ports: protocols: masquerade: no forward-ports: source-ports: icmp-blocks: rich rules:
[root@cos7 etc]# cat exports /home/egreshko 2001:b030:112f:2::0/64(rw,sync,insecure,no_root_squash,no_subtree_check) /home/egreshko 192.168.0.0/16(rw,sync,insecure,no_root_squash,no_subtree_check)
On an F33 VM I have the following in /etc/fstab
#[2001:b030:112f:2::41]:/home/egreshko /mnt nfs defaults 0 0 [2001:b030:112f:2::41]:/home/egreshko /mnt nfs nfsvers=3,defaults 0 0
And, I am able to mount using either nfs-v3 or nfs-v4
Here is the V3 result
[root@f33k etc]# mount /mnt [root@f33k etc]# df -T | grep greshko [2001:b030:112f:2::41]:/home/egreshko nfs 29599744 5193472 24406272 18% /mnt
Here is the V4 result
[root@f33k etc]# mount /mnt [root@f33k etc]# df -T | grep greshko [2001:b030:112f:2::41]:/home/egreshko nfs4 29599744 5193472 24406272 18% /mnt
--- The key to getting good answers is to ask good questions.
On 11/25/20 5:23 AM, Ed Greshko wrote:
On 25/11/2020 09:40, Ed Greshko wrote:
Can you confirm that the NFS server is on a Centos7 system?
Is NFS-v3 required, or can you switch to NFS-v4?
FWIW, I installed a Centos7 VM. And enabled nfs-server.service. The server has the following:
[root@cos7 etc]# firewall-cmd --get-active-zones public interfaces: eth0 [root@cos7 etc]# firewall-cmd --info-zone=public public (active) target: default icmp-block-inversion: no interfaces: eth0 sources: services: dhcpv6-client mountd nfs nfs3 rpc-bind ssh ports: protocols: masquerade: no forward-ports: source-ports: icmp-blocks: rich rules:
[root@cos7 etc]# cat exports /home/egreshko 2001:b030:112f:2::0/64(rw,sync,insecure,no_root_squash,no_subtree_check) /home/egreshko 192.168.0.0/16(rw,sync,insecure,no_root_squash,no_subtree_check)
On an F33 VM I have the following in /etc/fstab
#[2001:b030:112f:2::41]:/home/egreshko /mnt nfs defaults 0 0 [2001:b030:112f:2::41]:/home/egreshko /mnt nfs nfsvers=3,defaults 0 0
And, I am able to mount using either nfs-v3 or nfs-v4
I can mount fine. The problem is that various programs (LyX, LibreOffice) freeze when attempting to open files on those filesystems. In LyX, I was able to trace the problem to a call to lockd, which never returns. So it seems there is some problem with file locking, which it was suggested involved some firewall problem. Which it does, since disabling the firewall on the server solves the problem. (There's no firewall on the client.)
Riki
On 25/11/2020 22:51, Richard Kimberly Heck wrote:
I can mount fine. The problem is that various programs (LyX, LibreOffice) freeze when attempting to open files on those filesystems. In LyX, I was able to trace the problem to a call to lockd, which never returns. So it seems there is some problem with file locking, which it was suggested involved some firewall problem. Which it does, since disabling the firewall on the server solves the problem. (There's no firewall on the client.)
OK, sorry, I was only thinking about the initial post.
I tested and found some problems opening a file with LibreOffice in "edit" mode when using NFS-v3. It would only open "read only". Get a message from lockd that user is "unknown". Too early in my AM to track this down. It maybe related to idmapd. Will see if I can find the cause later today.
However, I had no problems when using NFS-v4. Can open, edit, and save documents.
Can you switch to mounting NFS-v4?
FWIW, I did change the fstab entry to...
[2001:b030:112f:2::41]:/home/egreshko /mnt nfs rw,soft,fg 0 0
--- The key to getting good answers is to ask good questions.
On 11/25/20 12:56 PM, Ed Greshko wrote:
On 25/11/2020 22:51, Richard Kimberly Heck wrote:
I can mount fine. The problem is that various programs (LyX, LibreOffice) freeze when attempting to open files on those filesystems. In LyX, I was able to trace the problem to a call to lockd, which never returns. So it seems there is some problem with file locking, which it was suggested involved some firewall problem. Which it does, since disabling the firewall on the server solves the problem. (There's no firewall on the client.)
OK, sorry, I was only thinking about the initial post.
I tested and found some problems opening a file with LibreOffice in "edit" mode when using NFS-v3. It would only open "read only". Get a message from lockd that user is "unknown". Too early in my AM to track this down. It maybe related to idmapd. Will see if I can find the cause later today.
Thanks for the help. It's much appreciated.
Just to clarify: Stopping the firewall solves my problem.
I enabled firewall debugging and tried to open a LyX file on the nfs share. I started seeing lots of these:
Nov 25 16:18:54 master kernel: FINAL_REJECT: IN=enp2s0 OUT= MAC=00:30:18:a6:7c:04:18:31:bf:4e:e7:a3:08:00 SRC=192.168.1.51 DST=192.168.1.2 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=21519 DF PROTO=TCP SPT=872 DPT=36389 WINDOW=64240 RES=0x00 SYN URGP=0 Nov 25 16:18:57 master kernel: FINAL_REJECT: IN=enp2s0 OUT= MAC=00:30:18:a6:7c:04:18:31:bf:4e:e7:a3:08:00 SRC=192.168.1.51 DST=192.168.1.2 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=49938 DF PROTO=TCP SPT=872 DPT=36389 WINDOW=64240 RES=0x00 SYN URGP=0
192.168.1.51 is the client; .2 is the server.
I then rebooted and tried a LibreOffice file:
Nov 25 16:24:15 master kernel: FINAL_REJECT: IN=enp2s0 OUT= MAC=00:30:18:a6:7c:04:18:31:bf:4e:e7:a3:08:00 SRC=192.168.1.51 DST=192.168.1.2 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33548 DF PROTO=TCP SPT=853 DPT=38641 WINDOW=64240 RES=0x00 SYN URGP=0 Nov 25 16:24:18 master kernel: FINAL_REJECT: IN=enp2s0 OUT= MAC=00:30:18:a6:7c:04:18:31:bf:4e:e7:a3:08:00 SRC=192.168.1.51 DST=192.168.1.2 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=58996 DF PROTO=TCP SPT=853 DPT=38641 WINDOW=64240 RES=0x00 SYN URGP=0
I don't know if it matters, but I also see
Nov 25 16:24:37 master kernel: FINAL_REJECT: IN=enp2s0 OUT= MAC=00:30:18:a6:7c:04:18:31:bf:4e:e7:a3:08:00 SRC=192.168.1.51 DST=192.168.1.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=19835 DF PROTO=UDP SPT=810 DPT=57620 LEN=64
pretty much as soon as I boot.
I will see if I can reconfigure the server to use NFS-v4.
Riki
On 11/25/20 4:28 PM, Richard Kimberly Heck wrote:
On 11/25/20 12:56 PM, Ed Greshko wrote:
On 25/11/2020 22:51, Richard Kimberly Heck wrote:
I can mount fine. The problem is that various programs (LyX, LibreOffice) freeze when attempting to open files on those filesystems. In LyX, I was able to trace the problem to a call to lockd, which never returns. So it seems there is some problem with file locking, which it was suggested involved some firewall problem. Which it does, since disabling the firewall on the server solves the problem. (There's no firewall on the client.)
OK, sorry, I was only thinking about the initial post.
I tested and found some problems opening a file with LibreOffice in "edit" mode when using NFS-v3. It would only open "read only". Get a message from lockd that user is "unknown". Too early in my AM to track this down. It maybe related to idmapd. Will see if I can find the cause later today.
Thanks for the help. It's much appreciated.
Just to clarify: Stopping the firewall solves my problem.
One other data point. If I reload the firewall, I get a lot of errors:
Nov 25 16:41:16 master firewalld[715]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -w --table mangle --delete POSTROUTING --out-interface virbr0 --protocol udp --destination-port 68 --jump CHECKSUM --checksum-fill' failed: iptables: No chain/target/match by that name. Nov 25 16:41:16 master firewalld[715]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -w --table nat --delete POSTROUTING --source 192.168.122.0/24 --destination 224.0.0.0/24 --jump RETURN' failed: iptables: Bad rule (does a matching rule exist in that chain?). Nov 25 16:41:16 master firewalld[715]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -w --table nat --delete POSTROUTING --source 192.168.122.0/24 --destination 255.255.255.255/32 --jump RETURN' failed: iptables: Bad rule (does a matching rule exist in that chain?). Nov 25 16:41:16 master firewalld[715]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -w --table nat --delete POSTROUTING --source 192.168.122.0/24 -p tcp ! --destination 192.168.122.0/24 --jump MASQUERADE --to-ports 1024-65535' failed: iptables: No chain/target/match by that name. Nov 25 16:41:16 master firewalld[715]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -w --table nat --delete POSTROUTING --source 192.168.122.0/24 -p udp ! --destination 192.168.122.0/24 --jump MASQUERADE --to-ports 1024-65535' failed: iptables: No chain/target/match by that name. Nov 25 16:41:16 master firewalld[715]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -w --table nat --delete POSTROUTING --source 192.168.122.0/24 ! --destination 192.168.122.0/24 --jump MASQUERADE' failed: iptables: No chain/target/match by that name. Nov 25 16:41:16 master firewalld[715]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -w --table filter --delete FORWARD --destination 192.168.122.0/24 --out-interface virbr0 --match conntrack --ctstate ESTABLISHED,RELATED --jump ACCEPT' failed: iptables: Bad rule (does a matching rule exist in that chain?). Nov 25 16:41:16 master firewalld[715]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -w --table filter --delete FORWARD --source 192.168.122.0/24 --in-interface virbr0 --jump ACCEPT' failed: iptables: Bad rule (does a matching rule exist in that chain?). Nov 25 16:41:16 master firewalld[715]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -w --table filter --delete FORWARD --in-interface virbr0 --out-interface virbr0 --jump ACCEPT' failed: iptables: Bad rule (does a matching rule exist in that chain?). Nov 25 16:41:16 master firewalld[715]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -w --table filter --delete FORWARD --out-interface virbr0 --jump REJECT' failed: iptables: No chain/target/match by that name. Nov 25 16:41:16 master firewalld[715]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -w --table filter --delete FORWARD --in-interface virbr0 --jump REJECT' failed: iptables: No chain/target/match by that name. Nov 25 16:41:16 master firewalld[715]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -w --table filter --delete INPUT --in-interface virbr0 --protocol udp --destination-port 53 --jump ACCEPT' failed: iptables: Bad rule (does a matching rule exist in that chain?). Nov 25 16:41:16 master firewalld[715]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -w --table filter --delete INPUT --in-interface virbr0 --protocol tcp --destination-port 53 --jump ACCEPT' failed: iptables: Bad rule (does a matching rule exist in that chain?). Nov 25 16:41:16 master firewalld[715]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -w --table filter --delete OUTPUT --out-interface virbr0 --protocol udp --destination-port 68 --jump ACCEPT' failed: iptables: Bad rule (does a matching rule exist in that chain?). Nov 25 16:41:16 master firewalld[715]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -w --table filter --delete INPUT --in-interface virbr0 --protocol udp --destination-port 67 --jump ACCEPT' failed: iptables: Bad rule (does a matching rule exist in that chain?). Nov 25 16:41:16 master firewalld[715]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -w --table filter --delete INPUT --in-interface virbr0 --protocol tcp --destination-port 67 --jump ACCEPT' failed: iptables: Bad rule (does a matching rule exist in that chain?).
I have not attempted to customize this at all, so maybe none of these issues matter.
Riki
On 11/25/20 4:28 PM, Richard Kimberly Heck wrote:
On 11/25/20 12:56 PM, Ed Greshko wrote:
On 25/11/2020 22:51, Richard Kimberly Heck wrote:
I can mount fine. The problem is that various programs (LyX, LibreOffice) freeze when attempting to open files on those filesystems. In LyX, I was able to trace the problem to a call to lockd, which never returns. So it seems there is some problem with file locking, which it was suggested involved some firewall problem. Which it does, since disabling the firewall on the server solves the problem. (There's no firewall on the client.)
OK, sorry, I was only thinking about the initial post.
I tested and found some problems opening a file with LibreOffice in "edit" mode when using NFS-v3. It would only open "read only". Get a message from lockd that user is "unknown". Too early in my AM to track this down. It maybe related to idmapd. Will see if I can find the cause later today.
Thanks for the help. It's much appreciated.
Just to clarify: Stopping the firewall solves my problem.
One more piece: I tried using rpcdebug to see if there were any error messages on the server. Nothing, at least with 'rpcdebug -m nfsd -s auth -s lockd -s fileop'. I also tried 'rpcdebug -m nlm -s all'. Again nothing.
On the client, I tried 'rpcdebug -m nlm -s all'. That produced:
Nov 25 16:52:56 rkhstudy systemd[1540]: Started LibreOffice Writer - Word Processor. Nov 25 16:52:57 rkhstudy kernel: lockd: get host 192.168.1.2 Nov 25 16:52:57 rkhstudy kernel: lockd: get host 192.168.1.2 Nov 25 16:52:57 rkhstudy kernel: lockd: nsm_monitor(192.168.1.2) Nov 25 16:52:57 rkhstudy kernel: lockd: call procedure 2 on 192.168.1.2 Nov 25 16:52:57 rkhstudy kernel: lockd: nlm_bind_host 192.168.1.2 (192.168.1.2) Nov 25 16:52:57 rkhstudy kernel: lockd: next rebind in 60000 jiffies Nov 25 16:52:57 rkhstudy kernel: lockd: server 192.168.1.2 not responding, still trying
It seems like it must be a firewall issue. But nothing has changed on the server (except my enabling more things on the firewall since all this started).
Riki
This problem seems to have been solved. I believe that the issue was a misconfiguration of the NFS server. I had:
/home/rikiheck/files 192.168.1.0/24(rw,sync,no_subtree_check,fsid=0) /home/nancy/files 192.168.1.0/24(rw,sync,no_subtree_check) /home/photos 192.168.1.0/24(rw,sync,no_subtree_check) /multi 192.168.1.0/24(rw,sync,no_subtree_check) /git 192.168.1.0/24(rw,sync,no_subtree_check)
But /home/rikiheck/files was an ordinary directory that I want to export, not the root for NFSv4. It was being mounted as NFSv3 (trying to mount with nfs4 would fail). But I'm guessing that it was being treated inconsistently between the client and the server. Removing the 'fsid=0' tag solved the problem. It's still a bit of a mystery what exactly the firewall was blocking, but it wasn't a firewall problem, in the end.
Thanks to everyone who offered suggestions and advice, especially Ed Greshko.
Probably I will take the opportunity to move anyway to NFSv4, now that I've done all this reading about how to do that!
Riki
On 11/25/20 4:55 PM, Richard Kimberly Heck wrote:
On 11/25/20 4:28 PM, Richard Kimberly Heck wrote:
On 11/25/20 12:56 PM, Ed Greshko wrote:
On 25/11/2020 22:51, Richard Kimberly Heck wrote:
I can mount fine. The problem is that various programs (LyX, LibreOffice) freeze when attempting to open files on those filesystems. In LyX, I was able to trace the problem to a call to lockd, which never returns. So it seems there is some problem with file locking, which it was suggested involved some firewall problem. Which it does, since disabling the firewall on the server solves the problem. (There's no firewall on the client.)
OK, sorry, I was only thinking about the initial post.
I tested and found some problems opening a file with LibreOffice in "edit" mode when using NFS-v3. It would only open "read only". Get a message from lockd that user is "unknown". Too early in my AM to track this down. It maybe related to idmapd. Will see if I can find the cause later today.
Thanks for the help. It's much appreciated.
Just to clarify: Stopping the firewall solves my problem.
One more piece: I tried using rpcdebug to see if there were any error messages on the server. Nothing, at least with 'rpcdebug -m nfsd -s auth -s lockd -s fileop'. I also tried 'rpcdebug -m nlm -s all'. Again nothing.
On the client, I tried 'rpcdebug -m nlm -s all'. That produced:
Nov 25 16:52:56 rkhstudy systemd[1540]: Started LibreOffice Writer - Word Processor. Nov 25 16:52:57 rkhstudy kernel: lockd: get host 192.168.1.2 Nov 25 16:52:57 rkhstudy kernel: lockd: get host 192.168.1.2 Nov 25 16:52:57 rkhstudy kernel: lockd: nsm_monitor(192.168.1.2) Nov 25 16:52:57 rkhstudy kernel: lockd: call procedure 2 on 192.168.1.2 Nov 25 16:52:57 rkhstudy kernel: lockd: nlm_bind_host 192.168.1.2 (192.168.1.2) Nov 25 16:52:57 rkhstudy kernel: lockd: next rebind in 60000 jiffies Nov 25 16:52:57 rkhstudy kernel: lockd: server 192.168.1.2 not responding, still trying
It seems like it must be a firewall issue. But nothing has changed on the server (except my enabling more things on the firewall since all this started).
Riki
On 26/11/2020 06:40, Richard Kimberly Heck wrote:
This problem seems to have been solved. I believe that the issue was a misconfiguration of the NFS server. I had:
/home/rikiheck/files 192.168.1.0/24(rw,sync,no_subtree_check,fsid=0) /home/nancy/files 192.168.1.0/24(rw,sync,no_subtree_check) /home/photos 192.168.1.0/24(rw,sync,no_subtree_check) /multi 192.168.1.0/24(rw,sync,no_subtree_check) /git 192.168.1.0/24(rw,sync,no_subtree_check)
But /home/rikiheck/files was an ordinary directory that I want to export, not the root for NFSv4. It was being mounted as NFSv3 (trying to mount with nfs4 would fail). But I'm guessing that it was being treated inconsistently between the client and the server. Removing the 'fsid=0' tag solved the problem. It's still a bit of a mystery what exactly the firewall was blocking, but it wasn't a firewall problem, in the end.
Thanks to everyone who offered suggestions and advice, especially Ed Greshko.
Probably I will take the opportunity to move anyway to NFSv4, now that I've done all this reading about how to do that!
Happy to hear it is all working.
Be careful that you don't read articles that are outdated on NFSv4. Things have changed and there is no need to do bind mount or something like that.
--- The key to getting good answers is to ask good questions.
On Nov 25, 2020, at 17:41, Richard Kimberly Heck rikiheck@lyx.org wrote:
But /home/rikiheck/files was an ordinary directory that I want to export, not the root for NFSv4. It was being mounted as NFSv3 (trying to mount with nfs4 would fail). But I'm guessing that it was being treated inconsistently between the client and the server. Removing the 'fsid=0' tag solved the problem. It's still a bit of a mystery what exactly the firewall was blocking, but it wasn't a firewall problem, in the end.
With NFSv4, the locking (NLM) is handed directly through the same rpc protocol as the mount, while in v3 and earlier, it’s a separate rpc protocol which was likely being blocked by the firewall. This is why v4 is so much easier to configure with firewalls.
-- Jonathan Billings
On Wed, Nov 25, 2020 at 11:41 PM Richard Kimberly Heck rikiheck@lyx.org wrote:
This problem seems to have been solved. I believe that the issue was a misconfiguration of the NFS server. I had:
/home/rikiheck/files 192.168.1.0/24(rw,sync,no_subtree_check,fsid=0) /home/nancy/files 192.168.1.0/24(rw,sync,no_subtree_check) /home/photos 192.168.1.0/24(rw,sync,no_subtree_check) /multi 192.168.1.0/24(rw,sync,no_subtree_check) /git 192.168.1.0/24(rw,sync,no_subtree_check)
But /home/rikiheck/files was an ordinary directory that I want to export, not the root for NFSv4. It was being mounted as NFSv3 (trying to mount with nfs4 would fail). But I'm guessing that it was being treated inconsistently between the client and the server. Removing the 'fsid=0' tag solved the problem. It's still a bit of a mystery what exactly the firewall was blocking, but it wasn't a firewall problem, in the end.
Thanks to everyone who offered suggestions and advice, especially Ed Greshko.
Probably I will take the opportunity to move anyway to NFSv4, now that I've done all this reading about how to do that!
You have to be careful when using "fsid=0".
1) If you don't set it for any of the shares:
a) "/" is the "fsid=0" filesystem by default (this wasn't the case in early nfsv4 implementations; verify with "cat /proc/fs/nfsd/exports")
b) If you have "/srv/nfs/share1 nfs_client(nfs_options)" in "/etc/exports", you mount this share with "mount nfs_server:/srv/nfs/share1 nfs_mountpoint1"
2) If you set it for "/srv/nfs":
a) If you have "/srv/nfs/share1 nfs_client(nfs_options)" in "/etc/exports", you mount this share with "mount nfs_server:/share1 nfs_mountpoint1"
b) If you want to share "/home/user/files":
- you bind-mount "/home/user/files" to "/srv/nfs/share2"
- you add "/srv/nfs/share2 nfs_client(nfs_options)" to "/etc/exports"
- you mount this share with "mount nfs_server:/share2 nfs_mountpoint2"
On 11/26/20 3:26 PM, Tom H wrote:
On Wed, Nov 25, 2020 at 11:41 PM Richard Kimberly Heck rikiheck@lyx.org wrote:
This problem seems to have been solved. I believe that the issue was a misconfiguration of the NFS server. I had:
/home/rikiheck/files 192.168.1.0/24(rw,sync,no_subtree_check,fsid=0) /home/nancy/files 192.168.1.0/24(rw,sync,no_subtree_check) /home/photos 192.168.1.0/24(rw,sync,no_subtree_check) /multi 192.168.1.0/24(rw,sync,no_subtree_check) /git 192.168.1.0/24(rw,sync,no_subtree_check)
But /home/rikiheck/files was an ordinary directory that I want to export, not the root for NFSv4. It was being mounted as NFSv3 (trying to mount with nfs4 would fail). But I'm guessing that it was being treated inconsistently between the client and the server. Removing the 'fsid=0' tag solved the problem. It's still a bit of a mystery what exactly the firewall was blocking, but it wasn't a firewall problem, in the end.
Thanks to everyone who offered suggestions and advice, especially Ed Greshko.
Probably I will take the opportunity to move anyway to NFSv4, now that I've done all this reading about how to do that!
You have to be careful when using "fsid=0".
- If you don't set it for any of the shares:
a) "/" is the "fsid=0" filesystem by default (this wasn't the case in early nfsv4 implementations; verify with "cat /proc/fs/nfsd/exports")
b) If you have "/srv/nfs/share1 nfs_client(nfs_options)" in "/etc/exports", you mount this share with "mount nfs_server:/srv/nfs/share1 nfs_mountpoint1"
- If you set it for "/srv/nfs":
a) If you have "/srv/nfs/share1 nfs_client(nfs_options)" in "/etc/exports", you mount this share with "mount nfs_server:/share1 nfs_mountpoint1"
b) If you want to share "/home/user/files":
you bind-mount "/home/user/files" to "/srv/nfs/share2"
you add "/srv/nfs/share2 nfs_client(nfs_options)" to "/etc/exports"
you mount this share with "mount nfs_server:/share2 nfs_mountpoint2"
Thanks. I appreciate the info.
Riki
On Sat, Nov 28, 2020 at 4:20 AM Richard Kimberly Heck rikiheck@lyx.org wrote:
On 11/26/20 3:26 PM, Tom H wrote:
You have to be careful when using "fsid=0".
- If you don't set it for any of the shares:
a) "/" is the "fsid=0" filesystem by default (this wasn't the case in early nfsv4 implementations; verify with "cat /proc/fs/nfsd/exports")
b) If you have "/srv/nfs/share1 nfs_client(nfs_options)" in "/etc/exports", you mount this share with "mount nfs_server:/srv/nfs/share1 nfs_mountpoint1"
- If you set it for "/srv/nfs":
a) If you have "/srv/nfs/share1 nfs_client(nfs_options)" in "/etc/exports", you mount this share with "mount nfs_server:/share1 nfs_mountpoint1"
b) If you want to share "/home/user/files":
you bind-mount "/home/user/files" to "/srv/nfs/share2"
you add "/srv/nfs/share2 nfs_client(nfs_options)" to
"/etc/exports"
- you mount this share with "mount nfs_server:/share2
nfs_mountpoint2"
Thanks. I appreciate the info.
You're most welcome. But I made a mistake. "cat /proc/fs/nfsd/exports" will be empty unless at least one share's been mounted. "cat /var/lib/nfs/etab" (or "exportfs" or "showmount -e") will list the shares set up by "exportfs -r" in "nfs-server.service".
In the nfsv4 case, "cat /proc/fs/nfsd/exports" varies:
NFSv4 only - fsid=0 set
root ~ # rpcinfo -s | grep 'nfs ' 100003 4 tcp6,tcp nfs superuser
root ~ # cat /etc/exports.d/nfs.exports /srv/nfs 127.0.0.1/8(ro,...,fsid=0) /srv/nfs/share 127.0.0.1/8(rw,...)
root ~ # cat /var/lib/nfs/etab /srv/nfs/share 127.0.0.1/8(rw,...) /srv/nfs 127.0.0.1/8(ro,...,fsid=0,...)
root ~ # mount 127.0.0.1:/share /mnt
root ~ # cat /proc/fs/nfsd/exports # Version 1.1 # Path Client(Flags) # IPs /srv/nfs 127.0.0.1/8(ro,...,fsid=0,...)
root ~ #
NFSv4 only - fsid=0 unset
root ~ # rpcinfo -s | grep 'nfs ' 100003 4 tcp6,tcp nfs superuser
root ~ # cat /etc/exports.d/nfs.exports /srv/nfs/share 127.0.0.1/8(rw,...)
root ~ # cat /var/lib/nfs/etab /srv/nfs/share 127.0.0.1/8(rw,...)
root ~ # mount 127.0.0.1:/srv/nfs/share /mnt
root ~ # cat /proc/fs/nfsd/exports # Version 1.1 # Path Client(Flags) # IPs /srv/nfs/share 127.0.0.1/8(rw,...) /srv 127.0.0.1/8(ro,...,v4root,...) / 127.0.0.1/8(ro,...,v4root,fsid=0,...) /srv/nfs 127.0.0.1/8(ro,...,v4root,...)
root ~ #
NFSv3 only
root ~ # rpcinfo -s | grep 'nfs ' 100003 3 tcp6,tcp nfs superuser
root ~ # cat /etc/exports.d/nfs.exports /srv/nfs/share 127.0.0.1/8(rw,...)
root ~ # cat /var/lib/nfs/etab /srv/nfs/share 127.0.0.1/8(rw,...)
root ~ # mount 127.0.0.1:/srv/nfs/share /mnt
root ~ # cat /proc/fs/nfsd/exports # Version 1.1 # Path Client(Flags) # IPs /srv/nfs/share 127.0.0.1/8(rw,...)
root ~ #
-
Ed Greshko -
Jonathan Billings -
Richard Kimberly Heck -
Stephen Morris -
Tom H