I am banging my head against the wall on this one… I am using TrueNAS to export an NFS volume to a Proxmox cluster. I can mount it and write to it no problem. However, there are certain operations (like dealing with templates) that fail. It looks like when Proxmox tries to use extended attributes like chattr it fails.
I did a quick test and confirmed this behavior:
rroot@pve-node02:/mnt/pve/proxmox-test# ls -l
total 3
drwxrwxrwx 2 root root 2 Jan 20 17:21 dump
drwxrwxrwx 2 root root 2 Jan 20 17:21 images
drwxrwxrwx 2 root root 2 Jan 20 17:21 import
drwxrwxrwx 2 root root 2 Jan 20 17:21 private
drwxrwxrwx 2 root root 2 Jan 20 17:21 snippets
drwxrwxrwx 4 root root 4 Jan 20 17:21 template
root@pve-node02:/mnt/pve/proxmox-test# touch testfile
root@pve-node02:/mnt/pve/proxmox-test# ls -l
total 4
drwxrwxrwx 2 root root 2 Jan 20 17:21 dump
drwxrwxrwx 2 root root 2 Jan 20 17:21 images
drwxrwxrwx 2 root root 2 Jan 20 17:21 import
drwxrwxrwx 2 root root 2 Jan 20 17:21 private
drwxrwxrwx 2 root root 2 Jan 20 17:21 snippets
drwxrwxrwx 4 root root 4 Jan 20 17:21 template
-rw-r--r-- 1 root root 0 Jan 20 17:54 testfile
root@pve-node02:/mnt/pve/proxmox-test# chattr +t testfile
chattr: Operation not supported while reading flags on testfile
root@pve-node02:/mnt/pve/proxmox-test# lsattr testfile
lsattr: Operation not supported While reading flags on testfile
root@pve-node02:/mnt/pve/proxmox-test# ls -l
total 4
drwxrwxrwx 2 root root 2 Jan 20 17:21 dump
drwxrwxrwx 2 root root 2 Jan 20 17:21 images
drwxrwxrwx 2 root root 2 Jan 20 17:21 import
drwxrwxrwx 2 root root 2 Jan 20 17:21 private
drwxrwxrwx 2 root root 2 Jan 20 17:21 snippets
drwxrwxrwx 4 root root 4 Jan 20 17:21 template
-rw-r--r-- 1 root root 0 Jan 20 17:54 testfile
I have tried almost every single possible combination of permissions and settings and I can’t make this work. Does anyone know how to fix this?
One thing you might try is under your NFS share settings in TrueNAS, go to advanced option, and select MaprootUser = root
I was having a lot of problems with NFS shares, and once I did this all my problems went away. I am not sure if it will solve your issue or not, but its worth a try.
Thanks for sharing the video but unfortunately it doesn’t address the issue I am having. That said, it may be the case that it is not such a big deal after all. I encountered this when trying to manually migrate some templates and noticing that at the end of the migration, Proxmox will throw the chattr error when trying to apply the immutable flag. After experimenting a bunch more, this seems related to the NFS volume not supporting the xattr option and thus, disallowing extended attributes. I confirmed that this also happens in with my Synology NFS shares (I just never noticed it before).
Anyways, this does not seem to affect the running VMs or creating VMs. So, I am just going to let it be.
Thanks for the suggestion. I was also force mapping to root and other users trying to sort this out but that wasn’t it either.
I had to do the force mapping to solve a different permission issue I was having with providing NFS storage for my Kubernetes cluster. After much frustration there as well, I am going to experiment with the democratic-csi driver to let K8S talk to TrueNAS and provision its own storage.
I haven’t seen this problem in my clusters, but it does make me wonder about a couple things.
First, on the NFS export’s underlying filesystem, how are xattrs set up (zfs get xattr). And also, what version of NFS do you have set up? It looks like xattrs are only supported on nfsv4.
My understanding is that xattrs should be supported by NFS v4.1 but even if I forced NFS v4.2 (using TrueNAS), I couldn’t get the xattrs to work. Not sure what else to set to make this work. I would probably need to spend much more time experimenting and trying things to get an answer.
Yeah, they are both negotiating NFSv4.2 and I tried mounting from the CLI using the right NFS options for xattr but it rejected them. At this point, I am moving on from this. It may be just and edge case and it doesn’t seem to affect daily operation.
I may come back to it later and if I figure it out, I’ll report back.
