ZFS iSCSI Zvols and Xen/XCP - Best Practices

Hey everybody!

I hope everyone is doing well and staying sane in the new year!

I have a two host XCP-ng pool connected to my TrueNAS box. The shared storage on the TrueNAS box is configured as an iSCSI Zvol connected via redundant links using MPIO. This setup has been happily humming along.

I’m aware of the rule of thumb that you don’t want to use more than 50% when it comes to block storage/Zvols on ZFS to allow room for scrubbing, dedup, etc. It occurred to me though that I’m not actually sure if the 50% rule applies to how much space from the pool is allocated to the Zvol, or if it applies to utilization of the Zvol itself. Thankfully I’m only at about 40% utilization of my Zvol, and the Zvol has just over 50% allocation of the pool’s capacity, so I haven’t seen any performance impact up to this point (guesstimate: I’m probably utilizing about 25% of the pool’s total capacity). I’m curious, though, what are everyone’s thoughts on this use case? Especially as things grow. At some point I’ll need to increase the pool’s capacity, but in the interim (as needed) would it be better to expand the Zvol and allocate closer to 100% of the pool’s capacity, or keep the Zvol as is and just eat up the remaining free space? (I should probably know this, but I inherited this setup and I’m trying to learn more about how ZFS works on a deeper level)

I know NFS is another option, but I’m not looking to make that move in the foreseeable future. I’d like to test with a production work load before I’d feel comfortable with this change.

I look forward to the discussion!

It’s the free space of the pool that matters. For best performance and to avoid excessive fragmentation, keep the used space of the pool below 80% when using iSCSI.