Hi,
Anyone here having experimented with using fast SSDs (maybe optanes) for metadata, with raidz2 vdevs, and have made some benchmarks? It looks like it could be a killer feature, provided the requirements for redundancy is met (loose the vdev, and loose the pool - just like “normal” vdevs).
Being so new a feature in TrueNAS finding benchmarks isn’t easy yet - the best I’ve seen have been the impact on DDT use (quite impressive).
I am also interested in learning more about the ZFS special allocation.
You may have already seen this, but Tom made a brief comment about it in another thread indicating that he plans to make a video on the topic. The thread he linked over on the L1 forum has some interesting discussion on the topic.
I am interested to see how worthwhile this would be for someone wtih a quite simple FreeNAS system. Not sure if this or a L2ARC/ZIL would be a better use of resources. I currently have none of the aforementioned items- simply a 2 disk mirror.
Saw his comment and I’m looking forward to the results 
Tom has some nicer toys to play with than many of us 
For a simple setup like the one you describe I wouldn’t use it - maybe ZIL/L2ARC if the numbers/workload adds up.
For my personal setup I’ve got an Optane SLOG in front of a 6disk raidz2, because I need sync writes (photos so don’t want to risk anything). I don’t need L2ARC right now, but persistence looks promising, so might bump the server to max RAM and add L2ARC 
The major thing to remember with special-vdevs are the need for redundancy. Loose the metadata vdev, and your pool is toast.
The setup I’m considering using special-vdevs for is one where there’s a lot of files pr. directory and a lot of updating of these. The pool will be 3x 6disk raidz2. Persistent L2ARC might also be able to help a lot in such a scenario.
Agreed. This sounds like a project for when I eventually revamp my TrueNAS system.
At this point I do not want to dedicate 2 of my 4 SATA ports to a special device vdev. My 2x4TB mirror has finally become nearly full and I will likely need to add another vdev to my pool shortly.
Agreed maybe an L2ARC/SLOG device would be an interesting thing to experiment with instead.