To add to @Astraea 's comments.
Correct - the only expansion method is to add vDevs or increase the drive sizes within an existing vdev.
@Nightwalker712 The first thing I did when setting up my first FreeNAS server was reading Tom’s fourm post on FreeNAS ZFS pools along with the documentation he linked at the bottom of his post. There is some really informative stuff in there that cleared up a LOT of my confusion on how ZFS works.
I think it’s very common that people avoid FreeNAS/ZFS because it isn’t easily expandable - but I disagree with the idea that it is not easy to expand. It’s very easy to expand as long as you are aware of the limitations in how you expand your storage.
I agree with you - for homelab purposes, I also do not wish to purchase loads of HDDs at once.
I am assuming you already have 4x 1TB disks that you’d like to use? I will lay out some options below that you could use with 4x drives. Please correct me if this assumption is incorrect. The below picture (taken from Tom’s post linked above) will be useful when understanding the below configurations.
- Mirrored pairs means multiple vdevs, each with 2x drives. In your case with 4x drives total, this would be 2x vdevs each with 2x drives. The ZFS Calculator from Wintelguy tells us that this configuration would provide 1.54TB of storage and still leave 20% free space.
- Mirrored pairs is essentially what some call “Raid-10” however ZFS does not technically use Raid-10.
- You can expand your ZFS pool by buying 2x drives at a time. Thus your next expansion might be adding 1x vdev to your pool consisting of 2x 2TB drives. This would result in a ZFS Pool with 3x vdevs: 1x vdev consisting of 2x 2TB drives and 2x vdevs each consisting of 2x 1TB drives.
- A configuration like this can sustain 1 drive failure per vdev. However mirrored drives re-silver MUCH faster than drives in a raidz2 configuration.
- Mirrored pairs is my preferred method of configuring ZFS Pools for home use because it is the least expensive method of slowly expanding the ZFS Pool while still offering reasonable protection from hardware failure.
- Eventually you could fill your HP server with 12 vdevs - each consisting of 2x drives. If all your drives were only 1TB - you’d have 11.5 TB of usable space. 1TB drives are usually not the best value for the dollar so I’d consider looking at 4TB drives instead. With 12x vdevs of 2x 4TB drives each, you would have 46TB of usable space.
- I think Raid-Z2 is a nice option for someone who can afford to buy 6 drives at once when expanding storage. Raid-Z2 has a minimum of 4 drives so you could put your 4x1TB into 1x raid-z2 vdev. This would provide you with 1.49TB of space.
- RaidZ2 can sustain 2x drive failures per vDev.
You could also use RaidZ1 or RaidZ3. In my opinion RaidZ1 does not provide ample protection from hardware failure. These options both require purchasing more than 2x drives to add a vdev to your pool which is not ideal for storage expansion in my opinion.
Well this turned into a wall of text, but hopefully it helps with your FreeNAS considerations. Happy to discuss further if you are interested.