A few months ago, I was entirely happy with my homelab set-up: a Ubuntu server running docker, and a bunch of docker-compose stacks to provide:
- email server
- three websites
- photo gallery
- database server for gnucash
- jitsi conferencing
- home automation (home assistant, esphome, app daemon)
- file server (samba and nfs)
- iSCSI server
and probably some more I’ve swept under the carpet.
Anyhow, I stumbled across LTS’s Youtube videos on xcp-ng, Proxmox and TrueNas and thought “there’s something interesting to play with”, also know in our household as “breaking the email”.
Initially I decided on xcp-ng as my homelab, with TrueNAS as a virtual machine to provide file services, and a Debian virtual machine running docker to provide the services.
I discovered one big gotcha here, which is probably so obvious that it doesn’t appear in the YouTube advice. It’s a bad idea to make your xcp-ng dependent on a virtual machine it is running. But you kind of want to do that because xcp-ng isn’t really a filesystem manager. It supports zfs, although only weakly, if you want to use zfs snapshots and replication you have to do this from a ssh command line. I want to mount an isos filesystem from the TrueNas server.
Ultimately I decided I needed the hypervisor to have no dependencies on the virtual machines it runs. So I switched to TrueNas running on the bare metal, providing zvol disks and nfs exports to virtual machines. TrueNas don’t have anything like the richness of Proxmox or xcp-ng defining and running virtual machines. But it’s OK for me and my single-machine home-lab.
Actually, it’s not a single-machine home-lab because I also have a backup server that is woken and put back to sleep from cron jobs on TrueNas, and acts as a replication target.
I hope these experiences are interesting/helpful to others in a similar situation to me. I’m sure that’s a lot different to the professionals out there with racks of equipment.
Adrian Stephens, Cambridge UK, retired software engineer and now full-time meddler and fettler