Booting natively encrypted ZFS?

Hey @LTS_Tom!

I am looking to move from my plain Arch on partitions install to a “fully” encrypted ZoL setup.

I wish I could encrypt the boot partition but that seems like a longshot for now.

Anyway, running a zroot with natively encrypted “root” and “home” partitions (datasets) should be feasible.

The problem is that the GRUB devs are not going to play ball with natively encrypted ZFS. GRUB straight up does not recognize natively encrypted ZFS partitions and as such cannot boot them.

Before I switch, I am waiting for the SIMD debacle to be resolved but I want to be ready to go once that is done.

So, do you know of any other way (systemd-boot, efi boot or whatever) that can boot a natively encrypted ZFS root?

I haven’t read the full documentation, but this guide for Debian Buster mentions setting up ZFS native encryption (not LUKS) and booting from it: https://github.com/zfsonlinux/zfs/wiki/Debian-Buster-Root-on-ZFS

Also the Arch wiki talks about this: https://wiki.archlinux.org/index.php/Installing_Arch_Linux_on_ZFS#Native_encryption

1 Like

What I Have recently done is setup a simple encrypted software raid1 for the root/boot using mdadm and then using ZFS on the data storage drives.

@rootpeer
Did you ever find a solution to this problem? I’ve researched on arch wiki and scoured internet. I don’t see anything that would suggest this is feasible right now. Grub guys don’t seem to want much to do with ZFS.

Hello @kevdog !

This is what I have understood so far:

You need an EFI partition for your “bootloader” of choice. That could be GRUB, rEFInd, systemd-boot or an EFISTUB kernel.

The optimal configuration is: enable the encryption flag on the pool, create unencrypted dataset and under that dataset create an encrypted /root and an encrypted /home dataset (e.g. /tank/root, tank/home). You could also instead create an encrypted dataset under the top unencrypted one and then create /root and /home there so that you only use your password once (so e.g. /tank/enc/root, /tank/enc/home). Tank being the unencrypted dataset.

If your kernel and initramfs is placed in the EFI partition, then GRUB (or any other method) can boot it normally and then the kernel will load the zfs driver (you have to tell it so in the GRUB config AFAIK) and will ask you to unlock the encrypted partitions.

The problem arises when you want to have your kernel and initramfs in an unencrypted ZFS /boot dataset. GRUB will not try to load the kernel from a ZFS filesystem that has the encryption flag (or some other flags) enabled, even if the kernel is not on an encrypted dataset.

Then you have these options:

  1. Modify GRUB so it ignores the fact that the pool has the encryption flag enabled
  2. Use rEFInd with a third-party ZFS EFI driver
  3. Possibly use your motherboard’s EFI shell, or modify its BIOS (VERY DANGEROUS) with the third-party ZFS EFI driver and boot the kernel directly

At the moment, I don’t think that there is a way to boot the kernel from an encrypted ZFS /boot dataset.

Here are some threads that may be of assistance:

https://github.com/zfsonlinux/zfs/issues/7153

1 Like

Yea I came across the first link researching the topic. Wasn’t extremely helpful. Maybe best option is installing zfs on top of dm-crypt and go that method. Kind of klunky but I think that is how its been historically done. I guess if you don’t require the /boot partition to be encrypted, then you could avoid the dm-crypt/LUKS route.

I wouldn’t think that this is the best option. You could natively encrypt the /root and /home datasets and have a /boot or /efi partition for GRUB and a third LUKS1 vfat or ext partition for your kernel and initramfs. That way GRUB can read and ask you to unlock the encrypted partition where the kernel is and then the kernel can read and ask you to unlock the ZFS /root and /home.

This obviously means that you lose the snapshotting and checksum ability of the kernel and initramfs (and might cause the wrong IO scheduler to load for ZFS - check that github thread) but I believe it is more “future-proof”, i.e. when in the near future we get some way to boot from encrypted ZFS, you won’t have to reconfigure the whole disk (which would be needed to remove LUKS) but just the /boot or /efi and the kernel partition.

That is what I would do but I do get to mess with, break and fix my systems quite often, so maybe my opinion should not be that trustworthy.

Anyway, good luck if you are going to try any of this!

This is what I was looking for… If anyone finds a way to boot from a ZFS root while /boot is part of the encrypted ZFS pool, then do let us know :slight_smile:
Thanks!

Hey! Unfortunately, after a lot of testing, I came to the conclusion that you cannot boot from pools with certain features enabled, either with GRUB or rEFInd.

The solution is simple. You create 3 partitions on each drive of the “system” pool. One partition is formatted as fat for the boot loader/manager, one partition is part of a special ZFS boot pool that contains the unencrypted kernel and initramfs and one partition is part of the ZFS root pool that can have native encryption (as well as any other feature/flag you want) enabled.

Basically, you need two ZFS pools to avoid booting the kernel from fat and possibly running into problems.

This means that you lose the ability to encrypt the kernel and initramfs (unless you use ZFS over LUKS but I have not tested that) but you have the ability to take snapshots and backup the boot pool with native zfs send/receive. Since you are booting from ZFS, you are also using the correct scheduler.

Here is an excellent guide on how to do this with Debian that can be adapted for your preferred Linux distro (I used it with CentOS 8): https://github.com/zfsonlinux/zfs/wiki/Debian-Buster-Root-on-ZFS

Basically, the problem for complete ZFS FDE right now is bootloader support. There have been discussions about porting FreeBSD bootloaders to Linux to possibly avoid the need for two pools, but native encryption has not landed on FreeBSD yet AFAIK, so native encryption of the kernel and initramfs is a long way away currently.

Edit: Many thanks to @brwainer for originally linking to the guide!