XOA Custom Ubuntu Template - What am I missing?

1

Hey all,

I’ve been hitting my head against a wall for a week and don’t seem to be any closer to a solution, so I’m turning to my friends in the community.

I’m trying to create a basic custom template of Ubuntu 22.04 in Xen Orchestra/XCP-ng for use with cloud-init/cloud config drive.

Steps taken to create template:

  • Create base VM with specs I want the template to have
  • Install Ubuntu from ISO using mostly default settings (details below)
    • Disable LVM on filesystem configuration
    • Leave OpenSSH disabled (my understanding is cloud-init should handle SSH enablement)
    • Set temporary hostname, username: ubuntu, set temporary password
    • No additional packages selected for installation
  • With installation complete I apply my customizations (detailed below, only includes guest tools install)
    • Install XCP-ng guest tools from disk (not available via repo yet).
  • Reboot VM
  • Cloud-init prep
    • “dpkg-reconfigure cloud-init”, deselect everything except ConfigDrive
    • “systemctl stop cloud-init && systemctl enable cloud-init && systemctl start cloud-init” (to re-apply cloud-init config and ensure it is enabled)
    • “cloud-init clean --log”
  • Shutdown VM and convert to template

These are the steps I’ve found in every guide to creating custom templates, no additional steps are taken other than those listed above. Newly created VMs from this template show the temporary hostname is still there, the temporary user account is still there, and none of the cloud config or network config items have been applied. If I import the cloud ready OVA image from the Canonical site, convert it to a template, then create a VM from the template with my config files, all config is applied and the VM works exactly as expected, so I know the issue does not lie with my config files, it’s something to do with the custom template I’m creating. I would just use the cloud ready OVA template, except that I would have to install guest tools after deployment for each VM which is exactly what I’m trying to avoid.

Clearly I’m missing something here, but what? I’ve read the cloud-init documentation, countless guides and videos to creating custom templates, and each test has the same result. I’m happy to provide additional details, but the steps listed above are inclusive of everything I’ve done so far.

Any assistance provided or even pointers in the right direction are greatly appreciated! Thanks in advance!

2

Unless there is something specific you need in your setup then I would just download the official cloud-init image from ubuntu.

https://cloud-images.ubuntu.com/releases/22.04/release/ubuntu-22.04-server-cloudimg-amd64.ova

3

Hey @xMAXIMUSx,

Thanks for the reply!

In the original post I mentioned that I’ve downloaded the official cloud ready image from the Canonical site (i.e. from Ubuntu’s site) and it works great! That’s how I know my cloud config files work as expected. However, because the cloud ready image is bare bones I have to manually install the XCP-ng guest tools from CD because the guest utilities have not made it into Ubuntu repo’s for anything newer than 20.04 as of yet. $Boss wants everything migrated to clean installs of 22.04 ASAP (for some reason). If it were only a few VM’s it wouldn’t be a big deal, but I’m looking at 40-50 VM’s needing to be created from this template, so when you scale to these numbers, having to install guest tools from CD on each one isn’t really an efficient option. I have to imagine there is a better way. Preferred option would be a custom template created from scratch, it just seems like I’m missing a step; the equivalent of sysprep on a Windows machine. I thought cloud-init clean would be it, but doesn’t appear to be the case

4

I have not watched it, but Jay has a video on CloudInit. I know there were lots of quirks he ran into about how it’s handled which is why he did a video about it.

5

Thank you for this; not sure why I didn’t look to Jay in the first place!

I got it working 99.9% of the way. The only issue I’m running into now is that if I remove the config disk created by XO with my user and network data, the VM refuses to proceed past the init-local stage of boot (that could even be by design, idk). But if needing to keep a 10MB config disk for each VM created from the template is all that’s required then I can live with it.

Documenting the exact steps I took to create my template below for the next poor unfortunate soul. Some steps may not be necessary, but they didn’t hurt.

  1. Create base VM with specs you want the template to have
  2. Install Ubuntu from ISO using mostly default settings (details below)
    • Optionally: disable LVM on filesystem configuration (keeping it enabled should be fine, but does not suit my needs)
    • Leave OpenSSH disabled (my understanding is cloud-init handles SSH enablement)
    • Set temporary hostname, username: ubuntu, set temporary password
    • No additional packages selected for installation
  3. Configure after initial boot:
    • Log in as temporary ubuntu account
    • Set root password: “sudo passwd root”
    • Log out of temporary account and log in as root with newly created password
    • Delete temporary account: “userdel -r ubuntu”
    • Purge cloud-init: “apt remove --purge cloud-init”
    • Update and upgrade apt: “apt update && apt full-upgrade -y”
    • Insert guest-tools disk and install XCP-ng Guest Tools (unmount and remove disk once complete)
    • Install additional cloud-init packages: “apt install cloud-initramfs-dyn-netconf cloud-initramfs-copymods cloud-initramfs-growroot” (some of these may be pre-installed, including all for reference)
  4. Reboot VM
  5. Apply any additional customization then reboot
  6. Cloud-init prep
    • Reinstall cloud-init: “apt install cloud-init”
    • Ensure cloud-init is enabled: “systemctl enable cloud-init”
    • Remove/clear files
      • Installer config files
        • rm /etc/cloud/cloud.cfg.d/99-installer.cfg
        • rm /etc/cloud/cloud.cfg.d/subiquity-disable-cloudinit-networking.cfg
        • rm /etc/netplan/00-installer-config.yaml
      • Temp directories: “rm -rf /tmp/* && rm -rf /var/tmp/*”
      • SSH keys: “rm -f /etc/ssh/ssh_host_*”
      • Machine ID and hostname files: “truncate -s 0 /etc/hostname && truncate -s 0 /etc/machine-id”
    • Modify /etc/cloud/cloud.cfg > comment out all lines that don’t roll up to a defined module
    • Autoremove packages and clean apt: “apt autoremove && apt clean”
    • Run cloud-init cleanup: “cloud-init clean && cloud-init clean --logs”
  7. Disable root password: “passwd -l root”
  8. Shutdown VM
  9. Make any changes to the VM settings you may want included in the template (networking) and convert to template
6

Cloud-Init I believe is read from at each start up so you will need to keep that drive connected as it contains all of your cloud-init configs for boot.

Also, you may be able to work around some of this using the runcmd option to run the commands needed to install from the disk.

7

@gsrfan01 good call on the config drive. Like I said, if all I need to do to make this work is keep those config drives connected, no big deal.

I’m actually using the runcmd module for a few other tasks and it’s working great! I don’t know that this would work the way you’re expecting for the guest tools install from disk. I’d need to make sure the disk is inserted each time a new VM boots which seems to interfere a bit with cloud-init (causes delay in my testing) and kind of defeats the purpose of having this be as ‘no-touch’ deployment as possible. For me, baking guest tools into the base image is working just fine :slight_smile:

Hopefully xe-guest-utilities will make it’s way into the Ubuntu 22.04 repositories in the future so I’ll be able to just use the Ubuntu provided cloud ready image! I’ll keep my fingers crossed, but I’m not holding my breath lol @olivier.lambert :eyes: