Xen/ XOA Cloud-init issues

bjerrecs · May 12, 2020, 4:30pm

Dear Forum.

I’m currently testing the cloud-init/ Cloud Config of Xen Orchestra on my XCP-NG 8.0 server.

But I’m having some trouble getting my VM’s to apply the changes in the Cloud Config box in XOA.

I have made a new VM from the default templates.
Installed Ubuntu 20.04 LTS server edition.
Run the apt-get install cloud-init cloud-initramfs-growfs
I have not made any changes to the /etc/cloud/cloud.cfg
I have run the dpkg-reconfigure cloud-init after installing cloud-init

when selecting with options to pick from I chose these:

Are these the correct boxes to check?

This is what I have in the XOA Cloud Config tab:

And this is the yaml, just in text

#cloud-config
growpart: {mode: 'on'}
locale: C.UTF-8
preserve_hostname: false
resize_rootfs: true
ssh_pwauth: false
users:
- gecos: itvadmin
  groups: [sudo]
  lock-passwd: false
  name: itvadmin
  passwd: PASSWORD HASH
  shell: /bin/bash
- gecos: sb
  groups: [sudo]
  lock-passwd: false
  name: sb
  passwd: PASSWORD HASH
  shell: /bin/bash

any ideas why this is not just working?

I’m more than happy to send logs and other information if need’ ed

bjerrecs · May 13, 2020, 6:51am

@LTS_Tom This might be up your ally with XCP-NG and XOA, any ideas?

barendbotes · July 30, 2020, 8:29am

Have you had any luck with this?

barendbotes · July 30, 2020, 10:46am

I managed to get my Ubuntu 18 Template working…

After a clean install of Ubuntu 18 on Xen Orchestra/XCP-ng, I did the following(could be unnecessary):

Purged all cloud-init applications:
sudo apt purge cloud-init cloud-initramfs-dyn-netconf cloud-initramfs-copymods
reboot

Prepared my server with all my apps and Tom’s dotfiles.

Installed XCP-ng tools:
mount guest-tools

mount /dev/cdrom /mnt
dpkg -i /mnt/Linux/xe-guest-utilities_X.X.X-XXXX_amd64.deb
umount /mnt

Installed cloud-init again:
apt-get install cloud-init cloud-initramfs-growroot

Choose cloud-init source: NoCloud only

dpkg-reconfigure cloud-init

systemctl enable cloud-init

shutdown server and converted to template. Once done, I created a cloud config file:
#cloud-config
hostname: {name}
fqdn: {name}.domain.com
manage_etc_hosts: true
growpart: {mode: ‘on’}
preserve_hostname: false
resize_rootfs: true

and on the network side to remove ‘#’ from network: and version: 1

network:
  version: 1
#  config:
#  - type: physical
#    name: eth0
#    subnets:
#      - type: dhcp

created a new VM and set the name, resources and disks and confirmed that it all worked.

barendbotes · August 1, 2020, 8:59am

I forgot to add that you need to remove the cloud-init files after the purge:
sudo rm -rf /etc/cloud/; sudo rm -rf /var/lib/cloud/

and then reboot

p4-k4 · August 21, 2020, 12:26pm

@barendbotes hey mate - Struggling to produce the same success that you have here.

I’m using an Ubuntu Server 18.0.4 official live image for my template, prior to stumbling on your messages, I didn’t purge or remove the cloud-init files, not that it made a difference because I still had no success anyway. But even doing both of these, no results.

XO creates the source drive fine and the guest OS see’s it. Mounting the drive and looking in it contains the correct meta-data so that’s fine however it seems that the metadata is just not getting injected on boot. The logs for cloud-init show that it’s loading config from the installation (user created by installation etc). Nothing to show that it’s even looking at the Cloud config drive created by XO though.

Any clues?

barendbotes · August 21, 2020, 1:15pm

When spinning up the VM are you removing the “#” from the network config? I remember when creating a new template that I had to apt purge/remove the cloud-init install for some reason and then reinstall it only selecting “NoCloud” for it to actually work.

p4-k4 · August 21, 2020, 3:29pm

Hey mate thanks for getting back @barendbotes

I don’t even bother feeding it any network config since I can’t even get it to inject basic things like hostname or user etc. I’m using this same one…

#cloud-config
hostname: test
user: test
password: test
chpasswd: {expire: false}

Yeah I’ve tried to remove/purge then reinstall all cloud-init related packages, have also tried without purging/removing. I’m well over spinning up over 50 test VM’s now using multiple variations.

I hit up Oliver over on their forums he seems to think it’s nothing to do with XO.
What build of XO are you running? I’m on xo-server 5.65.0.

I mean, it’s a fairly straight forward process but for whatever reason, no matter how many VM’s I make, none of them let XO inject the cloud-init config. I’ve been at it for over 8 hours now

Is there anything you might have missed in your process above?

Thanks again

p4-k4 · August 21, 2020, 3:50pm

Hmmm seems I got it to work just thought I’d let ya know before you reply.

It’s either one of the following that caused the issue which I am testing now:

Network config MUST be set. I never done this until now or
I copied your line to purge exactly and noticed that the line to reinstall is missing one of the packages, so I added that.

Well, there goes bloody 8 hours of my day. I can finally take a bath and eat lol!

bjerrecs · August 28, 2020, 11:33am

@p4-k4 & @barendbotes

Thanks for sharing your findings.
I will try mine with the 20.04 image later this next month!

barendbotes · January 12, 2021, 7:58pm

Did you manage to get it to work with Ubuntu 20.04? I just can’t get it to work on it, it boots into initramfs. I have been too lazy to do a deep dive into it, I am still happy with my 18.04.5 template.

hansaya · June 17, 2022, 10:30pm

I got the network config to work with Ubuntu 20.04 but however I cannot get normal config(hostname, ssh keys) to work with nocloud. Then I enabled nocloud and ‘openstack’ but now only openstack works. Does anyone able to config the sshkeys as well as network config?

What I’m trying to do is use terraform to deploy K3S cluster. For that I need to be able to setup all the ssh keys and setup static ip’s so that I can easily deploy ansible to setup the cluster.

LTS_Tom · June 18, 2022, 11:41am

I have not used Cloud Init but Jay has a video on it here:

hansaya · June 21, 2022, 7:55pm

Thanks @LTS_Tom

I managed to get ssh keys and networking working with nocloud, I just had to start from scratch. This worked with Ubuntu Server 20.04 and 22.04.

However I cannot get the storage to work currently with terraform. xcp-ng goes and changes the size of the disk while VM is running but this leads to a unbootable disk in the vm. I have yet to find the cause of it. My guess is this has nothing to do with cloudinit but with how xcp-ng changes the size of the disk and leading to some sort of a corruption. Does anyone have any lead on expandable storage?

Thanks @barendbotes for your tips. This is what I did to get the cloud-init to work.

# Install Ubuntu 20.04 or 22.04 normally

# Mount guest_tools.iso and install the guest tools
sudo mount /dev/cdrom /mnt
sudo dpkg -i /mnt/Linux/xe-guest-utilities_X.X.X-XXXX_amd64.deb
sudo umount /mnt

# Update and reboot
sudo apt update && sudo apt upgrade -y
sudo reboot

# Purge existing cloud-init configs and packages
sudo apt purge cloud-init cloud-initramfs-dyn-netconf cloud-initramfs-copymods -y
sudo rm -rf /etc/cloud/; sudo rm -rf /var/lib/cloud/

# Start fresh with cloud-init
sudo apt update && sudo apt install cloud-init cloud-initramfs-growroot -y

# Reconfigure and only select nocloud
sudo dpkg-reconfigure cloud-init
sudo systemctl enable cloud-init

# Shutdown and convert into a template
sudo shutdown now

This is what I sent in through terraform. First cloud_config

#cloud-config
    hostname: k3s-master-2
    fqdn: k3s-master-2.example.com
    package_upgrade: true
    ssh_authorized_keys:
      - ssh-rsa AAAXXXXXXXXXXXXXXXX

Network config

#network:
  version: 1
  config:
    - type: physical
      name: eth0
      mtu: 9000
      subnets:
        - type: static
          address: 172.16.1.182/24
          gateway: 172.16.1.1
          dns_nameservers:
            - 172.16.1.1

hansaya · June 21, 2022, 9:16pm

I figured it out, seems like storage modification leads to deleting the old drive and creating a new(by terraform). I’m putting the VM on to a ext drive which is thin provisioning compatible, if that matters. I’m fairly new to xcp-ng but I cannot even figure out how to modify the storage allocation manually. What am I missing?

I publish what I got on terraform into GitHub - hansaya/xcp-ng-k3s-terraform: Deploy K3S/K8S cluster into xcp-ng using Terraform. If anyone interested in look into helping me out ironing out any issues.