I went to update my XCP-NG system and I’m having some problems, can’t migrate the only VM running, and I was having problems installing the new patches. I stopped the VM, then shut down all three hosts, and shut down my Truenas. Brought them all back up in reverse order. VM can start and run as normal, but still can’t migrate it. XO also still shows 5 patches but when I look at the pool or individual hosts there are none there. Also going to the servers and terminal a “yum update” shows no packages.
Digging deeper I found the following error:
March 14, 2022, 4:39 PM (last month) POOL_CPU_FEATURES_DOWN The pool-level CPU features have changed. Some features have gone away.
That makes me think my very old processors are finally unsupported. Anyone else seeing this or might be able to offer suggestions?
Likely. I never used XCP-NG so take what I say with a grain of salt. My server with a similar processor, x5570 is not supported with the latest VMware ESXi. They are just old processors in general so not having new features is to be expected.
I’m actually thinking this may be a secure boot issue that I had a couple months ago. I’m going to try the fix posted in that thread and see what happens when I turn secure boot back on.
The funny thing is that if I go to the local monitor/keyboard for each host, I can move this VM to any of my three servers, just like I could do with XO before. I’ll need to keep digging before I shell out more money on slightly newer servers. Newest I can afford would be Xeon E5-26xx v2, and even then getting a decent supply of RAM will be difficult.
I applied new certificates to get secure boot back, now the VM boots with secure boot again. But still can’t migrate the stupid thing. Going to build a new VM and see if I can move it around, something might be broken in this older install. Here is the error I get, probably going to need to post this on the XCP-NG forums, and keep pricing out newer servers.
“message”: “operation failed”,
“stack”: “XoError: operation failed
at operationFailed (/opt/xo/xo-builds/xen-orchestra-202204141530/packages/xo-common/src/api-errors.js:21:32)
at Object.migrate (file:///opt/xo/xo-builds/xen-orchestra-202204141530/packages/xo-server/src/api/vm.mjs:471:3)
at Api.callApiMethod (file:///opt/xo/xo-builds/xen-orchestra-202204141530/packages/xo-server/src/xo-mixins/api.mjs:314:20)”
The first error makes it look like the VM has an invalid GUID, but kind of guessing here.
I fresh VM is able to be migrated without issue… Looks like the original VM is broken in some way.
We run XCP-ng for all our internal systems and have a few clients that use it too. We can not live migrate running guest VMs between XCP-ng hosts with different CPU architectures. Our work around is to shut the guest VM down and then Export the VM to the new host. Live migration works only with similar host hardware and other similar configurations.
Example 1: Trying to live migrate a guest Debian based VM running on a Dell R710 to a new Dell R720 XCP-ng systems with the same XCP-ng versions and patch levels does not work, and the related error message says something to the effect that the CPUs are different.
Example 2: Trying to live migrate a guest Debian based VM running on a Dell R720 to another R720. Both of the Dell R720s have the same CPUs but one has a 10G NIC with the running VM configured to use the 10G NIC, while the other R720 does not have a 10G NIC. The migration fails here too.
Not sure where you are located at, but there are some cheaply priced servers and things like memory here in FL. Just picked up (2) R720s for under $100.
I’m certainly not seeing R720 for under $100 each unless they have no cpu and no ram.
These are 3 hosts that are identical, migrating worked until a couple of updates ago. I just need to dig into it and see what happened. Might be a conflict between the drivers, I seem to recall that if you have some Citrix and some XCP-NG drivers, this might happen. So I need to go through that VM and sort it out, especially since the new VM moves around without issue (same Server 2022 OS).
Eventually I’ll move up to newer hardware, but I’m seeing around $200 each with E5-2600v2 processors and different amounts of RAM. Not a huge priority right now as I have bigger projects going on, and none of those projects are replacing my old production servers.