XCP-NG and Xen Orchestra Backup

My Backups just stopped working. “Job canceled to protect the VDI chain”. I have waited days and it still has not started working again. I don’t know what to do at this point. Here is a log from one of the backups. Any help would be greatly appreciated.

{
“data”: {
“mode”: “full”,
“reportWhen”: “failure”
},
“id”: “1554821833235”,
“jobId”: “970a64e5-88b2-48b1-9007-160aa935e096”,
“jobName”: “XOA Only”,
“message”: “backup”,
“scheduleId”: “872df4d4-56f0-459b-a44a-c2352062fd71”,
“start”: 1554821833235,
“status”: “skipped”,
“tasks”: [
{
“data”: {
“type”: “VM”,
“id”: “8598cee8-d5cd-8c09-b2c2-dc7178552e82”
},
“id”: “1554821833238”,
“message”: “Starting backup of XOA. (970a64e5-88b2-48b1-9007-160aa935e096)”,
“start”: 1554821833238,
“status”: “skipped”,
“end”: 1554821833242,
“result”: {
“message”: “unhealthy VDI chain”,
“name”: “Error”,
“stack”: “Error: unhealthy VDI chain\n at Xapi._assertHealthyVdiChain (/opt/xen-orchestra/packages/xo-server/src/xapi/index.js:859:14)\n at Xapi._assertHealthyVdiChain (/opt/xen-orchestra/packages/xo-server/src/xapi/index.js:863:9)\n at Xapi._assertHealthyVdiChain (/opt/xen-orchestra/packages/xo-server/src/xapi/index.js:863:9)\n at /opt/xen-orchestra/packages/xo-server/src/xapi/index.js:872:11\n at arrayEach (/opt/xen-orchestra/node_modules/lodash/_arrayEach.js:15:9)\n at forEach (/opt/xen-orchestra/node_modules/lodash/forEach.js:38:10)\n at Xapi._assertHealthyVdiChains (/opt/xen-orchestra/packages/xo-server/src/xapi/index.js:871:12)\n at /opt/xen-orchestra/packages/xo-server/src/xo-mixins/backups-ng/index.js:1012:9\n at Generator.next ()\n at asyncGeneratorStep (/opt/xen-orchestra/packages/xo-server/dist/xo-mixins/backups-ng/index.js:82:103)\n at _next (/opt/xen-orchestra/packages/xo-server/dist/xo-mixins/backups-ng/index.js:84:194)\n at /opt/xen-orchestra/packages/xo-server/dist/xo-mixins/backups-ng/index.js:84:364\n at Promise._execute (/opt/xen-orchestra/node_modules/bluebird/js/release/debuggability.js:313:9)\n at Promise._resolveFromExecutor (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:483:18)\n at new Promise (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:79:10)\n at /opt/xen-orchestra/packages/xo-server/dist/xo-mixins/backups-ng/index.js:84:97\n at BackupNg._backupVm (/opt/xen-orchestra/packages/xo-server/src/xo-mixins/backups-ng/index.js:926:19)\n at BackupNg. (/opt/xen-orchestra/node_modules/golike-defer/src/index.js:60:48)\n at /opt/xen-orchestra/packages/xo-server/src/xo-mixins/backups-ng/index.js:640:27\n at Generator.next ()\n at asyncGeneratorStep (/opt/xen-orchestra/packages/xo-server/dist/xo-mixins/backups-ng/index.js:82:103)\n at _next (/opt/xen-orchestra/packages/xo-server/dist/xo-mixins/backups-ng/index.js:84:194)\n at /opt/xen-orchestra/packages/xo-server/dist/xo-mixins/backups-ng/index.js:84:364\n at Promise._execute (/opt/xen-orchestra/node_modules/bluebird/js/release/debuggability.js:313:9)\n at Promise._resolveFromExecutor (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:483:18)\n at new Promise (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:79:10)\n at /opt/xen-orchestra/packages/xo-server/dist/xo-mixins/backups-ng/index.js:84:97\n at handleVm (/opt/xen-orchestra/packages/xo-server/src/xo-mixins/backups-ng/index.js:621:20)\n at /opt/xen-orchestra/@xen-orchestra/async-map/src/index.js:32:16\n at Promise._execute (/opt/xen-orchestra/node_modules/bluebird/js/release/debuggability.js:313:9)\n at Promise._resolveFromExecutor (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:483:18)\n at new Promise (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:79:10)\n at /opt/xen-orchestra/@xen-orchestra/async-map/src/index.js:31:6\n at /opt/xen-orchestra/node_modules/lodash/_baseMap.js:17:23\n at /opt/xen-orchestra/node_modules/lodash/_createBaseFor.js:17:11\n at baseForOwn (/opt/xen-orchestra/node_modules/lodash/_baseForOwn.js:13:20)\n at /opt/xen-orchestra/node_modules/lodash/_createBaseEach.js:17:14\n at baseMap (/opt/xen-orchestra/node_modules/lodash/_baseMap.js:16:3)\n at map (/opt/xen-orchestra/node_modules/lodash/map.js:50:10)\n at asyncMap (/opt/xen-orchestra/@xen-orchestra/async-map/src/index.js:30:8)\n at /opt/xen-orchestra/packages/xo-server/src/xo-mixins/backups-ng/index.js:696:23\n at Generator.next ()\n at asyncGeneratorStep (/opt/xen-orchestra/packages/xo-server/dist/xo-mixins/backups-ng/index.js:82:103)\n at _next (/opt/xen-orchestra/packages/xo-server/dist/xo-mixins/backups-ng/index.js:84:194)\n at tryCatcher (/opt/xen-orchestra/node_modules/bluebird/js/release/util.js:16:23)\n at Promise._settlePromiseFromHandler (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:512:31)\n at Promise._settlePromise (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:569:18)\n at Promise._settlePromise0 (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:614:10)\n at Promise._settlePromises (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:694:18)\n at Promise._fulfill (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:638:18)\n at PromiseArray._resolve (/opt/xen-orchestra/node_modules/bluebird/js/release/promise_array.js:126:19)\n at PromiseArray._promiseFulfilled (/opt/xen-orchestra/node_modules/bluebird/js/release/promise_array.js:144:14)\n at Promise._settlePromise (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:574:26)\n at Promise._settlePromise0 (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:614:10)\n at Promise._settlePromises (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:694:18)\n at _drainQueueStep (/opt/xen-orchestra/node_modules/bluebird/js/release/async.js:138:12)\n at _drainQueue (/opt/xen-orchestra/node_modules/bluebird/js/release/async.js:131:9)\n at Async._drainQueues (/opt/xen-orchestra/node_modules/bluebird/js/release/async.js:147:5)\n at Immediate.Async.drainQueues (/opt/xen-orchestra/node_modules/bluebird/js/release/async.js:17:14)\n at runCallback (timers.js:810:20)\n at tryOnImmediate (timers.js:768:5)\n at processImmediate [as _immediateCallback] (timers.js:745:5)”
}
}
],
“end”: 1554821833242
}

https://xen-orchestra.com/docs/backup_troubleshooting.html

What type of backup are you trying to do? Delta, full, or any other?

I personally have had a large problem with the delta backups – they work for awhile and then one day a process will freeze and the job will never complete. For whatever reason, most of the time the xcp-ng server kill kind of lock up as will the VM. It’s really annoying.