Defective Boot device and System Restore

Computer Hardware & Server Infrastructure Builds
1

My problem is quite complex.

  1. I had to reinstall Truenas Core on a device. The system reported for some time that the NVMe was unreliable. Of course, I backed up the configuration of my system. Not alone, the boot environment was defective. Likewise, a hard disk of a mirrored pool showed errors and was already taken offline before the renewal. The replacement of this hard disk I wanted to do of course after the renewal of the boot device.

  2. before restoring my backed up configuration, I import the pools recognized by the system. I have created two mirrored pools in my system. For the first pool, the import took a very short time. Meanwhile, when importing the pool whose one hard drive was detected as faulty and degraded, the import takes 3 hours. Well, in this pool there are directories created, in which a very large amount of files is stored. Is the new hard disk I installed undergoing a resilver process at the same time?

  3. I have not encrypted the hard drives. The pools do not have a padlock. I assume that the data is not encrypted, and I do not need a decryption key.

  4. what dangers are lurking for me? What do I have to watch out for? Is the recovery of my system at risk?

Thank you very much for your support

2

You can see the status of the resilver under “Pools” then click the gear icon and choose “status”

3

OK,
I must have expressed myself in a misleading way.
After backing up my configuration, I reinstalled the system by creating a new, error-free bootdevice.
Afterwards I exchanged the defective disk of the pool, consisting of two mirrored disks, for a new one.
After rebooting this system, I tried to restore my saved configuration without delay. I have never been able to restore the pools. After importing the backed up configuration, neither the disks were displayed, nor the dashboard.

So before restoring the backed up system, I now started importing the two existing pools. The first pool was recognized. The import of the second pool sent in nirvana.
The next time I tried to restore the system, I left it at the import of the first pool and then imported the backed up configuration.
After that the system stopped with this message:

photo_2023-09-16_13-28-45
photo_2023-09-16_13-28-451280×720 207 KB

metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 3331284, spa CKSpiegel, vdev_id 0, ms_id 41, smp_length 14832, unflushed_alloc 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 43014 ms, loading_time 21 ms, ms_max_size 8589893632, max size

I have already started the attempt NOT to interrupt this job. But the display did not change after a long wait of 2 days. It should therefore have been stuck.

What should I do best? reinstall the defective hard drive and then first restore the system which had a disk offline? I then interrupted the above process with CTRL+C each time to at least get more info.

What is the best way to proceed? Can I read the data in any other way?

And I can’t get to a section under storage, shares, disks or pools unfortunately. All the windows remain without content.

Your help is very welcome.

4

The Task Manager window shows the following info:

photo_2023-09-16_13-28-454
photo_2023-09-16_13-28-454720×1280 180 KB

And the percentage display does not change.

5

I am assuming the second pool you are trying to import is corrupt which also means you will need to restore the data on those drives from backup.

6

Is this necessarily the case? This is a pool that consists of two mirrored hard disks.
I must be able to read the data from the intact hard disk somehow. Otherwise mirroring makes no sense at all.
And then the system generates the following message:

Current alerts:

  • ck-truenas.cadkon.local had an unscheduled system reboot. The operating system
    successfully came back online at Sat Sep 16 15:06:02 2023.

  • smartd is not running.

  • Attempt to connect to netlogon share failed with error: [EFAULT] could not
    obtain winbind interface details: Winbind daemon is not available. could not
    obtain winbind domain name! failed to call wbcPingDc: Winbind daemon is not
    available…

  • Domain validation failed with error: [EFAULT] Failed to retrieve machine
    account status: Failed to set machine account:
    NT_STATUS_CANT_ACCESS_DOMAIN_INFO

  • IPMI SEL low space left: 16 bytes (99% used).

  • Pool CKSpiegel state is OFFLINE: None

  • Pool BigSpender state is OFFLINE: None

  • Pool Cache state is OFFLINE: None

  • New ZFS version or feature flags are available for pool(s) CKSpiegel,
    BigSpender, Cache. Upgrading pools is a one-time process that can prevent
    rolling the system back to an earlier TrueNAS version. It is recommended to
    read the TrueNAS release notes and confirm you need the new ZFS feature flags
    before upgrading a pool.

  • Communication with UPS ups lost.

7

The short answer is that if there is too much data corrupted the pool can not be mounted.
Wendell from Level1Techs dives into this topic here: