I am having trouble with a ZFS pool in TrueNAS Scale. I’m relatively new to TrueNAS, I have a lot of technical experience but I am completely self-taught, so I am sure there are large gaps in my knowledge.
I Initially built the NAS about 18+ months ago.
Dell R720
64 GB ECC Memory
(2) SSD Boot Drives
(5) Seagate Exos X16 16TB SAS Drives in Raid Z2
I then built a low powered Ryzen 3 machine with (7) WD Red Pro 10TB Drives in a Raid Z1 setup. That machine is at my parent’s house, and the Dell has replication tasks set up to duplicate the data to the Ryzen 3 machine.
Life became very busy, so I haven’t monitored the machines much at all since then, except a number of months in the Dell machine started having network issues. With being so busy I simply swapped the drives into a Ryzen 5 machine I had, and everything worked well.
Recently, on a Monday morning the family was trying to access work and school files and the NAS was not available. I couldn’t access the Web Interface, and it wasn’t responding to SSH. Rebooted the machine and nothing.
Digging deeper I found during startup the machine would hang. I eventually found that if I unplugged the pool drives it would boot and I could access the Web interface.
With the drives installed it would fail with the error. [end Kernel panic – not syncing: error 52 from bpobj_iterate]
Once booted without the Data pool, one of the boot drives had errors. I ended up replacing both boot pool drives.
I was running TrueNAS Scale 24.04.2.5
I found that if I hot swapped the drives into the system after it was booted the drives would be discovered. It recognized that they were part of the pool named “Data”, but it was listed as (Export). Under Storage – Import Pool no pools were found to import.
I replaced a number of parts.
LSI Cards (tried multiple ones)
Power Supply
Breakout Cables
The same problem persisted
I Installed the boot drives and pool drives in a different system. Same results.
I then installed TrueNAS Scale 25.04.2.6. I did not restore my old configuration. At this point the system would boot with all drives installed. I think there is some sort of problem with the config. It saw all the drives, with the pool labeled as (Export). Now though under import pool it saw the Data pool as available.
When trying to import the pool it would bring up a progress bar and then after a while reboot the system and the pool would not be imported.
Once I was booted into the system with all the drives connected, I ran a s.m.a.r.t. tests on the drives. All the drives past the short test. I then ran a long test and one of the drives did have an error on a long test.
I found instructions online that I might have to import the pool in the terminal. I did that, with no luck. I then tried to do a force import. I used the command “sudo zpool import -f Data”. I also tried it with the ID number of the pool. No luck. The system would work on something for a while, reboot and the pool is still not imported.
I then read that I may need to upgrade the pool. Before doing that, I thought of reinstalling 24.04.2.5 and not restoring my old config. I did that, it booted with all drives connected, but when trying to import the pool it acted the same as 25.04.2.5.
I attempted to upgrade the pool but none of what I thought were the correct commands were correct.
I do have most of my data on the backup NAS. I’m doing all of this as one important folder was not backing up properly even though it said the backups were successful. Secondly, we put some other important files on the NAS on Sunday and then it failed before the backup Sunday night.
Does anyone have any suggestions on additional steps to take to get the pool back online?
If I cannot get the pool back online. Is there a way to recover the lost data? The data is not encrypted.
Thank you for any help.