UDM Pro V1.9.3 Unifi OS Corruption

I know how Tom always tends to not recommend Unifi Routers. I have had my UDM Pro for over a year and I started with v.1.8.5 and updated incrementally through all the intermediate releases finally to v1.9.3. The system has been stable overall and I have several switches and APs.

I also have several VLANs, I use L2TP for remote access, and I have used site to site VPN now and then. They all work great.

I have nightly backups for the SDN Controller and they work well.

Over time, I had become complacent regarding taking local backups.

A week ago, it looked like my ISP connection had gone down. As it turned out, my UDM Pro became unresponsive both via the web address and the LCD front panel. I had no choice but to cycle power.

After a reboot, the front panel indicated it was taking longer to start then expected and eventually went into a mode where it never completed a boot up and suggested recovery mode.

So, I initiated recovery mode. I eventually had to use recovery mode to reload the v1.9.3 Unifi OS firmware. When performed this recovery option (with a PC connected to UDM Pro Port 1) the device went into setup mode.

The setup prompted me for my Ubiquiti Cloud account and password. I entered them correctly and kept getting a login error for a bad password. This is because the setup process initiated from “recovery” prompted me for a username and password only and my Ubiquiti account was set for 2fa. In the initial setup, there is no way to perform a setup with a local account or bypass 2fa. To make matters worse, my 2fa key runs from my NAS on a LAN instance of Bitwarden which requires my NginX Reverse proxy which also loads its SSL certificates and requires WAN access. This “recovery” setup provides no default WAN access like out of the box setup and so I could not correct the problem.

Ultimately, I had to create a new Ubiquiti cloud account and restore my last SDN backup that I manually made in April which was on a PC. That gave me connection to the Internet and brought up my NginX reverse proxy and then my Bitwarden worked. I signed on to my original Ubiquiti cloud account using the now working 2FA and disabled 2FA for that account.

After that, I factory reset my UDM Pro and performed a new setup/install with my original Ubiquiti account and once it was up, I restored the April backup and worked from there.

One big takeaway from this activity was that the nightly backups get destroyed in the event that recovery mode is needed because Unifi OS is corrupted as in my case. The other huge problem is that you won’t have any network access to do 2FA unless your password manager is cloud hosted and you have another interface connection to get the 2FA pin. Even if you do that, recovery mode only prompts for username and password which renders an existing Ubiquiti account with 2FA enabled useless in the event you need to perform a recovery.

Also, in the Unifi OS Advanced setup screen there is an option for “Back up device”. In my previous configuration of 1.9.3, there was no restore option at all on this menu. It does say that Backup device is Beta, but I never had a restore option. Now, with my total wipe and reload, magically I have a restore device option on this menu.

Also, a friend of mine does not see this restore option on his 1.9.3 UDM Pro and pressing the backup button just results in an error. In my case, I am able to backup successfully with my new, newer 1.9.3 and I also have the restore button.

I don’t know if the restore button works and whether it restores things like the entries in the users screen. I am not about to try. Also, this message on the above screen is completely wrong and mostly ambiguous.

As far as I know, your SDN controller options are backed up only from the backup option inside the SDN controller. So, “Store backup file containing your Unifi application configurations” does not sound right. I agree that such a backup would store “general Unifi OS Settings and user roles”. I also think that the “Currently this feature is only supported by the Network Controller” is trying to tell the user that the SDN controller settings are backed up in the SDN. This statement of theirs so so massively semantically flawed that it would be better if it were not there at all.

Long and short of it is that I am back up and running.

I will tell you that prior to the lockup and corruption that occurred all at once, the internal storage was at about 75%. So, perhaps this constrained the Unifi OS and resulted in a crash/corruption. Also, the recovery mode has a disk check which I did first before anything else and it came back clean.

It may be that 1.9.3 has an issue that can result in sudden and total failure. Like I said, I have five VLANs, lots of routing rules, I have all the security features enabled and about 85 network clients at any one time. The UDM Pro is not overloaded by any means, but I fear that others may see this happen and I am hoping this posting may help someone else.

As Tom has indicated many times in his YouTube videos, the Unifi routers may lack more complex router functions needed for many business customers. I’d like to say that the Unifi model with its single pane of glass management coupled with its no pay software license support makes a compelling case as a fine integrated solution.

I find it easy to fix the myriad of problems that I have had, but article shows that the overall software design has not really passed muster as a true piece of production software and instead has overtones of a community supported open software development effort.

So, bottom line is user beware.

P.S. I had to cut all my screen shots except one to post.

1 Like