Weird ESXI Problem

Computer Hardware & Server Infrastructure Builds
1

Hi Guys,

I have a ESXI host (Dell T620) that was being working away from approx 2 years without an issue.

Last week I ran into a problem:

loading /b.b00
loading /jumpstart.gz
loading /useropts.gz
loading /features.gz
loading /k.b00
loading /vc_intel.b00
loading /procfs.b00
loading /vmx.v00
CRC error during decompression. Received CRC (0x8ae2a503) != calculated CRC (0x886cc395)
gzip_extract failed for /vmx.v00 (size 0): CRC error
Error 20 (CRC error) while loading module: /vmx.v00
Compressed MD5: 0000000000000000000000000000
Fatal error: 28 (CRC Error)

and during a subsequent reboot I saw:

Mutibit ECC errors were detected on the RAID controller
If you continue, data corruption can occur.

I Download a clean ISO of 7.0U2B and installed on clean USB drive on the internal port.
Issue are resolved, no data corruption on the datastores.

This morning I arrive at the office and users have no access to the internet (pfsense). I find some VM’s are frozen and not accessible, even via host web UI. Try a soft reboot and it doesn’t respond. I power cycle via power button and it hangs on reboot:

nfsclient loaded sucessfully

I try to replicate fix from last week and the installer freezes:

Loading /EFI/Boot/boot.cfg
UEFI Secure Boot is not enabled
Failed to load crypto64.efi : not found
Falling back to internal crypto suite
Loading /b.b00
Fatal error: 15 (Not Found)
  • USB drive (Internal Port) was swapped and the install on a clean drive was again not sucessful.
  • Life Cycle Controller was in Maintenance Mode
  • USB driver were placed into the rear of server chassis, and install progressed to user options screen.
  • Attempted to boot from original USB, originally placed in the internal USB port from a rear port. Boot stalled as before.
  • Commenced install of 7.0U3B which was successful.

I guess that these errors are something to do with internal USB port, but I am completely guessing. Has anybody ever seen anything like this, or have any idea of what I can do to narrow down the root cause. It is working now, but I would like to try understand the cause to prevent it happening again.

Thanks!

2

I’ve had issues running ESXi on a USB drive in the past. Now I will only use HDD. SD Cards seem to work better too.

3

Perhaps not directly related but you should be aware VMWare has pulled the 7.0 U3 release due to security concerns.
" "Dear Customer,
As you may be aware, we have identified critical issues following the vSphere 7.0 U3 GA release, leading to two express patches. After further review, additional resolution complexities have come to light which we have been working around the clock to resolve, test and validate. To protect you from further impact and reduce the potential for further complexity until we have a properly vetted path forward, the decision has been made to put a hold on the full ESXi 7.0 U3 release (incl. ESXi U3, U3a and U3b) and vCenter Server U3b, removing it for download at this time. vCenter Server 7.0 U3 GA and U3a will remain available as a viable upgrade path and ESXi host backwards compatibility remains unchanged.
For now, we are asking you to hold on any moves to ESXi 7.0 U3. Please reference the following FAQ (VMware Knowledge Base) which goes into more detail and also gives guidance for those who have already updated in any form. For transparency and awareness, a banner is also being maintained directly on the download page. .
We are committed to getting this fully resolved in an upcoming release.