Currently setting up a home lab with cast off servers and eBay purchases Upgrade parts like SSD are not going to be supplied. I have a requirement for 6 concurrent VMs with OpenSuse or Ubuntu desktop, Wine running a Windows App. Plan is pfsense, XCP-ng and OpenNAS.
TLDR; I didn’t know a home lab would have so much potential for hardware upgrade problems. So far with UPS, Memory, Storage, BIOS, Firmware, Video hardware.
In the dim dark past I had gone with Synology boxes to back up our PCs. In discussions with a friend he had recommended FreeNAS as an alternative. Recently when discussing bare metal options XCP-ng had been recommended. A plan was hatched for a home lab of pfsense, XCP-ng and OpenNAS.
Over the years I’ve used Smoothwall, IPCop, IPFire and pfsense. So pfsense in a suitable box (Xeon, ECC, mirrored SSDs and Intel interfaces) already exists. My training and experience is electronics and networks but I haven’t touched servers since before Netware 2.12 (whatever version required the keycard) through 3 on HP LM series hardware. That translates as the 286/386/486 era.
In this town DL380 Gen6 and Gen7 servers are basically heading to the scrap heap while still perfect for a home lab. I’ve ended up with a HP UPS, multiple HP DL380s, (3x Gen6, 2x Ggen7, 1x Gen8) via auction houses and eBay. Due to COVID restrictions it was not possible to inspect items, all you could see was a poor quality photo with a description of ‘Contents of Rack’. Lucky dip time…
The HP UPS came with a 10 year old battery pack. It couldn’t hold charge. My guess is somebody substituted a dead pack when they were preparing the sale. Built my own pack. UPS batteries are a whole topic in themselves. Previously I’d been caught with sealed Lead-Acid batteries that were just about to hit their end of life. At end of life there is a steep drop in capacity. Or, cheap sealed Lead-Acid batteries that had greater AHr capacity but less capacity for liquid. End result was a shorter life. These days I only buy Yuasa or Varta. After Y2K there was a flood of APC 1200 UPS units sold through the auction houses. Currently the house has 7 or 8 UPS systems. The only maintenance performed has been feeding them sealed Lead-Acid batteries. Basically everything here has a UPS, even the TV (PC with DVB-T card) and some of the audio amps. I’ve had one APC UPS die which is not bad for units that were second hand $25 - $50 back in 2001. Hardware upgrades involving UPS capacity, batteries and pitfalls could be a topic.
Installed memory varied from 4 GBytes on up. One eBay server was dead on arrival due to a invalid mix of EEC memory. First step was to strip one Gen6 for parts and strip all ECC memory from all Gen6 and Gen7 servers. Discovered what a weird and varied mixture of parts had been used. Rebalanced memory so that the Gen6 servers have 20 GBytes for FreeNAS/TrueNAS and Gen7 servers have 88 GBytes for XCP-ng. Gen8 came with 64 GBytes.
This was the first hardware upgrade. It required knowledge of handling static sensitive items, ECC types, valid memory configurations from service manuals, etc.
All systems had been stripped of some drives given the missing drive carriers. Plus some drives were DOA. Stripped all drives from GEN6 and Gen7 servers and found a mix of SAS 72, 146 and 300 GByte drives with 2x SATA 500 GByte. End result is additional drives are going to be required.
Removed some SAS 72 GByte drives from their carriers and installed spare domestic (Samsung, etc.) SATA SSDs . Intention was to mirror the SSDs for the operating systems. One SSD of a pair was rejected by the Gen7 systems as over temperature. Gen6 and Gen7 share the same carriers but the Gen8 had moved to ‘smart’ carriers that are not compatible and likely to cause problems if I try to install non-HP drives.
Second upgrade is storage and there are all sorts of potential problems such as the carriers, drive firm ware, system configuration based for ZFS, RAID arrays, RAID controllers, HBA mode, etc…
At this point need to update BIOS to current recommended versions. Utilities restricted to systems under contract. Gen7 ILO version so old it requires installing an intermediate version before being able to load the recommended version. Again would like access to utilities limited to contracted systems. BIOS has been upgraded, yet to do ILO firmware.
Want to confirm the Windows application would run on this hardware. Dual Xeon processors are not supported by Windows 10 Home, requires Windows 10 Pro. Had a spare Windows 10 Pro retail pack from the dim dark past. Downloaded current Microsoft Windows 10 installer and launched into the install. Motherboard video hardware not supported by Windows 10 no provision for sound. Need low profile video card due to physical limitation of the available slot. Only cards in shops are NVIDIA Geforce GT 1030 variants. Added a mid-range Creative card for sound. EverQuest II ran perfectly with 24 cores (Hyperthreading enabled) and 88 GBytes RAM.
Next step was confirming the application runs under Linux. Installed a clean, plain Ubuntu 20.04 LTS Desktop. NVIDIA driver upgraded …Steam install grief, Wine install grief, PlayOnLinux Bash script grief… needs wrapper for new Bash Script… Bah humbug… Result was OpenSuse was installed then configured. These software trials and tribulations are irrelevant to hardware upgrade topic. Down load of data appears to be throttled which can also stall the video. Not a problem as frame rate above 60 fps.
Installed XCP-ng on the Gen8 without SSDs. Need to rebuild the 1Gbps wired network, have a clean install of the current stable pfsense and need to configure. Run multiple wiFi networks to isolate hardware that I cannot update or maintain e.g. air conditioner, solar inverter, etc. A network for home automation, and so on.
Totally off topic… I’m a licenced ham (amateur) radio operator and have an Icom 2m/70cm IDAS (NXDN) repeater, WIRES hardware, etc. I investigated the options for linking this repeater to an existing USA NXDN network via the internet. I could have obtained the appropriate approvals and obtained the required pieces of paper for a slice of 44.x.x.x address space, my ISP said they would be happy to configure and my Draytek modem had the smarts. Two catches: One, there could not be any commercial type traffic and my partner was working from home. That killed the project. Second trivial catch was BGP would be required and the minimum address space on offer would have been 255 addresses. Obviously no guarantees what will be available to other individuals. If you are a licensed Ham Operator it might be worth investigating for a home lab linked to amateur radio.