Video Suggestion: Home Lab Server Hardware Upgrade Problems and Solutions

Site Feedback & Video Suggestions
1

I would like to see a segment on the pitfalls of old hardware and how to avoid painful experiences.

As a home lab I have some HP DL380 Gen6, Gen7 and Gen8 servers that I picked up cheap through auction houses and eBay. The plan had been to upgrade everything to Gen7 to standardize the hardware. I had an eBay seller screw up a Gen7 purchase and a Gen8 was supplied.

My preference is a couple of mirrored SSD drives for operating systems. Everything else can be whatever makes sense with the weird combination of SAS hard drives supplied with the systems. Installing some SSD drives has thrown up weird problems. In the Gen7 server the disk controller reports one SSD drive is overheating. Needless to say it was not hot. Google finds multiple discussions containing recommendations to buy HP branded drives to preserve warranty and support. This is not a valid reason in my situation.

I would like to upgrade ILO but the HP utility appears to be locked away behind a pay wall.

Windows 11 requirements list TPM 2.0 but my PC motherboards donā€™t have that option.

Iā€™m not suggesting a narrowly focused program on HP but a session on upgrade problems experienced and their solutions.

2

Seems to be a gotcha with Dell servers and vmware editions that can be run, Iā€™ve run the latest ESXi on an Lenovo desktop with no issues. Windows 11 is obviously aimed at latest kit.

3

We always buy the servers with the parts we need to avoid all the hardware compatibility and possible licence issues that come with upgrading existing systems. There are SO MANY things you can run into with each brand. We mostly use Dell servers both new and used from places such as Tech Supply Direct so itā€™s also not something that I deal with very often.

1 Like
4

Currently setting up a home lab with cast off servers and eBay purchases Upgrade parts like SSD are not going to be supplied. I have a requirement for 6 concurrent VMs with OpenSuse or Ubuntu desktop, Wine running a Windows App. Plan is pfsense, XCP-ng and OpenNAS.

TLDR; I didnā€™t know a home lab would have so much potential for hardware upgrade problems. So far with UPS, Memory, Storage, BIOS, Firmware, Video hardware.

In the dim dark past I had gone with Synology boxes to back up our PCs. In discussions with a friend he had recommended FreeNAS as an alternative. Recently when discussing bare metal options XCP-ng had been recommended. A plan was hatched for a home lab of pfsense, XCP-ng and OpenNAS.

Over the years Iā€™ve used Smoothwall, IPCop, IPFire and pfsense. So pfsense in a suitable box (Xeon, ECC, mirrored SSDs and Intel interfaces) already exists. My training and experience is electronics and networks but I havenā€™t touched servers since before Netware 2.12 (whatever version required the keycard) through 3 on HP LM series hardware. That translates as the 286/386/486 era.

In this town DL380 Gen6 and Gen7 servers are basically heading to the scrap heap while still perfect for a home lab. Iā€™ve ended up with a HP UPS, multiple HP DL380s, (3x Gen6, 2x Ggen7, 1x Gen8) via auction houses and eBay. Due to COVID restrictions it was not possible to inspect items, all you could see was a poor quality photo with a description of ā€˜Contents of Rackā€™. Lucky dip timeā€¦

The HP UPS came with a 10 year old battery pack. It couldnā€™t hold charge. My guess is somebody substituted a dead pack when they were preparing the sale. Built my own pack. UPS batteries are a whole topic in themselves. Previously Iā€™d been caught with sealed Lead-Acid batteries that were just about to hit their end of life. At end of life there is a steep drop in capacity. Or, cheap sealed Lead-Acid batteries that had greater AHr capacity but less capacity for liquid. End result was a shorter life. These days I only buy Yuasa or Varta. After Y2K there was a flood of APC 1200 UPS units sold through the auction houses. Currently the house has 7 or 8 UPS systems. The only maintenance performed has been feeding them sealed Lead-Acid batteries. Basically everything here has a UPS, even the TV (PC with DVB-T card) and some of the audio amps. Iā€™ve had one APC UPS die which is not bad for units that were second hand $25 - $50 back in 2001. Hardware upgrades involving UPS capacity, batteries and pitfalls could be a topic.

Installed memory varied from 4 GBytes on up. One eBay server was dead on arrival due to a invalid mix of EEC memory. First step was to strip one Gen6 for parts and strip all ECC memory from all Gen6 and Gen7 servers. Discovered what a weird and varied mixture of parts had been used. Rebalanced memory so that the Gen6 servers have 20 GBytes for FreeNAS/TrueNAS and Gen7 servers have 88 GBytes for XCP-ng. Gen8 came with 64 GBytes.

This was the first hardware upgrade. It required knowledge of handling static sensitive items, ECC types, valid memory configurations from service manuals, etc.

All systems had been stripped of some drives given the missing drive carriers. Plus some drives were DOA. Stripped all drives from GEN6 and Gen7 servers and found a mix of SAS 72, 146 and 300 GByte drives with 2x SATA 500 GByte. End result is additional drives are going to be required.

Removed some SAS 72 GByte drives from their carriers and installed spare domestic (Samsung, etc.) SATA SSDs . Intention was to mirror the SSDs for the operating systems. One SSD of a pair was rejected by the Gen7 systems as over temperature. Gen6 and Gen7 share the same carriers but the Gen8 had moved to ā€˜smartā€™ carriers that are not compatible and likely to cause problems if I try to install non-HP drives.

Second upgrade is storage and there are all sorts of potential problems such as the carriers, drive firm ware, system configuration based for ZFS, RAID arrays, RAID controllers, HBA mode, etcā€¦

At this point need to update BIOS to current recommended versions. Utilities restricted to systems under contract. Gen7 ILO version so old it requires installing an intermediate version before being able to load the recommended version. Again would like access to utilities limited to contracted systems. BIOS has been upgraded, yet to do ILO firmware.

Want to confirm the Windows application would run on this hardware. Dual Xeon processors are not supported by Windows 10 Home, requires Windows 10 Pro. Had a spare Windows 10 Pro retail pack from the dim dark past. Downloaded current Microsoft Windows 10 installer and launched into the install. Motherboard video hardware not supported by Windows 10 no provision for sound. Need low profile video card due to physical limitation of the available slot. Only cards in shops are NVIDIA Geforce GT 1030 variants. Added a mid-range Creative card for sound. EverQuest II ran perfectly with 24 cores (Hyperthreading enabled) and 88 GBytes RAM.

Next step was confirming the application runs under Linux. Installed a clean, plain Ubuntu 20.04 LTS Desktop. NVIDIA driver upgraded ā€¦Steam install grief, Wine install grief, PlayOnLinux Bash script griefā€¦ needs wrapper for new Bash Scriptā€¦ Bah humbugā€¦ Result was OpenSuse was installed then configured. These software trials and tribulations are irrelevant to hardware upgrade topic. Down load of data appears to be throttled which can also stall the video. Not a problem as frame rate above 60 fps.

Installed XCP-ng on the Gen8 without SSDs. Need to rebuild the 1Gbps wired network, have a clean install of the current stable pfsense and need to configure. Run multiple wiFi networks to isolate hardware that I cannot update or maintain e.g. air conditioner, solar inverter, etc. A network for home automation, and so on.

Totally off topicā€¦ Iā€™m a licenced ham (amateur) radio operator and have an Icom 2m/70cm IDAS (NXDN) repeater, WIRES hardware, etc. I investigated the options for linking this repeater to an existing USA NXDN network via the internet. I could have obtained the appropriate approvals and obtained the required pieces of paper for a slice of 44.x.x.x address space, my ISP said they would be happy to configure and my Draytek modem had the smarts. Two catches: One, there could not be any commercial type traffic and my partner was working from home. That killed the project. Second trivial catch was BGP would be required and the minimum address space on offer would have been 255 addresses. Obviously no guarantees what will be available to other individuals. If you are a licensed Ham Operator it might be worth investigating for a home lab linked to amateur radio.

5

Comms Upgrades
The house comms wiring had been installed by our telco. At the point their underground telephone cableā€™s conduit reached the wall of the house the telephone line went up into the roof space behind the plaster/dry wall. All the house telephone points in the house were cabled back to a central point at the front of the house. The longest run of telephone cable (lets call it CAT 3) was to the study at the rear of the house. 1200/75 Baud modem for my partnerā€™s PC clone or my VMS system was the highest speed available. There was no internet access, just access to bulletin boards and our university VMS systems and mainframe.

When ADSL was rolled out to our neighborhood I would have been the first signup in our street. The telco installed a central splitter and used the spare pair of the longest CAT 3 cable to provide an ADSL point to the study. The internet had arrived at about 1Mbs download. Catch was our house was at the line limit for ADSL.

Free ADSL2 upgrade by the telco was worthless as we had more reliable comms by locking our modem down to ADSL. In ADSL2 the modems would continually retrain due to line noise.

xDSL is rolled out and the Plain Old Telephone Service (POTS) is removed. The telco removes the central splitter and our Draytek modem is painlessly switched to xDSL mode. A higher speed internet has arrived. The upgrade path has been from dial-up to an ADSL modem then ADSL/xDSL modem.

At this point the catch is the telco has used the existing CAT 3 cabling in the house with all the unterminated telephone points acting as antennas picking up random noise. The Draytek modem reports detailed statistics on xDSL line state. Disconnecting the unterminated lines from the xDSL service delivered a measurable improvement in line noise. This translates into a more reliable, higher speed internet connection. The actual change was only a few dB which was lower than expected when based on reports by Dr. Google.

Both the ADSL Draytek Vigor2830 and ADSL/xDSL Vigor286x series modem/routers support USB dongles for fallback on DSL line failures. A facility rarely used but they have been worth every penny.

The hardware upgrade was the disconnection of the un-terminated telephone lines. For over 20 years I was employed by our local telco in a variety of roles. I do have the experience and certification to work on my own cabling. Normally laziness wins out when it comes to crawling around my own roof space. Iā€™ve been horrified by the low level of technical knowledge possessed by some of their cabling sub-contractors. Obviously skill levels will vary. At the time of the initial cabling of the house the star wiring configuration was not the standard or recommended for new, domestic installs.

My best guesses are the initial telco cabler cut corners and used star wiring to speed up the job by simplifying the cable layout. While words fail me when it comes to the xDSL cabler connecting all the un-terminated lines in parallel with the xDSL line.

6

OP, for home lab, I moved away from any old/used servers like Dell or HP because when you want to try some new adapters and what not, it generally will not work properly because it isnā€™t Dell or HP ā€œstampedā€ and will induce lots of issues that arenā€™t really existing.

When AMD Zen 2 3900X processor came out, I bought a couple of those with a good X570 motherboard (Aorus Master) with RAMS (ECC for some build, non-ECC for others) and it was the best thing I have ever done (plus lower the heat and nose in my labs too). My ESXi farms consist of 2 x 3900X with 64GB each with NVMe & SSDs to do whatever the VMs needs. I would have required 4-5 HP Gen 7 to have the same horsepower that these 2 offer me - minus the headaches of compatibility with other modern adapters.

For 24/7 lab requirement, I use AsRock Rack X470D with ECC RAM that let me log in via their IPMI remote access.

In the end, I am saving on useless problem solving but also on electricity bills, heat, noise, and also I moved away from Intel all in one clean swipe. I also end up with more room in my server cabinet with less cables to manage. With 10GBE adapters in the 3900X systems (which wasnā€™t possible with my HPs and the cards I was using), I can try and test also whatever solutions I want.

7

Iā€™m retired and money is tight. Iā€™m willing to trade off speed to save money. At todayā€™s prices Iā€™ve spent less on all my hardware than 64 GBytes of ECC memory from our local suppliers.

Our study was built to house my old personal computers. These systems included an 11/750. Electricity consumption, heat and noise are not an issue with these modern HP systems.

8

It is well documented consumer ssdā€™s dont play well with many of the Proliant raid cards from that era. You absolutely need to upgrade the cards to the latest firmware. I still even tend to buy hp branded spinning rust for my Proliantā€™s just to avoid problems.

9

Wow, i had an 11/750 at home many years ago now. I was still supporting vax systems up until 10 years ago.

10

I moved sideways into computer hardware maintenance and my old PCs included a 10 with LPS-11 running PTS and CAPS-11, 34A with RSX-11M, 750 with VMS, some Alphas.plus some MIPs DECStations with NetBSD. We were renting an old farmhouse and the weight of the 750 caused compression cracks in the walls. When we had our house built it had to be concrete slab with French Doors.

1 Like
11

I never had the chance to work with pdpā€™s but I have supported OpenVMS and Tru64 clusters. Ultrix was my first introduction to Unix. I still have a Vaxstation 4000/90 in my collection.

12

The photos of the VAX-11/750 were pre-digital and lost somewhere in storage boxes. I was trained for the first generation but only ever maintained second generation hardware (B5500) for an infinitesimally short time before moving onto third generation systems. If you have a first generation system in your basement Iā€™m probably qualified to maintain it.

[16 new items added to shared album] Home PCs plus Other Bits and Pieces
13

Love the hockey puck mice. I have some of those myself but not brand new. Love the Microvax II as well.

14

Lost somewhere in a box is a photo of my VAX750 sitting on seagrass matting.

15

Returning to the Hardware Upgrade Problems topic.

I bought secondhand a complete Icom NXDN (IDAS) repeater installation in a small rack. The intention was using the IC-FR5000 in the radio shack (such as it is). This gave me a small rack that I could re-purpose as a home lab.

Rackstuds looked much less painful than cage nuts butā€¦
Rackstuds are sold for two different thicknesses of rack hardware metal. I made the mistake of buying the thin metal Rackstuds (Red coloured) for an oddball, secondhand, communications (radio) equipment rack which had thicker metal. I had to revert to metal cage nuts.

The colour coded Rackstud sizes are:
Red up to 2.2mm/0.086"
Purple 2.7mm/0.106" to 3.2mm/0.125"

Suggest you confirm your rackā€™s rail thickness before buying any Rackstuds.

On a completely different topic with only a tenuous link.- Housing upgrades.
Of course using the word ā€˜Housingā€™ may cause confusion. Iā€™m not going to be discussing the outer shell of an item like a rack. The upgrade being discussed is to your home.

Our move was from a rental property into our new home. In the Contract signed with our builder I inserted a clause that at the stage his electrician had run the power cables to the future power points and before the plasterboard (dry wall) was installed there would be a two week period for the installation of comms, media, etc., cabling. This may seem excessive but in my case that really was not enough due to circumstances outside of my control. In our country the installation of comms cabling is regulated and I have always held the appropriate licence(s). Iā€™m old enough to predate the introduction of our licencing regime and was grandfathered into the system i.e. no written exams.

If your comms and media cabling is in the builders contract it is advisable to have access specified in the contract to take photographs of the cable installation prior the plasterboard (dry wall) installation. Depending on your contract you may not have the right to enter a house under construction. At this stage of construction any water or gas pipelines should be visible and detailed photos of the house will make it much easier to avoid serious mistakes when making modifications to wiring.

In our case the sales guy was boasting how we would be surprised by how many power points were going to be installed by the builder. That boast went out the window when we sat down to do the the location of power points. I would have liked 3 phase power but that like some other items was outside our budget. Do make certain that you have too many power points for your computer and comms needs as too few can be more expensive and/or unsightly.

Power is one regulated area that does not translate well between countries due to different standards e.g. voltages and current. While comms has its own issues that prevent recommendations i.e. ADSL, xDSL, Cable, Fibre, Sattelite, etc. I would argue, in the past, a comms cabinet or closet was not needed in the average home. These days planing for comms is essential.

to be continuedā€¦