Truenas - Build is Finally Happening! Critique my setup

Computer Hardware & Server Infrastructure Builds
1

Hey everyone! Although I’m disabled, I have a friend coming into town on the 3rd of April to help me rack all my new hardware! This is a once in a lifetime opportunity for me given my situation, so I’m thrilled as can be (got injured at work and lost my entire career, but it is what it is… I suffer from extreme pain every day… but that’s a whole other story. bottom line I doubt I’ll ever be able to afford server hardware again. trying to keep it positive in this chat).

Anyway, please critique my build plans… I’m especially interested in hearing what I can do to increase search speed. I already have 2 smaller Truenas (one Scale, one Core), and it seems to take forever to search for something. I’m not familiar with databases, indexing, etc, but I feel like I need something like that.

So here’s what I have. It’s here in big heavy boxes, waiting for my friend to put it in my 42u for me…

Supermicro 4U-X11DPH-T-36BL-S3-2PS
SAS3 Expander 12GBPS

  • Integrated Intel X540 Dual Port
    10GBase-T
    2x Intel skylake 6130 2.1Ghz 16 Core (32 cores total)
    256GB DDR4 ECC REG
    1x AOC-S3008L-L8e HBA 12Gb/s
    36x 3.5"Supermicro caddy (With Rear 2qty 2.5" wired to on board sata)
    Dual SQ Power supply
    Rail Kit 4U

The original deal on an X10 system ran OOS on me while I was waiting for my money so I basically had to buy an X11 system… seems like a waste for just a nas… but it is what it is. I’m coming from (2) Truenas’ running on old ass R720XD’s, one an xeon E5v1 buld, another a E5v2 build… so this is a massive upgrade

I also have the following drives and plans:
35 16TB Seagate Exos X18 SATA drives
Sata was much much cheaper than SAS, so I went SATA
Going to set up as 4 groups of 6 drives, RAIDz2.
I’m aware I’m losing 1/3 of my drives to parity, but I’m going for security vs maximizing available space. After that and chopping off the 20% overhead Truenas wants you to keep I’m really losing 1/2 of my space, but at least I need to lose 3 drives in any group before data loss occurs. I’m also going to have a few hot spares, and several cold spares.

firefox_31XHBvVZd4
firefox_31XHBvVZd41465×1061 31.5 KB

I’m also going to have 6.4TB of L2ARC via a P4610 U.2 SSD. I actually bought 4 of these U.2’s… 2 are going in a Proxmox server as RAIDz1 (mirrored), 1 is in this Truenas as described, and the last one is a cold spare shared between the Truenas and the Proxmox.

Any thoughts so far??? Also, I thought Tom mentioned in a vid (can’t remember which one) that there’s a way to add a ssd to increase speed of searches? Is that true? Does it index everything in all the datasets? Does it work with SMB?

Thanks for anyone that took the time to read this. Any constructive ideas are greatly appreciated.

2

Special Metadata VDEVS can help if you have a few hundred thousand files but Windows is not fast at search so it may not work well depending on your use case. Also, those Metadata drives need to at least be mirrored because if you lose them you lose the whole pool.

3

Thanks for the reply, Tom. I’ll take a look at the vid. and your mention of the need for redundancy is noted.

4

Supermicro X10 is getting old, do not regret buying the X11.

5

Only as another thought, but I use TreeSize (by Jam Software) sometimes, it has a super fast search feature that works on networked drives. I actually mainly use it for finding duplicates.

Screenshot 2024-03-28 122557
Screenshot 2024-03-28 1225571306×700 40.9 KB

Screenshot 2024-03-28 122615
Screenshot 2024-03-28 1226151008×723 29.9 KB

I hope that helps, and of course my sympathies for your unfortunate life changing event, but my hat off to you for getting on with things :clap:

I hope this assists.

6

You made my three Supermicro X10 servers very sad :pensive:.

You’re right, of course. But with the most recent motherboard, 4C/8T CPU, and 32GB of ECC RAM, purchased this year, costing all of $80, my home lab budget still has a great big smile. :slight_smile:

7

I’m buying even older stuff for my lab, but not older than X10, x9 is no longer getting bios updates unless a big bug presents itself, and the processors are getting very old too.

My lab is a bunch of used HP dl360 gen8 because the price point was right. I do have several X10 based servers in production still, but looking forward to being able to replace them sometime in the next few years.

8

sorry for the delayed response… lots of family things going on along w/ my friend being in town to help redo my rack.

Thanks for letting me know. I barely remember using that app a few years ago I believe if it’s the one I’m thinking of. I forgot all about it and didn’t realize it worked on networked drives. I remember it being a lot faster than windirstat.

ETA: I was thinking of something different. Do you need professional or will personal work? They reference windows “corporate” networks a lot in their comparison. IDK what makes a network “corporate” other than AD.

9

Fair enough. sometimes I feel like I’m overbuying.
Heck… even on my pfsense that’s running on a R420 w/ an E5v2 Xeon (well dual E5’s), I honestly thought the CPU reading was broken b/c it read 0% for so long. Then I finally saw it jump to 1% utilization and realized it was fine.

I need to watch it when I’m checking (downloading) my off site backups. I also assume it will go up if and when netbird comes out w/ a pfsense integration… which is now officially on their roadmap.

1 Like
10

Your r420 is about the same as my HP gen8, both are Xeon E5 v2 and kind of the oldest I suggest right now. But even that may be changing as Linux starts to force the use of newer processors. I can’t find the link, but I remember Brodie talking about this possibility in a video a couple months ago. Something about the newer processor code and taking advantage of them for more speed. There was talk of Ubuntu forcing this, and Fedora not able to push this into Fedora 40 because the timeline was too close to do proper research, but Fedora 41 may not run on these older Xeon v2 processors as they may not have the newer code, I think E5v3 does.

[edit] remembered enough keywords to find one link https://www.reddit.com/r/linuxhardware/comments/s2x60j/how_do_i_know_if_my_cpu_supports_x86_64v3/