cancel
Showing results for 
Search instead for 
Did you mean: 

Rig Showcase Discussions

scythefwd
Adept III

Home lab, aka the 32 headed beast

Part
Spec
CPUThreadripper 1950X
MOTHERBOARDASRock x399 Taichi
RAMG.skill Trident Z 32gb ddr4-3200
STORAGE1x seagate 1tb platter, 1x 1tb WD platter, 250 gbSamsung 850 EVO
VIDEOXFX Rx570 4gb
CPU COOLEREnermax Liqtek tr4 240
PSUCoolermaster 1000 watt
CASENZXT s340

Future plans, 3x 256gb samsung 960

1tb samsung 850

(creates the possibility to demonstrate tiered storage)

meshing the front of the case to provide better air flow through the radiator

pics to follow

She's not as clean as she could be, but she runs cool, seems to not mind being a VM host

44 Replies
scythefwd
Adept III

Re: Home lab, aka the 32 headed beast

She's now running 3.9ghz, 1.25v

Ram is running at 3466 vs 3200.. no ill effects yet.

Front of case is half modded...rear cut in liner is done, mesh is fitted.  Front cut still needs to happen to actually make the functional port.  Debating leaving grate unpainted.. or clear coated.

scythefwd
Adept III

Re: Home lab, aka the 32 headed beast

Well, now she can breath correctly ...  Edges could be filed.. could use a paint on the front.. but she's a workhorse, not a showdog.

nzxt_grill.jpg

kxuping
Forerunner

Re: Home lab, aka the 32 headed beast

Awesome homelab PC.

What settings did you use for the ram?

What do you run on the PC?

0 Kudos
scythefwd
Adept III

Re: Home lab, aka the 32 headed beast

This will be running win 10 pro with hyper-v role active.  As my main PC, I'll be interacting with that instance of win 10...

It also has a free ipa instance on it.  Once I decide on my network configurations, there will be a smoothwall firewall/router and 3 security zones, file share based off of windows, an AD domain controller, probably an email server.. havent decided on which open source one I'll be using, a linux client or 2 (centOS and Ubuntu most likely).  There will also probably be a squid server on there.  Current project is to attempt to get active directory federated services to authenticate with HOTP or TOTP two factor authentication using the freeipa server as the authentication server.. and then having my windows AD client authenticated using 2fa to the ipa server.. Meaning native support for HOTP or TOTP in a windows domain.. and using a central authentication server without samba to authenticate my linux clients and windows clients.  Once I get that authentication working, then its on to secure baseline configs using gpo.. would really rock if I could get the LDAP accounts recognized in AD for ad group membership.. gpo filters, etc...

IPA will probably be doing dns and dhcp roles, though I may kick dhcp over to smoothwall (using reservations.. dhcp is just practice playing with one).

There may be other uses that happen.. but first I have to get authentication setup.. then on to pxe boot servers hosting various images.. pxe boot clients that are inexpensive, remote patching capabilities (preferable to manage both windows and linux environments.. patch level reporting is required)... After that, probably Identity management tied to a sql speaking database (postgre, mysql, or ms sql lite)...

In short, its starting with the host os, and 3 vm's, moving on to probably 7-10 vm's in the future.. Low load so Im gonna keep core count down to 2-4 wherever possible..

Thats the short term goals.. the environment will be fluid based on what I want to focus on...

Ram settings.. it seems happy using the xmp 2.0 3466 profile that is pre-configured on the mobo.. The ram itself is rated for 3200 by gskill.. It just happens to deal with 3466 in my system.. Nothing special  was done.  I havent tweaked the settings manually.  Im at 3.9ghz and 1.25 volts, it will do 4ghz at 1.3..

Adding the grill up front raised my temps 2-3C from no front on the case (exposed radiator), so there is an air flow restriction (duh.. look at it), but its still 7-10 degrees cooler than the front on without the grill.

kxuping
Forerunner

Re: Home lab, aka the 32 headed beast

I activated Hyper-V on Windows 10 pro by mistake when installing latest version of Docker. (When activating Hyper-V windows 10 is run inside a VM, for others reading this).

I thought my PC had a virus or was defective. All games run like crap with stuttering and crazy lag even on notepad++, videos on Utube were unwatchable even on 360p. Network did not work for the first 4 minutes that windows has the desktop up and running, and download/upload speeds were really bad compared to my laptop just in the next ethernet port. sometimes it stopped sending/receiving network packets for minutes, then it came back. changing video resolution on 3 monitors took minutes.

I realized I was running that awful creation when trying to run a Virtualbox VM the PC crashed hard every time not even a blue screen, black screen and weird sounds, had to power off. Vmware workstation pro did not crash the OS, but did not work anymore.

After uninstalling Hyper V all started working normally. But some Hyper V network interfaces and other disposals were left on the system so a backup restore was necessary.

Docker toolbox still uses Virtualbox instead of Hyper-V on windows, so that is what I use to run dockers when running windows.

After my experience I understand why Vmware is still number 1 in paid virtualization and KVM is number 1 in free virtualization.

I use Docker in Mac and Linux and none had this type of problems, they use their native virtualization options (non-hyper-V).

Please share your experience after you have it running.

scythefwd
Adept III

Re: Home lab, aka the 32 headed beast

I have run hyper-v on both a 3770 (with a ubuntu guest) and on the threadripper (with a centos guest) at home.  I have ran hyper-v on 2012 and 2016 supporting roughly 100 vm's in a cluster with no problems as well.

I run hyper-v with a centos guest at work just to dink around too.. and thats on a mediocre i5...   The virtual nics that get left over.. those were used to connect into the virtual switches, and you can delete those via powershell or through device manager.

Now, with hyper-v.. they recommend provisioning at least 10% more on hardware.. so if your desktop was a little short on ram, and then you spin up another vm.. then bam.. you're going to have stutter problems due to low ram (and possibly swapping all to hell).  I also refuse to thin provision either disks or ram, and I run minimum 2 procs per vm period.  I have played pcsx2 in two instances (one as admin, one as unprivileged user).. running ffx-2 (have two ripped copies on my desktop just to test this... Yes I own the game, no.. it's not pirating) and keeping 60 fps on both windows (at 720p)..

I do find I run into some slight stuttering when only running at 3.4ghz on the TR, but kicking it up to 3.6 base clock and she runs smooth. 

scythefwd
Adept III

Re: Home lab, aka the 32 headed beast

Alo, if you didn't have the virtualization extensions enabled in the bios, and if your proc didn't support slat, it may have activated the feature.. but been broke as all hell from the get go.

kxuping
Forerunner

Re: Home lab, aka the 32 headed beast

Thx for answering.

Not being able to run vms in virtualbox or vmware is a no go, so I did not even try to check Hyper-V configuration/documentation.

My PC has 64GB Ram and all vm optimizations active in BIOS, core info reports:

AMD Ryzen 7 1700X Eight-Core Processor

AMD64 Family 23 Model 1 Stepping 1, AuthenticAMD

Microcode signature: 08001129

HYPERVISOR      -       Hypervisor is present

SVM             *       Supports AMD hardware-assisted virtualization

NP              *       Supports AMD nested page tables (SLAT)

0 Kudos
scythefwd
Adept III

Re: Home lab, aka the 32 headed beast

Interesting.. I did notice some latency till i oc'd it.  At current clock, I ran prime 95 torture test.. not small fft... Even my aio cannot keep up with that one... but the default torture test and I only saw 60 C...

Best of luck on the ryzen.. had I been expecting to only run 2-3 vm's, is probably had gone with an 1800x or 1700x myself.

I have heard that running 32 bit guests in hyper v was causing soft locks of both host and guest... But I've only ran 64 bit guests.. and been fine here

0 Kudos