I have this issue as well. Due to it and other issues, my Threadripper build has been a giant unmitigated disaster unfortunately. I was extremely excited to do this build. I had hyped up the possibilities that all the cores offered specifically with VMs, Pass-through, Compilation, etc to everyone I knew, and they'd all be anticipating seeing how it turns out. Unfortunately, I had to report to them that my build has been a complete failure due to this.
For a high-end part @ $999 and affiliated motherboard (Zenith) @ $549, this is not acceptable. If I had bought the Core i9 instead, I would not be having this issue. If I go back to my older i7, I don't have this issue. I want to give Team AMD a chance as I appreciate what you guys are doing to up the available core count, but the parts have to work.
Several things greatly concern me right now after I've looked into it further that I am looking for AMD to address:
- The Ryzen had PCI passthrough broken as well at launch, and you had to fix it with AGESA 1006. Since the Threadripper is two joined 8-core Ryzens, why didn't you learn from this? It is extremely disappointing to see this issue repeated on the higher-end more expensive part, where this functionality is most valuable. For a high end processor, this functionality should've been tested and working day 1.
- I've been reading about the Nested Page Tables bug (NPT) in kvm_amd that has been around for...... 9 years? Why? This is an absolutely critical bug; AMD engineering should be assisting with fixing this. Having your options be that either the GPU or the CPU performance terrible in a VM is not an acceptable option and botches the entire potential of high core parts for these applciations. See: 196409 – kvm_amd nested pagetable gpu passthrough performance oddities
- Ryzen had a critical issue with high core loads related to compilation that ended up being a hardware fault. AMD says this is a "performance marginality problem" (what?). Affected users have to get their physical processor exchanged through support. This makes me *really* nervous. One of the first things I thought when I say Threadripper's core count is, man that's going to be great for compiling AOSP. While the article claims Threadripper is not affected, the fact that something like this was just discovered on Ryzen makes me really nervous about anything broken on my expensive Threadripper See: AMD Confirms Linux 'Performance Marginality Problem' On Ryzen - Slashdot
- My expensive motherboard won't even boot with certain common PCIe cards (USB controllers, for example). The torx screws was too short on the socket making threading near impossible. The memory slots are so sensitive that it took multiple re-seats of every DIMM to get it to boot with 128GB. The boot order of my HDDs is randomly lost with a SATA card installed (bugs reported: ROG Zenith Extreme (X399, socket TR4) - info, experience, updates - Page 8 ). This has been the hardest and most frustrating build I've done in the last 20 years.
None of these issues are problems on Team Intel. I want to give Team AMD a chance again... I really do. The Threadripper has amazing potential if the issues are fixed, and makes the price much more affordable. But again, the features required to utilize them have to work. Like the original poster, if I don't see both recognition and traction from AMD on these issues soon, then I will be RMA'ing as well at the end of the window and going with the Core i9. I hope that AMD takes these issues seriously and starts to get fixes in place. Especially because, since you're using the same architecture for Epyc, those processors are going to have a hard time in the server market with things like this broken.
With that said, on to information about this issue. Here is what I've collected from my motherboard. Please pass on to engineering:
- OS: Linux Mint 18.2 Cinnamon Edition (Ubuntu 16.04)
- CPU: AMD Ryzen Threadripper 1950x
- MB: Asus Zenith Xtreme (BIOS 0503)
- MEM: 8x16GB (128GB) Crucial Ballistix BLS4K16G4D240FSC
- GPU: 2x EVGA 1080Ti FTW3 Hybrid
- Other: Inatek USB PCIe (KTU3FR-5O2U, disabled due to BIOS bugs), Supermicro AOC-SAS2LP-MV8 8-port SATA, Asus 10GEth PCIe (Bundled w/ Zenith)
Root problem:
- Pass-through GPU stuck in D3 state, no output. VM hangs. Let me know if there is any more information I can provide.
Kernels tried:
Versions tried (same result on all of them):
- QEMU 2.5, Libvirt 2.5
- QEMU 2.9, Libvirt 2.5
- QEMU 2.10rc4, Libvirt 3.6.0
Logs:
