1 Reply Latest reply on Apr 28, 2018 11:21 AM by linux_monkey

    Reddit: EPYC Server, RAM and NVMe RAID question

    jesse_amd

      I came across this question on r/amdhelp and believe that once our Gurus answer it, it will be of great benefit to our community:

       

      https://www.reddit.com/r/AMDHelp/comments/8dn4lp/epyc_server_ram_and_nvme_raid_question/

       

      "Hi guys!

      Finally we're going to get our Epyc Server!

      Mainboard Supermicro H11DSi-NT with a AMD EPYC 7451 (2,30GHz, 24-Core, 64MB) - we need high clocks

      Now I'm in the last steps of my planning and can't grasp some issues with this, maybe you can help.

      We get one CPU and save the second socket for the future. Bad thing is, the PC Builder only supports 2400 ECC RAM right now, so I decided to get the RAM with 2666 separately and order the server without RAM.

      Epyc has 8-Channel, so - if I read the manual the right way - I need to use ALL RAM slots of my CPU. We only need 64 GB RAM (and already have double of what we have now), so I would get 8x8 GB Samsung RDIMM ECC. But the compatibility list from Supermicro only lists 16 GB RAM... price difference would be around 70 €. So, anyone knows if I really need 8 sticks or lower? And better to get 128 GB instead of 64? Right now we have two servers with 24 and 32 GB RAM.

      Second I would like to use 4 NVMe in 2x Raid 1 combination. Is this possible with Epyc? I know of TR with support, but EPYC I can't really find any information about it.

      Btw. the Server will be virtualised running Windows Server 2012 R2 (HyperV Replica <3) and 4 VM, our ERP System, our Mail Server (Kerio) and two TerminalServers (2008 R2 OR 2012 R2, need to test it) with around 15-20 Users working on it.

      It will be awesome, can't wait "

        • Re: Reddit: EPYC Server, RAM and NVMe RAID question
          linux_monkey

          First, on the memory. There are a number of 8 GB RDIMMs qualified on other EPYC platforms. You could obtain some through a separate channel and try them on the board and they will likely work just fine. Of course, if there is a problem with the memory, SuperMicro will rightfully tell you that you are using a non-approved DIMM and will not provide any support.

          While it is not necessary to populate all 8 channels, it will be more performant than leaving some channels un-populated. You could use 4x 16GB instead, following the SuperMicro rules such that one channel of each channel pair (e.g. A/B, C/D, E/F, G/H) is populated. The degree of performance impact will be directly correlated to the degree to which your applications are sensitive to memory bandwidth. Definitely do not use 5, 6 or 7 DIMMs. Use 8 or 4.

           

          Second on the RAID, EPYC does not include any additional hardware for RAID. You can use any operating-system-supported software RAID solution, however you would not be able to use a software RAID drive as a boot drive. Check the SuperMicro diagram for which PCI-e slots are tied to which of the EPYC socket NUMA nodes. If possible, install and configure each of your x2 RAID 1 drives onto the same node (e.g. two NVMe drives on first RAID drive on node 1 and the other two NVMe drives on the second RAID drive on node 2). The node numbers I chose are arbitrary - it doesn't matter which ones you use, just that your performance will be best if the two member drives for each RAID drive are on the same node and the two RAID drives are on different nodes.

           

          -Monkey