PC Processors

[mstolet]

I have been facing a hard to fix issue in the last few days: very low performance when running some applications inside a virtual machine when using AMD processors. I have two machines connected to each other using an AMD Epyc 7452 32-core processor. In each of these machines I assign 20 cores to a VM using QEMU and then start a server that uses 12 threads in one machine and and a client that uses 12 threads on the other machine. The client sends RPC requests with 1024 byte messages to the server and I measure the throughput per second. I have been seeing a huge decrease in performance when running things inside a VM. For example, I get 36 Gbps when running the client and server on the bare-machine and only 8 Gbps when running inside a VM. In comparison, when running the same configuration using an Intel CPU I get 32 Gbps on the bare machine, actually worse than with the AMD Epyc CPU, but a much improved 24 Gbps when running the client and server inside the VM. 

The overhead with the AMD CPU is much higher than anticipated when running things inside a VM, especially since I am externalizing the network stack from the guest VM and running it on the host: the guest VM deposits packets in a shared memory region (QEMU IVSHM) using PCI-passthrough, so I was expecting very minimal I/O overhead. There seemed to be an issue with non-write combining mappings (https://community.amd.com/t5/server-gurus-discussions/amd-terrible-performance-with-qemu-ivshmem/m-p...), so I tried mapping the bar for the shared memory region through resourceX_wc. This improved performance a bit (8 Gbps -> 10 Gbps) but it's still far off the expected ~24Gbps, so I think something is still not configured properly. Has anyone faced similar virtualization issues with AMD processors? Does anyone know of any potential fixes or misconfigurations?

This is the command I used to start my VM with QEMU:

qemu-system-x86_64 \
    -nographic -monitor none -serial stdio \
    -machine accel=kvm,type=q35 \
    -cpu host \
    -smp 20 \
    -m 10G \
    -device virtio-net-pci,netdev=net0 \
    -netdev user,id=net0,hostfwd=tcp::2220-:22 \
    -chardev socket,path="/run/agent",id="agent" \
    -device ivshmem-doorbell,vectors=1,chardev="agent" \
    -drive if=virtio,format=raw,file=base.img \
    -drive if=virtio,format=raw,file=seed.img

KVM seems to be enabled

lsmod | grep kvm
kvm_amd               155648  0
kvm                  1032192  1 kvm_amd
ccp                   106496  1 kvm_amd

[elstaci]

That link to AMD SERVER GURUS is the forum you should be asking about your EPYC processor. That Forum concerns with everything to do with EPYC processors, hardware & Software.

[mstolet]

Ok, I posted there too. Thanks for pointing it out.