PC Processors

[jmandawg]

Hi,

The latest Agesa update broke GPU VFIO IOMMU gpu passthrough virtualization.  It's been broken for 4 months or so and still no fix in sight.  After Agesa update we receive error: "Unknown PCI header type 127"  Can you give any ETA on this.  

The issue is documented here:

https://www.reddit.com/r/Amd/comments/bh3qqz/agesa_0072_pci_quirk/

https://www.reddit.com/r/VFIO/comments/c357kg/msi_x370_bios_update_breaks_gpu_passthrough/

https://www.reddit.com/r/VFIO/comments/c0hnmf/vfio_agesa_0072_survivor_wants_to_foolishly/

https://www.reddit.com/r/VFIO/comments/bvqxnt/psa_amd_agesa_1001_does_not_fix_the_pci/

[mz01]

Wanted to add- this thread documents 13 motherboards that have been affected.  This includes the new x570.  And it is preventing some users from upgrading to the latest Ryzen 3 line of processes on the x470 motherboards.

To add- the workaround to this video card pass-through error is to downgrade to a motherboard (AGESA) BIOS that was before the prep for the Ryzen 3000 series of CPUs.  So the BIOS has to be pre-May 2019.

Win10 VM graphics pass-through broke after AMD BIOS update - General Support - Unraid 

This also shows it happening - Upgrading my Bios ready for Nextgen Ryzen Broke my Passthrough Twice! - YouTube 

[calebwill]

Hello, first time posting here.  Chiming in to say that I would also appreciate an update that fixes this problem.

I just bought an AMD CPU for the first time only to find my upgraded system incapable of passing my video card to my virtual machine using IOMMU/AMD-Vi.  I definitely would not have purchased AMD if I had known that this would be the case, and certainly won't be recommending or buying AMD again if it remains unfixed.

• Ryzen 2700X
• ASUS PRIME X470-PRO
  • BIOS version 5204 2019/07/31
• Debian 10
• Linux 4.19.37-5+deb10u1

'lspci -nnk' for GPU:

0a:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 980] [10de:13c0] (rev a1)
Subsystem: eVga.com. Corp. GM204 [GeForce GTX 980] [3842:2983]
Kernel driver in use: vfio-pci
Kernel modules: nouveau
0a:00.1 Audio device [0403]: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)
Subsystem: eVga.com. Corp. GM204 High Definition Audio Controller [3842:2983]
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel

Relevant log entries:

Aug 03 21:01:03 hill kernel: AMD-Vi: IOMMU performance counters supported

Aug 03 21:01:03 hill kernel: iommu: Adding device 0000:0a:00.0 to group 17

Aug 03 21:01:03 hill kernel: AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40
Aug 03 21:01:03 hill kernel: AMD-Vi: Extended features (0xf77ef22294ada):
Aug 03 21:01:03 hill kernel: PPR NX GT IA GA PC GA_vAPIC
Aug 03 21:01:03 hill kernel: AMD-Vi: Interrupt remapping enabled
Aug 03 21:01:03 hill kernel: AMD-Vi: virtual APIC enabled
Aug 03 21:01:03 hill kernel: AMD-Vi: Lazy IO/TLB flushing enabled

Aug 03 23:40:13 hill kernel: vfio-pci 0000:0a:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
Aug 03 23:40:13 hill kernel: vfio_pci: add [10de:13c0[ffff:ffff]] class 0x000000/00000000
Aug 03 23:40:13 hill kernel: vfio_pci: add [10de:0fbb[ffff:ffff]] class 0x000000/00000000

Aug 03 23:36:24 hill kernel: vfio-pci 0000:0a:00.0: enabling device (0000 -> 0003)
Aug 03 23:36:24 hill kernel: vfio_ecap_init: 0000:0a:00.0 hiding ecap 0x1e@0x258
Aug 03 23:36:24 hill kernel: vfio_ecap_init: 0000:0a:00.0 hiding ecap 0x19@0x900
Aug 03 23:36:24 hill kernel: vfio-pci 0000:0a:00.1: enabling device (0000 -> 0002)
Aug 03 23:36:26 hill kernel: vfio_bar_restore: 0000:0a:00.1 reset recovery - restoring bars
Aug 03 23:36:26 hill kernel: vfio_bar_restore: 0000:0a:00.0 reset recovery - restoring bars
Aug 03 23:36:27 hill kernel: vfio_bar_restore: 0000:0a:00.0 reset recovery - restoring bars
Aug 03 23:36:27 hill kernel: vfio_bar_restore: 0000:0a:00.1 reset recovery - restoring bars
Aug 03 23:36:27 hill kernel: vfio_bar_restore: 0000:0a:00.0 reset recovery - restoring bars
Aug 03 23:36:27 hill kernel: vfio_bar_restore: 0000:0a:00.1 reset recovery - restoring bars
Aug 03 23:36:27 hill kernel: vfio_bar_restore: 0000:0a:00.0 reset recovery - restoring bars
Aug 03 23:36:27 hill kernel: vfio_bar_restore: 0000:0a:00.1 reset recovery - restoring bars
Aug 03 23:36:27 hill kernel: vfio_bar_restore: 0000:0a:00.0 reset recovery - restoring bars
Aug 03 23:36:27 hill kernel: vfio-pci 0000:0a:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff
Aug 03 23:36:27 hill kernel: vfio_bar_restore: 0000:0a:00.0 reset recovery - restoring bars
Aug 03 23:36:27 hill kernel: vfio_bar_restore: 0000:0a:00.0 reset recovery - restoring bars
Aug 03 23:36:27 hill kernel: vfio_bar_restore: 0000:0a:00.0 reset recovery - restoring bars
Aug 03 23:36:27 hill kernel: vfio_bar_restore: 0000:0a:00.0 reset recovery - restoring bars

…

Aug 03 23:37:42 hill kernel: vfio-pci 0000:0a:00.1: Refused to change power state, currently in D3

The display attached to the GPU does not power on at all when the virtual machine is booted.  This worked without a problem on my old Intel system in both Debian 9 and Debian 10.

[calebwill]

ASUS finally released a UEFI with AGESA 1.0.0.3 ABB for my PRIME X470-PRO motherboard.  Unfortunately this does not resolve the issue at least with my Ryzen 2700X.

The only solutions I have found so far are less the optimal: patching the kernel to work around the bug, or downgrading to a UEFI from before AGESA 0.0.7.2.

AMD-Vi/IOMMU is still broken for Ryzen 2000 on AGESA 1.0.0.3ABB for anyone out there researching a new VFIO/passthrough build.

[tmartins]

Hi!

 I'm also being affected by this problem.

 I've updated the MOBO BIOS from here:

 X470 AORUS ULTRA GAMING (rev. 1.0) | Motherboard - GIGABYTE Global 

 To the F42a version, no more PCI Passthrough.

 Then, I downgraded to F6 and I can use my NVIDIA within a QEMU VM again!

Best,

Thiago

[widde]

This is broken on my motherboard aswell, Asus ROG Strix B350-F Gaming.

BIOS that I have tested it on: 5008, 5204.

Same problem as the thread owner. 
I can currently not use my setup without GPU Passthrough!

[gb_l_fr]

Please AMD , you definitely need to allow this again, this is just software bugs, please do something, my setup can't upgrade bios either (asus C7H)

[sss123next]

i have same problem with new agesa code, i have two testing boards, x470 and x570 based, also i have posted more detailed information here: 204975 – AMD-Vi: Command buffer timeout 

[psycmos]

Somebody know if new AMD ComboPI1.0.0.4 Patch B (SMU v46.54) FIX IT?

[calebwill]

Yes this seems to finally be fixed by AGESA 1.0.0.4 patch B, at least on my ASUS PRIME-X470 PRO motherboard with UEFI version 5406.

[psycmos]

I have one MSI X470 Gaming Pro! Now problem was fixed!!

With Last update:  7B79v1E  (2019-11-12)

- Update AMD ComboPI1.0.0.4 Patch B (SMU v46.54)

All working good!

[siiruhs]

ASRock x570M Pro4 on BIOS 3.0 (AGESA Combo-AM4 V2 1.0.8.0), no luck