I remember now why I never wanted to remove the GPU from my server. This case and the GPU I/O plate has these prongs that are supposed to slot into the case as you install the GPU. These prongs gave me the fight of my life. They are straight on the the rear GPU I/O plate but they just would not line up with the case. I pushed as hard as I could reasonably without damaging the GPU and it slammed into place with a loud clang. I spent hours benchmarking and stress testing the PC, and everything seems fine, but if I ever remove that GPU again I am going to get a vertical GPU mount from Lian Li.
After all of this I read up on the issue and it seems to be relatively common. I heard you can loosen the motherboard screws and pull the motherboard more to the right and it will work so maybe before I invest in the vertical mount I will try that, but I felt I had to share this experience. It was terrifying.
Ryzen 7 7700X, MSI MAG X670E Tomahawk Wifi, Corsair DOMINATOR® TITANIUM RGB 2x16GB DDR5 DRAM 6000MT/s CL30, Sapphire Pulse AMD Radeon RX 7900 XTX, Corsair HX Series™ HX1000, Corsair MP600 PRO NH 4TB