So, I experienced this issue with a DP55SB board and a RX 460 card. For me, it was true that Windows was starting up despite the screen showing 0_ on the bottom right corner. You said your windows didn't start?
You could get a second video card, plug it in, then plug in your AMD card in the second PCIe slot, and in windows (using the video outout of the non-AMD card), install the drivers for the R9 card.
I was able to get my RX 460 to work by using a second video card to completely uninstall AMD drivers, installing TeamViewer on the PC, set TeamViewer to start with windows, set a permanent password for TeamViewer (teamviewer asks you to do this when you set it to start with windows), write down the ID for TeamViewer, shut down, remove the second video card, install the RX 460, boot the PC, then go to my secondary PC, install teamviewer, then use teamviewer to log into the PC with the no video. And what do you know, windows was fully working on the PC, despite still showing 0_ on the screen. So, from there, I installed the latest drivers for the RX 460, then Restart. On the restart, 0_ still shows after the bios, but the screen came on when Windows loaded. So, it's not a PCIe slot problem, a backward compatibility problem, video card problem, it's a motherboard bios problem.
Anyway, I can use the PC now, just without visibility of the PC until windows loads.