Have same card, same problems.
My issue seems to be related to two issues. 20.x drivers were worse for me, and i'm using a HDMI cable
My fix: safe mode -> DDU -> install 19.7.5 (full install) -> turn on HDMI link ?? setting (sorry forgot complete setting name)
You are likely better to use the latest drivers from the AMD site.
The 20.11.3 drive is out today in fact. When you install that don't choose the express install, choose Factory Reset and see if that helps.
There are a lot of complaints about the drivers and black screen issues not sure how this new driver is.