If you are using the latest(20.9.2) I'd recommend going back to 20.8.3 for now. I've seen a quite a few people having this issue on the new drivers, & also have had it myself. It completely went away when I went back to 20.8.3 so I'm staying there until the next driver comes.
Should probably send a report in to AMD too using the bug report tool in the driver software(upper right corner) too.