I have a RX 580 on a X470 Board and had similar problems. It even git worse when Windows were extreme slow after boot, I mean 3 Minutes to open the start menu etc.
After some days now, I found the problem: the radeon software!
So I uninstalled the radeon software which uninstalled the driver automatically, in the device manager i had the windows default graphic driver after the uninstallation of the radeon software.
Now I installed the driver via device manager, you can find the driver in C:\AMD\
In my case it was C:\AMD\AMD_Radeon_Installer_20.1.3\Packages\Drivers\Display\WT6A_INF\
I had to remove the .cat and .inf starting with a C so only the one starting with U were present.
I dont know what there are for, the C inf is larger than the U inf and had lots of R300 config in, maybe support for older cards?!
After that I could install the driver via the device manager without radeon software.
That was 2 days ago and my system is running better than ever before. No more lags in windows, no black screens, netflix in Edge does not fail or stutter anymore etc.
All the evil comes from this bloatware!