At the outset, I realize the 5600 isn't a great 4K card but I'm also not playing games that push the envelop a ton (Civ 6, Cities Skylines and that type). Plus I know for sure some games were working at 4K and aren't working now all of a sudden (Project Cars 2 for one).
That said, I'm finding that when I try to run most games at 4K my system is hard crashing and rebooting. Some I can get into the main menu and they crash when I try to then start the game (Cities, Project Cars), others just crash outright (Blood Rage digital). However, for those that can lower the resolution on, if I play them at 1080p they seem to work fine. For those like Blood Rage Digital, if I reset my screen resolution before entering the game they then work. This issue seems to be exclusively with games too, I've had no issues whatsoever in Windows running at the native resolution.
Radeon drivers are up to date, in fact I've tried wiping the drivers and reinstalling almost every version since RX 5600 XT support was added to no avail. Windows 10 is up to date. I've tried with scaling turned off and with HDR mode both on and off. I've monitored temperatures as well and don't see anything out of the ordinary there either.
Rest of the machine:
Ryzen 9 3900x
64 GB RAM (DDR4-3600)
Asus TUF Gaming x570-Plus Wi-Fi motherboard (running most recent BIOS)
MSI Mech OC RX 5600 XT (w/ VBIOS update to 14Gbps)
Monitor is a TCL 43s425 4K TV connected via a brand new HDMI 2.1 cable (it does not have a display port connection)