I had a similar issue with a 480 that was fixed by simply replacing the thermal paste.
In my case, the original thermal paste was covering like 2/3 of the CPU, the temperature was surprisingly not bad, so I assume that either the temperature sensor was full of BS or that some spot in the die was getting hot so fast that the card would crash before it have the time to show me that the temperature was high, unlikely, but I have no other explanation.
Also, not sure if they fixed things on 24.1.1 but on 23.12 the drivers/pc would crash by simply enabling stuff in AMD's Adrenalin > tuning (default values), testing older drivers 23.8 ~ 23.9 would magically fix the issue, so that something that you could try.