I found this https://www.amd.com/en/support/kb/faq/pa-105#faq-Workarounds, but the info seems not very match: my motherboard's BIOS is up to date, AUS TUF Gaming B550M plus, BIOS version 1804.
The issue on the machine is when reboot, it will always ask for keyboard selection for which Language, while during normal reboot, no such step, and then after select the language we want, it booted in to the Windows, and then we can see the Windows is not updated.