Doesn't tell me what resolution you game at .. I'm assuming 1080p with a 5500xt with probably low'ish settings in the game you play
Doesn't tell me what speed your RAM is running at (Task Manager>Performance>Memory)
Doesn't tell me if you have the latest BIOS installed for your motherboard
Doesn't tell me what make/model power supply you are running
You say you updated your drivers ..does that include the AM4 chipset driver from AMD.com?
IF you haven't already .. you can run the program called "DDU" and remove all past/present AMD GPU drivers .. reboot . .and THEN install the latest AMD GPU drivers from AMD.com
Make sure your Windows install is up to date
How are your CPU/GPU temps?