To give an update on my problem, I was able to fix the VRAM issue by doing a clean reinstall of Windows 10 2004. Originally I had just built a new pc and had installed a clean version of 1909 a few weeks before 2004 had initially released and at the time I didn't have any problems with game performance. But when 2004 did release I didn't receive the update right away and decided "what the heck why don't I just force it". Figured it wouldn't be that much of a problem. I didn't really have problems at first but as I updated my GPU drivers I started to notice I was starting to have stutters and freezing that got progressively worse. It wasn't until just recently that it was starting to get really bad.
For whatever reason, I think forcing the upgrade in my case (from 1909 t0 2004) when my system wasn't ready was probably the culprit because now with a clean install of 2004 and drivers installed my games are running fast and smooth like they did when I was on 1909.
Is it possible your problem is software related like my mine was and maybe you need to do a clean reinstallation of Windows?
