I have this same issue, on 18.12.3. Thanks for the HBCC memory segment trick, that worked for me, hopefully this bug gets squashed for good in the next update
edit: HBCC toggle seems to be a fix for only 5 minutes before the card goes back to 835/500. <- possibly because of high memory overclocks.
edit #2: after a few days of generally fiddling around (unrelated to this issue) I replicated this bug after an extended gaming session (approx 2 hours). Still on the latest beta drivers, clockspeeds came down to 849 core / 700 HMB2 from overclocked 1600/900. GPU and HMB temps were always <60 degrees celcius so not likely a throttling issue. I'll do more 'research' into this and see if I can give you guys some more info.
edit #3: radeon settings 18.2.1 did not exhibit this issue while I had it installed and playing assassins creed odyssey. Tried keeping it for metro exodus but experienced bizarre visual glitches so had to upgrade to 19.2.2, which brought back this issue. Besides restarting the computer (which I would rather not do) -> HBCC on/off is the quickest workaround that exists to date and I have to do that once or twice a day.
edit #4: beyond the issues, a few thoughts on vega that I will share from elsewhere --
non-oc 1660 is 17% cheaper in Aus than what I paid for vega 56, OC 1660 is maybe 5-10% cheaper, or even more expensive. But vega is way faster right? Especially with +50% power, undervolted, oc to 1600 core clock etc etc... ? Gotta say my vega card has been great for me, radeon chill keeps it running at ~1000 mhz clock in metro exodus, on ultra, constant 75fps @ 50 c temp! I like a quiet system https://gpu.userbenchmark.com/Compare/Nvidia-GTX-1660-Ti-vs-AMD-RX-Vega-56/4037vs3938
edit #5 - haven't played video games much recently but I did tonight and I hesitantly say this issue has gone away with driver updates.