Can you please identify which edges specifically you consider "messed up"? To me, it looks like normal 4x antialiasing with some sharpen filter on top. Is it antialiasing in the highlited areas that bothers you? Have you made sure to disable MLAA and Sharpening in the driver settings?
I think it's more or less known by now, that A LOT of games have problems with Radeon AntiAliasing: https://community.amd.com/t5/drivers-software/antialiasing-is-not-taking-effect-in-direxctx-9-applic...
It can't be properly overriden / enhanced in any mode unless the game itself supports multisampling. This is, unfortunately, a common issue. The worst part is that many older games don't have native AA support, meaning you're royally screwed. For lower resolution monitors, there might be some remedy in using VSR for supersampling, which goes up to 5120x2880 (5760x4320 and 8K VSR when, AMD?!), but on native 4K it looks fugly, because interpolation. IDK about others, but to me, driver-based MLAA also looks disgusting - it blurs like there's no tomorrow but provides surprisingly little actual antialiasing.
I would advise you get a nice juicy ReShade with an SMAA / FXAA / NFAA shaders and use those where normal AA is unavailable. The best SMAA setting for me, personally, is Chroma Edge Detection, EdgeDetectionThreshold = 0.10, DepthEdgeDetectionThreshold = 0.001, MaxSearchSteps = 32, MaxSearchStepsDiagonal = 0, CornerRounding = 50, Predication = Enabled, PredicationThreshold = 0.01, PredicationScale = 2.0, PredicationStrength = 0.5; diagonal search creates weird edge artifacts sometimes, be sure to disable it. FXAA is simplier - only 3 parameters to set, make them 1, 0, 0 - it will have as much, if not a bit more blur than driver MLAA but FAR better actual antialiasing quality. NFAA is by far the best of all injectable AA filters I've seen, but it has the greatest demerit of all as well - creates weird dot-like, or sand grain type, arifacts within textures (takes a while to notice, but once you do - can't unsee). If someone updated NFAA to remove that issue, it would've become the best injectable post-AA method ever, period (well, unless someone manages to write a proper TAA / TSSAA code for ReShade).
So, in conclusion: if you feature a 1080p or 1440p display - use VSR for extra antialiasing. If you're at 4K - get a ReShade. AMD driver sucks.
