I'm having this exact same problem as you described. My radeon overlay doesnt open in game and if i try forcing record it, it crashes the game, sometimes it crashes the whole computer.
Thanks for replying. I have found that rolling back to 18.2.1 allows me to use relive although the overlay is still randomly unresponsive. I just replaced my i5 with an i7 so I updated to 18.6.1 to test again, and I still immediately crash when rocket league opens.
Are you using an ultrawide monitor or a normal 16:9 monitor?