I have been having this problem for a year now, hoping someone would come up with a fix.. It is possible to fix on Nvidia cards!
For some reason, my computer is not capturing window capture - Apparently this is an old issue that could be fixed by going to radeon additional setting and selecting the integrated graphic driver, for running OBS itself. - However, you can't do that in the new Radeon Drivers "Additional setting missing" Or "Moved to Display" But! - When you are not on a computer with a build in graphics card, like a normal PC, then there is no option to do or change anything - You simply can't do window capture.
I have worked out that I can use Display Capture, but why is no one @ AMD addressing this, there is a lot of streamers out there!
I mean I know why, but please do something about it!
So for now, I guess Display capture is fine.
Please bump the subject, if you have the same or similar problem.
this is most-likely an OBS bug - so please post it here https://obsproject.com/forum/list/windows-support.32/
here some tips:
1. start OBS as ADMIN - this can maybe fix your problem (because now it is allowed to use other elevated areas in ram)
2. de-install driver with latest DDU in safe-mode && install new driver after second reboot && try again
3. talk to OBS FORUM
btw i use OBS since years - with x264 or AMD AMF - also working with OBS Devs and Plugin Devs and knowing the AMD Driver Dev for AMD AMF
when you go to window capture, what is coming up when you choose properties? I know i used to have a problem with it auto captureing lock screen. Does it allow you to choose the game you have available. Also have you tried running the game in windowed mode as some games work better for capturing in windowed full screen.