Archives Discussions

[nuclear_bro]

Need help with OpenGL API inerception

Hello, dear community.

I'm a rookie programmer and the problem i ran into seems too deep for me, so i need help.

I have an issue with OpenGL API interception in Win7 x64 with newest Catalyst (10.11) drivers. My program integrates into OpenGL based CAD application and intercept calls to opengl32.dll for a purpose to inject own graphics in render context during frame rendering. The way it works is quite simple: the CAD application loads my dll and when it calls my startup function i patch an Import Address Table (IAT) of the particular module in the process to intercept some OpenGL function calls from this module and render my own graphics. This is possibly the one and only method to integrate into this specific application, because i cannot rely on other non render-specific function calls.

This simple trick worked fine until i ran into a problem with newest Catalyst drivers in Win7. It seems that in terms of optimization, when CAD application start up, someone (loader, or ati driver) is patching it, so after loading into memory it is no more calling opengl32 API thru IAT. In those places of code where it should do that, now 'call ptr' commands have new arguments (i compared asm codes of both loaded and unloaded images of same module). This argument leads to an element of new table, placed somewhere between other loaded modules. This is address table of internal functions inside atio6axx module. So now, each opengl call leads to ati driver directly, skipping opengl32 interleaf. This is truly an optimization issue because every call to opengl32 in my own 'unpatched' module after few instructions jumps to corresponded atio6axx function anyway. It's even more obvious when taking into account a fact that when i run CAD application from debugger, the loaded modules are unpatched, in other words: every call to OpenGL API goes thru IAT, as it supposed to do.

The thing that surprises me most, is when i attach to the process with debugger, that specific ati address table resets to the addresses of original opengl32 functions. This costed me few weeks of understanding why hooking directly opengl32 module with hooking libraries wont work if debugger hasn't attached to the process yet, until i started to inspect process memory dumps.

So my questions are:

1. Is it vendor specific optimization, or should i look inside Windows7 DWM and WDDM specs to deal with it?

2. Is there a way to turn that optimization off? Switching Catalyst A.I. options didn't help.

3. Is there a way to locate this specific address table of ATI functions and find out which function represents which opengl call?

4. Is there another methods to intercept OpenGL calls, or to find out it was supposed to be called?

I would appreciate any your suggestions or help. Thank you in advance.

[nuclear_bro]

...ok, little update then.

I've managed to trace who and when patches the host process. After loading into memory the host process seems to be fine, but after it calls first opengl instruction (in my case it is wglChoosePixelFormat) the following modules dynamically loads into the memory: atig6pxx.dll atio6axx.dll atig6txx.dll aticfx64.dll atiadlxx.dll. And after that, the host process is patched and every opengl call from host process leads directly to atio6axx.dll module. This is just for your interest though.

I've also traced that all places where opengl call should occur are patched in one time simultaneously, even before those calls executed at least once. It indicates, that ATI driver has a database with something like a 'call pattern' of each OpenGL based application and it simply patches it in one time according to that pattern. It would be nice to find out what should i do to prevent ATI from using that database on particular application.

But still then, any thoughts how to trace opengl calls in the case where all calls lead directly to vendor's driver?

[pboudier]

the best way to intercept calls is indeed to create  your own opengl32.dl. the ICD will detect this case and skip the IAT patching.

this optimization is IHV specific, although it has been implemented by several vendors already.

regards,

Pierre B.

[nuclear_bro]

Thank you for your reply. I'll definately try this.