we want to report an ATI OpenGL driver issue regarding "outline" rendering in OpenGL, i.e. rendering black lines around objects for a Comic/CellShading look.
we are using the "normal" method to render outlines:
1.) render objects normally
2.) setup OpenGL to render backfaces and as lines ( glCullFace(GL_FRONT); glPolygonMode(GL_FRONT_AND_BACK, GL_LINE); )
2.1.) switch to a simple shader that just draws black or disable lighting and use glColor(black) in fixed function
3.) render objects again
4.) reset (glCullFace(GL_BACK); glPolygonMode(GL_FRONT_AND_BACK, GL_FILL);)
this works fine on Mac OS X with all cards, and with NVIDIA on windows and Linux. it also works fine with most ATI cards on windows/linux. however on lower end "mobility" ATI cards there are issues - the black object outlines are not drawn around the objects, but rather appear as random lines around the screen, completely destroying the image - additionally performance goes way down.
more specifically this issue appears with an ATI Radeon HD 2600 Pro Mobility in an iMac, even with the latest catalyst drivers (on windows and linux).
on a Radeon HD 4850M (another iMac) this issue exits up to Catalyst from september 2011, but seems to be mostly solved in newer versions.
we've tried various changes to the outline rendering algorithm, e.g. render the back faces first, or properly drawing lines instead of using glPolygonMode(), but none of the changes completely solved the issue on the affected cards.
you should be able to reproduce the issue on affected cards using our game:
what we would like:
0.) confirmation of this problem with with a list of all affected cards
1.) this issue to be fixed in december catalyst on all cards ;-)
2.) workaround code for outline rendering on the affected cards. note that rendering black outlines as a post process (or even with geometry shaders) is not possible on the affected cards - doing black outlines as post process is much slower on the Radeon 2600 presumably because of bandwidth issues.
p.s. if this isn't the proper place to report ATI driver issues, sorry. there doesn't seem to be a proper bug-report place.