Our application uses DX9 to render a fairly large section of terrain with a large number of fairly complex objects.
We think the problem we are seeing is related to a known Microsoft bug with DX9 (http://support.microsoft.com/kb/2026022), in that we are limited in the amount of GPU memory we can utilise before we start experiencing major stuttering, however this value changes with the amount of system memory installed. By monitoring GPU memory usage with Process Monitor we see that when the stutters occur the amount of Dedicated GPU memory and System GPU memory is in an almost constant state of flux as we move around the terrain, even though total amount of GPU memory in use is well below the amount installed on the GPU(s). The timing of the stutters coincides with the changes in GPU memory utilisation.
Altering the amount of system memory installed in the PC changes this behaviour. For example on a PC with 8GB of system memory and a single 4GB 6990 with one particular terrain map which utilises around 800MB of GPU memory we see the stutters and changing memory utilisation, however if we drop the installed system memory down to 4GB this behaviour goes away and we can run stutter free without any variation in GPU memory utilisation.
If we lower the amount of GPU memory we attempt to use (e.g. by running with lower resolution textures, reducing the amount and/or complexity of objects rendered or by dropping the level of SSAA) then we are sometimes able to run stutter free with 8GB or even 16GB of system memory.
Another point worth noting is that if we run with Crossfire disabled the amount of GPU memory we are able to utilise before the stuttering starts increases substantially.
We have tested for this problem across a range of AMD GPUs including single and dual 5870s, single and dual 6990s, single and dual 7970s and a triple GPU system with a 6990 and a 6970 and have noticed that issues occurs on all of them, however the amount of GPU memory we can utilise before the stutters occur does change depending on the amount of memory on both the system and GPU.
We always run under Windows 7 Pro x64, however we have also tested with the Windows 8 release preview and see the same issue.
Interestingly with GPUs from other manufacturers we do not see this problem at all, we are able to utilise as much of the installed GPU memory as we require without seeing any changes in GPU memory utilisation or stutter.
Are there any known workarounds for this issue?