I have a kernel that executes a few times per second. There are 2 anomalies that I can't figure out.
1 (less important). Every 3-4 seconds the gap between the end of the kernel execution and the start of the next kernel execution gets bigger, then backs to normal. I can reduce the time of the gap by using multiple queues, but anomaly stays.
2 (more important). Every 7-8 seconds the execution time of the kernel gets bigger for one iteration then backs to normal. It doesn't matter if the kernel executes once or ten times per second. A slowdown always comes after 7-8 seconds.
Thank you for reporting this. It would be helpful for us if you can share some profiling data (like application timeline trace using CodeXL) that manifests the above issues. Also please share your setup details.
Thanks for the reply,
It seems that I have found the cause of the problem no. 2.
It has nothing to do with the driver, at least there is no direct connection. It looks that the main cause of the slowdown is GPUz (when running). It calls the driver on a very regular interval and generates small slowdown.