Archives Discussions

[SiegeLord]

As the title says, after I reboot my machine, I find my OpenCL programs run 2-3x faster on both the CPU and GPU. After awhile (I haven't been able to figure out precisely when, but when testing I just repeatedly invoke the programs, alternating between CPU and GPU) the performance rapidly drops back to "normal" levels and stays that way indefinitely afterwards (until I reboot again). I managed to reproduce this with at least one SDK sample. For example, here's the (edited) output right after boot:

./ScanLargeArrays -q -t -x 2048 -i 100 --device gpu
-------------------------------------------
Elements                 Time(sec)                kernelTime(sec)         
2048                     0.53979                  0.00079

./ScanLargeArrays -q -t -x 2048 -i 100 --device cpu
-------------------------------------------
Elements                 Time(sec)                kernelTime(sec)         
2048                     0.08662                  0.00062

And after the slowdown happens:

./ScanLargeArrays -q -t -x 2048 -i 100 --device gpu
-------------------------------------------
Elements                 Time(sec)                kernelTime(sec)         
2048                     0.56563                  0.00263

./ScanLargeArrays -q -t -x 2048 -i 100 --device cpu
-------------------------------------------
Elements                 Time(sec)                kernelTime(sec)         
2048                     0.11062                  0.00162

The times are representative.

My specs:

OS: Ubuntu 10.10 x86_64

Driver: 11.3

CPU: AMD Phenom II X6 1090T

GPU: ATI Radeon HD 5830

SDK: 2.4

EDIT:

So my question is... why does this happen? Is there any way I can make the speedup permanent? I tested a bit more and it seems that I can just let the system wait for 5 or so minutes, and the first time I run the application it will run in the slow "mode".

[himanshu_gautam]

That is really weird.

Do you see this with all the samples or this particular sample.

[adm271828]

What does cpufreq report?

A conservative governor could explain the variation at least for the CPU, and perhaps for the GPU if communication between CPU and GPU dominates pure kernel execution. When I do some timing test, I always use the performance governor. That way, there is no surprise.

Regards,

Antoine

[SiegeLord]

Originally posted by: adm271828 What does cpufreq report?

A conservative governor could explain the variation at least for the CPU, and perhaps for the GPU if communication between CPU and GPU dominates pure kernel execution. When I do some timing test, I always use the performance governor. That way, there is no surprise.

Regards,

Antoine

You're right, thanks. If I use the 'performance' governor, it runs at the fast speed, and if I use the 'ondemand' governor it runs at the slower speed.

I wonder if this is indicative of the 'ondemand' governor not being fast enough to switch to a higher frequency (perhaps the kernel runtimes are too small) or is it that OpenCL on the CPU/GPU don't really tax the CPU enough for the scaling to kick in?

[nou]

it can be mix of both.when i run my program i have 99% of GPU load but zero CPU load.

[himanshu_gautam]

Hi,

Can you tell me how the system behaves when you run a kernel which runs for a considerable time like running the matrix mul for large sizes and large iterations. Do you see some improvement when you average the performance?

[SiegeLord]

I didn't want to increase the matrix size, since it's already somewhat large. Running it for 10000 iterations (the whole call now took a few seconds) yielded the same result as above.

Either way, in my actual use case the runtime of the program is very short so these sorts of tricks won't be of any use. I will make a "turbo button" for when I need maximum performance.