0 Replies Latest reply on Oct 17, 2011 7:02 PM by cadorino

    Strange memory measures: gpu timer vs cpu timer

    cadorino
      I get some incredible different results when I try to measure bandwidth and completion time using cpu timer vs gpu timer

       

      Hi,

      I'm performing some memory tests on a pc (cpu + discrete gpu) and on an apu.

      In particular, my test consists in writing Y bytes X times to find out the completion time and the average bandwidth. I do this for all the possible allocation strategies for the source and the destination buffers.

      I tried to compare the results given by using cpu timers (on windows, queryPerformanceCounter) to those obtained using gpu timers, by now only on the cpu + discrete gpu.

      The difference between thos two measures is so huge that I'm sure I made some mistakes.

      Here is an example:


      Testing transfer of 1024 bytes 16 times...

      Unpinned -> Unpinned

      CPU timer: 6259.02 Mbytes/s (total time: 0.02 ms)

      GPU timer: 6368.19 Mbytes/s (total time: 0.02 ms)

      Unpinned -> Device

      CPU timer: 4.10 Mbytes/s (total time: 4.65 ms)

      GPU timer: 13611.35 Mbytes/s (total time: 0.03 ms)

      Pinned   -> Unpinned

      CPU timer: 3.94 Mbytes/s (total time: 4.33 ms)

      GPU timer: 7492.48 Mbytes/s (total time: 0.03 ms)

      Pinned   -> Pinned

      CPU timer: 3.73 Mbytes/s (total time: 5.23 ms)

      GPU timer: 9359.60 Mbytes/s (total time: 0.04 ms)

      Pinned   -> Device

      CPU timer: 3.30 Mbytes/s (total time: 5.64 ms)

      GPU timer: 12743.39 Mbytes/s (total time: 0.03 ms)

      Device   -> Unpinned

      CPU timer: 4.70 Mbytes/s (total time: 3.69 ms)

      GPU timer: 11231.09 Mbytes/s (total time: 0.03 ms)

      Device   -> Pinned

      CPU timer: 4.37 Mbytes/s (total time: 3.79 ms)

      GPU timer: 8819.22 Mbytes/s (total time: 0.04 ms)

      Device   -> Device

      CPU timer: 7.78 Mbytes/s (total time: 2.15 ms)

      GPU timer: 8876.45 Mbytes/s (total time: 0.04 ms)


       

       

      In case of 16 times 16 Mbytes, I get:


      Pinned   -> Device

      CPU timer: 3317.04 Mbytes/s (total time: 81.06 ms)

      GPU timer: 3837185.08 Mbytes/s (total time: 0.10 ms)


      i.e. 3Tbytes of bandwidth, which is practically impossible, especially for a transfer host->device, which should be limited by the PCI bandwidth.

       

       

      I really need and help to find the mistake, or to be told why I get such different results.

      I show you the piece of code where I compute the completion time for a transfer between a pinned source buffer and a destination buffer allocated on the device. The other cases are really similar.

      Some hints regarding the code:

      1) DATATYPE is a macro actually set to "int"

      2) The struct Timer is contained in an utility library. I report the code of timer (just in case the mistake is in there) at the end of the post

      Thank you very much!

       

      //profile with gpu timer if(!gpu_timer) { timer.start(); src_pointer = (DATATYPE*)clEnqueueMapBuffer(queue, src, CL_FALSE, CL_MAP_READ, 0, size * sizeof(DATATYPE), 0, NULL, NULL, NULL); for(int i = 0; i < NUM_TRANSF; i++) clEnqueueWriteBuffer(queue, dst, CL_FALSE, 0, size * sizeof(DATATYPE), src_pointer, 0, NULL, NULL); clFinish(queue); time = timer.get(); } //profile with cpu timer else { src_pointer = (DATATYPE*)clEnqueueMapBuffer(queue, src, CL_FALSE, CL_MAP_READ, 0, size * sizeof(DATATYPE), 0, NULL, &transfer_event, NULL); clWaitForEvents(1, &transfer_event); clGetEventProfilingInfo(transfer_event, CL_PROFILING_COMMAND_QUEUED, sizeof(cl_ulong), &start, 0); for(int i = 0; i < NUM_TRANSF; i++) clEnqueueWriteBuffer(queue, dst, CL_FALSE, 0, size * sizeof(DATATYPE), src_pointer, 0, NULL, &transfer_event); clFinish(queue); clWaitForEvents(1, &transfer_event); clGetEventProfilingInfo(transfer_event, CL_PROFILING_COMMAND_END, sizeof(cl_ulong), &end, 0); time = (double)1.0e-9 * (end - start); } double bandwidth = ((double)(NUM_TRANSF * size * sizeof(DATATYPE)) / (double)time) * 1000.0 / 1000000.0; result.total_time = time + alloc_time; result.bandwidth = bandwidth; //Code of struct timer typedef struct Timer { LARGE_INTEGER frequency; LARGE_INTEGER start_time; void start() { QueryPerformanceFrequency(&frequency); QueryPerformanceCounter(&start_time); } double get() { LARGE_INTEGER end; QueryPerformanceCounter(&end); double elapsedTime = (end.QuadPart - start_time.QuadPart) * 1000.0 / frequency.QuadPart; return elapsedTime; } } Timer;