cancel
Showing results for 
Search instead for 
Did you mean: 

Archives Discussions

tugrul_512bit
Adept III

How can I query number of remaining(unfinished) commands in a command queue (with or without events)?

I'm adding producer - consumer design into an OpenCL program for multiple GPUs and I managed to make it work at the host-device synchronization level. Now I need to add finer grained queue handling for commands and without adding any synchronization between host and device, only way I can think of is querying a device's command queue's remaining commands and finding the device with minimum commands remaining and assign a command to that device.

How can I know that how many commands are waiting to be processed in a command queue?

This is for OpenCL 1.2.

For now, I'm learning how to use callbacks with markers to count things. Does a firing callback halt its command queue? I wish not because I need finer grained control with less bubbles.

Trying something like this:

callback.cpp · GitHub

but I'm not sure if its the way to do it.

Regards.

0 Likes
2 Replies
tugrul_512bit
Adept III

Now I implemented event callback based control logic, but it has more than 150 microseconds between kernels. Before this, there was only 2-3 microseconds between consecutive kernels(without sync on host, also without control logic). Pure synced(wait,finish) control logic has better load sharing between GPUs but it has the most gap between kernels like 300-400 microseconds. Is there a way to decrease overhead of callback on device side?  Maybe decreasing number of callbacks per kernel (from 1 to 0.1 for example) can make it faster but this time performance-awareness would decrease.

  • no sync, no control logic = 2-3 microseconds (only works with identical GPUs)
  • (I am looking something to fit this position)
  • no sync, callback logic   = 150-200 microseconds (asymmetric multi-GPUs are balanced with error of %2.5 of total kernels)
  • sync                                = 300-400 microseconds (best balance, worst performance)

I will test for multiple command-queues per device when I have time. Maybe this could hide the gap latency to 10-20 microseconds with 10+ queues.

0 Likes

Tested multiple queues, it drops to only about 100 microseconds and only few times. Could all the callbacks be serialized when going to host side even though they are on different command queues?

The latest shape of command queue class is this:

test.cpp · GitHub

0 Likes