OpenCL

[eric880212]

I am trying to test the OpenCl 2.0 SVM features. My machine has the following SVM capabilities:

• CL_DEVICE_SVM_COARSE_GRAIN_BUFFER: Yes
• CL_DEVICE_SVM_FINE_GRAIN_BUFFER: Yes
• CL_DEVICE_SVM_FINE_GRAIN_SYSTEM: No
• CL_DEVICE_SVM_ATOMICS: No

I am trying to write a code that the OpenCl kernel will keep computing until the host side change the value of the flag to stop the kernel.

On the host side:

n is the buffer that I have to compute, and garbage is just a useless buffer, stop_compute is a flag for the host side to inform the device side to stop compute.

I use a while loop to check eventStatus, once the kernel finished, then it will be set to 0, then the host side will leave the loop.

The host side will change *stop_compute to 1 after 250 us .

I launch only 3 work items for testing.

int* n = (int*) clSVMAlloc(context, CL_MEM_READ_WRITE | CL_MEM_SVM_FINE_GRAIN_BUFFER, sizeof(int) * 6, 0);
int* stop_compute = (int*) clSVMAlloc(context, CL_MEM_READ_WRITE | CL_MEM_SVM_FINE_GRAIN_BUFFER, sizeof(int) * 1, 0);
int* garbage = (int*) clSVMAlloc(context, CL_MEM_READ_WRITE | CL_MEM_SVM_FINE_GRAIN_BUFFER, sizeof(int) * 3, 0);

memset(n, 0, sizeof(int) * 6);
memset(stop_compute, 0, sizeof(int));
memset(garbage, 0, sizeof(int)*3);

//set kernel argument and enqueue launch kernel
double timer = currentSeconds();
cl_int eventStatus = 1;

while(eventStatus) {
    clGetEventInfo(event, CL_EVENT_COMMAND_EXECUTION_STATUS, sizeof(cl_int), (void *)&eventStatus, NULL);
    fprintf(stderr, "eventStatus is %d\n", eventStatus);
    if((currentSeconds() - timer) * 1000000 >= 250) {
        *stop_compute = 1;
        clFinish(commands);
        fprintf(stderr, "kernel stop !! \n");
    }
}

On the device side:

It keeps computing in the array n, then leave the loop after *stop_compute be changed.

printf("before loop, *stop_compute is %d\n", *stop_compute);
while(true) {
    if(*stop_compute == 0) {
        if(n[tid] <= 100)
            n[tid]++;
    }
    else {
        *garbage += 1;
        break;
    }
    barrier(CLK_LOCAL_MEM_FENCE | CLK_GLOBAL_MEM_FENCE);
}
printf("after loop, *stop_compute is %d\n", *stop_compute);

The behavior what I expected of the device side is that the array n be added many times, and the buffer stop_compute be changed, then leave the loop. So maybe I would see the value of the array n be changed to 100 , and the value of garbage will be 1 on the host side.

But it turns out that the kernel hangs and printf() doesn't work on the device side.

As for the host side, it can't leave the while loop and keeps printing "eventStatus is 2".

I want to ask, is the problem caused by not using atomic? If my GPU has the atomic svm capability and I used CL_MEM_SVM_ATOMICS while creating svm buffers, will the behavior of my code act as what I expected? If so, I also want to ask which AMD product supports the svm capability of atomic. Is it must be an APU (an integrated Graphics card rather than a dedicated Graphics card) to support atomic or there is also a dedicated Graphics card that can support atomic?

[dipak]

Hi @eric880212 ,

Thanks for your query. I have moved the post to the OpenCL forum and whitelisted you for the AMD Devgurus community.

Thanks.

[dipak]

From the above description, yes, it looks like you need to use SVM atomics operations in this case. As the spec says:

"If SVM atomic operations are supported, they provide memory consistency for loads and stores by the host and kernels executing on devices supporting SVM. This means that the host and devices can concurrently read and update the same memory. The consistency provided by SVM atomics is in addition to the consistency provided at synchronization points."

For the other SVM atomics support related query, could you please share the OS and driver information?

Thanks.

[eric880212]

Thanks for replying me. My OS is centos 7. 
I'm not sure how to get my driver information, but I find a directory named "amdgpu-pro-20.20-1089974-rhel-7.8", and I think it's the driver I'm using. 

[dipak]

Thanks for the information. I will check with the OpenCL team regarding your query about the SVM atomics support on AMD products.

Thanks.

[eric880212]

I'm looking forward to your reply, thanks !

[dipak]

I have forwarded the query to the OpenCL team. As soon as I get any information on this, I will share with you.

Thanks.

[eric880212]

Hi, did you get any information? Or may I get the contact information to the OpenCL team then I can ask them myself, thank you.

[dipak]

Below is the feedback from the OpenCL team on the SVM atomics support related query:

"dGPUs can support SVM atomics if PCIE atomics are available. However, there is an extra side note. Fine-grain buffer means system memory. Hence, the app must be careful what to mark as coarse-grain, fine-grain and fine-grain atomics, because each level will have performance impact."

Thanks.

[eric880212]

Which AMD products support SVM atomics ? Or where can I get this information ? I think I can buy one graphic card that supports SVM atomic.

Thanks.

[eric880212]

Hi, can you tell me which dGPU that AMD sells can support SVM atomics ?

And how do I know if PCIE atomics are available or not ?

I really appreciate your help very much.

[dipak]

Here is an old documentation that provides some insights about PCIE atomics.

https://github.com/RadeonOpenCompute/ROCm_Documentation/blob/master/Installation_Guide/More-about-ho...

Thanks.