cancel
Showing results for 
Search instead for 
Did you mean: 

Archives Discussions

fredriko
Journeyman III

OpenCL multi device memory problem

When using OpenCL together with AMD Radeon HD 7990 that is supposed to have 3 GB memory on each of its two GPUs, I can use (nearly) all of the 3 GB when using one GPU but only about half of the total 6 GB in total while using both GPUs.

I am using separate OpenCL contexts for the devices.

Any idea about what can be the problem?

0 Likes
15 Replies
hsaigol
Adept III

Are you using Linux or Windows.
I  believe in linux only half the memory on the GPU is available so your 6GB card only shows 3GB.

0 Likes

hsaigol, we're on a linux box, Ubuntu 13.04.


thanks!

0 Likes

Here is a post by the member nou who which backs up my claim of half the Vram available under linux environment in opencl

Re: Blender Cycles(Opencl on AMD GPUS)

search for the text:

OpenCL memory size under Linux is currently limited to 50% of total VRAM.

0 Likes

Have you tried adding this on your environment variables:

export GPU_MAX_ALLOC_PERCENT=100

0 Likes
himanshu_gautam
Grandmaster

Are you on a 32-bit system (or) 64-bit system?

0 Likes

Hi,

I'm on a 64-bit system, so the address space shouldn't be a problem.

0 Likes

Are you talking about buffer size limitation of single clCreateBuffer() call?

(or)

Is this the sum total of all allocations you can possibly make on it?

If you can give some repro case, it would be useful to check with Engg team.

0 Likes
fredriko
Journeyman III

Lantis: Yes I have tried a few different versions of that and GPU_MAX_HEAP_SIZE

A new development has occurred. I have less problems with the memory in the GPU when doing manual garbage collection regularly, I am using JavaCL. But that is a quite bad workaround.

0 Likes

JavaCL is just a binding to OpenCL right? So, Garbage collection occurs only for code running on CPU.

It should not really matter to the GPU.

Are you talking about buffer size limitation of single clCreateBuffer() call?

(or)

Are you talking about the sum total of all allocations you can possibly make on it?

If you can give some repro case, it would be useful to check with Engg team.

0 Likes

I know that it should not matter with garbage collection. I am thinking that it might be some pointer in java or something that might not be correctly released or something.

It is the sum of all allocations that is the problem. I am trying to do some debugging with CodeXL but it does not work with my java program for some reason. I get SIGSEV directly, outside the debugger it works.

I will see if I can strip it down to a reproducible test I can send you, right now it uses jboss and a lot of  3rd party jars.

0 Likes

I think why what you see -- could actually be not a bug....

We need to understand that a "cl_mem" object is associated with the Context and not with the device.

Say the Runtime allows you to allocate cl_mem objects -- sum total of which is ~6GB spanning 2 devices....

Say you launch a kernel that requires all these buffers -- then the Runtime cannot actually place all these buffers inside 1 device..

So, there is no point in exposing 6GB of RAM....

This is not a very hypothetical situation.. Consider this - max buffer allocation should at least be 256MB on these devices.

12 such buffers will saturate 3GB. 24 for 6GB.

If you write a kernel with 24 arguments, each pointing to a buffer, it is technically impossible for the Runtime to schedule this with all buffers on the device.... In such a case, the Runtime has to keep 3GB on device and remaining 3GB on some pinned memory locations... This will complicate things a lot...

So, Possibly, the runtime has limited this...I will go check.

-

Bruhaspati

0 Likes
himanshu_gautam
Grandmaster

oh.. I did not realize you had separate context for these 2 devices....

If you could give me a repro-case, I can try it over here...

Thanks,

0 Likes

I have a problem with getting you a reproducible case that can be packaged in a form that are not to big. I will come back later on this case.

0 Likes

Sure, Thanks. Will wait...

- Bruha...

0 Likes

Reviving the thread.  Do you still see this issue?  Can you provide a reproducible test case?

--Prasad

0 Likes