cancel
Showing results for 
Search instead for 
Did you mean: 

Archives Discussions

Gunter
Journeyman III

8 GPUs in one PC

Hi,

we are planning to build a PC with 4 HD 6990 in it, for a total of 8 GPUs. In the first instance, this PC will run Windows 7 Enterprise 64-bit, and Stream SDK 2.3.

Is this number of GPUs supported by the driver/SDK? If not, what is the limit?

Thx

0 Likes
14 Replies
genaganna
Journeyman III

Originally posted by: Gunter Hi,

we are planning to build a PC with 4 HD 6990 in it, for a total of 8 GPUs. In the first instance, this PC will run Windows 7 Enterprise 64-bit, and Stream SDK 2.3.

Is this number of GPUs supported by the driver/SDK? If not, what is the limit?

It is possible. You will face bandwidth issue if all PCIE are not working at 16x.  Please use latest driver and SDK.  Presently available SDK is 2.5 and driver is 11.8.

0 Likes

Originally posted by: genaganna

It is possible. You will face bandwidth issue if all PCIE are not working at 16x.  Please use latest driver and SDK.  Presently available SDK is 2.5 and driver is 11.8.

I have a AMD Radeon HD5970 and wasn't aware that using both GPUs on these kinds of cards was officially supported yet.  If there was news about this would you please post a link.  If it's unofficially supported, please let me know what I can do to make it work.

0 Likes

Originally posted by: settle

 

I have a AMD Radeon HD5970 and wasn't aware that using both GPUs on these kinds of cards was officially supported yet.  If there was news about this would you please post a link.  If it's unofficially supported, please let me know what I can do to make it work.

 

OpenCL should work without any issue on x2 cards. Are you facing any issue?

What are you looking in link?

0 Likes

Will you see both devices on a 6990? The release notes don't really say whether this issue has been fixed or not. We're looking at building a similar machine for research, but don't want to waste money on 6990s if we can't use them all. Also, do they make power supplies big enough to drive 4 6990s and 2 magney cours (my guess is you'd want 2-2.5kW).

0 Likes

Originally posted by: rick.weber Will you see both devices on a 6990? The release notes don't really say whether this issue has been fixed or not.


If you check the system requirements for APP SDK 2.5 you can find 5970 listed but not 6990.

http://developer.amd.com/sdks/AMDAPPSDK/pages/DriverCompatibility.aspx

0 Likes

Originally posted by: settle

 

I'm using Catalyst 11.8 and APP SDK 2.5 on my openSUSE 11.4 x86_64 machine and I see two GPU devices corresponding to my 5970.  My concern though is if both devices are fully supported or will they conflict as I don't know how they're sharing the global memory.

 

I was also hoping that that the driver would abstract away the two devices with 20 compute units each into a single device with 40 compute units.  Is there any reason why this route wasn't chosen?  Nothing in the OpenCL specifications seems to conflict with this and I think it would strongly appeal to developers.  Maybe it has to do with that the device has 4 GB of global memory but each GPU can only access 2 GB each, but I'm not sure if that's true or not.

 

OpenCL treats two devices as separte. User is responisble to estabish communication b/w them.  Functionally there is no difference b/w X2 cards and devices connected with crossfire.

0 Likes

Originally posted by: genagannaOpenCL treats two devices as separte. User is responisble to estabish communication b/w them.  Functionally there is no difference b/w X2 cards and devices connected with crossfire.


 

Doesn't OpenCL treat dual CPU socket systems as a single CPU device (maybe I misunderstood this part)? However, if that's true then it would make sense to try to treat a dual GPU socket board as a single GPU device.

http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5782450&tag=1

0 Likes

Originally posted by: settle

 

 Doesn't OpenCL treat dual CPU socket systems as a single CPU device (maybe I misunderstood this part)? However, if that's true then it would make sense to try to treat a dual GPU socket board as a single GPU device.

 

http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5782450&tag=1

 

Yes OpenCL treat dual CPU socket system as a single CPU device and X2 card as two separate devices. Good question? I will clarify as early as possible.

0 Likes

Originally posted by: genaganna

 

Yes OpenCL treat dual CPU socket system as a single CPU device and X2 card as two separate devices. Good question? I will clarify as early as possible.

 

X2 cards will be treated as separate GPU devices.  Dual CPU systems have a much simpler memory architecture that make it much easier to share data between them.  Dual GPU cards have distinct memory for each GPU and no easy mechanism for sharing data.

Jeff

0 Likes

Originally posted by: jeff_golds

X2 cards will be treated as separate GPU devices.  Dual CPU systems have a much simpler memory architecture that make it much easier to share data between them.  Dual GPU cards have distinct memory for each GPU and no easy mechanism for sharing data.

In the AMD Accelerated Parallel Processing OpenCL™ Programming Guide (v1.3c) in Appendix D Device Parameters, Hemlock 5970 is listed as L2 Cache Size / GPU = 2 x 512k and Frame Buffer = 2 GB.  Written that way seems to imply that the two GPU dies share a single global memory pool, similar to two CPU dies share a single host memory pool.  I'm not a GPU hardware expert but I'm very interested in learning more.  Maybe the Frame Buffer should read 2 x 1 GB or 2 x 2 GB if they have distinct global memory pools?  If you could describe the X2 card memory architecture a little more I'd really appreciate it.  Thanks!

0 Likes

IMHO, it should be written as 2 x 1GB. I will confirm and report.

0 Likes

Originally posted by: himanshu.gautam IMHO, it should be written as 2 x 1GB. I will confirm and report.

 

Hi Himanshu,

Have you been able to confirm if the global memory is in fact 2GB or 2 x 1GB?

0 Likes

settle,

I have requested a correction to change 2GB to 2x1GB. It will be updated in document.

Thanks for your query.

0 Likes

Originally posted by: genaganna

 

OpenCL should work without any issue on x2 cards. Are you facing any issue?

 

What are you looking in link?

 

Going back as far as this thread it was an issue on all x2 cards, but I'm not aware of when/if it was fixed.  I didn't find it anywhere in the Catalyst driver pdf files or APP SDK pdf files.

http://forums.amd.com/forum/messageview.cfm?catid=390&threadid=140518&highlight_key=y&keyword1=2x%20GPU

 

 

0 Likes