cancel
Showing results for 
Search instead for 
Did you mean: 

Archives Discussions

Arakageeta
Journeyman III

Opteron 8-GPU systems?

Do Opteron systems w/ 8 double-wide GPUs exist?

I'm a grad student looking to purchase a many-GPU system for our research lap.  I'd like to maximize both CPU core and GPU count.  The best I can find that maximizes both of these is a Tyan Intel-based system (http://www.tyan.com/product_SKU_spec.aspx?ProductType=BB&pid=412&SKU=600000188), which supports 12 CPU cores and 8 double-wide GPUs.  Is there any sort of equivalent (or better) in the AMD world?  The best I can find is a SuperMicro system that can sport 4 double-wide GPUs.  I've read that a company called Aprius had planned to produce an 8-way GPU system, but it looks like the product may have been cancelled.


I would much prefer an Opteron-based system.  I believe it's memory/cache hierarchy may provide more deterministic program execution (something that is important in our research).


Any recommendations?  Please tell me these systems exist!

0 Likes
21 Replies
empty_knapsack
Adept II

Opteron 8-GPU systems?

ATI drivers doesn't support more than 4 GPUs within single system. End of story.

0 Likes
gaurav_garg
Adept I

Opteron 8-GPU systems?

Ignore this message. Forum was giving an error when I replied.

0 Likes
gaurav_garg
Adept I

Opteron 8-GPU systems?

ATI drivers doesn't support more than 4 GPUs within single system. End of story.


This is not true. We have recently installed a 8-GPU (Firestream 9250) per node cluster on a customer site.

We are using 2 PCIe expansion system (each containing 4 GPUs) to add total 8 GPUs on one node.

0 Likes
Arakageeta
Journeyman III

Opteron 8-GPU systems?

What is the effect of using a PCI expansion box?  I presume it limits the memory/communication bandwidth available?

0 Likes
jross
Adept I

Opteron 8-GPU systems?

Originally posted by: gaurav.garg

 

This is not true. We have recently installed a 8-GPU (Firestream 9250) per node cluster on a customer site.

 

We are using 2 PCIe expansion system (each containing 4 GPUs) to add total 8 GPUs on one node.

 

Gaurav, did this actually work?  Runing FindNumDevices returns 8?  Can you give details of the Linux OS/kernel, ATI driver version, and other configuration details?

0 Likes
empty_knapsack
Adept II

Opteron 8-GPU systems?

Yes, Gaurav, can you provide more details? If situation finally changed and 4+ GPUs supported by drivers it's definitely good news.

... Though support for 5970 is still under big question...

0 Likes
gaurav_garg
Adept I

Opteron 8-GPU systems?

Following was the configuration:

OS: Red Hat 5.3 64-bit
ATI driver: Catalyst 10.4
CPU: Two Quad-Core AMD Opteron per node
Chipset: NVIDIA nForce Professional 3600 and 3050
32 GB RAM

0 Likes
gaurav_garg
Adept I

Opteron 8-GPU systems?

Gaurav, did this actually work?  Runing FindNumDevices returns 8?  Can you give details of the Linux OS/kernel, ATI driver version, and other configuration details?


Yes, FindNumDevices as well as OpenCL showed 8 GPUs. We also ran a OpenCL program that used all 8 GPUs.

0 Likes
rotor
Journeyman III

Opteron 8-GPU systems?

Hi Gaurav,

Can you give us a hint of which pci-expansion card/box do you use and which interface do you use to hook up the pc-expansion card to the system? And as other asked: do you lost bandwith in the expansion card?

Thanks,

Roto



0 Likes