cancel
Showing results for 
Search instead for 
Did you mean: 

Archives Discussions

pwvdendr
Adept II

maximum number of GPUs?

Is there a maximum number of GPUs that can be used for OpenCL computing?

I heared there is a hard limit of 8 with the current linux drivers, but I'm not sure on windows. Technically it's not so difficult to build a compute cluster consisting of 7 dual GPUs on a single mother board (say 7x HD6990 or soon even 7x HD7990). But I just want to verify that the drivers will actually support this under windows, since I heared they don't support it under linux (only up to 8).

54 Replies

Ok, FirePro is out.

Are there now any new Informations???

0 Likes
pwvdendr
Adept II

So... we're 6 months later, so let's ask again. Are there any updates on this? Is there any work on making the drivers support more cards?

0 Likes

Nothing i know.

To hard, that AMD is so quiet.

0 Likes

Ok, there are some new Infos:

Nvidia and Advanced Micro Devices on Monday announced high-performance graphics chips for supercomputers.

Nvidia announced GPUs (graphics processing units) called K20 and K20X, with the latter being used in Titan, a 20-petaflop supercomputer at the U.S. Department of Energy's Oak Ridge National Laboratory. AMD announced the FirePro SM10000 graphics processor, which is targeted at high-performance computers and servers in virtualized environments.

Co-processors like GPUs are considered more powerful that CPUs for specific tasks such as scientific and math applications. The GPUs are important in providing more computing power for simulation and experimentation in research areas such as biosciences, climate, energy and space. IBM and Intel also offer accelerators for supercomputers.

[ Get served with the latest developments on data centres and servers in Computerworld's Storage newsletter ]

Some of the world's fastest supercomputers today harness the processing power of CPUs and graphics chips for complex calculations. The Titan supercomputer pairs 18,688 Nvidia Tesla K20X GPUs with 18,688 AMD 16-core Opteron 6274 CPUs, with the GPUs handling 80 to 90 percent of the processing load. Other supercomputers that pair CPUs and GPUs include the Tianhe-1A at the National Supercomputer Center in Tianjin, China.

Nvidia has a big lead over AMD in GPUs for supercomputing, said Dan Olds, principal analyst at Gabriel Consulting Group.

Nvidia pushed parallel programming tools many years ago so coders could write applications for GPUs, Olds said. AMD has virtually no presence in the supercomputing market and needs to foster a programming environment for parallel frameworks like OpenCL to be a worthy alternative to Nvidia, Intel and other companies, Olds said.

Nvidia's K20 has 5GB of memory and delivers 1.17 teraflops of double-precision performance and 3.52 teraflops of single-precision performance. Double-precision performance is more important for supercomputing applications as it carries higher precision for floating-point calculations than single-precision calculations. The faster K20X has 6GB of memory and delivers 1.31 teraflops of double-precision performance. The K20X is three times faster than its predecessor, the Tesla M2090, which was released in the middle of last year.

The K20 products will be available in computers from companies such as Hewlett-Packard, IBM, Asus, Fujitsu, Tyan, Quanta Computer and Cray. Nvidia declined to provide pricing, saying the GPUs would be sold through server vendors.

The new chips have thousands of small processing cores that will be able to more effectively execute application code simultaneously. The Hyper-Q feature will speed up execution of legacy code through smarter scheduling of code execution.

AMD claimed that its FirePro SM10000 delivered 1.48 teraflops of peak double-precision performance. The graphics card has 6GB of memory.

The SM10000 is designed for multiple server deployments, AMD said in a slide presentation. GPUs in servers are capable of deploying virtual desktops to client devices like PCs and tablets. The GPU can accelerate graphics on the server side for full high-definition virtual desktops on client devices.

The faster processing speed of SM10000 could also help deploy virtual machines at a faster rate in computing environments, AMD said. A single graphics card will be able to deploy many virtual machines, and AMD has worked with Citrix, VMware and Microsoft to boost virtualization performance on the GPU.

AMD is also targeting the FirePro SM10000 at workstations. The company did not immediately comment on questions related to single-precision performance and pricing for the graphics card.

Agam Shah covers PCs, tablets, servers, chips and semiconductors for IDG News Service. Follow Agam on Twitter at @agamsh. Agam's e-mail address is agam_shah@idg.com

Join the Computerworld Australia group on Linkedin. The group is open to IT Directors, IT Managers, Infrastructure Managers, Network Managers, Security Managers, Communications Managers.

http://www.computerworld.com.au/arti...upercomputing/

So the here announced "multi-GPU" project seems to be the W10000 card with 2 Tahiti Chips.

Would be nice to read something about this card here.

0 Likes
antonyef
Adept II

If it would help, a few words about  our group's experience.

Half a year ago we faced a question: which operating system to choose for development.

Under Windows we didn't manage to launch more than four cards. Tried Windows 7 and Windows Server 2008, all drivers, tricks like VGA dummies, - with no results.

Under Linux 8 cards work  well without additional shaman dances and even produce correct calculation results

http://devgurus.amd.com/servlet/JiveServlet/showImage/2-1284414-1176/amd_stand.png

We didn't have an opportunity to test 9 cards or more, when we do, we'll write about it.

antonyef wrote:

If it would help, a few words about  our group's experience.

Half a year ago we faced a question: which operating system to choose for development.

Under Windows we didn't manage to launch more than four cards. Tried Windows 7 and Windows Server 2008, all drivers, tricks like VGA dummies, - with no results.

Under Linux 8 cards work  well without additional shaman dances and even produce correct calculation results

http://devgurus.amd.com/servlet/JiveServlet/showImage/2-1284414-1176/amd_stand.png

We didn't have an opportunity to test 9 cards or more, when we do, we'll write about it.

This is similar to my experience: 8 cards work perfect under linux without much (software) issues, but Windows is much more restricted. I did get 5 working though under Windows, not sure why you strand at 4.

I'm curious whether you will get 9 working. As problems may start as early as the bios boot.

0 Likes
Skysnake
Adept II

Since when is this topic moderated?

I try to post a link to a 16 GPU-Server, and the moderator say no...

People like pwcdendr would be very happy to read something about this. And i think they would contact this people for some questions.

0 Likes

Did a human moderator say no, or did your post appear as "awaiting moderator approval" and did it not get accepted?

Sometimes the AMD forum triggers strange censoring rules. E.g. one cannot type the expression 'to get the ba_ll rolling' without underscore, due to the word ba_ll (without underscore) being banned. Perhaps your post triggered a similar strange rule?

0 Likes

Perhaps, because it was like you say. First the "awaiting moderator approval" and than some time/hours later "PLOP" and it was no longer there...

I don't really like something like this..

Especial, because this is a very helpful thread, that is now nearly 3/4 years old, and there a people who really really long talks about what AMD now do...

So its not really good marketing what AMD do here.

0 Likes

Thx, for me it was not possible to post this

And some mor informations what is possible with something like this monster

http://passwords12.at.ifi.uio.no/Jeremi_Gosney_Password_Cracking_HPC_Passwords12.pdf

0 Likes
nexusts
Journeyman III

The GPU limit in Windows 7 is 32 GPUs per OS. Three are reserved for remote services and the primary desktop.

0 Likes

Interesting, it looks like no such kind of hard limit is set in modern Linux system. Instead, there is a kernel configuration option named "Maximum number of GPUs" under Linux, and we can set it to an arbitrarily large value freely.

In practice, I can confirm that more than 16 GPUs running on a single motherboard is possible, for I have just built such a rig by myself (with 18 GPUs, on a Supermicro X9DRX board). See https://devtalk.nvidia.com/default/topic/649542/cuda-setup-and-installation/18-gpus-in-a-single-rig-...

However, just like the FASTRA2, my rig is also built with NV cards. Hope AMD can solve the driver issue so we can build a similar or even more powerful rig with AMD cards.

0 Likes


Thanks for the suggestion and input.

AMD will definetely come back on this if required.

0 Likes

Revisiting this question again.

Is there still a 8 AMD GPU limit within linux/windows? I would like to pack more than 4 295x2 within a system...