cancel
Showing results for 
Search instead for 
Did you mean: 

OpenCL

Skysnake
Adept II

Re: maximum number of GPUs?

very interesting.

So you have NO problems with linux?

Can you post you hardware? I think you will have a UEFI Systems. Perhaps the MB facturers have solved the problem without the knowledge of it.

Have you build the linux bye yourself?

Just post as much informations as you can please. To solve your problem without any informations is not possible.

On the other hand. How have you solved the physical fitting problem? PCI-E riser cables, or single-Slot desing?

Have you checked, if you can use with a OpenCL program all the 8 devices? If yes, please let it know. Be shuhre, that you use all 8 devices please at the same time.

0 Kudos
Reply
pwvdendr
Adept II

Re: maximum number of GPUs?

So you have NO problems with linux?

Several, but all solvable with my limited linux knowledge (just following some good guides). I wrote the steps for installing the drivers here. It's tedious, I've had to re-install a few times for unclear reasons, but in the end this simple instruction set works to get everything up and running.

Can you post you hardware? I think you will have a UEFI Systems. Perhaps the MB facturers have solved the problem without the knowledge of it.

The motherboard (MSI Big Bang Marshall B3) does clearly struggle with the cards. When all 8 are attached, for example, I cannot enter the bios. But the machine does boot up (this only breaks at 9+ cards iirc). I'm writing an article about it with all the hardware, but the only relevant ones for the windows driver problem I mention seem to be the GPUs themselves, which are 8x HD7970.

Have you build the linux bye yourself?

Nope, stock ubuntu 12.04 beta2 with all default settings. But no updates. Updating broke it.

On the other hand. How have you solved the physical fitting problem? PCI-E riser cables, or single-Slot desing?

PCI-e extender cables with external power support (since the MB is not supposed to be able to feed 8x75W and you might fry it with unsupported extenders). But details like this will be in the article I'm writing about it.

Have you checked, if you can use with a OpenCL program all the 8 devices? If yes, please let it know.

Of course I have. Computations have been up and running all weekend.

0 Kudos
Reply
Skysnake
Adept II

Re: maximum number of GPUs?

Great News!

So i have to call some guys i know, and inform them, that a equal project now can start

Linux should be OK for me. The leak of windows hurts, but it is not such important.

So perhaps i can saw also something in the near feature about a 8 GPU System

If you have a nice application, it would be nice if you could post something about it.

Also please let it us know, when you have finished your paper. I really interested to read it

0 Kudos
Reply
Skysnake
Adept II

Re: maximum number of GPUs?

Hi,

since i have some positiv informations, i started to get support for testing a 7 GPU System. It looks like i get everything for this, i want to ask, if AMD would support me to look after the usability/stabebility of such a system.

My maintarget ist to fit everything into a single case with default hardware. So no PCI-E risercables and so on. It would be a great help if AMD could give some hardwaresupport, because of the GPU shortage, it is really difficult to get support from the Boardpartners.

It would be great if AMD could contact me.

0 Kudos
Reply
Skysnake
Adept II

Re: maximum number of GPUs?

Have you some new informations for us?

nVidia have announced there new Tesla cards yesterday. I hope we hear something about the FirePro very soon.

Also they named a GPU with up to 4 GPU-DIEs on a single PCB.

So i really hope, we hear something about the Mutligpu support situation from AMD.

I have now everything together to write a introduction how to build a 24/7 7 GPU system that fit into a single Case. Only 6 7970 left until now

Would be great if i could hear something from you.

0 Kudos
Reply
Skysnake
Adept II

Re: maximum number of GPUs?

Ok, the FirePro are anounced. Are there any new informations about this "multi-GPU" project from AMD???

I am still here, and would test everything, write a how-to and so an, if AMD would provide me with the GPUs.

I hope we hear very soon something about this AMD project. XeonPhi is now paper"launched", and GK110 will be there at Q4 this year or Q1 2013.

0 Kudos
Reply
Starglider
Adept I

Re: maximum number of GPUs?

I am running a grid compute team at a large company and I am trying (once again) to write a business case for using AMD GPUs in the next hardware tranche. It is difficult to justify because of pointless driver limitations. If we could plug eight 7990s into a Tyan barebones and have all 16 GPUs actually work reliably in OpenCL, then I would place an order for 20 nodes (i.e. 320 GPU sales for AMD) immediately. As usual though, AMD have decent hardware but utterly defeat themselves in the professional market with hopeless drivers. There is the pointless and unnecessary 8 GPU limit (which Nvidia cards don't have), the pointless and unnecessary crippling of OpenCL on the 5970, 6990 and most likely 7990 as well (second GPU is unusable for compute due to crossfire locked on - fix promised for the last three years - never delivered - meanwhile Nvidia 590 GTX works just fine), and the bizarre performance drops for even modest multi-GPU setups (again, Nvidia has no such issue).

If AMD could fix even just the 8 GPU limit then we could make a reasonable case that double density (16-GPU) AMD nodes installed now will still have superior price/performance to a standard (8 GK-110) Tesla nodes available later this year, and go ahead with the purchase. Being able to use double-GPU cards to avoid the engineering time of installing PCI-E extenders and splitters would make the case quite compelling. Unfortunately several years of complete AMD indifference to these issues suggests that they will continue to go unresolved and we will end up buying more Nvidia as usual. That's about half a million $ a year from just one customer that could go to AMD, but goes to Nvidia instead because of driver shortcomings,

realhet
Miniboss

Re: maximum number of GPUs?

Hi,

Are you saying that OpenCL is not working on the second GPUs of thouse double-chip cards?

I'm curious because I've never tested my programs with OpenCL only on multiple single-chip cards.

0 Kudos
Reply
Meteorhead
Challenger

Re: maximum number of GPUs?

It is working, just not officially. I have not found a single use-case where it actually produced incorrect results. But if someone has such a case, please (re-)post it, I might have overlooked. We have multiple 5970s in a single machine and they work fine.

0 Kudos
Reply
Skysnake
Adept II

Re: maximum number of GPUs?

Yeah, that is right, but again, AMD say: Yeah we do something" and than you hear nothing from them....

I really can't understand what they do.

0 Kudos
Reply