cancel
Showing results for 
Search instead for 
Did you mean: 

Archives Discussions

riza_guntur
Journeyman III

About HD5970 and 4 boards on one motherboard

I've read from some source about 5970, will it support 4 boards on one motherboard? I mean like 8 GPUs on one motherboard, will it detected properly by CAL runtime this time?

0 Likes
21 Replies
eduardoschardong
Journeyman III

The card has been launched, and so?

May I order 4 of them? Will it work? Motherboard recommendations?

 

0 Likes

you want to buy it eduardo?

I need nome infos since I want to put "but the 5970 can be installed 4 cards in a board, unlike 4870x2" on my paper

0 Likes

Yes... If the 8 gpu per node works and avaliability allows...

 

Performance enough for show.

 

0 Likes

Just buy 4 5970s and give it a try ...

lol 😉

No but seriously, I did that with 4870x2's.  No luck.  I had a msi gd70-790fx and an asrock x58 deluxe running 2 4870x2's and a nvidia gtx260 all doing brook or cuda work.  The minute I go over 4 ATI gpus, the thing crashes.  I was rather exhaustive on my search to get this working, to no avail. It would be great if someone from AMD could comment on this issue.  If 8 GPUs is not going to be supported, I would go with 4 5870s. If it is, I would naturally go with 4 5970s.  I already have a big enough power supply.  Then again, there's the option of getting 2 5970s and putting 2 fermi's in the other 2 slots ... the nvidia architecture is good for some problems ... maybe the two pairs of cards could cooperate 😉

 

 

 

 

 

 

0 Likes

I have a rather encouraging news. I have 5 GPUs working together: two 5970s and one 5870, on 64-bit Linux, with sdk 1.4, and the 9.10 drivers. Although I notice a weird symptom. One of the 5 GPUs seems to run more slowly than the others. For example the "ilperf" program I posted in another thread runs a loop of MAD instructions on all 5 GPUs in parallel. It reports the expected GFLOPS perf for all GPUs except it reports about 800 GFLOPS for one of the 5. The GPU affected by this perf pb seems to be picked at random depending on the order of the video cards in the PCI-E slots. Although in my tests it seems to always be one of the 2 GPUs on one of the 5970s. I can't verify that the 9.11 drivers fix the pb because they make X.org segfault.
0 Likes

Thank you zpdixon, good news, it would be nice if someone from AMD answer this.

 

BTW, for the driver team, why they are threated as graphics devices in Windows (and then problems like remote desktop)? Tesla don't even have a video output and, IIRC, the first stream processors didn't have too, for this system ok, one of the GPUs will be the graphic, but production it's more like a server with a rage or so for Graphics and as many as possible GPUs on PCIe slots...

 

0 Likes

For me it simply amazing that company which pretend to be No. 1 GPU vendor simply don't care at all about their hardware support.

 

It's so easy having access to all new GPUs to grab 4x of them, put into motherboard, get some CPU (which are also produced by same company btw) and make some tests. Why it haven't been done even with 4870x2 -- I have no idea.

0 Likes

@ zpdixon

Thanks for the very encouraging news.  Which linux are you running, and have you tried the beta drivers provided with OpenCL 2.0beta4 ? 

You say 9.11 crashes X.org ... is this the case also for <=4 GPUs?

 

 

0 Likes

Originally posted by: zpdixon I have a rather encouraging news. I have 5 GPUs working together: two 5970s and one 5870, on 64-bit Linux, with sdk 1.4, and the 9.10 drivers. Although I notice a weird symptom. One of the 5 GPUs seems to run more slowly than the others. For example the "ilperf" program I posted in another thread runs a loop of MAD instructions on all 5 GPUs in parallel. It reports the expected GFLOPS perf for all GPUs except it reports about 800 GFLOPS for one of the 5. The GPU affected by this perf pb seems to be picked at random depending on the order of the video cards in the PCI-E slots. Although in my tests it seems to always be one of the 2 GPUs on one of the 5970s. I can't verify that the 9.11 drivers fix the pb because they make X.org segfault.


Maybe other slot run in slower speed, for example 4x or 8x

Because when my motherboard got short circuit, it detected as ax, the card run super slow, less than half normal performance. Though after reboot it is fine again

Can you clarify this?

0 Likes

To answer the various questions:

- I am running Ubuntu 8.04 amd64

- I have not tried the beta drivers that ship with the OpenCL 2 beta, however I doubt they will fix the pb...

- I was doing something wrong when I installed the 9.11 drivers. I have noticed that fglrx.ko seems to fail to fully reinitialize the hw because sometimes my cards are in such a state that even restarting X.org, rmmod'ing and modprobe'ing fglrx is not sufficient to be able to start X.org (it segfaults). I had to hard reset the box for it to work. So the 9.11 drivers now work but I experience the same symptoms as before: one of the 5 GPUs runs more slowly than the others.

- It is not due to PCI-E link width differences. Not only my workload is not bandwidth intensive (purely ALU-bound), but also all my cards are on x1 PCI-E links (via flexible PCI-E adapters). So it cannot explain the 5th outlier GPU.

0 Likes

I bought a 3rd HD5970 and tried running 3 of them on one motherboard but I observed the same weird performance symptoms.

The 9.11 Linux drivers detect 6 GPU devices. My CAL IL code can use them. But some of the GPU devices seem to be operating more slowly than the others. More precisely, 3 of the 6 GPU devices (the 2nd GPU of each HD5970) take longer than the 3 others to run my "ilperf" benchmark tool whose source code I posted in this thread:

'ilperf' Source code


The 3 slow GPUs report ~700 GFLOPS instead of the theoretical 2320 GFLOPS, while the 3 other GPUs report almost 2320 GFLOPS as expected (more precisely 2270).

I can't verify the shader clocks with aticonfig because the tool report an error against the HD5970 (aticonfig: No supported adapters detected). My guess is that for some reason the HD5970 is throttling the shader clock to around 220 MHz instead of 725 MHz. This would explain the ~700 GFLOPS.

When I modify ilperf to only execute on the first 2 devices (2 GPUs of the first card), the same assymetric performance can be observed: 1st device report ~2320 GFLOPS, 2nd device reports ~700 GFLOPS.

Any idea, anyone? This is an interesting pb.

0 Likes

what about only one 5970 in system? if it is problem with too much GPU or it just that second core is underclocked. and try newest catalyst 9.12 with hotfix.

0 Likes

zpdixon, I have reported this to the correct team to look at but since it is the holidays I don't expect it to be worked on until the new year. Most likely CAL is not correctly clocking the second device up when executing on that device.
0 Likes

nou: as I said earlier, with 1 or 2 HD5970 in my system, all GPU devices run at full speed as expected.

Micah: thanks for reporting this to the team!

In the mean time I will give a try to the 9.12 drivers, and maybe the drivers that ship with the SDK 2.0 (if any - I haven't looked at how this SDK is packaged yet)

0 Likes

Micah: any news from your team, about supporting 3 HD5970 on 1 system?

0 Likes

zpdixon,
They are working on a solution for this currently and it will part of a future catalyst release when it is fixed.
0 Likes

We need to set up a machine with 3x5970 cards for a project. Are there any specific motherboards that would be recommended for this ? I was thinking of some high end Asus.

 

Micah: Also when can we expect to have drivers that will support all GPUs (Preferably Linux)

 

 

0 Likes

@frankas

For >4 GPUs, I would suggest a dual quad core xeon motherboard like this one:

http://www.tyan.com/product_board_detail.aspx?pid=641

Though I haven't tried it.  I have dual 5970, and my experience has been that one wants at least 1 core dedicated per GPU ... and at least 1 free to do other OS things ... at least for my applications.

I have tried 2 5970 and 1 5770, and it freezes as it did for my attempted 4x 4850x2 system way back when (Ubuntu 9.10).  If 3 5970 is a want rather than a need, I would suggest 4x5870 ... and an asus p6t7 or an msi 790fx-gd70 ... The former is 4xPCIe 16x but I have not tested.  The latter is 4xPCIe 8x or 2xPCIe 16x and its working great right now!

But please inform if you get >2 5970 working at some point. I've given up.

 

 

0 Likes

Originally posted by: MicahVillmow zpdixon, They are working on a solution for this currently and it will part of a future catalyst release when it is fixed.


Any progress?

0 Likes

If OTOY's announcement of a render cluster based on 4 HD 5970 per server (500 cards, 125 servers) is correct, then presumably AMD has or will soon have drivers supporting 4 such cards per server:

http://www.otoy.com/media/press/launch.html

However technical errors in the press release may indicate they meant HD 5870 instead of HD 5970 ("2.7TFLOPS per card").

OTOY is partnering with AMD on this project.

0 Likes

Any news on that issue? That is, can I fully run 3 or more 5970 on a single motherboard?

 

I have an Asus P6T7 WS Supercomputer motherboard running GNU/Linux and I want to know how many 5970s I am able to plug in that motherboard in order to decide which video card to buy.

0 Likes