7 Replies Latest reply on Jul 21, 2010 7:22 AM by johnrdore

    Accessing both GPUs on a 4870x2 (using OpenCL) [RESOLVED]

    johnrdore
      and a few other questions

      Hi

      I'm fairly new to GPU programming and OpenCL. I was wondering if anyone could give me some advice on a few questions.

      I have two Sapphire Radeon HD 4870x2 cards installed (with CrossFire disabled) on a 64-bit Ubuntu 10.04 machine. I'm using the ATI Catalyst 10.6 driver and Stream SDK v2.1.

      Firstly, each Radeon card has two GPU cores on it (aticonfig --lsa shows four adapters), but it seems that ./FindNumDevices only detects two "devices" (i.e. each dual-core card is a single device, not two separate devices). From what I've read on the other threads in this forum, this is the correct behavior and it is not currently possible to use Stream to access each GPU core individually. However:

      1) Are there any plans to add this functionality to the SDK in the future?

      2) Does anyone know if both GPU cores are actually being used when a card is processing a task, or is Catalyst only driving one of the GPU cores on each card?

      On an unrelated note, I looked through the output of ./CLInfo and was wondering:

      3) The "image support" is set to "no" for my CPU device and both my GPU devices. Is this normal?

      4) The global memory size for each GPU device is reported as "268435456". This translates to 256 megabytes if that figure is in bytes, but I thought that each 4870x2 card has 2GB memory (1GB per core). Am I confusing bits and bytes here, or being otherwise silly?

      Many thanks

      John

        • Accessing both GPUs on a 4870x2 (using OpenCL)
          nou

          1. did you run aticonfig --initial --adapter=all without it show only first card.

          2. each GPU should report as one device

          3. yes image support only Evergreen aka 5xxx series card.

          4. this is somehow artifical limit to 1/4 of available VRAM in current SDK.

            • Accessing both GPUs on a 4870x2 (using OpenCL)
              johnrdore

              Hi Nou

              In reverse order

              4) Thanks for letting me know. Since 256 MB is 1/4 of 1GB, that would suggest that Catalyst is only driving one GPU core on each card, not both.

              3) Again, thanks for telling me this.

              2) Are you sure? In the following thread (although it's about two years old now)

              http://forums.amd.com/devforum/messageview.cfm?catid=328&threadid=100006

              an AMD engineer wrote "We wouldn't be able to use both the GPUs in that card until [internal] crossfire is disabled". So the internal bridge between the two GPU cores on the 4870x2 stops us from accessing each core individually. However, the information could well be out of date. I have just PM'd him asking about this issue.

              1) I tried running aticonfig --initial --adapter=all as you suggested, but when I rebooted the X server my monitor got "No Signal" and I couldn't see anything. I only have one monitor connected to the PC, and it still did not receive a signal when I tried plugging its HDMI cable into different ports at the back of the computer (I have tried all four available ports). Am I doing something wrong?

              For what it's worth, this is my working /etc/X11/xorg.conf file (generated by aticonfig --initial)



              Section "ServerLayout"
                  Identifier     "aticonfig Layout"
                  Screen      0  "aticonfig-Screen[0]-0" 0 0
              EndSection

              Section "Files"
              EndSection

              Section "Module"
              EndSection

              Section "Monitor"
                  Identifier   "aticonfig-Monitor[0]-0"
                  Option        "VendorName" "ATI Proprietary Driver"
                  Option        "ModelName" "Generic Autodetecting Monitor"
                  Option        "DPMS" "true"
              EndSection

              Section "Device"
                  Identifier  "aticonfig-Device[0]-0"
                  Driver      "fglrx"
                  BusID       "PCI:3:0:0"
              EndSection

              Section "Screen"
                  Identifier "aticonfig-Screen[0]-0"
                  Device     "aticonfig-Device[0]-0"
                  Monitor    "aticonfig-Monitor[0]-0"
                  DefaultDepth     24
                  SubSection "Display"
                      Viewport   0 0
                      Depth     24
                  EndSubSection
              EndSection




              This is the xorg.conf file generated by aticonfig --initial --adapter=all

              Section "ServerLayout"
                  Identifier     "aticonfig Layout"
                  Screen      0  "aticonfig-Screen[0]-0" 0 0
                  Screen         "aticonfig-Screen[1]-0" RightOf "aticonfig-Screen[0]-0"
                  Screen         "aticonfig-Screen[2]-0" RightOf "aticonfig-Screen[1]-0"
                  Screen         "aticonfig-Screen[3]-0" RightOf "aticonfig-Screen[2]-0"
              EndSection

              Section "Files"
              EndSection

              Section "Module"
              EndSection

              Section "Monitor"
                  Identifier   "aticonfig-Monitor[0]-0"
                  Option        "VendorName" "ATI Proprietary Driver"
                  Option        "ModelName" "Generic Autodetecting Monitor"
                  Option        "DPMS" "true"
              EndSection

              Section "Monitor"
                  Identifier   "aticonfig-Monitor[1]-0"
                  Option        "VendorName" "ATI Proprietary Driver"
                  Option        "ModelName" "Generic Autodetecting Monitor"
                  Option        "DPMS" "true"
              EndSection

              Section "Monitor"
                  Identifier   "aticonfig-Monitor[2]-0"
                  Option        "VendorName" "ATI Proprietary Driver"
                  Option        "ModelName" "Generic Autodetecting Monitor"
                  Option        "DPMS" "true"
              EndSection

              Section "Monitor"
                  Identifier   "aticonfig-Monitor[3]-0"
                  Option        "VendorName" "ATI Proprietary Driver"
                  Option        "ModelName" "Generic Autodetecting Monitor"
                  Option        "DPMS" "true"
              EndSection

              Section "Device"
                  Identifier  "aticonfig-Device[0]-0"
                  Driver      "fglrx"
                  BusID       "PCI:3:0:0"
              EndSection

              Section "Device"
                  Identifier  "aticonfig-Device[1]-0"
                  Driver      "fglrx"
                  BusID       "PCI:4:0:0"
              EndSection

              Section "Device"
                  Identifier  "aticonfig-Device[2]-0"
                  Driver      "fglrx"
                  BusID       "PCI:10:0:0"
              EndSection

              Section "Device"
                  Identifier  "aticonfig-Device[3]-0"
                  Driver      "fglrx"
                  BusID       "PCI:11:0:0"
              EndSection

              Section "Screen"
                  Identifier "aticonfig-Screen[0]-0"
                  Device     "aticonfig-Device[0]-0"
                  Monitor    "aticonfig-Monitor[0]-0"
                  DefaultDepth     24
                  SubSection "Display"
                      Viewport   0 0
                      Depth     24
                  EndSubSection
              EndSection

              Section "Screen"
                  Identifier "aticonfig-Screen[1]-0"
                  Device     "aticonfig-Device[1]-0"
                  Monitor    "aticonfig-Monitor[1]-0"
                  DefaultDepth     24
                  SubSection "Display"
                      Viewport   0 0
                      Depth     24
                  EndSubSection
              EndSection

              Section "Screen"
                  Identifier "aticonfig-Screen[2]-0"
                  Device     "aticonfig-Device[2]-0"
                  Monitor    "aticonfig-Monitor[2]-0"
                  DefaultDepth     24
                  SubSection "Display"
                      Viewport   0 0
                      Depth     24
                  EndSubSection
              EndSection

              Section "Screen"
                  Identifier "aticonfig-Screen[3]-0"
                  Device     "aticonfig-Device[3]-0"
                  Monitor    "aticonfig-Monitor[3]-0"
                  DefaultDepth     24
                  SubSection "Display"
                      Viewport   0 0
                      Depth     24
                  EndSubSection
              EndSection



              I have noticed in this recent thread

              http://forums.amd.com/devforum/messageview.cfm?catid=390&threadid=135339

              you have suggested that the /dev/vga_arbiter file must be deleted. I have not done this yet, but is that the cause of the problem?

              In addition, I read in the Stream SDK Developer Release Notes:

              To access multiple GPUs under Linux, ensure the DISPLAY environment variable is set to :0 instead of :0.0.

              But when I run "export DISPLAY=:0" in a shell and then try running ./CLInfo or ./FindNumDevices it makes no difference. I still see 2 devices, not 4.

                • Accessing both GPUs on a 4870x2 (using OpenCL)
                  nou

                  on linux crossfire is disabled by default. this is currently problem on windows where secon core is return wrong result.

                  as you can see in xorg.conf which you posted there is four Device section. for each core one. and yes you need delete /dev/vga_arbiter for each start of system.

                    • Accessing both GPUs on a 4870x2 (using OpenCL)
                      johnrdore

                      Hi

                      My monitor shows "No Signal" when I use the 4-device xorg.conf, so my system becomes un-usable. I have to use the 1-device xorg.conf to actually see anything. Will deleting the /dev/vga_arbiter file in an init script make my monitor work properly with the 4-device xorg.conf? How should I do that?

                      Also, if I install the 4-device xorg.conf and delete /dev/vga_arbiter, does this mean that ./FindNumDevices will detect 4 GPU devices?

                      Thanks

                      John

                        • Accessing both GPUs on a 4870x2 (using OpenCL)
                          nou

                          just create custom init script. there is ton of tutorials.

                          and yes if you install four devices xorg.conf t should work and FindNumDevices should return 4 devices.

                            • Accessing both GPUs on a 4870x2 (using OpenCL)
                              johnrdore

                              Hi nou

                              Thanks for your help so far.

                              I installed an init script in /etc/rcS.d which runs "rm /dev/vga_arbiter" just before the screen-cleanup and x11-common scripts execute. This allows me to boot X using the four-screen config file I pasted into an earlier post.

                              However, when I run ./FindNumDevices it tells me that there is only one GPU device, whereas I had two before!

                              I was wondering if anything is wrong with my xorg.conf file? As I mentioned earlier, I only have one monitor connected to the system. Do I need to plug a monitor into each GPU before it can be detected by ./FindNumDevices?

                              Many thanks

                              John

                                • Accessing both GPUs on a 4870x2 (using OpenCL)
                                  johnrdore

                                  Ahhh...

                                  I'm being silly. I actually identified the solution to this problem in an earlier post:

                                  To access multiple GPUs under Linux, ensure the DISPLAY environment variable is set to :0 instead of :0.0.

                                  If I install the 4-device xorg.conf, delete /dev/vga_arbiter before the X server starts, and set DISPLAY=:0, I can now see 4 GPUs from an OpenCL program.

                                  Hopefully this information can be of use to someone else.