3 Replies Latest reply on Jul 5, 2017 1:27 PM by ray_m

    amdpro-gpu linux driver doesn't seem to support southbridge pcie slots

    fastpoke

      I've been mining using Xubuntu 13.10 and the amd-catalyst-13.11-beta V9.4-linux-x86.x86_64 drivers for years on a variety of motherboards using R9 290's.

       

      Since Xubuntu 13.10 has reached end of life, I decided to upgrade.  I've installed Xubuntu 17.04 and the amdgpu-pro-17.10-414273 drivers.  Things work great except for one thing.

       

      Generally, I have cards connected to most or all of the pcie slots on a given motherboard.  When switching to the new op system and driver, some slots stopped working.  Sometimes Xubuntu will just hang while booting, while other times it will display console debugging messages before hanging or rebooting.  These scroll by fast, but if I remember, they had to do with IOCTRL loop timeouts relating to AMDiv6(?).

       

      By experimenting, I can figure out which R9 290's to remove from a given motherboard to get it to work.  Once I get the right combination, it runs great.

       

      I don't think it's a hardware issue, because if I put the GPUs back in and boot with the old setup, all of them work.  If I reboot with the new setup, I have the problems mentioned above.

       

      In researching this, I noticed that the PCIE slots that no longer work are all connected to the southbridge.  I've confirmed this on two different Gigabyte motherboards, the F2A85X-UP4 and the F2A88X-D3H Rev 3 (I have 2 of these motherboards, same problem).  I suspect this is also happening on ASUS boards of the same era, like the A88X-PRO, but I haven't experimented enough to be sure.

       

      What else I've tried:

      It seems that this is a problem in either Ubuntu or the amdgpu-pro driver.  My goal is to get a combination of a supported Xubuntu and AMD driver that will work as well as my original setup.  One alternative is to use the frlgx driver.  The latest version of Ubuntu that supports this driver (has an Xorg that is compatible) is 14.041.  So I tried this version of Xubuntu with the radeon-crimson-15.12-15.302-151217a-297685e driver.  It had the same problems as Xubuntu 17.04 + amdgpu-pro 17.10.

       

      I tried the amd-catalyst-13.11-beta V9.4-linux-x86.x86_64 with Xubuntu 14.041 but it won't compile.  There are 3rd party patches available, but the one I tried ended up with a driver that was unusable. I would have tried a slightly later driver, but I can't find one, they seem to have all disappeared.

       

      Several questions:

      Is this a known limitation of the amdgpu-pro drivers?  If so, will it be addressed?  If not, is anyone interested in the logs relating to this problem?  If so, kindly tell me what log files you'd need.

       

      Why can't I find any of the previous catalyst distros for linux anywhere?  I understand they're discontinued, but for someone like me, who's purchased many r9 290's, what would it hurt to make the old versions available if they might help me?

       

      Is there a third party git hub you could recommend that will work for my problem?  I've seen a couple with different approaches to solving the compile problem with getHandles() in klr_aspci.c.

       

      Is it possible that this is a kernel problem, and that I need to find the appropriate chipset drivers and build them into the kernel?  This doesn't seem likely, since this wasn't necessary for Xubuntu 13.10. I understand that this forum's place isn't to deal with that, but it would help me to narrow the problem down to know whether this was a possibility.

       

      Thanks in advance for any help anyone could provide.

        • Re: amdpro-gpu linux driver doesn't seem to support southbridge pcie slots
          fastpoke

          Perhaps I shouldn't have used the term "southbridge" which is kind of archaic since the northbridge functions have been taken over by the cpu.  In my case, thepcie slots connected to the AMD A85X and A88X don't seem to work.

           

          GA-F2a85X-UP block diagram.PNG

          In the case of the GA-F2A85X-UP4 motherboard, the PCIE slots that don't work are PCIEX4 and PCIEX1_3.  I used to be able to run 6 cards on this board, now I can only do 4.  This doesn't seem to be a general problem with the A85X, since the USB ports are working.  I have this same thing happen on two different instances of this board.

           

          GA-F2a88X-D3H block diagram.PNG

          In the case of the GA-F2A88X-D3H, I lose the PCIEx4 slot, which is attached to the AMD A88X.  Again, USB's attached to this chip are working.

           

          I also have problems with ASUS boards that use this chip. Unfortunately, I can't find a block diagram indicating which slots are attached to the CPU and which are attached to the A88X.  I can report that some slots stop working when I changed op sys and driver, but without a block diagram I can't confirm if they were the ones attached to the A88X.

            • Re: amdpro-gpu linux driver doesn't seem to support southbridge pcie slots
              fastpoke

              More info:

              When I run dmidecode | grep "PCI" on the GA-F2A85X-UP4, I get:

                      Type: x16 PCI Express

                      Type: x1 PCI Express

                      Type: x8 PCI Express

                      Type: x16 PCI Express

              Looking at the detail, all of these are reported as in use.  Two are missing from the list, presumably the ones attached to the A85X.

               

              Running this on the GA-F2A88X-D3H

                      Type: x16 PCI Express

                      Type: x1 PCI Express

                      Type: x8 PCI Express

                      Type: x16 PCI Express

              These are also the ones reported in use.  The x4 is missing from the list, which is the one that is attached to the A88X.

               

              Since Ubuntu isn't seeing the slots, this makes me think that it's an op sys issue.  I'm going to check their forum to see if I can find a solution, or at least a definitive "not supported anymore" answer.