1 Reply Latest reply on Mar 6, 2018 8:12 AM by karatekid430

    AMD graphic cards - do they support PCIE hotplug?

    mdec

      Hello,

       

      I use AMD's cards on Linux. I'm hoping to soon drop fglrx driver in favor of amdgpu. However, I don't have much faith in GPU drivers for Linux in general, so I think a good way to save power would be to actually detach the GPUs when they aren't needed instead of relying on the driver. I've read that the PCIE standard generally supports this, but are AMD's cards supporting that as well in hardware and software?

       

      Best regards,
      Michael

        • Re: AMD graphic cards - do they support PCIE hotplug?
          karatekid430

          I have been looking at this, mainly in the context of Linux and Thunderbolt external graphics. Technically it can be hot-plugged, but the drivers and operating systems are a very different story. For it to actually work in reality, these need to change.

           

          The short answer with Windows, is no - because Windows is still buried deep within ACPI and firmware, and cannot allocate the resources on the fly.

          Linux automatically uses the vast 64-bit address space, so this is not a problem here. Windows RS3 has poorly documented "Native Express" mode which does native PCI bus enumeration - find the references in the Thunderbolt Driver release notes. This might fix the problem - but it is too early to tell, since the Titan Ridge Thunderbolt controllers are still not in the wild. The other option is to apply a DSDT patch to Windows to force it to use large memory resources for allocation - 36-bit I think. But this feels dirty as all hell, and requires test mode enabled.

           

          Hot-plugging can be problematic - some motherboards deliberately shut down if they detect hot-attach on large PCIe slots. They generally tolerate it in PCIe x1. The way around this is to plug in a PCIe to SFF-8643 card (PCIe 3.0 x4) without the cable attached, and then attach the U.2 cable to the GPU on a SFF-8639 to PCIe x4 open-end slot - generally hot-plugging the U.2 port does not trigger the motherboard.

          Linux AMDGPU driver still cannot handle surprise removal - it gives a stack dump and leaves the PCI subsystem locked in the kernel (presumably mutex-related), requiring a reboot, which often hangs after this, requiring a "Reboot Even If System Utterly Broken" sysrq.

          But you can use the information in the lspci command with:

          echo 1 | sudo tee /sys/bus/pci/devices/0000\:????????/remove

          on the devices corresponding to the GPU and its audio device and then gracefully remove the hardware without any issues. You just cannot yank it out unexpectedly.

          I would expect this to change because of Thunderbolt, which has already brought forward extensive changes to the kernel with tightened locking and technical changes. But only time will tell how long we have to wait.

           

          The other thing with Linux is I have had trouble getting the desktop environment to use the newly-introduced device - but then again, that was with Thunderbolt and the driver was giving some weird atombios error from the drm kernel object, so that could have been the reason.

          1 of 1 people found this helpful