2 Replies Latest reply on Nov 10, 2016 9:30 AM by greenview

    Driver/Setup Ubuntu 16.04 with vGPU (FirePro S7150) and vmWare

    greenview

      Hi,

      in addition to some windows desktops I try to setup some Linux (ubuntu 16.04LTS) desktops for vmware horizon view.

      For windows there is an guest-driver that works, but in my ubuntu machines the vgpu is detected, but the vmware emulated gpu is used, leading to very bad 3d performance:

       

      # lspci -vn

      ...

      00:0f.0 0300: 15ad:0405 (prog-if 00 [VGA controller])

              Subsystem: 15ad:0405

              Flags: bus master, medium devsel, latency 64, IRQ 16

              I/O ports at 1070 [size=16]

              Memory at e8000000 (32-bit, prefetchable) [size=128M]

              Memory at fe000000 (32-bit, non-prefetchable) [size=8M]

              [virtual] Expansion ROM at c0200000 [disabled] [size=32K]

              Capabilities: [40] Vendor Specific Information: Len=00 <?>

              Capabilities: [44] PCI Advanced Features

              Kernel driver in use: vmwgfx

              Kernel modules: vmwgfx

      ...

      0b:00.0 0300: 1002:692f (prog-if 00 [VGA controller])

              DeviceName: pciPassthru0

              Physical Slot: 192

              Flags: fast devsel

              Memory at d0000000 (64-bit, prefetchable) [size=256M]

              Memory at e7400000 (64-bit, prefetchable) [size=4M]

              Memory at fd000000 (32-bit, non-prefetchable) [size=4M]

              Capabilities: [58] Express Endpoint, MSI 00

              Capabilities: [a0] MSI: Enable- Count=1/1 Maskable+ 64bit+

              Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>

              Capabilities: [150] Advanced Error Reporting

              Capabilities: [2b0] Address Translation Service (ATS)

              Capabilities: [328] Alternative Routing-ID Interpretation (ARI)

              Kernel modules: amdgpu

       

       

      dmesg (reduced to the parts that might be interesting, can provide full):

      [    0.000000] Initializing cgroup subsys cpuset

      [    0.000000] Initializing cgroup subsys cpu

      [    0.000000] Initializing cgroup subsys cpuacct

      [    0.000000] Linux version 4.4.0-42-generic (buildd@lgw01-13) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.2) ) #62-Ubuntu SMP Fri Oct 7 23:11:45 UTC 2016 (Ubuntu 4.4.0-42.62-generic 4.4.21)

      [    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.4.0-42-generic root=UUID=acfd066f-602a-421c-baa9-e8c993074cc0 ro quiet splash

      [    0.000000] KERNEL supported cpus:

      [    0.000000]   Intel GenuineIntel

      [    0.000000]   AMD AuthenticAMD

      [    0.000000]   Centaur CentaurHauls

      [    0.000000] Disabled fast string operations

      [    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256

      [    0.000000] x86/fpu: Supporting XSAVE feature 0x01: 'x87 floating point registers'

      [    0.000000] x86/fpu: Supporting XSAVE feature 0x02: 'SSE registers'

      [    0.000000] x86/fpu: Supporting XSAVE feature 0x04: 'AVX registers'

      [    0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.

      [    0.000000] x86/fpu: Using 'eager' FPU context switches.

      [    0.000000] e820: BIOS-provided physical RAM map:

      [    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009f3ff] usable

      [    0.000000] BIOS-e820: [mem 0x000000000009f400-0x000000000009ffff] reserved

      [    0.000000] BIOS-e820: [mem 0x00000000000dc000-0x00000000000fffff] reserved

      [    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007feeffff] usable

      [    0.000000] BIOS-e820: [mem 0x000000007fef0000-0x000000007fefefff] ACPI data

      [    0.000000] BIOS-e820: [mem 0x000000007feff000-0x000000007fefffff] ACPI NVS

      [    0.000000] BIOS-e820: [mem 0x000000007ff00000-0x000000007fffffff] usable

      [    0.000000] BIOS-e820: [mem 0x00000000f0000000-0x00000000f7ffffff] reserved

      [    0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000fec0ffff] reserved

      [    0.000000] BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved

      [    0.000000] BIOS-e820: [mem 0x00000000fffe0000-0x00000000ffffffff] reserved

      [    0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000003ffffffff] usable

      ...

      [    2.422381] [drm] Initialized drm 1.1.0 20060810

      [    2.424219] VMware vmxnet3 virtual NIC driver - version 1.4.7.0-k-NAPI

      [    2.424235] vmxnet3 0000:03:00.0: # of Tx queues : 2, # of Rx queues : 2

      [    2.462479] Fusion MPT base driver 3.04.20

      [    2.462480] Copyright (c) 1999-2008 LSI Corporation

      [    2.465893] ahci 0000:02:01.0: version 3.0

      [    2.473866] Fusion MPT SPI Host driver 3.04.20

      [    2.474017] [drm] DMA map mode: Using physical TTM page addresses.

      [    2.474124] [drm] Capabilities:

      [    2.474125] [drm]   Rect copy.

      [    2.474125] [drm]   Cursor.

      [    2.474126] [drm]   Cursor bypass.

      [    2.474127] [drm]   Cursor bypass 2.

      [    2.474128] [drm]   8bit emulation.

      [    2.474128] [drm]   Alpha cursor.

      [    2.474129] [drm]   Extended Fifo.

      [    2.474130] [drm]   Multimon.

      [    2.474131] [drm]   Pitchlock.

      [    2.474131] [drm]   Irq mask.

      [    2.474132] [drm]   Display Topology.

      [    2.474133] [drm]   GMR.

      [    2.474134] [drm]   Traces.

      [    2.474134] [drm]   GMR2.

      [    2.474135] [drm]   Screen Object 2.

      [    2.474136] [drm]   Command Buffers.

      [    2.474137] [drm]   Command Buffers 2.

      [    2.474137] [drm]   Guest Backed Resources.

      [    2.474138] [drm] Max GMR ids is 64

      [    2.474139] [drm] Max number of GMR pages is 65536

      [    2.474140] [drm] Max dedicated hypervisor surface memory is 0 kiB

      [    2.474141] [drm] Maximum display memory size is 65536 kiB

      [    2.474142] [drm] VRAM at 0xe8000000 size is 4096 kiB

      [    2.474153] [drm] MMIO at 0xfe000000 size is 256 kiB

      [    2.474154] [drm] global init.

      [    2.474562] ahci 0000:02:01.0: AHCI 0001.0300 32 slots 30 ports 6 Gbps 0x3fffffff impl SATA mode

      [    2.474564] ahci 0000:02:01.0: flags: 64bit ncq clo only

      [    2.510247] [TTM] Zone  kernel: Available graphics memory: 7184290 kiB

      [    2.510249] [TTM] Zone   dma32: Available graphics memory: 2097152 kiB

      [    2.510250] [TTM] Initializing pool allocator

      [    2.510253] [TTM] Initializing DMA pool allocator

      [    2.510359] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).

      [    2.510361] [drm] No driver support for vblank timestamp query.

      [    2.510547] [drm] Screen Target Display device initialized

      [    2.510587] [drm] width 640

      [    2.510592] [drm] height 480

      [    2.510598] [drm] bpp 32

      [    2.511680] [drm] Fifo max 0x00040000 min 0x00001000 cap 0x0000077f

      [    2.512506] [drm] Using command buffers with DMA pool.

      [    2.512511] [drm] DX: no.

      [    2.515307] fbcon: svgadrmfb (fb0) is primary device

      [    2.517361] Console: switching to colour frame buffer device 100x37

      [    2.518913] vmxnet3 0000:03:00.0 eth0: NIC Link is Up 10000 Mbps

      [    2.522503] scsi host2: ahci

      [    2.524989] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4

      [    2.525149] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3

      [    2.525638] [drm] amdgpu kernel modesetting enabled.

      [    2.525676] scsi host3: ahci

      [    2.560665] AMD IOMMUv2 driver by Joerg Roedel <jroedel@suse.de>

      [    2.560666] AMD IOMMUv2 functionality not available on this system

      [    2.562745] CRAT table not found

      [    2.562747] Finished initializing topology ret=0

      [    2.562771] scsi host4: ahci

      [    2.562913] kfd kfd: Initialized module

      [    2.563120] amdgpu 0000:0b:00.0: enabling device (0000 -> 0002)

      [    2.564310] scsi host5: ahci

      [    2.564459] scsi host6: ahci

      [    2.564710] [drm] Initialized vmwgfx 2.9.0 20150810 for 0000:00:0f.0 on minor 0

      [    2.565901] [drm] initializing kernel modesetting (TONGA 0x1002:0x692F 0x0000:0x0000 0x00).

      [    2.565910] [drm] register mmio base: 0xFD000000

      [    2.565911] [drm] register mmio size: 4194304

      [    2.565914] [drm] doorbell mmio base: 0xE7400000

      [    2.565915] [drm] doorbell mmio size: 4194304

      [    2.565931] [drm:amdgpu_device_init [amdgpu]] *ERROR* Unable to find PCI I/O BAR

      [    2.565979] [drm] probing gen 2 caps for device 15ad:7a0 = 100602/0

      [    2.565986] [drm] probing mlw for device 15ad:7a0 = 100602

      [    2.566011] amdgpu 0000:0b:00.0: BAR 6: can't assign [??? 0x00000000 flags 0x20000000] (bogus alignment)

      [    2.566202] amdgpu 0000:0b:00.0: BAR 6: can't assign [??? 0x00000000 flags 0x20000000] (bogus alignment)

      [    2.566218] [drm:amdgpu_get_bios [amdgpu]] *ERROR* Unable to locate a BIOS ROM

      [    2.566237] amdgpu 0000:0b:00.0: Fatal error during GPU init

      [    2.566252] [drm] amdgpu: finishing device.

      [    2.566254] [TTM] Memory type 2 has not been initialized

      [    2.566450] amdgpu: probe of 0000:0b:00.0 failed with error -22

       

       

      Tried to install the "AMD FirePro™ Unified Driver", but that also fails with strange errors like "No write permission to hQZ" (the path changes everytime I run it).

      Tried to blacklist the vmware "vmwgfx" driver, but that results only in a black screen and failing logins with the horizon client.

       

      Any hints how to get ubuntu using the vgpu?

        • Re: Driver/Setup Ubuntu 16.04 with vGPU (FirePro S7150) and vmWare
          greenview

          Short follow-up:

          After observing what the unified driver installer did, I found a temporary directory that is created during the install, in which a shell-script and a helper tool exists, that detects supported gpus:

          ~/fglrx-15.302.2301/fglrx-install.AcrY8u# ./amd_dcm64

          ./amd_dcm64: No supported adapters detected

           

          So if the "AMD FirePro™ Unified Driver" is wrong, what is the right one?

          • Re: Driver/Setup Ubuntu 16.04 with vGPU (FirePro S7150) and vmWare
            greenview

            Update: Newer Kernel (tried 4.9.0-rc4) doesn't help:

             

            [ 9629.699485] [drm] amdgpu kernel modesetting enabled.

            [ 9629.701873] [drm] initializing kernel modesetting (TONGA 0x1002:0x692F 0x0000:0x0000 0x00).

            [ 9629.701927] [drm] register mmio base: 0xFD000000

            [ 9629.701928] [drm] register mmio size: 4194304

            [ 9629.701931] [drm] doorbell mmio base: 0xE7400000

            [ 9629.701931] [drm] doorbell mmio size: 4194304

            [ 9629.701955] [drm:amdgpu_device_init [amdgpu]] *ERROR* Unable to find PCI I/O BAR

            [ 9629.701981] [drm] probing gen 2 caps for device 15ad:7a0 = 100602/0

            [ 9629.701987] [drm] probing mlw for device 15ad:7a0 = 100602

            [ 9629.701996] [drm] VCE enabled in physical mode

            [ 9629.724187] [drm:amdgpu_device_init [amdgpu]] *ERROR* Unable to find PCI I/O BAR; using MMIO for ATOM IIO

            [ 9629.724189] ATOM BIOS: C76304

            [ 9629.724195] [drm] GPU post is not needed

            [ 9629.727433] amdgpu 0000:0b:00.0: VRAM: 960M 0x0000000000000000 - 0x000000003BFFFFFF (960M used)

            [ 9629.727435] amdgpu 0000:0b:00.0: GTT: 7016M 0x000000003C000000 - 0x00000001F28187FF

            [ 9629.727442] [drm] Detected VRAM RAM=960M, BAR=256M

            [ 9629.727443] [drm] RAM width 256bits GDDR5

            [ 9629.727455] [drm] amdgpu: 960M of VRAM memory ready

            [ 9629.727456] [drm] amdgpu: 7016M of GTT memory ready.

            [ 9629.727470] [drm] GART: num cpu pages 1796120, num gpu pages 1796120

            [ 9629.729269] [drm] PCIE GART of 7016M enabled (table at 0x0000000000040000).

            [ 9629.729285] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).

            [ 9629.729286] [drm] Driver supports precise vblank timestamp query.

            [ 9629.729511] amdgpu 0000:0b:00.0: amdgpu: using MSI.

            [ 9629.729607] [drm] amdgpu: irq initialized.

            [ 9629.729614] Can't find requested voltage id in vdd_dep_on_sclk table!

            [ 9629.732850] Invalid VDDGFX value!

            [ 9629.732851] Get EVV Voltage Failed.  Abort Driver loading!

            [ 9629.732852] amdgpu: powerplay initialization failed

            [ 9629.732887] [drm:amdgpu_device_init [amdgpu]] *ERROR* sw_init of IP block <amdgpu_powerplay> failed -22

            [ 9629.732889] amdgpu 0000:0b:00.0: amdgpu_init failed

            [ 9629.735327] [drm] amdgpu: ttm finalized

            [ 9629.735330] amdgpu 0000:0b:00.0: Fatal error during GPU init

            [ 9629.735332] [drm] amdgpu: finishing device.

            [ 9629.735333] [TTM] Memory type 2 has not been initialized

            [ 9629.735840] amdgpu: probe of 0000:0b:00.0 failed with error -22

             

            And I got an email, telling me more or less explicit that I have to buy Nvidia M60 if I want GPU acceleration for Linux systems, is that true?