AnsweredAssumed Answered

Supermicro X9DRi-F + AMD FirePro S7150 VMs fail to start with MxGPU

Question asked by sbi on Apr 19, 2017
Latest reply on Nov 27, 2017 by sbi

Hi everyone,

 

I'm trying to setup AMD FirePro S7150 MxGPU on Supermicro X9DRi-F MB with ESXi 6.5.

I've managed to enable next features from the AMD MxGPU requirements list on my X9DRi-F MB:

 

  • Intel VT-d;
  • SR-IOV Support;
  • Above 4G Decoding (without this option system boot up stuck on "System initializing... 91");

Also I failed to find next one:

  • ARI Forwarding;
  • MMIO High Size;
  • MMIOH Base;
  • SVM capability;
  • IOMMU capability.
After AMD drivers setup I've managed to configure some VMs with SR-IOVed devices:

 

Inline image 1

Inline image 1

Unfortunately VM fails to start with the next error message: "Module 'DevicePowerOn' power on failed."

Here are some LOGs:

 

VM's vmware.log

 

2017-04-19T20:27:34.263Z| vmx| I125: PCIPassthru: 0000:83:02.3 : Device BAR 0 requested 64-bit memory address that exceeded MPN type (61708076843008).

 

2017-04-19T20:27:34.263Z| vmx| I125: PCIPassthru: 0000:83:02.3 : Device BAR 2 requested 64-bit memory address that exceeded MPN type (61709834256384).

2017-04-19T20:27:34.263Z| vmx| I125: Module 'DevicePowerOn' power on failed.

2017-04-19T20:27:34.263Z| vmx| I125: VMX_PowerOn: ModuleTable_PowerOn = 0

 

vmkernel.log during VM start

 

2017-04-19T20:27:34.263Z cpu20:69873)VSCSI: 6532: handle 8193(vscsi0:0):Destroying Device for world 69874 (pendCom 0)

 

2017-04-19T20:27:34.390Z cpu11:65634)WARNING: IOMMU: 2219: Unable to unset device 0000:83:02.3 since it hasn't been set.

 

 

cat /var/log/vmkernel.log | grep amdgpuv

 

VMB: 323:    name: /amdgpuv.v00

VMB: 323:    name: /amdgpuv_.v00

2017-04-19T20:38:50.732Z cpu0:65536)VisorFSTar: 1954: amdgpuv.v00 for 0x70228 bytes

2017-04-19T20:38:50.734Z cpu0:65536)VisorFSTar: 1954: amdgpuv_.v00 for 0x507c6 bytes

2017-04-19T20:38:53.610Z cpu4:65893)ALERT: Failed to verify signatures of the following vib(s): [amdgpuv amdgpuv-cim]. All tardisks validated

2017-04-19T20:39:10.207Z cpu22:66086)Loading module amdgpuv ...

2017-04-19T20:39:10.211Z cpu22:66086)Elf: 2043: module amdgpuv has license VMware

2017-04-19T20:39:10.217Z cpu22:66086)Device: 191: Registered driver 'amdgpuv' from 24

2017-04-19T20:39:10.217Z cpu22:66086)amdgpuv_log: amdgpuv_device_list_init:41: [amdgpuv]: amdgpuv device list initialized successfully

2017-04-19T20:39:10.217Z cpu22:66086)amdgpuv_log: init_module:455: [amdgpuv]: amdgpuv module loaded successfully

2017-04-19T20:39:10.217Z cpu22:66086)amdgpuv_log: init_module:457: [amdgpuv]: AMD SRIOV GPU-V Device Driver 0.9.1 BUILD: Dec 23 2016

2017-04-19T20:39:10.217Z cpu22:66086)Mod: 4968: Initialization of amdgpuv succeeded with module ID 24.

2017-04-19T20:39:10.217Z cpu22:66086)amdgpuv loaded successfully.

2017-04-19T20:39:10.222Z cpu15:66023)amdgpuv_log: amdgpuv_attachDevice:678: [amdgpuv]: amdgpuv_attachDevice: attachDevice...

2017-04-19T20:39:10.222Z cpu15:66023)amdgpuv_log: amdgpuv_attachDevice:686: [amdgpuv]: attacheDevice of amdgpuv driver invoked, moduleID = 24, heapID = 0x43060eead000, num = 0

2017-04-19T20:39:10.222Z cpu15:66023)amdgpuv_log: amdgpuv_attachDevice:697: [amdgpuv]: vmkDev = 0x14b343052f2ab48e

2017-04-19T20:39:10.222Z cpu15:66023)amdgpuv_log: amdgpuv_attachDevice:706: [amdgpuv]: get pf pci device handle successfully

2017-04-19T20:39:10.222Z cpu15:66023)amdgpuv_log: amdgpuv_pci_match_device:68: [amdgpuv]: found the supported asic, deviceID = 0x6929

2017-04-19T20:39:10.222Z cpu15:66023)amdgpuv_log: amdgpuv_manage_pci_resource:138: [amdgpuv]: reg_idx=5, doorbell_idx=2

2017-04-19T20:39:10.222Z cpu15:66023)amdgpuv_log: amdgpuv_manage_pci_resource:185: [amdgpuv]: map mmio reg successfully, pDev->pIOReg = 0x41001d180000

2017-04-19T20:39:10.222Z cpu15:66023)amdgpuv_log: amdgpuv_manage_pci_resource:186: [amdgpuv]: pDev->regAperSize = 0x40000

2017-04-19T20:39:10.222Z cpu15:66023)amdgpuv_log: amdgpuv_manage_pci_resource:198: [amdgpuv]: map doorbell successfully, pDev->pDoorbell = 0x410015400000

2017-04-19T20:39:10.222Z cpu15:66023)amdgpuv_log: amdgpuv_manage_pci_resource:199: [amdgpuv]: pDev->doorbellAperSize = 0x200000

2017-04-19T20:39:10.224Z cpu15:66023)amdgpuv_log: amdgpuv_manage_pci_resource:210: [amdgpuv]: map FB successfully, pDev->pFramebuffer = 0x438087c00000

2017-04-19T20:39:10.224Z cpu15:66023)amdgpuv_log: amdgpuv_manage_pci_resource:211: [amdgpuv]: pDev->fbAperSize = 0x10000000

2017-04-19T20:39:10.224Z cpu15:66023)amdgpuv_log: KCL_Ih_Ctx_Init:63: [amdgpuv]: kcl ih ctx initialized successfully

2017-04-19T20:39:10.224Z cpu15:66023)amdgpuv_log: amdgpuv_attachDevice:740: [amdgpuv]: register interrupt successfully

2017-04-19T20:39:10.224Z cpu15:66023)amdgpuv_log: amdgpuv_read_VBIOS:313: [amdgpuv]: VBIOS starts:  0x55, 0xaa

2017-04-19T20:39:10.357Z cpu15:66023)amdgpuv_log: amdgpuv_read_VBIOS:337: [amdgpuv]: BIOS Version Major 0xF Minor 0x31

2017-04-19T20:39:10.357Z cpu15:66023)amdgpuv_log: amdgpuv_read_VBIOS:347: [amdgpuv]: Valid video BIOS image, size = 0x10000, check sum is 0x553a00

2017-04-19T20:39:10.357Z cpu15:66023)amdgpuv_log: ATOM_PostVBIOS:189: [amdgpuv]: VBIOS post: init ASIC

2017-04-19T20:39:10.680Z cpu15:66023)amdgpuv_log: ATOM_PostVBIOS:187: [amdgpuv]: VBIOS post: load FW

2017-04-19T20:39:13.060Z cpu15:66023)amdgpuv_log: amdgpuv_post_VBIOS:404: [amdgpuv]: Post VBIOS successfully!

2017-04-19T20:39:13.061Z cpu15:66023)amdgpuv_log: amdgpuv_set_thermal_cntl_enable:358: [amdgpuv]: Thermal Control Enable

2017-04-19T20:39:13.061Z cpu15:66023)amdgpuv_log: KCL_PCI_Find_Ext_Capability:324: [amdgpuv]: found the Capability 0xb @pos 0x100

2017-04-19T20:39:13.061Z cpu15:66023)amdgpuv_log: KCL_PCI_Find_Ext_Capability:324: [amdgpuv]: found the Capability 0xb @pos 0x400

2017-04-19T20:39:13.061Z cpu15:66023)amdgpuv_log: amdgpuv_pci_gpuiov_init:65: [amdgpuv]: gpuiov->pos = 0x400

2017-04-19T20:39:13.061Z cpu15:66023)amdgpuv_log: amdgpuv_pci_gpuiov_init:81: [amdgpuv]: Total FB size @pos = 0x424 is 8190 MB

2017-04-19T20:39:13.061Z cpu15:66023)amdgpuv_log: amdgpuv_pci_gpuiov_init:101: [amdgpuv]: initialize gpu iov capability successfully

2017-04-19T20:39:13.061Z cpu15:66023)amdgpuv_log: parse_conf_str:57: [amdgpuv]: Parse Configuration:16,B131

2017-04-19T20:39:13.061Z cpu15:66023)amdgpuv_log: get_fb_partition:192: [amdgpuv]: maximal framebuffer size: 8176 MB

2017-04-19T20:39:13.061Z cpu15:66023)amdgpuv_log: get_fb_partition:196: [amdgpuv]: minimal pf framebuffer size: 256 MB

2017-04-19T20:39:13.061Z cpu15:66023)amdgpuv_log: get_fb_partition:201: [amdgpuv]: pf framebuffer size: 256 MB

2017-04-19T20:39:13.061Z cpu15:66023)amdgpuv_log: get_fb_partition:205: [amdgpuv]: total available framebuffer size for VFs: 7904 MB

2017-04-19T20:39:13.061Z cpu15:66023)amdgpuv_log: get_vf_mem_ap_size:224: [amdgpuv]: the STRAP_BIF_VF_MEM_AP_SIZE limits the VF FB to 128 MB

2017-04-19T20:39:13.061Z cpu15:66023)amdgpuv_log: validate_vf_params:161: [amdgpuv]: idx=0,bus=131

2017-04-19T20:39:13.061Z cpu15:66023)amdgpuv_log: valid_fb_size:108: [amdgpuv]: Set the FB size as upper limit 7904 by default

2017-04-19T20:39:13.061Z cpu15:66023)amdgpuv_log: valid_sched_intv:137: [amdgpuv]: Invalid scheduler interval 0 from config

2017-04-19T20:39:13.061Z cpu15:66023)amdgpuv_log: valid_sched_intv:143: [amdgpuv]: Calculate the scheduler interval as 5000 us by default

2017-04-19T20:39:13.061Z cpu15:66023)amdgpuv_log: calibrate_vf_config:260: [amdgpuv]: set the default FB to 480 MB

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_enableVFs:133: [amdgpuv]: 16 VFs are enabled successfully

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_total_fb_consumed:214: [amdgpuv]: the amount of FB reserved for virtualizaion @pos = 0x426, is 7926 MB

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_context:137: [amdgpuv]: set gpuiov ctx @pos = 0x420, data = 0x100001

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_vf_frame_buffer:242: [amdgpuv]: set vf frame buffer @pos = 0x42c, size = 480, offset = 0x110

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_vf_frame_buffer:242: [amdgpuv]: set vf frame buffer @pos = 0x430, size = 480, offset = 0x2f0

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_vf_frame_buffer:242: [amdgpuv]: set vf frame buffer @pos = 0x434, size = 480, offset = 0x4d0

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_vf_frame_buffer:242: [amdgpuv]: set vf frame buffer @pos = 0x438, size = 480, offset = 0x6b0

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_vf_frame_buffer:242: [amdgpuv]: set vf frame buffer @pos = 0x43c, size = 480, offset = 0x890

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_vf_frame_buffer:242: [amdgpuv]: set vf frame buffer @pos = 0x440, size = 480, offset = 0xa70

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_vf_frame_buffer:242: [amdgpuv]: set vf frame buffer @pos = 0x444, size = 480, offset = 0xc50

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_vf_frame_buffer:242: [amdgpuv]: set vf frame buffer @pos = 0x448, size = 480, offset = 0xe30

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_vf_frame_buffer:242: [amdgpuv]: set vf frame buffer @pos = 0x44c, size = 480, offset = 0x1010

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_vf_frame_buffer:242: [amdgpuv]: set vf frame buffer @pos = 0x450, size = 480, offset = 0x11f0

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_vf_frame_buffer:242: [amdgpuv]: set vf frame buffer @pos = 0x454, size = 480, offset = 0x13d0

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_vf_frame_buffer:242: [amdgpuv]: set vf frame buffer @pos = 0x458, size = 480, offset = 0x15b0

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_vf_frame_buffer:242: [amdgpuv]: set vf frame buffer @pos = 0x45c, size = 480, offset = 0x1790

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_vf_frame_buffer:242: [amdgpuv]: set vf frame buffer @pos = 0x460, size = 480, offset = 0x1970

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_vf_frame_buffer:242: [amdgpuv]: set vf frame buffer @pos = 0x464, size = 480, offset = 0x1b50

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_set_gpuiov_vf_frame_buffer:242: [amdgpuv]: set vf frame buffer @pos = 0x468, size = 480, offset = 0x1d30

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: KCL_PCI_Find_Ext_Capability:324: [amdgpuv]: found the Capability 0x10 @pos 0x330

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_query_sriov_configuration:108: [amdgpuv]: sriov->numVFs = 16

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_query_sriov_configuration:109: [amdgpuv]: sriov->vfOffset = 0x10, sriov->vfStride = 1

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_query_sriov_configuration:110: [amdgpuv]: sriov->vfDeviceID = 0x692f

2017-04-19T20:39:14.947Z cpu15:66023)WARNING: amdgpuv_log: amdgpuv_attachDevice:866: [amdgpuv]:Success attachDevice Done

2017-04-19T20:39:14.947Z cpu15:66023)Device: 326: Found driver amdgpuv for device 0x14b343052f2ab48e

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_startDevice:1068: [amdgpuv]: amdgpuv_startDevice: startDevice...

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_init_device:168: [amdgpuv]: pDev->vmkDev = 0x14b343052f2ab48e

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_init_device:169: [amdgpuv]: pDev->pfPciDev = 0x43008805c030

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_init_device:170: [amdgpuv]: pDev->pIOReg = 0x41001d180000

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_init_device:171: [amdgpuv]: pDev->doorbell = 0x410015400000

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_ih_iv_ring_disable:340: [amdgpuv]: disable iv ring successfully

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_alloc_iv_ring:87: [amdgpuv]: ih->ivRingNumEntries = 256

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_alloc_iv_ring:90: [amdgpuv]: ih->ivRingSizeInBytes = 4096

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_alloc_iv_ring:94: [amdgpuv]: ih->ivRingAllocSizeInBytes = 4100

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_alloc_iv_ring:96: [amdgpuv]: iv ring page_cnt = 2

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: KCL_MEM_AllocPage:160: [amdgpuv]: Allocated one physical page, page_MA = 0x2083133000

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: KCL_MEM_AllocPage:160: [amdgpuv]: Allocated one physical page, page_MA = 0x2080f4a000

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_alloc_iv_ring:116: [amdgpuv]: ih->ivRing_MA.QuadPart = 0x2083133000

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_alloc_iv_ring:120: [amdgpuv]: ih->ivRing_VA = 0x41001c140000

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_alloc_iv_ring:122: [amdgpuv]: ih->ivRingWptrWB = 0x41001c141000

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_alloc_iv_ring:124: [amdgpuv]: ih->ivRingWptrWB_MA.QuadPart = 0x2080f4a000

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_alloc_iv_ring:135: [amdgpuv]: update rptr via doorbell

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_ih_iv_ring_init:261: [amdgpuv]: ih->rptrDoorbell = 0x4100154007a0

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_ih_iv_ring_init:262: [amdgpuv]: ih->rptrDoorbellOffset = 0x1e8

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_ih_iv_ring_hw_init:187: [amdgpuv]: the physical address of ring buffer: 0x20831330

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_ih_iv_ring_setupRPTR:399: [amdgpuv]: write mmBIF_DOORBELL_APER_EN: 0x1

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_ih_iv_ring_enable:309: [amdgpuv]: ih->ivRingWptr_Reg = 0x0

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_ih_iv_ring_enable:311: [amdgpuv]: ih->ivRingWptr = 0

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_ih_iv_ring_enable:313: [amdgpuv]: ih->ivRingRptr_Reg = 0x0

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_ih_iv_ring_enable:315: [amdgpuv]: ih->ivRingRptr = 0

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_ih_iv_ring_enable:317: [amdgpuv]: *(ih->rptrDoorbell) = 0x0

2017-04-19T20:39:14.947Z cpu15:66023)amdgpuv_log: amdgpuv_ih_iv_ring_init:267: [amdgpuv]: init iv ring successfully

2017-04-19T20:39:14.948Z cpu15:66023)CpuSched: 692: user latency of 66087 tq:amdgpuv_sched_timer 0 changed by 66023 vmkdevmgr -6

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_sched_init:842: [amdgpuv]: create timer queue for scheduler successfully

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_sched_init:860: [amdgpuv]: Scheduler uses 5000 us VF switching interval

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_sched_init:871: [amdgpuv]: Scheduler uses 3000 (ticks) timeout, 200 (ticks) to start hang detection, 100 (ticks) to judge GPU hang

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_sched_init:875: [amdgpuv]: Scheduler uses schema: fixed percentage

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_sched_init:884: [amdgpuv]: Scheduler uses switch method all-in-one

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_ih_irq_source_enable:520: [amdgpuv]: irq sourceID 0x89 get enabled

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_ih_irq_source_enable:520: [amdgpuv]: irq sourceID 0x88 get enabled

2017-04-19T20:39:14.948Z cpu15:66023)WARNING: amdgpuv_log: amdgpuv_startDevice:1086: [amdgpuv]:Success startDevice Done

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_scanDevice:891: [amdgpuv]: amdgpuv_scanDevice: scanDevice...

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:366: [amdgpuv]: Get vf pci device handle @vf_idx = 0, 0x430088073c90

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:377: [amdgpuv]: register VF successfully, vf_idx = 0

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:366: [amdgpuv]: Get vf pci device handle @vf_idx = 1, 0x4300880740f0

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:377: [amdgpuv]: register VF successfully, vf_idx = 1

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:366: [amdgpuv]: Get vf pci device handle @vf_idx = 2, 0x4300880746a0

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:377: [amdgpuv]: register VF successfully, vf_idx = 2

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:366: [amdgpuv]: Get vf pci device handle @vf_idx = 3, 0x430088074c50

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:377: [amdgpuv]: register VF successfully, vf_idx = 3

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:366: [amdgpuv]: Get vf pci device handle @vf_idx = 4, 0x430088075200

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:377: [amdgpuv]: register VF successfully, vf_idx = 4

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:366: [amdgpuv]: Get vf pci device handle @vf_idx = 5, 0x4300880757b0

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:377: [amdgpuv]: register VF successfully, vf_idx = 5

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:366: [amdgpuv]: Get vf pci device handle @vf_idx = 6, 0x430088075d60

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:377: [amdgpuv]: register VF successfully, vf_idx = 6

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:366: [amdgpuv]: Get vf pci device handle @vf_idx = 7, 0x430088076310

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:377: [amdgpuv]: register VF successfully, vf_idx = 7

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:366: [amdgpuv]: Get vf pci device handle @vf_idx = 8, 0x4300880768c0

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:377: [amdgpuv]: register VF successfully, vf_idx = 8

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:366: [amdgpuv]: Get vf pci device handle @vf_idx = 9, 0x430088076e70

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:377: [amdgpuv]: register VF successfully, vf_idx = 9

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:366: [amdgpuv]: Get vf pci device handle @vf_idx = 10, 0x430088077420

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:377: [amdgpuv]: register VF successfully, vf_idx = 10

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:366: [amdgpuv]: Get vf pci device handle @vf_idx = 11, 0x4300880779d0

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:377: [amdgpuv]: register VF successfully, vf_idx = 11

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:366: [amdgpuv]: Get vf pci device handle @vf_idx = 12, 0x430088077f80

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:377: [amdgpuv]: register VF successfully, vf_idx = 12

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:366: [amdgpuv]: Get vf pci device handle @vf_idx = 13, 0x430088078530

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:377: [amdgpuv]: register VF successfully, vf_idx = 13

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:366: [amdgpuv]: Get vf pci device handle @vf_idx = 14, 0x430088078ae0

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:377: [amdgpuv]: register VF successfully, vf_idx = 14

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:366: [amdgpuv]: Get vf pci device handle @vf_idx = 15, 0x430088079090

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_registerVFs:377: [amdgpuv]: register VF successfully, vf_idx = 15

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_scanDevice:951: [amdgpuv]: Register logical character device successfully

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_init_idh_queue:198: [amdgpuv]: Initialize a new idh queue

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_init_idh_queue:198: [amdgpuv]: Initialize a new idh queue

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_init_idh_queue:198: [amdgpuv]: Initialize a new idh queue

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_init_idh_queue:198: [amdgpuv]: Initialize a new idh queue

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_init_idh_queue:198: [amdgpuv]: Initialize a new idh queue

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_init_idh_queue:198: [amdgpuv]: Initialize a new idh queue

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_init_idh_queue:198: [amdgpuv]: Initialize a new idh queue

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_init_idh_queue:198: [amdgpuv]: Initialize a new idh queue

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_init_idh_queue:198: [amdgpuv]: Initialize a new idh queue

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_init_idh_queue:198: [amdgpuv]: Initialize a new idh queue

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_init_idh_queue:198: [amdgpuv]: Initialize a new idh queue

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_init_idh_queue:198: [amdgpuv]: Initialize a new idh queue

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_init_idh_queue:198: [amdgpuv]: Initialize a new idh queue

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_init_idh_queue:198: [amdgpuv]: Initialize a new idh queue

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_init_idh_queue:198: [amdgpuv]: Initialize a new idh queue

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_init_idh_queue:198: [amdgpuv]: Initialize a new idh queue

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_idh_init:1894: [amdgpuv]: pending signal thread untill it get invoked by interrupt

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: KCL_Thread_Start:149: [amdgpuv]: amdgpuv kernel thread 66088 start successfully

2017-04-19T20:39:14.948Z cpu8:66088)amdgpuv_log: amdgpuv_signal_scheduler:1620: [amdgpuv]: signal thread get invoked

2017-04-19T20:39:14.948Z cpu8:66088)amdgpuv_log: amdgpuv_signal_scheduler:1656: [amdgpuv]: get an IDH_PENDING event

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_bind_vf:246: [amdgpuv]: add a node to vf list, list->node_cnt = 1

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_bind_vf:246: [amdgpuv]: add a node to vf list, list->node_cnt = 2

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_bind_vf:246: [amdgpuv]: add a node to vf list, list->node_cnt = 3

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_bind_vf:246: [amdgpuv]: add a node to vf list, list->node_cnt = 4

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_bind_vf:246: [amdgpuv]: add a node to vf list, list->node_cnt = 5

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_bind_vf:246: [amdgpuv]: add a node to vf list, list->node_cnt = 6

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_bind_vf:246: [amdgpuv]: add a node to vf list, list->node_cnt = 7

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_bind_vf:246: [amdgpuv]: add a node to vf list, list->node_cnt = 8

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_bind_vf:246: [amdgpuv]: add a node to vf list, list->node_cnt = 9

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_bind_vf:246: [amdgpuv]: add a node to vf list, list->node_cnt = 10

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_bind_vf:246: [amdgpuv]: add a node to vf list, list->node_cnt = 11

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_bind_vf:246: [amdgpuv]: add a node to vf list, list->node_cnt = 12

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_bind_vf:246: [amdgpuv]: add a node to vf list, list->node_cnt = 13

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_bind_vf:246: [amdgpuv]: add a node to vf list, list->node_cnt = 14

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_bind_vf:246: [amdgpuv]: add a node to vf list, list->node_cnt = 15

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: amdgpuv_bind_vf:246: [amdgpuv]: add a node to vf list, list->node_cnt = 16

2017-04-19T20:39:14.948Z cpu15:66023)amdgpuv_log: init_vf:928: [amdgpuv]: INIT_GPU done on PF0

2017-04-19T20:39:14.951Z cpu15:66023)amdgpuv_log: SearchMcfg:473: [amdgpuv]: MCFG is found at system BIOS 60000000

2017-04-19T20:39:14.951Z cpu15:66023)amdgpuv_log: SearchP2PBridge:578: [amdgpuv]: Search P2P bridge for bus 83

2017-04-19T20:39:14.957Z cpu15:66023)amdgpuv_log: TraverseBus:515: [amdgpuv]: P2P bridge depth 0: vender 8086 device e08 BDF 80:03.0 CFG 68018000

2017-04-19T20:39:14.957Z cpu15:66023)amdgpuv_log: SearchP2PBridge:599: [amdgpuv]: P2P bridge found, dpeth 1

2017-04-19T20:39:14.957Z cpu15:66023)amdgpuv_log: SearchP2PBridge:633: [amdgpuv]: downstream P2P cfg 68018000

2017-04-19T20:39:14.957Z cpu15:66023)amdgpuv_log: SearchP2PBridge:638: [amdgpuv]: mask unrecoverable error at P2P cfg 68018000

2017-04-19T20:39:14.957Z cpu15:66023)amdgpuv_log: amdgpuv_mask_unrecoverable_errors:866: [amdgpuv]: cfg sapce mapping 68018000

2017-04-19T20:39:14.957Z cpu15:66023)amdgpuv_log: amdgpuv_mask_unrecoverable_errors:881: [amdgpuv]: ext cap pos 100

2017-04-19T20:39:14.957Z cpu15:66023)amdgpuv_log: amdgpuv_mask_unrecoverable_errors:881: [amdgpuv]: ext cap pos 110

2017-04-19T20:39:14.957Z cpu15:66023)amdgpuv_log: amdgpuv_mask_unrecoverable_errors:881: [amdgpuv]: ext cap pos 148

2017-04-19T20:39:14.957Z cpu15:66023)amdgpuv_log: amdgpuv_mask_unrecoverable_errors:886: [amdgpuv]: Advance error reporting found: 148

2017-04-19T20:39:14.957Z cpu15:66023)amdgpuv_log: SearchP2PBridge:644: [amdgpuv]: mask unrecoverable error at PF cfg 68300000

2017-04-19T20:39:14.957Z cpu15:66023)amdgpuv_log: amdgpuv_mask_unrecoverable_errors:866: [amdgpuv]: cfg sapce mapping 68300000

2017-04-19T20:39:14.957Z cpu15:66023)amdgpuv_log: amdgpuv_mask_unrecoverable_errors:881: [amdgpuv]: ext cap pos 100

2017-04-19T20:39:14.957Z cpu15:66023)amdgpuv_log: amdgpuv_mask_unrecoverable_errors:881: [amdgpuv]: ext cap pos 150

2017-04-19T20:39:14.957Z cpu15:66023)amdgpuv_log: amdgpuv_mask_unrecoverable_errors:886: [amdgpuv]: Advance error reporting found: 150

2017-04-19T20:39:14.957Z cpu15:66023)amdgpuv_log: amdgpuv_check_sclk_dpm:470: [amdgpuv]: SCLK DPM ARG = 0x0,ID=0x162, SCLK DPM disabled

2017-04-19T20:39:14.957Z cpu15:66023)WARNING: amdgpuv_log: amdgpuv_scanDevice:965: [amdgpuv]:Success scanDevice Done

Outcomes