We have a Supermicro dual socket Epyc 7401 2u server and see that BIOS assigns PCI Root Bus numbers of 0x00 0x10 0x20 0x30 0x40 0x50 0x60 and 0x70 for the 8 numa nodes. Each of the nodes would therefore have a total of 16 PCI buses available.
When IO devices are connected to the server (via a host adapter card connecting to a PCIe Switch), we see that BIOS can assimilate the PCI buses from adjacent nodes and assign it to the PCI Root Bus in need. For example, when a Celestica JBOF (requires 26+ PCI Buses) is connected to Root Bus 0x20, PCI buses from Root PCI Bus 0x30 and 0x40 are reassigned such that Root Bus 0x20 ends up with 48 PCI Buses.
A few questions....
1) only 128 of the 256 PCI Buses are being configured by BIOS. Is it possible to have PCI Buses 0x80-0xFF configured/assigned?
2) when additional IO is added to Root Bus 0x20 (e.g. a 2nd Celestica JBOF), the number of PCI Buses assigned does not go beyond 48. Is there a way to go beyond 48 PCI Buses? Using the last half (0x80-0xff) of the PCI Buses might be a start.
Regards