Archives Discussions

[Gunter]

Hi,

we have a typical NUMA-system with two Intel Xeon CPUs, each with a local main memory system. To each of the CPUs, an I/O-hub 5520 is connected. Each 5520 in turn connects to a set of PCIe-slots. We use multiple HD 6990 cards, so that one card is connected to the one 5520, and the other card is connected to the other.

Now I want to use calResCreate2D() to create resources in either memory. The memory for these resources has been allocated using VirtualAllocExNuma(), thereby placing it into the desired memory system. I can now place CPU-threads on cores close to this data structure for fast access.

My question is: if such thread performs a calDeviceOpen(..., id), how do I find the ids of the GPUs connected to the memory via the shortest path? In essence, I want to avoid that a DMA goes through one 5520, then through one CPU, then through the other CPU, then through its memory, then back the same way.

Related to that: how can I identify GPUs being on the same card?

Thx

[settle]

---

First, sorry for posting using attach code, but currently it‘s the only working method to post from my android device. Using the AMD APP SDK 2.5 you can query the device topography which will tell you the BDF (bus, device, and function). In a linux system these are the same numbers found from the command /sbin/lspci | grep VGA. I believe this works only for PCI devices so something will have to change if in the future there are dual socket APU systems because the GPUs in those APUs will be treated as two seperate devices while the CPUs are treated as a single device.

[Gunter]

Okay, thanks. But I'm not sure how this would solve my problem. Given BDF, how do I know what PCIe-slot or CPU-socket this relates to? I might be wrong, but aren't bus and device numbers even assigned dynamically?

Thx