如何发现电路板上的PCIe总线拓扑和插槽号？

例如，当我使用CUDA C / C ++和GPUDirect 2.0 P2P的多GPU系统时，我使用嵌套的PCI-Express交换机，如图所示，那么我必须知道PCI总线ID在任何两个GPU之间有多less交换机，优化数据传输和计算分配。

或者如果我已经知道PCIe交换机的硬件PCIe拓扑结构，那么我必须知道，板上的哪个硬件PCIe插槽连接了任何GPU卡。在这里输入图像说明

据我所知，即使我已经知道PCIe交换机的硬件PCIe拓扑结构，那么这些标识符就不会硬件绑定到板上的PCIe插槽上，这些ID可能会随着系统的运行而改变，

CUDA device_id
nvidia-smi / nvml GPU编号
PCI总线ID

在Windows和Linux上使用详细的设备树和板上PCIe插槽的数量来发现PCIe总线的拓扑结构的最佳方法是什么？

PCI设备（端点）具有唯一的地址。这个地址有3个部分：

BusID
设备ID
FunctionID

例如，总线3上的设备12的功能3被写入BDF概念： 03:0C.3 。扩展的BDF符号添加一个域（主要是0000）作为前缀： 0000:03:0c.3 。

Linux将这些设备列在/sys/bus/pci/devices

 paebbels@debian8:~$ ll /sys/bus/pci/devices/ drwxr-xr-x 2 root root 0 Aug 19 11:44 . drwxr-xr-x 5 root root 0 Aug 5 15:14 .. lrwxrwxrwx 1 root root 0 Aug 19 11:44 0000:00:00.0 -> ../../../devices/pci0000:00/0000:00:00.0 lrwxrwxrwx 1 root root 0 Aug 19 11:44 0000:00:01.0 -> ../../../devices/pci0000:00/0000:00:01.0 lrwxrwxrwx 1 root root 0 Aug 19 11:44 0000:00:07.0 -> ../../../devices/pci0000:00/0000:00:07.0 lrwxrwxrwx 1 root root 0 Aug 19 11:44 0000:00:07.1 -> ../../../devices/pci0000:00/0000:00:07.1 ... lrwxrwxrwx 1 root root 0 Aug 19 11:44 0000:00:18.6 -> ../../../devices/pci0000:00/0000:00:18.6 lrwxrwxrwx 1 root root 0 Aug 19 11:44 0000:00:18.7 -> ../../../devices/pci0000:00/0000:00:18.7 lrwxrwxrwx 1 root root 0 Aug 19 11:44 0000:02:00.0 -> ../../../devices/pci0000:00/0000:00:11.0/0000:02:00.0 lrwxrwxrwx 1 root root 0 Aug 19 11:44 0000:02:01.0 -> ../../../devices/pci0000:00/0000:00:11.0/0000:02:01.0 lrwxrwxrwx 1 root root 0 Aug 19 11:44 0000:02:02.0 -> ../../../devices/pci0000:00/0000:00:11.0/0000:02:02.0 lrwxrwxrwx 1 root root 0 Aug 19 11:44 0000:02:03.0 -> ../../../devices/pci0000:00/0000:00:11.0/0000:02:03.0 lrwxrwxrwx 1 root root 0 Aug 19 11:44 0000:03:00.0 -> ../../../devices/pci0000:00/0000:00:15.0/0000:03:00.0

在这里你可以看到sys-fs列出了总线02的设备00到03连接到总线00，设备11，功能0

从这些信息中，您可以重建完整的PCI总线树。除非添加或删除设备，否则启动后树总是相同的。

Windows设备管理器提供相同的信息。属性对话框向您显示设备类型，供应商和位置：例如PCI bus 0, device 2, function 0集成Intel HD 4600图形的PCI bus 0, device 2, function 0 。

目前，我不知道如何在Windows环境下通过脚本或编程语言来获取这些信息，但是在互联网上有提供这些信息的商业和免费工具。也许有一个API。

这是一个不需要解析注册表的脚本版本。所有的信息（这里使用的）都是在win32_pnpentity中可用的。

 Function Get-BusFunctionID { gwmi -namespace root\cimv2 -class Win32_PnPEntity |% { if ($_.PNPDeviceID -like "PCI\*") { $locationInfo = $_.GetDeviceProperties('DEVPKEY_Device_LocationInfo').deviceProperties.Data if ($locationInfo -match 'PCI bus (\d+), device (\d+), function (\d+)') { new-object psobject -property @{ "Name" = $_.Name "PnPID" = $_.PNPDeviceID "BusID" = $matches[1] "DeviceID" = $matches[2] "FunctionID" = $matches[3] } } } } }

在Windows上，您可以使用例如：以下Powershell脚本与Windows设备驱动程序工具包中的devcon.exe工具：

 Function Get-BusFunctionID { $Devices = .\devcon.exe find PCI\* for($i=0; $i -lt $Devices.length; $i++) { if(!($Devices[$i] -match "PCI\\*")) { continue } $DevInfo = $Devices[$i].split(":") $deviceId = $DevInfo[0] $locationInfo = (get-itemproperty -path "HKLM:\SYSTEM\CurrentControlSet\Enum\$deviceID" -name locationinformation).locationINformation $businfo = Resolve-PCIBusInfo -locationInfo $locationinfo new-object psobject -property @{ "Name" = $DevInfo[1]; "PnPID" = $DevInfo[0] "PCIBusID" = $businfo.BusID; "PCIDeviceID" = $businfo.DeviceID; "PCIFunctionID" = $businfo.FunctionID } } } Function Resolve-PCIBusInfo { param ( [parameter(ValueFromPipeline=$true,Mandatory=$true)] [string] $locationInfo ) PROCESS { [void]($locationInfo -match "\d+,\d+,\d+") $busId,$deviceID,$functionID = $matches[0] -split "," new-object psobject -property @{ "BusID" = $busID; "DeviceID" = "$deviceID" "FunctionID" = "$functionID" } } }

用法示例：

 Get-BusFunctionID | Where-Object {$_.PCIBusID -eq 0 -and $_.PCIDeviceID -eq 0} | Format-Table Get-BusFunctionID | Sort-Object PCIBusID, PCIDeviceID, PCIFunctionID | Format-Table -GroupBy PCIBusID Get-BusFunctionID | Sort-Object PCIBusID, PCIDeviceID, PCIFunctionID | Out-GridView

对于Windows，准备运行Powershell脚本：

 Function Get-BusFunctionID { #gwmi -query "SELECT * FROM Win32_PnPEntity" $Devices = get-wmiobject -namespace root\cimv2 -class Win32_PnPEntity for($i=0; $i -lt $Devices.length; $i++) { if(!($Devices[$i].PNPDeviceID -match "PCI\\*")) { continue } $deviceId = $Devices[$i].PNPDeviceID $locationInfo = (get-itemproperty -path "HKLM:\SYSTEM\CurrentControlSet\Enum\$deviceID" -name locationinformation).locationINformation $businfo = Resolve-PCIBusInfo -locationInfo $locationinfo new-object psobject -property @{ "Name" = $Devices[$i].Name; "PnPID" = $Devices[$i].PNPDeviceID "PCIBusID" = $businfo.BusID; "PCIDeviceID" = $businfo.DeviceID; "PCIFunctionID" = $businfo.FunctionID } } } Function Resolve-PCIBusInfo { param ( [parameter(ValueFromPipeline=$true,Mandatory=$true)] [string] $locationInfo ) PROCESS { [void]($locationInfo -match "\d+,\d+,\d+") $busId,$deviceID,$functionID = $matches[0] -split "," new-object psobject -property @{ "BusID" = $busID; "DeviceID" = "$deviceID" "FunctionID" = "$functionID" } } }

用法示例：

 Get-BusFunctionID | Where-Object {$_.PCIBusID -eq 0 -and $_.PCIDeviceID -eq 0} | Format-Table Get-BusFunctionID | Sort-Object PCIBusID, PCIDeviceID, PCIFunctionID | Format-Table -GroupBy