一、前言
在Linux
內(nèi)核2.6.13-rc3
以前,驅(qū)動(dòng)和設(shè)備之間的綁定和解綁只能通過(guò)insmod(modprobe)
和rmmod
來(lái)實(shí)現(xiàn),但是這種實(shí)現(xiàn)方法有一個(gè)弊端,就是一旦綁定或者解綁定都是針對(duì)驅(qū)動(dòng)與其所支持的所有設(shè)備之間進(jìn)行,無(wú)法實(shí)現(xiàn)驅(qū)動(dòng)單獨(dú)綁定或者解綁定一個(gè)設(shè)備。然而,在Linux
內(nèi)核2.6.13-rc3
以后,提供了在用戶(hù)空間動(dòng)態(tài)的綁定和解綁定驅(qū)動(dòng)與設(shè)備之間關(guān)系的功能,這樣就解決了前面提到的無(wú)法實(shí)現(xiàn)驅(qū)動(dòng)單獨(dú)綁定或者解綁定一個(gè)設(shè)備的問(wèn)題。
二、PCI總線
PCI
(peripheral component interconnect
,外圍設(shè)備互聯(lián))總線由PCI Host Bridge
(PCI
主橋)或者PCI Bus Bridge
(PCI
橋)管理,用來(lái)連接各類(lèi)PCI
設(shè)備,如聲卡、網(wǎng)卡和PCI
橋等。在一個(gè)處理器系統(tǒng)中,可以通過(guò)PCI
橋擴(kuò)展PCI
總線,形成PCI
總線樹(shù)型結(jié)構(gòu),如下圖:Bus0
上的Device2
是一個(gè)PCI
橋,由該橋拓展出Bus1
。
在處理器系統(tǒng)中有幾個(gè)HOST
主橋,就有幾棵PCI
總線樹(shù),而每一棵PCI
總線樹(shù)都與一個(gè)PCI Domain
(PCI
總線域)對(duì)應(yīng),上圖就是一個(gè)總線域。每個(gè)PCI
設(shè)備都有一個(gè)vendor ID
(供應(yīng)商號(hào))和device ID
(設(shè)備號(hào))。
lspci
用來(lái)顯示所有的PCI
總線和連接在PCI
總線上的設(shè)備,在一臺(tái)Linux
設(shè)備上敲一敲lspci
,通常得到類(lèi)似如下信息:
[root@localhost ~]# lspci
00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU
00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge
00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
00:05.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]
00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 61)
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 0
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 3
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 5
00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 6
00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 7
00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 0
00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 1
00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 2
00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 3
00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 4
00:19.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 5
00:19.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 6
00:19.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 7
01:00.0 Ethernet controller: Corigine, Inc. Device 4000
......
用其中最常見(jiàn)的以太網(wǎng)卡信息來(lái)剖析下各個(gè)參數(shù)的含義:
01:00.0 Ethernet controller: Corigine, Inc. Device 4000
- 01:00.0 :表示
PCI
總線域?yàn)?code>0000,bus
總線號(hào)為02
,槽位號(hào)為01
,功能號(hào)為0
(lspci
默認(rèn)省略總線域0000
,可以加參數(shù)-D
顯示)。 - Ethernet controller :指設(shè)備類(lèi)型,設(shè)備類(lèi)型包括
Device classes,Device subclasses,Program interfaces
(一般省略),通過(guò)加參數(shù)-nn
可以看到Ethernet controller
對(duì)應(yīng)的設(shè)備類(lèi)型號(hào)為0200
,其中02
指Device classes
,00
指Device subclasses
,通過(guò)The PCI ID Repository
(http://pci-ids.ucw.cz/read/PD/02/00
)在線網(wǎng)站也可以查看。
[root@localhost ~]# lspci -nn -D -s 01:00.0
0000:01:00.0 Ethernet controller [0200]: Corigine, Inc. Device [1da8:4000]
- Corigine, Inc. :表示
Corigine
公司,如上面所示,對(duì)應(yīng)了供應(yīng)商號(hào)1da8
和設(shè)備號(hào)4000
,可通過(guò)如下方式查看,也可以通過(guò)device hunt``(https://devicehunt.com/)
在線網(wǎng)站查看。
[root@localhost ~]# lspci -nvmms 01:00.0
Slot: 01:00.0
Class: 0200
Vendor: 1da8
Device: 4000
SVendor: 1da8
SDevice: 0bf9
PhySlot: 2
NUMANode: 0
IOMMUGroup: 13
注意 :在
Linux
中,PCI
設(shè)備的設(shè)備名稱(chēng)(Device Name
)通常以domain:bus:slot:function
的形式來(lái)表示,如圖所示:其中冒號(hào)分隔開(kāi)的各個(gè)數(shù)字具有以下含義:
domain
:表示PCI
設(shè)備所在的PCI
總線域(Domain
),通常為一個(gè)16
位的十六進(jìn)制數(shù),用于區(qū)分不同的PCI
總線域。在大多數(shù)情況下,這個(gè)值為0000
。bus
:表示PCI
設(shè)備所在的總線(Bus
),通常為一個(gè)8
位的十六進(jìn)制數(shù),用于區(qū)分不同的總線。一個(gè)系統(tǒng)可以具有多個(gè)總線。slot
:表示PCI
設(shè)備所在的插槽(Slot
),通常為一個(gè)5
位的十六進(jìn)制數(shù),用于區(qū)分不同的插槽。一個(gè)總線上可以有多個(gè)插槽。function
:表示PCI
設(shè)備的功能(Function
),通常為一個(gè)3
位的十六進(jìn)制數(shù),用于區(qū)分同一插槽上的不同功能。一個(gè)插槽上可以有多個(gè)功能。一個(gè)PCI
物理設(shè)備可以實(shí)現(xiàn)多個(gè)功能設(shè)備,且邏輯功能相互獨(dú)立,其實(shí)就是 硬件虛擬化 。通過(guò)這種編號(hào)方式,可以唯一標(biāo)識(shí)一個(gè)
PCI
設(shè)備的位置信息。在上述示例中,"0000:01:00.0" 表示該設(shè)備位于 PCI 總線域 0000,總線 01,插槽 00,功能 0 。請(qǐng)注意,這些數(shù)字可能會(huì)因系統(tǒng)配置而有所不同,具體取決于你的系統(tǒng)和相應(yīng)的
PCI
設(shè)備。
三、驅(qū)動(dòng)與單個(gè)PCI設(shè)備的綁定和解綁定
此處,我們以PCI總線的Corigine
網(wǎng)卡設(shè)備及其驅(qū)動(dòng)為例,講述網(wǎng)卡設(shè)備及其驅(qū)動(dòng)的綁定和解綁定過(guò)程。首先,執(zhí)行lspci
顯示所用機(jī)器上的所有網(wǎng)卡設(shè)備:
[root@localhost ~]# lspci | grep Eth
01:00.0 Ethernet controller: Corigine, Inc. Device 4000
21:00.0 Ethernet controller: Corigine, Inc. Network Flow Processor 3800
41:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
41:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
41:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
41:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
對(duì)于所有的Corigine
網(wǎng)卡設(shè)備,都可以在nfp
驅(qū)動(dòng)下看到。其中,bind
和unbind
文件就是涉及到綁定和解綁的關(guān)鍵文件。
[root@localhost ~]# ls -l /sys/bus/pci/drivers/nfp
total 0
lrwxrwxrwx 1 root root 0 Nov 14 05:17 0000:01:00.0 - > ../../../../devices/pci0000:00/0000:00:01.1/0000:01:00.0
lrwxrwxrwx 1 root root 0 Nov 14 05:17 0000:21:00.0 - > ../../../../devices/pci0000:20/0000:20:03.2/0000:21:00.0
--w------- 1 root root 4096 Nov 14 05:17 bind
lrwxrwxrwx 1 root root 0 Jun 28 07:58 module - > ../../../../module/nfp
--w------- 1 root root 4096 Nov 14 05:17 new_id
--w------- 1 root root 4096 Nov 14 05:17 remove_id
--w------- 1 root root 4096 Jun 8 17:34 uevent
--w------- 1 root root 4096 Nov 14 05:17 unbind
1、解綁定
這里我們以PCI
總線BDF
號(hào)為01:00.0
的Corigine
網(wǎng)卡為例,解綁該設(shè)備,只需將設(shè)備的帶PCI
總線域的BDF
號(hào)寫(xiě)入/sys/bus/pci/drivers/nfp/unbind
(不同的設(shè)備,驅(qū)動(dòng)不同,即這里的nfp
)即可:
[root@localhost ~]# echo 0000:01:00.0 > /sys/bus/pci/drivers/nfp/unbind
# 解除綁定成功,再查看目錄下文件,該驅(qū)動(dòng)下不再有對(duì)應(yīng)的設(shè)備。
[root@localhost ~]# ls -l /sys/bus/pci/drivers/nfp
total 0
lrwxrwxrwx 1 root root 0 Nov 14 05:17 0000:21:00.0 - > ../../../../devices/pci0000:20/0000:20:03.2/0000:21:00.0
--w------- 1 root root 4096 Nov 14 05:17 bind
lrwxrwxrwx 1 root root 0 Jun 28 07:58 module - > ../../../../module/nfp
--w------- 1 root root 4096 Nov 14 05:17 new_id
--w------- 1 root root 4096 Nov 14 05:17 remove_id
--w------- 1 root root 4096 Jun 8 17:34 uevent
--w------- 1 root root 4096 Nov 14 05:23 unbind
執(zhí)行解綁定操作以后,我們可以通過(guò)lshw
命令查看PCI
設(shè)備:
[root@localhost ~]# lshw -c network -businfo
Bus info Device Class Description
=======================================================
pci@0000:01:00.0 /dev/fb0 network Corigine, Inc.
pci@0000:21:00.0 ens1np0 network Corigine, Inc.
pci@0000:41:00.0 eno1 network I350 Gigabit Network Connection
pci@0000:41:00.1 ens16f1 network I350 Gigabit Network Connection
pci@0000:41:00.2 ens16f2 network I350 Gigabit Network Connection
pci@0000:41:00.3 ens16f3 network I350 Gigabit Network Connection
pci@0000:21:00.0 ens1np1 network Ethernet interface
通過(guò)上述結(jié)果可知,BDF
號(hào)為01:00.0
的設(shè)備,在解綁定nfp
驅(qū)動(dòng)和該設(shè)備以后,該設(shè)備接口名稱(chēng)已無(wú)法正常顯示,而同樣使用nfp
驅(qū)動(dòng)的其他PCI
設(shè)備,如上面BDF
號(hào)為21:00.0
的設(shè)備,則接口名稱(chēng)正常顯示,說(shuō)明驅(qū)動(dòng)和該設(shè)備正常綁定。另外,也可通過(guò)如下方式對(duì)比兩個(gè)PCI
設(shè)備此時(shí)的區(qū)別:
[root@localhost ~]# echo 0000:01:00.0 > /sys/bus/pci/drivers/nfp/unbind
[root@localhost ~]# lspci -s 01:00.0 -v
01:00.0 Ethernet controller: Corigine, Inc. Device 4000
Subsystem: Corigine, Inc. Device 0bf9
Physical Slot: 2
Flags: fast devsel, IRQ 117, NUMA node 0, IOMMU group 13
Memory at 340a0000000 (64-bit, prefetchable) [size=128M]
Memory at 340a8000000 (64-bit, prefetchable) [size=64M]
Memory at 340ac000000 (64-bit, prefetchable) [size=16M]
Expansion ROM at f5000000 [disabled] [size=16M]
Capabilities: [80] Power Management version 3
Capabilities: [b0] MSI-X: Enable- Count=256 Masked-
Capabilities: [c0] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Alternative Routing-ID Interpretation (ARI)
Capabilities: [150] Device Serial Number 88-3c-c5-a0-18-6d-10-ff
Capabilities: [200] Single Root I/O Virtualization (SR-IOV)
Capabilities: [300] Secondary PCI Express
Kernel modules: nfp
[root@localhost ~]# lspci -s 21:00.0 -v
21:00.0 Ethernet controller: Corigine, Inc. Network Flow Processor 3800
Subsystem: Corigine, Inc. Device 7feb
Physical Slot: 1
Flags: bus master, fast devsel, latency 0, IRQ 213, NUMA node 0, IOMMU group 29
Memory at 2c070000000 (64-bit, prefetchable) [size=128M]
Memory at 2c078000000 (64-bit, prefetchable) [size=64M]
Memory at 2c07c000000 (64-bit, prefetchable) [size=16M]
Expansion ROM at b0000000 [disabled] [size=16M]
Capabilities: [88] Power Management version 3
Capabilities: [b0] MSI-X: Enable+ Count=64 Masked-
Capabilities: [c0] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Alternative Routing-ID Interpretation (ARI)
Capabilities: [150] Device Serial Number 88-3c-c5-a0-32-1e-10-ff
Capabilities: [160] Power Budgeting ? >
Capabilities: [1b8] Latency Tolerance Reporting
Capabilities: [200] Single Root I/O Virtualization (SR-IOV)
Capabilities: [300] Secondary PCI Express
Capabilities: [400] Vendor Specific Information: ID=0001 Rev=1 Len=010 < ? >
Kernel driver in use: nfp
Kernel modules: nfp
根據(jù)上述結(jié)果可知,BDF
號(hào)為01:00.0
的PCI
設(shè)備較BDF
號(hào)為21:00.0
的PCI
設(shè)備明顯少了一行Kernel driver in use: nfp
,說(shuō)明BDF
號(hào)為01:00.0
的PCI
設(shè)備已與nfp
驅(qū)動(dòng)解綁定,即nfp
驅(qū)動(dòng)未在使用中。而BDF
號(hào)為21:00.0
的PCI
設(shè)備則正在與nfp
驅(qū)動(dòng)綁定中,即nfp
驅(qū)動(dòng)正在使用中。
2、綁定
綁定一個(gè)網(wǎng)卡設(shè)備,和解綁類(lèi)似,將設(shè)備的帶PCI
總線域的BDF
號(hào)寫(xiě)入/sys/bus/pci/drivers/nfp/bind
(不同的設(shè)備,驅(qū)動(dòng)不同,即這里的nfp
):
[root@localhost ~]# echo 0000:01:00.0 > /sys/bus/pci/drivers/nfp/bind
[root@localhost ~]# ls -l /sys/bus/pci/drivers/nfp
total 0
lrwxrwxrwx 1 root root 0 Nov 14 05:28 0000:01:00.0 - > ../../../../devices/pci0000:00/0000:00:01.1/0000:01:00.0
lrwxrwxrwx 1 root root 0 Nov 14 05:17 0000:21:00.0 - > ../../../../devices/pci0000:20/0000:20:03.2/0000:21:00.0
--w------- 1 root root 4096 Nov 14 05:27 bind
lrwxrwxrwx 1 root root 0 Jun 28 07:58 module - > ../../../../module/nfp
--w------- 1 root root 4096 Nov 14 05:17 new_id
--w------- 1 root root 4096 Nov 14 05:17 remove_id
--w------- 1 root root 4096 Jun 8 17:34 uevent
--w------- 1 root root 4096 Nov 14 05:23 unbind
執(zhí)行綁定操作以后,我們可以通過(guò)lshw
命令查看PCI
設(shè)備:
[root@localhost ~]# lshw -c network -businfo
Bus info Device Class Description
=======================================================
pci@0000:01:00.0 ens2np0 network Corigine, Inc.
pci@0000:21:00.0 ens1np0 network Corigine, Inc.
pci@0000:41:00.0 eno1 network I350 Gigabit Network Connection
pci@0000:41:00.1 ens16f1 network I350 Gigabit Network Connection
pci@0000:41:00.2 ens16f2 network I350 Gigabit Network Connection
pci@0000:41:00.3 ens16f3 network I350 Gigabit Network Connection
pci@0000:21:00.0 ens1np1 network Ethernet interface
pci@0000:01:00.0 ens2np1 network Ethernet interface
通過(guò)上述結(jié)果可知,BDF
號(hào)為01:00.0
的設(shè)備,在綁定nfp
驅(qū)動(dòng)和該設(shè)備以后,該設(shè)備接口名稱(chēng)已正常顯示,且使用如下命令時(shí),也顯示nfp
驅(qū)動(dòng)正在使用中。
[root@localhost ~]# echo 0000:01:00.0 > /sys/bus/pci/drivers/nfp/bind
[root@localhost ~]# lspci -s 01:00.0 -v
01:00.0 Ethernet controller: Corigine, Inc. Device 4000
Subsystem: Corigine, Inc. Device 0bf9
Physical Slot: 2
Flags: bus master, fast devsel, latency 0, IRQ 117, NUMA node 0, IOMMU group 13
Memory at 340a0000000 (64-bit, prefetchable) [size=128M]
Memory at 340a8000000 (64-bit, prefetchable) [size=64M]
Memory at 340ac000000 (64-bit, prefetchable) [size=16M]
Expansion ROM at f5000000 [disabled] [size=16M]
Capabilities: [80] Power Management version 3
Capabilities: [b0] MSI-X: Enable+ Count=256 Masked-
Capabilities: [c0] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Alternative Routing-ID Interpretation (ARI)
Capabilities: [150] Device Serial Number 88-3c-c5-a0-18-6d-10-ff
Capabilities: [200] Single Root I/O Virtualization (SR-IOV)
Capabilities: [300] Secondary PCI Express
Kernel driver in use: nfp
Kernel modules: nfp
四、總結(jié)
bind
是PCI
設(shè)備綁定驅(qū)動(dòng)過(guò)程中會(huì)使用到的文件,unbind
是解綁PCI
設(shè)備與驅(qū)動(dòng)過(guò)程中會(huì)使用到的文件。具體的綁定與解綁的過(guò)程就是向這兩個(gè)文件中寫(xiě)入規(guī)定格式的數(shù)據(jù)完成的。
對(duì) bind
文件寫(xiě)入每一個(gè)接口的 PCI
號(hào)意味著我們可以將一個(gè)網(wǎng)卡上的不同口綁定到不同的驅(qū)動(dòng)上。對(duì)unbind
文件寫(xiě)入接口的 PCI
號(hào)就會(huì)解除當(dāng)前綁定的驅(qū)動(dòng)。一個(gè)接口可以不綁定到任何驅(qū)動(dòng)上面,不過(guò)我們常常不會(huì)這樣去做。
這里,我們只講了bind
和unbind
文件,其實(shí)PCI
設(shè)備接口綁定驅(qū)動(dòng)可能還會(huì)與new_id
文件有關(guān),new_id
主要用于動(dòng)態(tài)的向PCI
設(shè)備驅(qū)動(dòng)中添加一個(gè)新的設(shè)備ID
,這種功能允許驅(qū)動(dòng)添加更多的硬件而非僅包含在編譯時(shí)驅(qū)動(dòng)靜態(tài)支持設(shè)備 ID
列表中的硬件。 寫(xiě)入這個(gè)文件的格式中,Vendor Id 與 Device ID 字段是必須的,其它的字段可以不指定 。格式如下:
[root@localhost ~]# echo "1da8 4000" > /sys/bus/pci/drivers/nfp/new_id
成功添加一個(gè)設(shè)備 ID
時(shí),驅(qū)動(dòng)會(huì)嘗試 probe
系統(tǒng)中匹配到的設(shè)備并嘗試綁定到它之上。
既然有添加新的PCI
設(shè)備的文件,那一定有移除新的PCI
設(shè)備的文件,即與new_id
位于同一目錄下的remove_id
。
remove_id
的寫(xiě)入格式與 new_id
的寫(xiě)入格式相同。寫(xiě)入 remove_id
可以用來(lái)確保內(nèi)核不會(huì)自動(dòng) probe
匹配到這個(gè)驅(qū)動(dòng)的設(shè)備。格式如下:
[root@localhost ~]# echo "1da8 4000" > /sys/bus/pci/drivers/nfp/remove_id
至此,關(guān)于驅(qū)動(dòng)與單個(gè)PCI
設(shè)備的綁定和解綁定就介紹完了。
-
處理器
+關(guān)注
關(guān)注
68文章
19369瀏覽量
230419 -
Linux
+關(guān)注
關(guān)注
87文章
11324瀏覽量
209938 -
總線
+關(guān)注
關(guān)注
10文章
2892瀏覽量
88210 -
LINUX內(nèi)核
+關(guān)注
關(guān)注
1文章
316瀏覽量
21686
發(fā)布評(píng)論請(qǐng)先 登錄
相關(guān)推薦
評(píng)論