如何配置IOMMU或者SWIOTLB

1. 前言

这篇文章说明了如何在Linux内核中启用和配置IOMMU和SWOTLB。

当今的计算或者嵌入设备使用一种内存分区的方法进行外设的管理，如显卡、PCI设备或USB设备，都将设备映射为一段内存，用于设备的读写。

传统意义上的IOMMU用于内存映射，在系统初始化时就已设置，并且在系统运行时不能动态更改，因此有一定的局限性，所以芯片制造商（如Intel和AMD）开发了更先进的内存管理方法。

在Linux内核中，我们可以使用Intel的SWOTLB和AMD架构的其他机制来操作IOMMU。64位的系统已经能够使系统使用大量内存范围，但是这些内存需要在使用之前都需要进行映射。

2. IOMMU

近年来，制造商已经将IOMMU集成到CPU中，这也为我们升级内存速度、类型提供了便利，从而不需要再更换CPU。当然，内核仍需要设置并读取映射才能有效地使用系统内存。

2.1 使能IOMMU

IOMMU是一个硬件模块，在使用之前需要先配置使能它，具体如下所示：

Device Drivers --->

[*] IOMMU Hardware Support --->

Generic IOMMU Pagetable Support ----

[*] AMD IOMMU support

[*] Export AMD IOMMU statistics to debugfs

AMD IOMMU Version 2 driver

[*] Support for Intel IOMMU using DMA Remapping Devices

[*] Support for Shared Virtual Memory with Intel IOMMU

[*] Enable Intel DMA Remapping Devices by default

[*] Support for Interrupt Remapping

2.2 配置IOMMU

需要在内核命令行中添加以下用于控制内存映射各方面的选项，以使其生效。具体操作取决于系统的引导加载程序。常见的引导加载程序包括GRUB和Lilo。

有关引导加载程序的更多信息可以在这里找到：Category:Bootloaders - Gentoo wiki

2.2.1 Generic options

Attribute	Description and options
iommu=off	This disables the IOMMU driver completely.
iommu=noforce	Don't force hardware IOMMU usage when it is not needed.
iommu=force	The use of the hardware IOMMU even when it is not actually needed (e.g. because < 3 GB memory).
iommu=soft	Use software bounce buffering (SWIOTLB) (default for Intel machines). This can be used to prevent the usage of an available hardware IOMMU. (read bellow for Intel SWIOTLB).

2.2.2 AMD64 systems

Attribute	Description and options
amd_iommu=nofullflush, amd_iommu=fullflush, amd_iommu=off, amd_iommu=force_isolation	Enable flushing of IO/TLB entries they are unmapped. Otherwise they are flushed before they will be reused, which is a lot of faster. off = do not initialize any AMD IOMMU found in the system. force_isolation = Force device isolation for all devices. The IOMMU driver is not allowed anymore to lift isolation requirements as needed. This option does not override iommu=pt.
amd_iommu_dump=0, amd_iommu_dump=1	This is a boolean option: 0 = disabled, 1 = enabled This is to dump the ACPI table for AMD IOMMU. With this option enabled, AMD IOMMU driver will print ACPI tables for AMD IOMMU during IOMMU initialization.
amd_iommu_intr=legacy, amd_iommu_intr=vapic	Specifies one of the following AMD IOMMU interrupt remapping modes: legacy = Use legacy interrupt remapping. mode.vapic = Use virtual APIC mode, which allows IOMMU to inject interrupts directly into guest. This mode requires kvm-amd.avic=1. (Default when IOMMU HW support is present.)

2.2.3 Intel systems

Intel generally adopts "an-always-enable-it-if-it-is-supported" rule so most options are to turn off or disable the IOMMU functions.

Attribute	Description and options
intel_iommu=on, intel_iommu=off	This is a boolean option: on = enabled, off = disabled.
intel_iommu=igfx_off	This option turns off mapping for a graphics card and is the default state for this option. The gfx is mapped as normal device. If a gfx device has a dedicated DMAR unit, the DMAR unit is bypassed by not enabling DMAR with this option. In this case the gfx device will use physical address for DMA.
intel_iommu=forcedac	With this option iommu will not optimize to look for io virtual address below 32-bit forcing dual address cycle on pci bus for cards supporting greater than 32-bit addressing. The default is to look for translation below 32-bit and if not available then look in the higher range.
intel_iommu=strict	The default setting for this is disabled. This option on every unmap_single operation will result in a hardware IOTLB flush operation as opposed to batching them for performance.
intel_iommu=sp_off	Super Page which is by default enabled if supported, you can turn this off using this option.
intel_iommu=ecs_off	By default, extended context tables will be supported if the hardware advertises that it has support both for the extended tables themselves, and also PASID support. With this option set, extended tables will not be used even on hardware which claims to support them.

3. SWIOTLB

SWOTLB是Intel的一项技术，它有点绕过了IOMMU，并提供了一个可配置内存管理的接口。无需深入探讨其工作原理，页面表被缓存到Lookaside Buffer，减少了不断访问物理内存以进行内存映射的需求。这项技术也被称为bounce buffer，因为内存映射的物理地址保存在这个虚拟空间中，I/O在物理I/O和物理内存之间通过virtual lookaside buffer进行bounce。这使得内存映射可以快速进行，更快地提供可用的物理内存空间。

每个IO TLB称为一个“slab”，可以在内核头文件swiotlb.h中找到：

/usr/src/linux-*/include/linux/swiotlb.h

* Maximum allowable number of contiguous slabs to map,

* must be a power of 2. What is the appropriate value ?

* The complexity of {map,unmap}_single is linearly dependent on this value.

#define IO_TLB_SEGSIZE 128

* log of the size of each IO TLB slab. The number of slabs is command line

* controllable.

*

这意味着1MB将是8个slabs，作为启动参数使用的值是以slabs为单位，而不是以大小为单位。

Attribute	Description and options
swiotlb=n'th amount of slabs	This specifies the amount of slabs to be used by IOTLB, each slab consists of 128K each which is 8 slabs per 1Mb(1024K), so a 64MB SWIOTLB would consist of 512 slabs. You can increase or decrease this value to allow for more buffering of virtual memory addresses in the buffer or not. Default is 64MB or 512 slabs.
swiotlb=force	This option will force all system IO through the SWIOTLB so there will be no IOMMU controlled by the BIOS or the IOMMU driver elsewhere if one existed.