Module Rig_amd_pci

The process takes the GPU's PCI function from Linux, boots the GPU from firmware files, and gives Rig_amd the GPU's memory, queues and interrupt ring (Rig_amd.path). The device then runs as every Rig_amd device does, and work reaches the GPU with no system call. This library does what a kernel driver would do. It boots the GPU, bounds how long work runs, reports faults from the interrupt ring, and resets a GPU whose state it cannot trust. Rig_pci holds the mechanism this path shares with other vendors' drivers, such as taking a function and holding a GPU.

A GPU passes through these states:

            detach                Rig.open_
amdgpu ---------------> detached ------------> held
   ^                     |  ^  ^                 |
   +-------attach--------+  |  +-----close-------+
                            |                    | fault, hang,
                            |                    v killed process
                            +------reset------- lost

• While amdgpu is bound, the kernel driver holds the GPU. detach takes it from the kernel driver, and attach gives it back.
• A detached GPU (Rig_pci.Driver) is bound to no kernel driver, or to vfio-pci, amdgpu has let go of it, and no process holds it.
• A held GPU belongs to this process, from Rig.open_ until its device stops (Rig_pci.Driver.hold). Closing the device stops the GPU's work and leaves it detached, marked clean, so that the next open boots it in part (Opening).
• A GPU is lost when a fault or a hang ended its device, a boot failed after its first write, or the process that held it was killed. Its state is then unknown. reset makes it detached, and its next open resets it first (Opening).

A program opens the first GPU with the firmware that the rig command downloaded (What an open needs):

let d = Rig.open_ (Rig_amd_pci.gpu 0)

Hardware status. Virtual functions and XGMI fabrics are written, never run on hardware.

What an open needs states what any open of a GPU with no kernel driver needs, and Rig_pci.Function how a process takes a GPU's function.

What an open needs

GPUs. The library boots GPUs whose blocks have versions it holds register tables and firmware for: GC 9.4.3 and 9.5.0 (the Instinct MI300 and MI350 series, physical and virtual functions), 11.0.0 and 11.0.2, and 12.0.0 and 12.0.1. It reads the GPU's discovery table without changing the GPU's state, and refuses a GPU with a block of another version before the boot writes.

Firmware. The images are files of linux-firmware's amdgpu/ directory at commit 0a6871b1, each pinned by its BLAKE2b-256 digest. They are named for the versions of the GPU's blocks: its security processor's psp_X_sos.bin, its power manager's smu_X.bin, its copy engines' sdma_X.bin, and its GC's gc_X_mec.bin and gc_X_rlc.bin, with gc_X_imu.bin from GC 11 and gc_X_pfp.bin and gc_X_me.bin from GC 12. The R9700, for one, boots with psp_14_0_3_sos.bin, smu_14_0_3.bin, sdma_7_0_1.bin and gc_12_0_1_{pfp,me,mec,imu,rlc}.bin. The library reads uncompressed files only. The rig command downloads the pinned images into rig's firmware directory (Rig_pci.Firmware.directory):

rig firmware amd

Host memory. Behind an IOMMU, the device's host memory counts against the process's locked-memory limit (RLIMIT_MEMLOCK). Taken physically, a GPU reaches only the host memory this library allocates for it, in 2 MiB huge pages from /dev/hugepages. The device takes one page for each 2 MiB block of its host memory, staging included, and the system must have them free (vm.nr_hugepages). The process also needs CAP_SYS_ADMIN, to read physical addresses in /proc/self/pagemap. Such a device does not declare maps_host (Rig_edge.facts).

GPUs

GPU i of a machine is the ith of its AMD GPUs in bus order (Rig_amd.is_gpu), whichever driver holds each, so it is the same GPU whichever path opens it. The GPUs are named "AMD-PCI", "AMD-PCI:1", "AMD-PCI:2", …. buses, gpu and reset also reach another machine (Rig_pci.Machine.t).

Opening

Times. A device measures the GPU's clock against the host's at open only, with stamps of its compute queue (Rig_amd.clock.Round_trip), so the times of its profiles drift from the host's clock at the clock's rate error since open.

Sessions. An open writes a mark, a value in one of the GC's scratch registers that says this library booted the GPU and whether its session ended cleanly. The process stops every GPU it holds at exit. A GPU whose work stopped with no fault is marked clean, and the next open, by this process or another, boots only its compute and copy blocks, from the state the last session left. A child of fork stops nothing at its exit.

The open resets the GPU as reset does, then boots it in full, when this library cannot vouch for the GPU's state, whoever left it. Such a GPU may run firmware without the mark, as its kernel driver leaves it, or have been left by a killed process, or by a process that lost it to a fault or a hang, this process or another. The open returns Error if firmware this library did not start still runs after the reset. A GC 9.5.0 GPU this library marked is the exception: a mode 1 reset over its live state can stop its fabric until a power cycle. Its open stops its queues, soft-resets its compute engines and boots them partially.

A boot turns the GPU's bus mastering on only once both of its memory hubs translate addresses, so that no address a boot step gets wrong reaches host memory.

Fabrics. This library cannot tell which fabric (XGMI) a GPU belongs to. It takes a machine's AMD GPUs for one fabric when their number is a fabric's size, and they then reach each other's memory over its links. Otherwise they reach each other through their memory BARs. Two halves of two fabrics are taken for one. A fabric's GPUs reset together, as their kernel driver resets them when it takes them back, so reset only stops such a GPU, and an open refuses one left running.

Other functions. Rig_amd.dma gives another function, such as a NIC that a process drives over PCI, a region's bus ranges (Rig_edge.dma.Bus), or the reason it cannot reach them, as Rig_pci.Memory.bus states: GPU memory through the memory BAR. The region's memory and the GPU then stay the process's until it exits, so no later open of the GPU, in this process or another, succeeds meanwhile. The device's facts name the GPU's function as bus.

Faults and hangs. The GPU reports faults on its interrupt ring: page faults with their address, shader errors, and fatal hardware errors with its machine-check banks. Rig_amd.sleep raises them. No kernel bounds the GPU's work, so the device declares hang_ms (Rig_edge.facts) of Rig_pci.Driver.hang_ms. A device stopped with the fault it was lost for, a hang included, leaves its GPU lost, since its stop does not wait for the queues to leave.

Domains. Any domain may call any value. Opens, resets and changes to the machine of AMD GPUs run one at a time, so a boot delays the others. Each holds a lock for its whole run, a boot included, so a call of rig's from a callback on its thread meanwhile is made inside a lock and raises Invalid_argument (Rig's domains section). An opener's make, detach, attach and reset, called from a callback inside a lock, raise Invalid_argument, having changed nothing (Drivers).

Changes to the machine

detach, attach and reset change the machine as Rig_pci.Driver describes, and their changes persist after the process. Each refuses a GPU that a process holds, this one included, over PCI or through a file of its kernel driver, such as the GPU's render node. detach and attach act on this machine alone, and need CAP_SYS_ADMIN and write access to the files under /sys/bus/pci they write, which root has. Each returns Error msg if the machine's PCI functions cannot be listed. An Error or a Part_way for a file names it.

References

• PCI-SIG. PCI Express Base Specification, 7.5.3.7 "Link Control Register": ASPM.