Module Rig_nv_pci

This library is the Rig_nv path for a GPU that no kernel driver holds. The process takes the GPU's PCI function, boots the GPU's system processor (GSP) with NVIDIA's signed firmware, and then acts as the kernel's half of the resource manager that the GSP runs. It gives the GSP the memory of the objects the GSP makes and runs the register sequences the GSP asks of it; the two talk through message queues in system memory. Once the GSP runs, a Rig_nv device runs submissions on the GPU as it does through NVIDIA's kernel driver.

This library is written, never run on hardware.

A machine is prepared once, outside the program, and a program then opens the GPU:

Rig.open_ (Rig_nv_pci.gpu 0)

                detach                 Rig.open_
kernel driver ---------> detached ------------------> held
      ^                   ^                            |
      +----- attach ------+---------- stop ------------+

detach takes the GPU from its kernel driver, and attach gives it back. An open holds the GPU until its device is stopped, and no other open, in this process or another, can take it meanwhile. A stop unloads the GSP's firmware and, on Ampere and Ada, takes down the protected memory (WPR2) the boot set up, which NVIDIA's driver refuses to boot a GPU with. attach resets a GPU first, as reset does. A GPU whose stop could not unload its GSP's firmware or take its boot down, or whose reset has failed, is lost, and its next open resets it first. A reset that leaves WPR2 up fails, so neither an open nor attach hands such a GPU on.

What an open needs states what any open of a GPU with no kernel driver needs.

Requirements

Opening requires Linux and a PCI function the process can take. How the process takes it depends on whether an IOMMU translates the GPU's accesses to system memory:

• With an IOMMU, bind the function to vfio-pci and grant access to its group's /dev/vfio/N. Rig_pci.Function.take states the group's rules.
• Without one, the process takes the GPU physically. Detach its kernel driver (detach) and grant write access to its files under /sys/bus/pci. The kernel must allow PCI mappings, which kernel lockdown refuses, and the process must read /proc/self/pagemap.

The chips whose boot this library lays out open: GA102, GA103, GA104, GA106 and GA107 (Ampere), AD102, AD103, AD104, AD106 and AD107 (Ada), and GB202, GB203, GB205, GB206 and GB207 (Blackwell), as the GPU's NV_PMC_BOOT_42 register names them. An open needs the firmware files Firmware lists.

Mapped memory (Memory) is the GPU's, through its memory BAR (BAR 1), the range of the GPU's memory that the host addresses over PCI. Pinned memory is the machine's system memory (Rig_pci.Function.alloc_dma). The GPUs' addresses and the process's addresses of system memory coincide, from 64 GiB to Rig_nv.windows, which the process reserves at its first open; every GPU it opens takes its addresses from that range. A device's stop gives back every address its open took, once the GPU's memory regions are freed, unless the GPU stopped answering (Rig_nv.stop), which keeps the addresses the GPU may still reach.

• Behind an IOMMU, system memory counts against the process's limit on locked memory (RLIMIT_MEMLOCK), and Rig_nv.alloc returns None once it is reached. Raise the limit to allocate more. Such a GPU's device has maps_host true.
• Taken physically, system memory lies in huge pages of 2 MiB, and Rig_nv.alloc returns None once none is free. Reserve more huge pages (vm.nr_hugepages) or free memory. Such a GPU's device has maps_host false, since the process's pages would go back to the system at its death with the GPU still writing them.

A boot also takes system memory for the GSP, most of it for the firmware's image, and it counts against the same limit. The GSP's logs and bootloader, and on Blackwell the FMC, each lie in one run of bus addresses.

The GSP's RM frees an object and every object under it when a device asks (Rig_nv.rm's free is Some), so a device of this path gives back the channels a growth replaces as Submitting states. With each set goes the GPU memory the path gave the GSP for it: the instance blocks of its two channels, the method buffer of their group and the compute engine's context buffers. An allocation the path or the GSP refuses gives back at once the memory the path gave for it, unless the GPU faulted meanwhile.

GPUs

GPU i here is GPU i (Rig_nv.is_gpu) of a machine. buses, count and reset also reach another machine through a Rig_pci.Machine.t transport, which an open refuses. Listing GPUs changes nothing on the machine.

Firmware

A boot loads files of linux-firmware's nvidia/ directory at commit 0a6871b1, each pinned by its BLAKE2b-256 digest: the GSP's firmware nvidia/ga102/gsp/gsp-570.144.bin, the same for every family, then its bootloader, the image that starts it, and on Ampere and Ada the booter that takes the boot down at its stop:

• Ampere: ga102/gsp/bootloader-570.144.bin, ga102/gsp/booter_load-570.144.bin and ga102/gsp/booter_unload-570.144.bin;
• Ada: the same three under ad102;
• Blackwell: gb202/gsp/bootloader-570.144.bin and gb202/gsp/fmc-570.144.bin.

They are read from directories the caller names, then from rig's firmware directory (Rig_pci.Firmware.directory), then from /lib/firmware (gpu), and this library downloads nothing. The rig command downloads them into rig's directory with curl:

rig firmware nv

Keep a directory of the caller's apart from /lib/firmware (What an open needs). On Ampere and Ada the boot also runs the FWSEC ucode of the GPU's VBIOS, which it reads from the GPU.

Opening

The GSP reports a channel it stopped and a fault its MMU queued, and the device raises them from Rig_nv.sleep, keeping a report through an exception in a sleep as Rig_edge.Driver.sleep requires. The GSP may also ask the process to run a sequence of register operations, which runs once, front to back: one an exception cuts short is a fault, which loses the GPU. Only the process uses a GPU it has booted, so the GPU's work waits for nobody else's, and rig loses the device once its timeline makes no progress for Rig_pci.Driver.hang_ms (the device's hang_ms).

Two GPUs map each other's memory only if both are taken physically, on one machine, each BAR reaches all of its GPU's memory, and the bus carries requests between them (Rig_pci.Memory.reaches).

Rig_nv.dma gives another function, such as a NIC that a process drives over PCI, a region's bus ranges (Rig_edge.dma.Bus), or the reason it cannot reach them, as Rig_pci.Memory.bus states: GPU memory through BAR 1, the memory BAR. The region's memory and the GPU then stay the process's until it exits, so no later open of the GPU, in this process or another, succeeds meanwhile. The device's facts name the GPU's function as bus.

A device reads the GPU's timer registers through BAR 0 (Rig_nv.clock.Registers), so its profiles' times drift from the host's clock as Submitting states.

Any domain may call any function. Opens and changes of NVIDIA GPUs run one at a time, so a boot delays the others. gpu's make, detach, attach and reset, called from a callback inside a lock (Rig's domains section), raise Invalid_argument, having changed nothing (Drivers).

Changes to the machine

These changes persist after the process, and each refuses a GPU this process holds. Each returns Error why if the machine's PCI functions cannot be listed. detach and attach act on this machine: they write its /sys/bus/pci files, and need CAP_SYS_ADMIN and write access to those files, which root has. Detaching or attaching another machine's GPUs needs a program there.