github ↗
docs menu
on this page

Module Rig_pci.Machine

Machines, their PCI functions, and transports to other machines.

Machines this process reaches, and their PCI functions.

A machine is a computer whose PCI functions this process reaches. It is this one (this), or another one that a transport reaches (make). Every operation of this library works on either machine, and a driver never learns which. This library knows no transport: another library implements one by giving the operations of ops.

A machine lists its functions (functions) without changing anything. Function.take takes one of them for the process, and reserve sets aside the process's addresses that the functions' system memory goes at.

Bus addresses

A function's bus address names it on its machine, as Linux spells it: "DDDD:BB:DD.F", its domain, bus, device and function numbers in lowercase hexadecimal, the domain in at least four digits.

#val compare_address : string -> string -> int

compare_address a b orders bus addresses in bus order: by domain, then bus, device and function, each as a number.

Raises Invalid_argument if a or b is not a bus address.

Machines

#type t

The type for machines.

#val this : t

this is the machine this process runs on.

#val name : t -> string option

name m is the name of m as its transport gives it, such as "host:7000", or None if m is reached without a transport.

#val failed : t -> string option

failed m is the reason m can no longer be reached, if it cannot. A machine that failed stays failed. A machine reached without a transport never fails.

#val page : t -> int

page m is the page size of m in bytes.

#type id = {
  1. #bus : string;
    (*

    Its bus address.

    *)
  2. #vendor : int;
    (*

    Its vendor's identifier, such as 0x1002 for AMD.

    *)
  3. #device : int;
    (*

    Its device identifier, assigned by its vendor.

    *)
  4. #class_ : int;
    (*

    Its 24-bit class code: base class, subclass and programming interface, such as 0x030000 for a VGA controller.

    *)
}

The type for the identity of a function.

#val functions : t -> (id list, string) Stdlib.result

functions m is the PCI functions of m, in bus order (compare_address). Listing them changes nothing on m. For a machine without a transport it is Ok [] where the system has no /sys/bus/pci, and a function whose files cannot be read, as while the kernel removes it, is left out.

Error why if the functions cannot be listed, such as when the process has no file left or the transport has failed.

#val reserve : t -> base:int -> int -> (unit, string) Stdlib.result

reserve m ~base n reserves the n addresses from base on, in the address space of the process that holds m's functions, so that only Function.alloc_dma maps memory there. A range stays reserved while that process runs, and reserving it again does nothing.

Error why if part of the range is in use, or if m is reached without a transport and its system is not Linux.

Routes

A machine's bus carries one function's requests to another function's BAR, as from a network adapter to a GPU's memory, where Linux's p2pdma allows it (Memory.reaches states the rule). The values below read each function's way up to its host bridge from the machine's files. A function is unread if its bus is not a bus address, if it names no function of the machine, or if its files cannot be read, and every function of a machine reached through a transport is.

#val forwards : t -> provider:string -> client:string -> bool

forwards m ~provider ~client is true iff the bus of m carries the requests of the function at client to a BAR of the function at provider, as Memory.reaches states Linux's p2pdma decides, for a process that reads every bridge's ACS controls. It is false if either function is unread.

A process without CAP_SYS_ADMIN cannot read the ACS controls past the first 64 bytes of configuration space, so for it forwards answers as the root complex's rule alone decides: false also where Linux carries the requests below a common bridge.

#type distance =
  1. #| Switch
    (*

    Below a bridge that both ways up to the host bridge pass, such as a switch's port or a root port.

    *)
  2. #| Bridge
    (*

    Below one host bridge, with no such bridge.

    *)
  3. #| System
    (*

    Below two host bridges.

    *)

The type for how far apart two functions of a machine are, nearest first.

#val distance : t -> string -> string -> distance option

distance m a b is how far apart the functions at a and b of m are, or None if either is unread.

Transports

A library that reaches another machine makes it a machine with make, from the operations below. A transport runs each operation on the other machine as the same operation of this runs here. A request it cannot run returns Error why, with why starting with the machine's name, and so does the reason its transport failed (failed). Window states what accesses give once the transport has failed.

Function refuses misuse before it calls an operation, and counts the windows and pins of each function. An operation is called only with arguments Function accepts. The size of alloc_dma is rounded up to page, and its va lies inside a range reserve reserved. A pin's range starts on a page. Once Function.release has begun on a function, no use of it (Function) starts, and the uses running finish before the give-back's own config, set_config and release. The release, the last use, a use refused after the release or Function.finish_releases makes them, never two at once. No free_dma, unpin or unmap follows the release's start but a use's, or a free_dma or unpin of a function kept taken for another function (Memory.bus) once its bus mastering is off: the take's release gives back what the take holds. An exception, such as an asynchronous one, may end release part-way, and Function then calls it again. Every pin and unpin of a live function is passed on, so a transport counts them too.

A use's operations run inside a lock of the calling thread (errors), as the give-back's do: alloc_dma, pin, free_dma and unpin but those above, the config of a wait's bound, and those of a request of Memory that writes a GPU's tables. Each must call nothing of rig's or of a driver's, and must return on the thread that called it.

#type addressing =
  1. #| Physical
    (*

    At physical addresses of its machine, which no IOMMU translates.

    *)
  2. #| Iommu
    (*

    Through an IOMMU, at device addresses the process maps for it alone.

    *)

The type for how a function reaches system memory.

#type fn = {
  1. #addressing : addressing;
    (*

    How it reaches system memory.

    *)
  2. #inherited : bool;
    (*

    Whether, when it was taken, the function still reached memory that a dead process left, which it may still write. It is read once, at the take.

    *)
  3. #config : int -> int -> int;
    (*

    config off n is the n-byte value at byte off of its configuration space, for n of 1, 2 or 4. Only Function's accessors call it.

    *)
  4. #set_config : int -> int -> int -> unit;
    (*

    set_config off n x writes the low n bytes of x at byte off, for n of 1, 2 or 4. Only Function's accessors call it.

    *)
  5. #bar : int -> (int * int) option;
    (*

    Function.bar.

    *)
  6. #map : combine:bool -> int -> int -> int -> (Window.t, string) Stdlib.result;
    (*

    map ~combine i off n is Function.map of BAR i.

    *)
  7. #unmap : Window.t -> unit;
    (*

    Function.unmap. It must raise only where an exception cuts it, and leaves what remains of a window it did not unmap to release.

    *)
  8. #interrupt : int -> bool;
    (*

    Function.interrupt.

    *)
  9. #reset : unit -> (unit, string) Stdlib.result;
    (*

    Function.reset, without waiting for the function to answer.

    *)
  10. #forget : unit -> (unit, string) Stdlib.result;
    (*

    forget () records that the function's device was reset, so that it reaches none of the memory that dead processes left. That memory goes once no other function reaches it.

    *)
  11. #alloc_dma : contiguous:bool -> va:int option -> below:int -> int -> ((Window.t * (int * int) list) option, string) Stdlib.result;
    (*

    Function.alloc_dma, whose runs end at or below below where the machine chooses their addresses, as behind an IOMMU (Function.set_dma_bits).

    *)
  12. #free_dma : Window.t -> unit;
    (*

    Function.free_dma.

    *)
  13. #pin : below:int -> int -> int -> ((int * int) list option, string) Stdlib.result;
    (*

    Function.pin, as alloc_dma places its runs.

    *)
  14. #unpin : int -> int -> unit;
    (*

    Function.unpin.

    *)
  15. #release : unit -> bool;
    (*

    Function.release, which turns the function's bus mastering off: false if the memory the function reached may still be written once given back, as when the system refused the write that turns its bus mastering off. It gives back everything the take holds: its windows, and, once its bus mastering reads off, its DMA memory and pins, those an exception kept Function from recording included. It must raise only where an exception cuts it. Called again after an exception ended it part-way, it must give back what it had not, and nothing twice.

    *)
}

The type for the operations on a function that a transport took.

#type ops = {
  1. #transport : Window.transport;
    (*

    The C accesses to the machine's addresses. The machine's windows use them, and their failed function is failed's.

    *)
  2. #page : int;
    (*

    page.

    *)
  3. #functions : unit -> (id list, string) Stdlib.result;
    (*

    functions, in any order.

    *)
  4. #take : string -> (fn, string) Stdlib.result;
    (*

    Function.take on the machine.

    *)
  5. #reserve : base:int -> int -> (unit, string) Stdlib.result;
    (*

    reserve.

    *)
}

The type for the operations of a transport.

#val make : name:string -> ops -> t

make ~name ops is the machine named name that ops reaches.