github ↗
docs menu

Module Rig_mlx5.Region

A region is memory the NIC may read or write.

#type nic := t
#type t

The type for registered memory.

#type memory =
  1. #| Host of Rig_mlx5_abi.buffer
    (*

    The bytes of a bigarray, whose pages stay resident while registered. The NIC keeps the bigarray reachable until a deregister of the region returns Ok () or the NIC closes.

    *)
  2. #| Dmabuf of {
    1. #fd : int;
    2. #offset : int;
    3. #bytes : int;
    }
    (*

    bytes bytes at offset in the dma-buf fd, a device's memory a driver exported. The kernel keeps the dma-buf while it is registered, so fd may be closed once register returns.

    *)

The type for memory to register.

#type access =
  1. #| Local
    (*

    Peers neither read nor write.

    *)
  2. #| Remote_write
    (*

    Peers write.

    *)
  3. #| Remote
    (*

    Peers read and write.

    *)

The type for what peers may do: to a region (register), and through a queue pair (Qp.connect). A peer reads or writes a region only through a queue pair whose access allows it, and only if the region's access allows it too. Whatever the access, the NIC reads a region for this process's writes and sends, and writes it for this process's reads and receives.

#type order =
  1. #| In_order
    (*

    The writes of one queue pair land in the order its entries and its peer's were posted. The bytes of a read or a receive of this process land in host memory before its completion, so they are there once it is polled.

    *)
  2. #| Any_order
    (*

    Writes may land in any order. A reader must not take one write's bytes as a sign that an earlier write's bytes have landed.

    *)

The type for the order in which the NIC's writes land in a region. They are a peer's writes, the bytes of this process's reads, and the bytes of a peer's sends that this process's receives take.

#val register : nic -> access -> order -> memory -> (t, string) Stdlib.result

register nic a o m registers m with nic for access a and write order o. It is Error msg if the kernel refuses, such as when the process's locked-memory limit (RLIMIT_MEMLOCK) cannot hold m's pages.

Raises Invalid_argument if m has no bytes, or its offset or descriptor is negative.

#val deregister : t -> (unit, string) Stdlib.result

deregister r deregisters r. The NIC reads and writes r no more once deregister returns Ok ().

It is Error msg if the kernel refuses. r then counts as deregistered, but the NIC may still read and write its memory until it closes, and keeps a Host bigarray reachable until then.

Warning. After such an Error, the program must not reuse that memory before close.

#val local : t -> int -> int -> Rig_mlx5_abi.Entry.local

local r at n is the n bytes at offset at of r, as this process's entries name them.

Raises Invalid_argument if they are not in r.

#val remote : t -> int -> Rig_mlx5_abi.Entry.remote

remote r at is offset at of r, as a peer's entries name it.

Raises Invalid_argument if at is not in r or r is Local.