docs menu

on this page

Module Rig_remote_proxy

Proxies: devices of another machine, driven through its agent.

A proxy is a device of this process that stands for a device an agent, a process on another machine, opened there (Rig_remote). The machine's host is one of them. The proxy's driver turns each call into a message on the machine's Link.t: a request, which has an answer, the release of one of the agent's objects, or a handover, the work of one value of the proxy's timeline. The agent runs the work and reports each value done, and the link's receiving thread advances the proxy's timeline word, which lies in this process's memory, as the reports come.

rig.remote opens proxies with Rig.open_, one per device the agent opened, and uses this library's Wire and Link for the job's connections. The library links rig.edge, rig.remote.abi and unix, and this module matches Rig_edge.Driver.

Every value may be called from several domains in parallel, except the driver's room and submit for one proxy, which rig calls one at a time (Rig_edge). A call that waits for the agent or the link releases the runtime while it waits.

#module Wire : sig ... end

The protocol between the processes of a job.

#module Link : sig ... end

This process's ends of its job's connections.

Proxies

#type t

The type for proxies.

#val device : Link.t -> Wire.account -> Rig_remote_abi.t -> t Rig_edge.opener

device l a c describes the proxy on l of the agent's device a, whose record is c. Its name is a.name. Constructing it registers nothing and sends nothing: a is the agent's answer for a device it opened.

Its make ~word registers the proxy on l, with the 8 bytes at word as its timeline word, and returns it. It sends nothing. The proxy's values start after the value those bytes hold.

make raises Invalid_argument if c is Host _ and a's id is not 0, if c is Device { id } and id is not a's, or if l has a proxy of a's device. Closing a proxy that handed work over does not permit another proxy of that account on the same link; closing one that handed none over does. A make that raises, an asynchronous exception included, leaves no proxy registered.

include Rig_edge.Driver with type t := t

#type region

The type for memory of a device.

#type image

The type for loaded code.

#exception Fault of string

Fault why reports that the device faulted. Any counted call may raise it.

#val key : t Stdlib.Type.Id.t

key identifies the driver. rig asks map_peer only of two devices with the same key.

#val facts : t -> region Rig_edge.facts

facts d is d's facts.

Memory

#val alloc : t -> Rig_edge.memory -> int -> region option

alloc d m n is n bytes of d's memory m, or None if d has none. n is positive and a multiple of 4, Rig.word, so the region holds a buffer's bytes past its length up to a whole word. Counted.

#val free : t -> region -> unit

free d r gives back r, which is a region alloc made, a mapping map_peer or map_host made, or d's word of the driver's own (Rig_edge.facts). rig frees r once no work of d that uses it can run, and may free it after stop.

#val locate : region -> Rig_edge.location

locate r is where r lies.

#val peer : t -> t -> bool

peer d d' is true iff map_peer d d' maps Device memory of d'. It must return its result from what d and d' learned when they opened, also once either is lost.

#val map_peer : t -> t -> region -> region Rig_edge.mapping

map_peer d d' r maps for d the memory of r, any memory of d'. d's GPU may already map r's memory, through r or another region. Each region must keep the memory mapped until rig frees it. Counted.

#val map_host : t -> int -> int -> region Rig_edge.mapping

map_host d p n maps for d the n bytes of host memory at p. p starts a page and n is positive. rig calls it only where d declares maps_host. The region must map the whole pages that hold the n bytes, so that work reaches through it each word of the allocation's last bytes (Rig's Words section), and they must stay mapped until the region is freed (free). Counted.

#val dma : t -> region -> fn:string -> (Rig_edge.dma, string) Stdlib.result

dma d r ~fn is how the PCI function at the bus address fn of d's machine, such as "0000:41:00.0", reaches all of r's bytes by DMA, or Error why if it cannot, why naming the cause, such as a memory BAR smaller than the GPU's memory. It is Error why, why naming fn, if fn is not a bus address or names no function of the machine. rig calls it only with a region alloc made. Counted.

The driver decides whether fn reaches the memory where it lies, and must return Error rather than move the memory where fn reaches it. It must make the memory visible to fn after each value's work that wrote it, and visible to the device's later work after fn wrote it and a call of rig's submits that work.

For Bus, the driver must keep the memory where it is, out of any other allocation, and keep d's function from any other open, until the process exits, whatever rig frees or stops meanwhile.

Code

val image : 
  t ->
  string ->
  ((region, image) Rig_edge.code, string) Stdlib.result

image d b takes the binary b, or is Error why if d refuses it. Counted.

#val entry : image -> string -> Rig_edge.entry_answer

entry i f is Entry e with what the driver needs to launch i's function f, or Absent if i has no function f. It must make what a launch of f needs, such as a compiled pipeline or f's scratch memory, so that the driver's submit allocates nothing to launch f. A second call for f must make nothing new. The first call may block. Counted.

It is No_memory n, having made nothing, if the device has no memory for the n bytes f's launches need. rig then runs the rounds of reclamation an allocation does, asks again after each, and raises Rig.Out_of_memory after the last.

Raises Invalid_argument with the driver's reason if f needs more than the device offers, where the driver finds that only here.

#val prepare : image -> string -> unit

prepare i f may start making what entry i f makes and return without waiting, so that the device makes the entries of several functions concurrently: it is entry's asynchronous start. It must report nothing, failures included: the next entry i f must wait for what it started and answer as if it made the entry itself. A prepare of an f that has an entry or a started prepare must start nothing, and a prepare that raises must have changed nothing. Doing nothing is a valid prepare, as on a device whose entry makes nothing slow. Counted.

#val unload : t -> image -> unit

unload d i releases what image, prepare and entry made for i. rig calls it once no work of d that runs i can run. The region of a Place is not i's, and rig frees it after. Counted.

Queues

#val grow : t -> Rig_edge.grown

grow d makes what d's queues need to hold the parts of the submission whose room last answered RIG_GROW on the calling thread (The C functions). It changes nothing the C functions read: a later room puts what it made into the queues. Once it has answered Grown, room must not answer RIG_GROW for those parts again, whatever other submissions' growths answer.

It is Short n if d has no memory for n bytes the growth takes. What it made stays for a later room. rig then runs a round of the reclamation an allocation runs, for d's budget and the host's, and asks room again, which answers RIG_GROW anew for parts that still need a growth.

rig calls it on the thread whose room answered RIG_GROW, after that answer and before that thread calls room again, while the C functions may run on other threads for other submissions. A submission asks at most four times, as an allocation tries: after the last, a Short n raises Rig.Out_of_memory with n. A driver whose room never answers RIG_GROW is never asked. Counted.

Timeline

#val signaled : t -> int

signaled d is the value in d's word, read with acquire order. rig calls it only for a word of the driver's own (Rig_edge.facts), also after stop, until it frees that word. Counted until stop. After it, rig drops its faults. rig may call it while it holds a lock of its own, so it must not block or call rig.

#val sleep : t -> seen:int -> still_ms:int -> unit

sleep d ~seen ~still_ms blocks on the device's events until the word differs from seen, and must return, or raise Fault, at the latest after still_ms milliseconds. It must return immediately if the word already differs. It may return earlier, on an event of other work or a timer of the driver's own, since rig reads the word again after every return and sleeps again while it is unmoved. rig may call it while the driver's submit or commit runs. Counted.

An exception other than Fault that escapes sleep loses nothing (Drivers). The driver must record each event it consumes, such as a fault's report, before it consumes it, so that the next call of sleep still raises Fault for it.

Raises Fault once the device has faulted.

Stopping

#val stop : t -> fault:string option -> unit

stop d ~fault stops the lost d and must not wait for its work. rig calls it only at d's close or at an open of its name, with no counted call of d in flight. fault is Some why if rig lost d for a Fault of a counted call or for its hang bound (hang_ms), and None otherwise.

The driver must write the last value given to its submit into the word, with release order, also when that submit failed, and only once no work of d runs, since rig then frees the memory that work uses. If no work runs when stop is called, the driver writes it before stop returns. Otherwise the queues' releases write it, or the driver does once that work runs no more.
The driver must drop the work d encoded that runs only once committed, such as an open command buffer.
stop must release nothing rig made. rig frees each region and unloads each image itself, also after stop.
stop must release what the driver made to run work, such as its queues, and must not wait for that work. A word of the driver's own, and what signaled needs to read it, must stay until rig frees the word.
Once the word shows the last value given to submit and stop has returned, no write of d or of its work to the word may land, since the word then passes to a later device.

After stop, rig calls free and unload only once the word reads that last value, also after a stop that raised, such as one an asynchronous exception ended before it ended the device's work. It calls them uncounted and drops their faults. They may come after an open of the same hardware has made a new device of this driver, and must touch nothing of the new device.

#val capability : t -> Rig_remote_abi.t

capability d is the record device was given, which its facts declare (Rig_remote_abi.declare).

Facts

Fact	Value
`arch`	The agent's device's `Rig.arch`, from its account
`budget`	The agent's device's `Rig.budget` when it opened
`queues`	`"COMPUTE:0"` and `"COPY:0"`
`copy_engine`	`Some 1`, `"COPY:0"`
`completion`	`Host`
`waits`	`hosts`; `most` `max_int`
`may_block`	`true`
`hang_ms`	`None`
`maps_host`	`false`
`host_addresses`	`false`
`cache_bytes`	The agent's device's `Rig.cache_bytes`
`capability`	`capability` (`Rig_remote_abi.declare`)
`word`	The word `make` was given
`bus`	`None`

Each queue runs Copy, and on the machine's host Words too. The link's receiving thread writes the timeline word as the agent reports (Timeline, failure and stopping). A wait on a proxy of the same link goes to the agent in the handover. submit sends its message itself, so it may block. A copy names this process's memory by its host address, and the proxy's own memory is another machine's.

Work

A proxy runs copies, and the machine's host also runs Rig.Submission.words parts, each a run of the rig.program code loaded on that host. The work goes to the agent and back as Rig_remote's order section states, a copy into this process's memory in a bytes message of the agent's (Messages), whose bytes the receiving thread writes where the copy names (Timeline, failure and stopping). A copy's bytes cross the link once. The agent runs the parts of both queues in one order, that of the handover, after the proxy's earlier values.

The driver's room returns:

RIG_NEVER for a part other than words and copies, for words on a proxy other than the machine's host, and for a submission with a copy from this process's memory after a copy into it, because submit reads the source bytes before the agent runs the earlier part;
RIG_LATER while the proxy has more than 64 MiB of copies and words submitted and not reported done;
RIG_FITS otherwise, so that a proxy with nothing in flight takes any submission it can run.

The driver's submit sends a Wire.handover message on the link: the waits, each as the device at and its value, and the parts, copies with their sides' handles and offsets, words with their words. It sends after the message the link is sending, and returns once its message is sent, so it also waits for the peer to take its bytes. Before a submission with a copy from this process's memory, it waits until the work the submission follows is done here: the timeline word of each wait's proxy reaches the wait's value, and d's own word the last of d's values with a copy into this process's memory. It then sends the copies' bytes from that memory, so the bytes they copy are final, and holds the submitting thread while they cross the link: in place where a copy's rows lie end to end, and otherwise packed into memory of its own, which it allocates for the call.

submit returns RIG_COMMITTED once its message is sent. It returns RIG_FAILED with the job's root cause if the job has failed, with "the job is closed" if the job closes, and with "out of memory for a hand-over" if memory runs out. The driver's commit does nothing, since each submit sends its message.

Memory

A region is an object of the agent, named by an id of the job (Link.fresh), or a proxy's timeline word. alloc d m n asks the agent for n bytes of m on its device. It is None if the agent lacks the room or refuses, and raises Fault if the job has failed. free sends the release of a region's object, after every message sent before, and the agent releases it once the work submitted before no longer needs it. It does nothing for a word.

locate of memory is the object's id as its handle, which submit sends as a copy's side, with no address and no host. Of a word it is Some i as its address, i the id of the word's device on the agent, which a wait's at carries and submit sends as the device to wait on; the word's host address as its host; and 0n as its handle.

peer d d' is true iff d and d' are proxies of one link and the agent's device of d reaches the own memory of d''s, as the account with the larger id describes (Wire.account). map_peer d d' r is None for proxies of two links. A word maps as itself. For memory, it asks the agent to map r's object on d's device as a new object, and is None if the agent cannot or refuses; it raises Fault if it asks and the job has failed. map_host is None. dma is Error, since the memory is another machine's, which its agent describes there.

Code

image d b loads b on the agent's host if d is the machine's host. It is Ok (Loaded i), or Error why with the agent's reason, rig.program's for a binary it does not load. On any other proxy it is Error why, why saying the proxy loads no code. It raises Fault if d is the machine's host and the job has failed. entry is the agent's answer for the function, None if the image has none or the agent refuses; it raises Fault if the job has failed. unload sends the release of the image, after every message sent before, and the agent releases it once the work submitted before no longer needs it.

Timeline, failure and stopping

The proxy's timeline word is the eight bytes of this process's memory that make was given. They hold, as an unsigned 64-bit integer in the host's byte order, the last value v the agent has reported done. The receiving thread writes v with release order, after it has written the bytes of the copies into this process's memory of every value up to v. After stop the word holds the last value submitted. It never decreases, and the edge's state holds its address. signaled reads it with acquire order. sleep returns once the word differs from seen, immediately if it does already, or after still_ms milliseconds; it raises Fault if the job has failed, before or meanwhile.

Proxies fail with their link's job, as Link states. Fault reports that the job has failed, its message the job's root cause, or that the job's close has begun (Link.close), its message "the job is closed". Once the job has failed, the calls that ask the agent (alloc, map_peer of memory, image, entry) and sleep raise Fault, and submit returns RIG_FAILED with the root cause. free and unload then send nothing, and the other calls behave as before. Once the close has begun, the calls that ask the agent raise Fault.

stop writes the last value submitted into d's timeline word with release order, unless the word holds it already, once no bytes of a copy into this process's memory may still land: immediately if none is pending or the link's receiving thread has ended, and otherwise as the last pending copy's bytes land or the thread ends. It waits for nothing, and ignores fault. Once the word holds that value, nothing writes it: the receiving thread writes it under the link's lock, as the stop does, and drops a report of a stopped proxy that the word has passed.