Module Nx
N-dimensional arrays as values on device sets.
A value of type ('v, 's, 'd) t is elements of one dtype, read as 'v and stored as 's, with a shape and a placement over the device set 'd. Nothing changes a value.
A device set is a module: module Gpu = (val Nx.devices [ d0; d1 ]) mints a brand Gpu.d, and values of two sets never meet in typed code. A value lies at a placement: on the host, on one device of its set, whole on every device of it, or cut into windows across them, where each device holds one window of the value. A value is made where it is asked: on the host, at a placement with an _at form, or at another value's placement with a _like form (Creation).
An operation computes where its operands lie. place moves a value, and an operation whose operands do not lie together raises, naming the place to write (Routing). Between devices an operation moves only bytes that no placement of its operands could have put where they are needed: the partial results of a reduction, a contraction, a scan, a gather or a scatter along a cut axis, and the parts of the result they close into, which exist only once it computes; a gather's or a scatter's positions, whose need depends on their values; and, on a set of devices of several architectures, each window that devices of several of them hold, which the devices of one compute and copy to the others (Reductions and scans, Routing). A contraction asked for its sum moves nothing: its result is the devices' terms, which place closes (Contraction).
Values
A value is a shape, a dtype and the devices its elements lie on. A shape has at most Nx_array.Layout.max_rank axes, no negative extent, and extents other than 0 that multiply to at most Nx_array.Layout.max_numel. A function given or making another shape raises Invalid_argument naming itself.
type (!'v, !'s, !+'d) tThe type for arrays of 's-format elements read as 'v, on the device set 'd.
type ('v, 's) dtype = ('v, 's) Nx_array.Dtype.tThe type for dtypes whose elements OCaml reads as 'v and which store them as 's.
Dtypes
A dtype says how a value stores its elements and how OCaml reads them: float32 stores 32-bit floats and reads them as float, int8 stores 8-bit integers and reads them as int. The operands of an operation share a dtype by their types; cast changes it. There is one value per constructor of Dtype.t, named in lowercase. Every NaN result follows the NaN rule (Stores): where this page leaves its sign and payload unspecified, it is still signalling only where an operand is a signalling NaN.
module Dtype = Nx_array.DtypeDtypes: their constructors, element types and properties.
Dtype's element types, under the same names: float32_elt is Dtype.float32_elt.
type float64_elt = Dtype.float64_elttype float32_elt = Dtype.float32_elttype float16_elt = Dtype.float16_elttype bfloat16_elt = Dtype.bfloat16_elttype float8_e4m3fn_elt = Dtype.float8_e4m3fn_elttype float8_e5m2_elt = Dtype.float8_e5m2_elttype float4_e2m1fn_elt = Dtype.float4_e2m1fn_elttype int64_elt = Dtype.int64_elttype uint64_elt = Dtype.uint64_elttype int32_elt = Dtype.int32_elttype uint32_elt = Dtype.uint32_elttype int16_signed_elt = Dtype.int16_signed_elttype int16_unsigned_elt = Dtype.int16_unsigned_elttype int8_signed_elt = Dtype.int8_signed_elttype int8_unsigned_elt = Dtype.int8_unsigned_elttype int4_elt = Dtype.int4_elttype uint4_elt = Dtype.uint4_elttype complex64_elt = Dtype.complex64_elttype complex32_elt = Dtype.complex32_elttype bool_elt = Dtype.bool_elttype bit_elt = Dtype.bit_eltval float64 : (float, Dtype.float64_elt) dtypeval float32 : (float, Dtype.float32_elt) dtypeval float16 : (float, Dtype.float16_elt) dtypeval bfloat16 : (float, Dtype.bfloat16_elt) dtypeval float8_e4m3fn : (float, Dtype.float8_e4m3fn_elt) dtypeval float8_e5m2 : (float, Dtype.float8_e5m2_elt) dtypeval float4_e2m1fn : (float, Dtype.float4_e2m1fn_elt) dtypeval int64 : (int64, Dtype.int64_elt) dtypeval uint64 : (int64, Dtype.uint64_elt) dtypeval int32 : (int32, Dtype.int32_elt) dtypeval uint32 : (int32, Dtype.uint32_elt) dtypeval int16 : (int, Dtype.int16_signed_elt) dtypeval uint16 : (int, Dtype.int16_unsigned_elt) dtypeval int8 : (int, Dtype.int8_signed_elt) dtypeval uint8 : (int, Dtype.int8_unsigned_elt) dtypeval int4 : (int, Dtype.int4_elt) dtypeval uint4 : (int, Dtype.uint4_elt) dtypeval complex128 : (Stdlib.Complex.t, Dtype.complex64_elt) dtypeval complex64 : (Stdlib.Complex.t, Dtype.complex32_elt) dtypeval bool : (bool, Dtype.bool_elt) dtypeval bit : (bool, Dtype.bit_elt) dtypeEach dtype value dt names the type of its values, dt_t: a 'd float32_t is a float32 value on the set 'd.
type 'd float64_t = (float, Dtype.float64_elt, 'd) ttype 'd float32_t = (float, Dtype.float32_elt, 'd) ttype 'd float16_t = (float, Dtype.float16_elt, 'd) ttype 'd bfloat16_t = (float, Dtype.bfloat16_elt, 'd) ttype 'd float8_e4m3fn_t = (float, Dtype.float8_e4m3fn_elt, 'd) ttype 'd float8_e5m2_t = (float, Dtype.float8_e5m2_elt, 'd) ttype 'd float4_e2m1fn_t = (float, Dtype.float4_e2m1fn_elt, 'd) ttype 'd int64_t = (int64, Dtype.int64_elt, 'd) ttype 'd uint64_t = (int64, Dtype.uint64_elt, 'd) ttype 'd int32_t = (int32, Dtype.int32_elt, 'd) ttype 'd uint32_t = (int32, Dtype.uint32_elt, 'd) ttype 'd int16_t = (int, Dtype.int16_signed_elt, 'd) ttype 'd uint16_t = (int, Dtype.int16_unsigned_elt, 'd) ttype 'd int8_t = (int, Dtype.int8_signed_elt, 'd) ttype 'd uint8_t = (int, Dtype.int8_unsigned_elt, 'd) ttype 'd int4_t = (int, Dtype.int4_elt, 'd) ttype 'd uint4_t = (int, Dtype.uint4_elt, 'd) ttype 'd complex128_t = (Stdlib.Complex.t, Dtype.complex64_elt, 'd) ttype 'd complex64_t = (Stdlib.Complex.t, Dtype.complex32_elt, 'd) ttype 'd bool_t = (bool, Dtype.bool_elt, 'd) ttype 'd bit_t = (bool, Dtype.bit_elt, 'd) tThe type for values whose dtype is hidden, such as a file's named values. Match on P for the value; unpack also fixes its dtype.
unpack dt p is the value in p if its dtype is dt.
Raises Invalid_argument naming both dtypes otherwise.
Device sets and placement
A value lies on a device set, a module minted by devices whose brand 'd keeps its values apart from other sets'. Placement says where a value lies within its set, and place moves a value between sets, and between placements of one. A value may also lie as a sum of terms across devices (Placement.partial), which place closes. The host is a set like any other, Host.
A brand names one set. A set's module unpacked twice names its brand twice, one type under two names, and two mints of one device list are two sets of two brands. Placement.same tells whether two brands name one set.
val devices : ?kernels:(module Nx_kernel.S) -> Rig.t list -> (module Devices)devices ~kernels ds mints a new set over ds, in their order, with a brand no other set has, computed eagerly by kernels. Without kernels, a set whose every device runs on the host (Rig.runs_on_host) is computed by nx.cpu, and any other set by none, as a GPU set is outside a compiler. On a set without kernels, a creation (Creation), a value whose elements come from numbers, is made on the host and placed at the placement the call names as place places it. Every operation other than a creation raises on such a set where it makes elements, naming the place onto a set with kernels or the compiler that would compute it, even where it needs no element of its operands, as matrix_power a 0 does. place, a view and a call that gives its operand itself make none. Under a compiler such as Rune.jit, the compiler receives a creation's operations.
Raises Invalid_argument if ds is empty or repeats a device, or if kernels does not compute on one of ds.
Routing
Each device computes its window of an operation's result from the windows of the operands it holds. Where some device's windows hold elements, a device whose windows of the results hold none computes nothing. The result lies at the union of the operands' cuts, joined in operand order, or, where that order cannot read an operand where it lies, with a gather's or a scatter's positions last. A cut of an axis that becomes a result axis cuts that axis over the same devices. A cut of an axis read whole cuts nothing. A cut of an axis that a reduction or a contraction sums makes the devices compute by partials where every operand holding the axis is cut alike, and cuts nothing in the result; so does one of the axis a gather reads at positions, and one of the axis along which a scatter reads its positions and updates. One along a scan's axis stays in it, and so does one along the axis a scatter writes at positions, each device writing the updates that land in its window. A product asked for its sum leaves the result a sum over the devices that cut the axis instead (Contraction). An operand whole on every device is read by a view of each device's window.
On a set whose devices are of several architectures (Rig.arch), such as a GPU beside a memory device under a compiler, each step that computes windows held by several devices, an operation's results, a fold's partials or a part of its closing, computes each such window on the devices of the architecture of the first device in the placement's order that holds it, and copies it to the others. Each element of a value whole on each device, or of a window repeated along a mesh's axis, so holds equal bits on every device that holds it. A closing cuts its result into parts across the devices, so a whole value's elements may come from devices of several architectures.
An operation raises Invalid_argument where two operands cut one result axis over different devices or tiles, where a device needs a window of an operand it does not hold, and where operands lie alone on two devices or on two sets. The message names the function, the operand, numbered from 0 in the order Prim.t lists them, where it lies, where it is read, and the place that puts it there, writing the set as s, a device by its name, a mesh's axes as "g0", "g1", … in the read placement's order, and the operands as x, y, z, then x3, x4, …:
Nx.matmul: operand 1 lies at split ~axis:1 [CUDA:0; CUDA:1] and is read at split ~axis:0 [CUDA:0; CUDA:1]; Nx.place (Placement.split ~axis:0 s) y
A function that computes by several operations over values it derives from its arguments, as reduce_ranges does by gathers and maps of its rows and bounds, raises in their place where one of them refuses an operand derived from an argument: the message names the function, the argument as its documentation names it, where it lies, and the call of the argument's placing function, place for a value, that reads the operand where the operation needs it, the grid moved back through each operation between them (Prim.composite). An argument that the remedy leaves where it lies is passed over for the next. Over lo and x split along their rows and columns over the same devices:
Nx.reduce_ranges: x lies at split ~axis:1 [CUDA:0; CUDA:1] and is read at on [CUDA:0; CUDA:1]; Nx.place (Placement.on s) x
An operation linear in its partial operands (Placement.partial), sums over the same groups, keeps them sums, each device computing on its terms: add, sub and where by a condition that is no sum, a value that is none joining on each group's first device, -0 or +0 standing in for it elsewhere; neg; mul and a float div by a value that is none; a cast between floats or between complex numbers; sum and a float mean along uncut axes; movements, copy and take; and the products (Contraction). A value made at a sum holds its elements on each group's first device. Any other operation raises, naming the place that closes the operand:
Nx.exp: operand 0 lies at partial [CUDA:0; CUDA:1], a sum the operation does not keep; Nx.place (Placement.on s) x
Closing a kept result equals closing first within the roundings each term passes, but where a term overflows that the sum would not, where terms that are zeros of both signs are negated or scaled, which may change a zero's sign, and where terms are scaled by an infinity or divided by zero, which may give a NaN where closing first gives an infinity.
Where neither order reads a gather's or a scatter's positions where they lie, the result lies where the other operands put it, and the positions move to where its devices read them before they compute. Each device receives the positions its window reads that it lacks, 8 bytes each, and a position the operation broadcasts moves once. No other operand moves.
A traced operation raises it as eager execution would (Prim.results).
val place : 'e Placement.t -> ('v, 's, 'd) t -> ('v, 's, 'e) tplace p x is x's elements at p. Each device of p holds its window over x's own memory where x already lies there, or where the host addresses both x's memory and that device's (Rig.reaches); otherwise it holds a copy of its window, allocating at most the window's bytes. Across sets it changes the brand; within one, the arrangement. Placing onto Host from another set is a wait on x's set; elsewhere it returns as Rig.Buffer.submit_copy returns for its copies.
A partial x placed where it is no sum closes: each element is its group's terms added in their order, from the first, by one addition of x's dtype per term after it, on whichever device adds it, so it is the same on every run. Where p is placement x with its summed mesh axes cutting, in their order, an axis placement x does not cut, each device adds its own window, receiving that window of each other term of its group, a reduce-scatter; it equals the closing at Placement.closed (placement x) placed at p, bit for bit. Elsewhere x closes at Placement.closed (placement x), each device of a group adding a part of its window and receiving the others', an all-reduce, then moves to p. A float or complex sum of t terms is within γ(t - 1, u) Σ|term| of its terms' exact sum, γ(n, u) = n u / (1 - n u) and u the dtype's unit roundoff, part by part for a complex one, and an integer sum wraps as one addition does. Every addition is the kernels' Add: the set needs kernels.
A value that is no sum placed at a partial p holds its window on the first device of each group, and the additive identity, -0 for floats and complex parts, 0 for integers, on every other, so that closing it gives its bits. A partial x placed at another partial p closes, then is placed so.
Raises Invalid_argument if p cuts an axis x does not have, or is partial and x's dtype is a boolean, bit, int4 or uint4, Rig.Lost for a lost device, and a failed check as a wait does.
val placement : ('v, 's, 'd) t -> 'd Placement.tplacement x is where x's bytes lie.
val beside : ('v, 's, 'd) t -> 'd Placement.tbeside x is Placement.uncut (placement x), whole on each device x lies on, and no sum. A value of another shape made there combines with x. At placement x such a value is cut along its own axes, which need not line up with x's. For x of shape [4; 3] split along axis 0, arange_at (placement x) float32 0 3 1 is split along its only axis, which add x reads along axis 1, and raises.
Creation
A value made from a dtype, a shape and numbers is a host value. The _at forms make it at a placement, each device computing its window; the _like forms make it at a value's placement, or whole on each of its devices where the shape has no axis the placement cuts. A creation holds its elements from the call on.
On a set without kernels, a creation is made on the host and placed where the call names (devices).
An _at form raises as its host form does, and raises Invalid_argument naming itself where its placement cuts an axis the shape lacks. A value made here combines with others through the functions of Arithmetic and their operators in Infix.
A function meant for values of every set makes its constants beside its operands, and names the set with a locally abstract type:
let shift (type d) (x : (float, 's, d) Nx.t) = Nx.add x (Nx.arange_f_at (Nx.beside x) (Nx.dtype x) 0. 3. 1.)
A host value written there in place of the _at form is then a type error at the operation that meets x. Without the annotation, OCaml infers x's set as host, and the error appears where the function is applied to a value of another set.
zeros dt s is the host value of shape s whose every element is zero.
Raises Invalid_argument if an extent is negative.
val zeros_at : 'd Placement.t -> ('v, 's) dtype -> int array -> ('v, 's, 'd) tzeros_at p dt s is zeros dt s at p.
ones dt s is the host value of shape s whose every element is one (true for booleans).
Raises Invalid_argument if an extent is negative.
val ones_at : 'd Placement.t -> ('v, 's) dtype -> int array -> ('v, 's, 'd) tones_at p dt s is ones dt s at p.
full dt s v is the host value of shape s whose every element is v, a float stored as Dtype.of_float says.
Raises Invalid_argument if an extent is negative or v is an int outside dt's range.
val full_at :
'd Placement.t ->
('v, 's) dtype ->
int array ->
'v ->
('v, 's, 'd) tfull_at p dt s v is full dt s v at p.
scalar dt v is the 0-d host value v.
Raises Invalid_argument if v is an int outside dt's range.
val scalar_at : 'd Placement.t -> ('v, 's) dtype -> 'v -> ('v, 's, 'd) tscalar_at p dt v is scalar dt v at Placement.uncut p, whole on each of p's devices however p cuts, since a 0-d value has no axis to cut.
arange dt start stop step is the 1-d host value start, start + step, … before stop, empty where stop is not past start in step's direction. Each value is exact, then stored in dt as cast stores an int64.
Raises Invalid_argument if step = 0, the range holds more than max_int values, or a value is outside dt's range for an integer dtype, or other than 0 and 1 for a boolean one.
val arange_at :
'd Placement.t ->
('v, 's) dtype ->
int ->
int ->
int ->
('v, 's, 'd) tarange_at p dt start stop step is arange dt start stop step at p.
arange_f dt start stop step is the 1-d host value of the ⌈(stop - start) / step⌉ values start + i step, each computed at float64 with one rounding, then stored in dt; empty where that count is not positive.
Raises Invalid_argument if step = 0. or the count is not finite.
val arange_f_at :
'd Placement.t ->
(float, 's) dtype ->
float ->
float ->
float ->
(float, 's, 'd) tarange_f_at p dt start stop step is arange_f dt start stop step at p.
linspace dt ~endpoint start stop n is the host value of n values evenly spaced from start to stop, stop included where endpoint (default true). They are computed at float64 and stored in dt: the first is start and, with endpoint and n >= 2, the last is stop, each as dt stores it, and for finite start and stop every value lies between them.
Raises Invalid_argument if n < 0.
val linspace_at :
'd Placement.t ->
('v, 's) dtype ->
?endpoint:bool ->
float ->
float ->
int ->
('v, 's, 'd) tlinspace_at p dt ~endpoint start stop n is linspace dt ~endpoint start stop n at p.
val logspace :
(float, 's) dtype ->
?endpoint:bool ->
?base:float ->
float ->
float ->
int ->
(float, 's, host) tlogspace dt ~endpoint ~base start stop n is the host value of base (default 10.) to the powers linspace float64 ~endpoint start stop n, each power computed at float64 and stored once in dt: logspace float32 0. 2. 3 is 1, 10 and 100.
Raises Invalid_argument if n < 0.
val logspace_at :
'd Placement.t ->
(float, 's) dtype ->
?endpoint:bool ->
?base:float ->
float ->
float ->
int ->
(float, 's, 'd) tlogspace_at p dt ~endpoint ~base start stop n is logspace dt ~endpoint ~base start stop n at p.
val geomspace :
(float, 's) dtype ->
?endpoint:bool ->
float ->
float ->
int ->
(float, 's, host) tgeomspace dt ~endpoint start stop n is the host value of n values in geometric progression from start to stop, stop included where endpoint (default true): the value at i is start (stop / start)^(i / (n - 1)), or ^(i / n) without endpoint. They are computed at float64 and stored in dt: the first is start and, with endpoint and n >= 2, the last is stop, each as dt stores it. geomspace float32 1. 1000. 4 is 1, 10, 100 and 1000.
Raises Invalid_argument if n < 0, or if start or stop is not positive and finite.
val geomspace_at :
'd Placement.t ->
(float, 's) dtype ->
?endpoint:bool ->
float ->
float ->
int ->
(float, 's, 'd) tgeomspace_at p dt ~endpoint start stop n is geomspace dt ~endpoint start stop n at p.
eye ~m ~k dt n is the n × m host value (m defaults to n) whose element at row i and column j is one where j - i = k (default 0) and zero elsewhere: k > 0 is a diagonal above the main one.
Raises Invalid_argument if n or m is negative.
val eye_at :
?m:int ->
?k:int ->
'd Placement.t ->
('v, 's) dtype ->
int ->
('v, 's, 'd) teye_at ~m ~k p dt n is eye ~m ~k dt n at p.
From a reference
These make a value of their argument's dtype where it lies. Its elements are never read.
Donation and copies
A program gives a value up with donate, which makes a handle of it: a value that one operation reads and may write its result over. The value given up is the handle's donor. The operation that reads a handle consumes it and is its consumer: once the operation has checked every operand, the donor and every handle of it are dead, whatever its set's kernels then answer, and a later operation on any of them raises Invalid_argument naming the consumer, while their shape, dtype and placement still answer. An operation refused by its checks kills nothing. One whose kernels decline raises with them dead and returns their memory to its device, as a consumer that raises does (Prim.Consumer.with_).
A movement, which rearranges elements without computing them (reshape, transpose, flip, broadcast_to, slicing by ranges, sliding_window), passes a new handle of the donor to its result, whether it makes a view or a copy, and kills nothing; the deaths wait for the final consumer. The handle it read stays a handle of the donor: once an operation consumes either, the other raises naming that consumer.
A call consumes each operand that reaches one operation, through movements: the call is the consumer the deaths name, as trace is for the diagonal it sums. An operand that several operations read is lent: the call reads it as a value over the donor's memory, which is shared from then on, and the donor and its handles live on, as a does in lstsq dt (donate a) b. That memory returns to its device once no value holds it. A sampler's key is lent (Rng). Two handles of one donor reaching one operation, the same handle twice, two donations of one value or a handle and one a movement passed on, are read as the donor beside its handle is: as the donor was when called, and it dies once. So add (donate x) x is x + x, and mul h h is x * x for h = donate x. Where they reach different operations of a call, which run in an order the call fixes, one that reads the donor or a handle after another consumed it raises naming the call.
An interpretation that receives an operation (Prim.interpret) reads each handle as its arrays and kills nothing: the donor lives on, and its memory is shared from then on.
On any number of domains, no value is consumed twice, and a call on another domain that overlaps a donor's consumer either reads the donor's elements as they were or raises Invalid_argument; it never reads the consumer's result. No consumer overwrites elements another call is reading: one that claims a handle while a call reads its memory computes into fresh memory. Racing consumers on several domains for the handles of one donor is a misuse: any of them may raise, naming another, and leave dead the other donors it claimed.
The consumer may write its result into the donated memory when it reads that memory only at the result's own index, through the handle or through the donor itself (an elementwise operand, the target of set or scatter, contract's init), the memory is C-contiguous with the result's shape and dtype, each device's memory with its window's for a value cut across devices where the donor lies as the result does, and the memory is not shared. Memory is shared once a value outside the donor's handles has been made over it (a view, a borrow by place, a Repr crossing, an interpretation's read, a read by a function of several operations), and stays shared. Otherwise the consumer computes into fresh memory, and a value that shares the memory keeps it. Its result is the same either way.
donate x is a handle of x (Donation and copies). x stays usable until an operation consumes a handle of x; then x and every handle of x are dead. A traced x is returned unchanged, and nothing dies.
Raises Invalid_argument if x is dead.
copy x is x stored afresh: equal elements in new memory.
From OCaml values
A value made from OCaml data lies on the host, the set Host.
create dt s vs is the host value of shape s holding vs in C order, a float stored as Dtype.of_float says.
Raises Invalid_argument if an extent is negative, Array.length vs is not the product of s, or an element is an int outside dt's range.
init dt s f is the host value of shape s whose element at index i is f i. f is called once per index, in C order, each time with a fresh array.
Raises Invalid_argument as create does.
map_item f x is the host value of x's shape whose element at each index is f of x's element there, a float stored as Dtype.of_float says. f is an OCaml function, so it runs on the host, once per element in C order. x, at any brand, is read as to_array reads it.
Raises Invalid_argument if f gives an int outside x's dtype's range, and as to_array does.
val of_bigarray :
('v, 's, Stdlib.Bigarray.c_layout) Stdlib.Bigarray.Genarray.t ->
('v, 's, host) tof_bigarray b is a host value holding a copy of b's elements, of b's shape and the dtype that stores its kind. Later writes to b do not reach it.
The copy is one pass over b, into memory the value owns, so its donation may write in place (donate). To read b's bytes in place instead, Repr.of_array Host.v (Nx_array.of_bigarray dt b) is the value over them (Nx_array.of_bigarray, Repr.of_array): a write to b from then on leaves what that value reads unspecified.
Raises Invalid_argument if b's kind is char, int or nativeint, which no dtype stores.
Reads
A read gives a value's elements to OCaml. It works at every brand: it is a wait on the value's set, which may raise a failed check, then copies the value's elements to the host and leaves every placement as it was. A partial value's elements are its terms' sums, so a read closes it as place does. A read reaches no interpretation.
Each raises Invalid_argument naming itself for a dead value, and for a traced one, as Nx.item: the value is traced by Rune.jit, or as Nx.item: a value of Rune.grad, used after Rune.grad returned once its interpretation has returned; item alone reads a traced value that has a twin, as its contract says.
item i x is x's element at i, one position per axis, a negative one counting from the end; item [] x reads a 0-d value. A traced x with a twin (Prim.twinned) reads as its twin where x's interpretation is around the call and no Extent interpretation lies between them, and raises as any traced value elsewhere.
Raises Invalid_argument if i does not have one position per axis of x or a position is outside its axis.
val to_bigarray :
('v, 's) Stdlib.Bigarray.kind ->
('v, 's, 'd) t ->
('v, 's, Stdlib.Bigarray.c_layout) Stdlib.Bigarray.Genarray.tto_bigarray k x is a fresh bigarray of x's shape holding its elements. A dtype Bigarray has no kind for is a type error.
Raises Invalid_argument if x has more than 16 axes, Bigarray's most.
Waiting and checks
A set's devices may compute while the caller goes on: an operation returns once its work is submitted. The host waits for a set only at a wait: sync, a read, or place onto Host. A place elsewhere returns as nx.array's placement does (Placement): where a device copies, as Rig.Buffer.submit_copy returns.
A device lost while it runs a value's work raises Rig.Lost at the next use of the memory that work wrote: an operation or a submit that reads it, a wait or a read. Every other error of an operation raises in its call. Memory that queued work reads stays allocated until that work is done, so a loop that never waits holds the values of the steps its devices' queues hold, and a submit waits for room once a queue is full.
A check is a condition on data whose failure raises the caller's exception: one stated with check, a sampler's parameter outside its domain (Rng), or a Prim.t.Check. On the host it raises at the call. On another set it is computed where its data lies, and its failure raises at the next wait on that set on the domain that issued it, so that no check makes the host wait. The systhreads of a domain share its checks. A wait drains the checks its domain issued on the set before it and raises the earliest failure among them; the later ones are dropped. A wait on one set raises nothing of another's checks; a check no wait follows never raises. A check keeps its data's memory until a wait drains it, or until a later check on its set is issued once its devices report its flags computed, so a donation of that data meanwhile computes into fresh memory. A device reports its work at most a fixed number of submissions late while work is submitted on it (Timelines), and a check on a device that gets no more work is found computed when the next check is issued, so the checks of a loop that never waits keep the data of a bounded window of steps. A failing check's data is read when its failure is found, at a wait or at a later check's issue, and that read may wait for the set. Once a check is known to fail, the checks after it release their data, and a check issued later on its set is not kept.
sync x waits for the work x depends on, then drains the checks pending on x's set, raising the earliest failure, as stated above.
Raises Invalid_argument naming Nx.sync for a dead value, and for a traced one, as item does: Nx.sync: the value is traced by Rune.jit, or Nx.sync: a value of Rune.grad, used after Rune.grad returned once its interpretation has returned.
Shapes, broadcasting and movements
Axes count from 0, and a negative axis from the end: -1 is the last. Shapes broadcast from their last axes: aligned there, each pair of extents is equal or one of them is 1, and missing leading axes count as 1.
A movement rearranges elements without computing on them. Each says whether it makes a view, a value over its operand's memory that costs no copy. That is a cost, never part of the result's meaning: a value never changes, so a view and a copy read the same.
Over a value cut across devices, a movement keeps the cut of an axis it keeps in order, as a permutation, a broadcast that does not stretch the axis and a slice of all of it do. get, squeeze, unsqueeze, slice's I and N, and rearrange for a name that is an axis of its own on both sides state which axes go and come, so each axis they keep keeps its cut on the axis it becomes, whatever its extent: a batch of one cut across devices lies, folds and scans as a batch of two does. An axis they drop is read whole. A reshape keeps the cut of an axis it factors or merges where the tiles divide both extents, each window then holding whole rows, and pairs axes of one element front to back, as it cannot know which is which. Any other axis it reads whole (Routing), so flatten or a merging reshape of a batch of one cut across devices raises, as one of three over two devices does.
dim a x is the extent of x's axis a.
Raises Invalid_argument if a is not an axis of x.
numel x is x's number of elements: the product of its extents, 1 for a 0-d value.
nbytes x is the bytes x's elements fill when stored contiguously, (numel x * b + 7) / 8 for a dtype of b bits.
reshape s x is x's elements in C order at shape s. One extent of s may be -1; it is inferred from the others. A view where strides express it, a copy otherwise.
Raises Invalid_argument if the element counts differ, an extent is below -1, or two are -1.
broadcast_to s x is x stretched to shape s: an axis of extent 1, or one x lacks, repeats x's elements along it. A view.
Raises Invalid_argument unless x's shape broadcasts with s to s.
val broadcast_shapes : int array list -> int arraybroadcast_shapes ss is the shape that ss broadcast to, [||] for [].
Raises Invalid_argument if an extent is negative or two of them do not broadcast.
broadcast_arrays xs is each of xs stretched to broadcast_shapes of their shapes, in order. Views.
Raises Invalid_argument if two of their shapes do not broadcast.
meshgrid ~indexing [x1; …; xn] is the coordinate grids of the 1-d x1 to xn: n views of one shape, the ith holding xi's element j wherever its axis for xi is at j. With `Ij, xi's axis is axis i, of shape [|n1; …; nn|]. With `Xy, the first two exchange, so that x1 runs along columns and x2 along rows as a plane's x and y do: [|n2; n1; n3; …; nn|].
Raises Invalid_argument if an operand is not 1-d.
squeeze ~axes x is x without the axes axes; without axes, without every axis of extent 1. A view.
Raises Invalid_argument if an axis of axes is not x's, repeats, or has an extent other than 1.
unsqueeze ~axes x is x with an axis of extent 1 at each position of axes, positions of the result: unsqueeze ~axes:[ 0; -1 ] of a [3] value is [1; 3; 1]. A view.
Raises Invalid_argument if a position is not an axis of the result or repeats.
flatten ~start_dim ~end_dim x is x with its axes start_dim to end_dim, both included, merged into one. start_dim defaults to 0 and end_dim to -1; a 0-d x flattens as the [1] value it holds. A view where strides express it, a copy otherwise.
Raises Invalid_argument if either is not an axis of x or start_dim comes after end_dim.
transpose ~axes x has x's axis List.nth axes i as its axis i; without axes, x's axes in reverse order. A view.
Raises Invalid_argument unless axes lists each of x's axes once.
moveaxis a b x is x with its axis a moved to position b, the other axes in their order. A view.
Raises Invalid_argument if a or b is not an axis of x.
swapaxes a b x is x with its axes a and b exchanged. A view.
Raises Invalid_argument if a or b is not an axis of x.
flip ~axes x is x with the order of elements along each axis of axes reversed; without axes, along every axis. A view.
Raises Invalid_argument if an axis of axes is not x's or repeats.
sliding_window ~axis ~window ~step x is x's windows of window consecutive elements along axis (default -1), one every step (default 1). Axis axis becomes the number of windows, (d - window) / step + 1 for an extent d, or 0 where window exceeds d, and an axis of extent window is appended: window w's element j is x's element w * step + j along axis. A view, whose windows share elements where step < window.
Raises Invalid_argument if axis is not x's, or window or step is below 1.
split ~axis n x is x cut along axis into n runs of consecutive elements, in order. With d the extent, the first d mod n runs have d / n + 1 elements and the others d / n. Views.
Raises Invalid_argument if axis is not x's or n < 1.
split_at ~axis offsets x is x cut along axis before each of offsets, in order. There is one run more than there are offsets; the first starts at 0 and the last ends at the extent d. A negative offset counts from the end; then each clips into [0, d], so that one past the end cuts at d. split_at ~axis:0 [ 2; 3 ] of [w; x; y; z] is [[w; x]; [y]; [z]]. Views.
Raises Invalid_argument if axis is not x's or an offset cuts before the offset preceding it in offsets.
concatenate ~axis xs is xs joined along their axis axis, in order. A copy. Joining two or more values reads axis whole on each device, so a value cut along it raises (Routing); a cut of another axis cuts the result alike.
Raises Invalid_argument if xs is empty, axis is not an axis of its first value, or their shapes differ off axis.
stack ~axis xs is xs joined along a new axis at position axis of the result (default 0), in order. A copy. A cut of an axis of xs cuts the result's axis it becomes (Routing).
Raises Invalid_argument if xs is empty, their shapes differ, or axis is not an axis of the result.
pad widths v x is x with fst widths.(i) elements v before its axis i and snd widths.(i) after it. A copy. It reads each axis it widens whole on each device, so a value cut along it raises (Routing); a cut of another axis cuts the result alike.
Raises Invalid_argument if widths does not have one pair per axis of x, a width is negative, or v is an int outside x's dtype's range.
roll ~axis k x is x with its elements along axis shifted k places toward the end, those past it wrapping to the start; a negative k shifts toward the start. Without axis, flatten x shifted, at x's shape. A copy.
Raises Invalid_argument if axis is not an axis of x.
tile reps x is x repeated reps.(i) times, end to end, along its axis i. With more entries in reps than x has axes, x first gains leading axes of extent 1. A view where strides express it, a copy otherwise.
Raises Invalid_argument if reps has fewer entries than x has axes or a negative one.
repeat ~axis n x is x with each element repeated n times in place along axis: repeat ~axis:0 2 of [a; b] is [a; a; b; b]. Without axis, flatten x repeated. A view where strides express it, a copy otherwise.
Raises Invalid_argument if axis is not an axis of x or n < 0.
Axis patterns
A pattern states a layout in words. Names are words separated by spaces, parentheses group axes merged into one, 1 is an axis of extent 1, and ... stands for the axes no name covers. One grammar serves rearrange, with one operand, contract, with two, and einsum, with one or more:
pattern ::= operands "->" layout [ "|" name { name } ]
operands ::= layout { "," layout }
layout ::= { axis }
axis ::= name | "(" name { name } ")" | "1" | "..."
name ::= a letter, then letters, digits and "_"let split_heads = Nx.Pattern.v "b t (h d) -> b h t d" let merge_heads = Nx.Pattern.inverse split_heads let q = Nx.rearrange ~sizes:[ ("h", heads) ] split_heads y
rearrange ~sizes p x is x split, permuted and merged as p says: the result's element at an index is x's element where each name takes the same position. A name alone on the left takes its axis's extent, and sizes gives others; a group on the left may hold one name whose extent neither gives, the quotient of its axis by the others. A view where strides express it, a copy otherwise.
Raises Invalid_argument if p has more than one operand, repeats a name or sums one, p's left side names more axes than x has or, without ..., other than x's rank, a 1 on the left meets an extent other than 1, a group's extents do not divide or multiply to its axis, a group has two unknown extents, a size disagrees with the extent x gives its name, a name in sizes is not in p or is there twice, or a size is negative.
Indexing and slicing
A position written in the program (I, L, the ranges) is checked: one outside its axis raises, and a negative one counts from the end. A position held in data (T, take, take_along_axis, scatter) never raises and never counts from the end: outside [0, d), negative included, it reads zero (+0., false, 0 + 0i) and its write is dropped. That makes a gather and an add-scatter adjoint; a program that wants a loud failure states it with a check. A D start, also held in data, clamps into [0, d - n] instead, so its window lies in the axis.
type 'd index = - #
| I of int(*One position; the axis goes.
*) - #
| L of int list(*Positions written in the program; a gather.
*) - #
| T of 'd int64_t(*Positions held in data, replacing the axis by their axes; a gather.
*) - #
| R of int * int(*
*)R (start, stop):starttostop - 1. A negativestartorstopcounts from the end; then both clip into[0, d]. - #
| Rs of int * int * int(*
*)Rs (start, stop, step):start,start + step, … towardstop,stopexcluded. A negativestartorstopcounts from the end; then both clip into[0, d]forstep > 0and into[-1, d - 1]forstep < 0, soRs (-1, -d - 1, -1)is the axis reversed. - #
| A(*The whole axis.
*) - #
| N(*A new axis of extent
*)1; it selects along no axis ofx. - #
| D of 'd int64_t * int(*
*)D (start, n):nconsecutive positions from the 0-dstartheld in data, the start clamped into[0, d - n]; a gather. - #
| M of 'd bool_t(*A mask over as many axes as it has, from the next one, of their shape: the positions where it holds, in C order, replace those axes by one; a gather. A 0-d mask covers no axis and adds one of extent
*)1or0. Its count is a length that is data (Lengths that are data).
The type for selections along one axis.
slice idx x selects from x axis by axis: each entry selects along the next axis, or a mask along its axes, independently of the others, N excepted, and axes past the list are whole. I, R, Rs, A and N give a view; L, T, D and M gather.
Raises Invalid_argument if idx addresses more axes than x has, a written position is outside its axis, a step is 0, a D start is not 0-d, a D length is negative or exceeds its axis, a mask's shape is not that of the axes it covers, or a mask is traced. Under a trace, T (positions ~size m) selects, or where computes without selecting.
get p x is slice (List.map (fun i -> I i) p) x, a view.
take ~axis p x replaces x's axis axis by p's axes, reading x at each position p holds; without axis, flatten x is read. A position outside [0, d) reads zero. A gather, which has the bits it has on one device wherever x lies, cut along axis too, but for an x of Nx_array.Layout.max_rank axes, or a complex128 one of one fewer, which is read whole along it (Routing). Positions that cannot be read where they lie move first to the devices that read them.
Raises Invalid_argument if axis is not an axis of x.
take_along_axis ~axis p x has p's shape broadcast with x's off axis; its element at j is x's element at j with axis axis replaced by p's element at j. A position outside [0, d) reads zero. A gather, which computes over a cut axis as take does.
Raises Invalid_argument if axis is not an axis of x, p's rank is not x's, or the shapes off axis do not broadcast.
unravel_index i s is the coordinates in the shape s, in C order, of each flat index of i: one value of i's shape per axis of s, [] for a 0-d s. An index k in [0, n), n the number of elements of s, has (k / t) mod s.(j) along axis j, t the product of the extents after j. An index outside [0, n) has each axis' extent, outside it, so take reads zero there and scatter writes nothing.
Raises Invalid_argument if s is not a shape.
Functional updates
set idx v x is x with slice idx x replaced by v, broadcast to the selection's shape. x is unchanged. Without donate it costs a copy of x; set idx v (donate x) writes only the selection, into x's memory, where donate lets the consumer write there. A position in T outside its axis writes nothing; where T repeats a position, the last update, in C order of the selection, wins. A D window clamps its start as slice does, so a start past d - n writes the last n positions: a decode step whose position reaches the cache's capacity overwrites its last row, and a program that can pass capacity states the bound with a check. A mask writes the positions where it holds. v is checked against its count only once the mask is read: a v that fits the selection but for that count raises with a donated mask consumed.
Where one entry selects by positions and every other is A, N or a range of its whole axis, the update is a scatter along that axis. Otherwise the positions of each L, T and D entry are read broadcast to the selection's shape, beside those of the entries before it, at the union of their cuts in the order of idx (Routing). A refusal names the entry's positions as operand 1, broadcast to the selection, a value the program does not hold, beside the entries before it as operand 0; the place to write puts that entry's positions, as given, whole on each device, Placement.on s.
Raises Invalid_argument as slice does, if v does not broadcast to the selection's shape, or if an L names one position twice, counting from the end.
type combine = Nx_kernel.combine = The type for how scatter combines a target with its updates.
val scatter :
?combine:combine ->
axis:int ->
'd int64_t ->
('v, 's, 'd) t ->
('v, 's, 'd) t ->
('v, 's, 'd) tscatter ~combine ~axis p u x is x with each element of u combined into x at its own index with axis axis replaced by p's element there; p and u have one shape. A target's updates apply in C order of u: Set (default) keeps the last; Add gives the element plus its updates, with no +0 added, so a -0. element and -0. updates give -0., a narrow float summed in float32 and rounded once; Max and Min give the largest or smallest of the element and its updates, or a NaN where one of them is. Where no update lands, the element keeps its bits, a signalling NaN's included. A position outside [0, d) writes nothing. A float Add gives the same bits on every run, adding in an order its kernel library states. For finite operands, barring overflow, it is within γ(n + 1) (|x| + Σ |ui|) of the exact sum of an element x and its n updates before a narrow float's rounding: γ(k) = k u / (1 - k u), u the unit roundoff of float32 for a narrow float, else of the dtype's parts. Without donate it costs a copy of x; scatter ~axis p u (donate x) writes only the updates, into x's memory, where donate lets the consumer write there. Where x is cut along axis, each device writes the updates whose positions fall in its window, reading p and u whole along axis (Routing): each element takes its updates in their order on one device. Where p and u are cut along axis, but for an x of Nx_array.Layout.max_rank axes, which reads them whole there, each device combines the updates it holds into its window of x, and these partial results close by combine in the order of the tiles of u, moving as a sum's over a cut axis do and never u: each tile of u holding updates sends a partial of each device's window of x, whatever the size of u. Two updates of one element differ only along axis, so the later tile holds the later in C order. Set keeps the last, from the last tile holding one, and Max and Min the bits one device gives, a NaN where one of them is. An Add's integer result is exact. For finite operands, barring overflow, an Add's float or complex element is within γ(k + t) (|x| + Σ|u|) of the exact sum before a narrow float's rounding, k its updates and t the tiles of u holding updates. An element no update lands on keeps its value; a NaN stays a NaN, its sign and payload unpromised.
Raises Invalid_argument if axis is not an axis of x, p and u differ in shape, their shape differs from x's off axis, or combine is Add and the dtype is boolean.
Arithmetic
Elementwise operations compute where their operands lie and give a new value. Binary operations broadcast: aligned at their last axes, two extents are equal or one of them is 1, which stretches. Integers wrap. The float formats narrower than 32 bits compute at float32 and round once.
Each raises Invalid_argument naming itself if the shapes do not broadcast, or if its operation does not take the operands' dtype: what each takes is stated with it. The host computes every dtype stated. Where a device's kernels do not compute one, as a GPU's may not compute complex numbers, the function raises naming itself. The operators of these functions, such as a + b for add a b, are in Infix (Operators), with those of the comparisons, the bitwise functions, matmul, get and slice.
add a b is the elementwise sum. Every dtype but booleans.
mul a b is the elementwise product, as add.
div a b is a / b: the IEEE 754 quotient on floats and complex numbers; on integers the quotient truncated toward zero, 0 for a divisor of 0, and a signed dtype's least value for that value by -1. Every dtype but booleans.
mod_ a b is the remainder of a / b, of a's sign: fmod on floats, and a for a divisor of 0 on integers, so that a = add (mul b (div a b)) (mod_ a b) on integers. Floats and integers.
pow a b is a to the power b. On complex numbers it is the principal value exp (b * log a): 1 where b is 0, 0 at a = 0 where b's real part is positive, with log's branch cut (Powers, exponentials and trigonometry); its error grows with |b log a|, within (3 + 3|b log a|) u |a{^b}|. On floats it is within 4 ulps at float32 and 2 at float64, with Annex F's special values (Powers, exponentials and trigonometry). Every dtype but booleans.
fma a b c is a * b + c rounded once. Floats and integers.
lerp a b w is a + w (b - a), by one fma: a + w (b - a) where w < 0.5, else b + (w - 1) (b - a), b - a rounded once first. Where b - a is finite, w = 0 gives a and w = 1 gives b, and lerp a a w is a for a finite w, each equal as a number: a -0. there may give +0.. Floats.
maximum a b is the greater element by the dtype's order (less). On floats it is the IEEE 754 maximum: a NaN gives NaN, and -0. orders below +0.. Every dtype.
minimum a b is the lesser element, as maximum.
recip x is 1 / x; on integers, x for 1 and -1 and 0 otherwise. Every dtype but booleans.
abs x is x's absolute value. Floats and integers: a complex number's modulus is a float, magnitude.
sign x is -1, 0 or 1 by x's sign, and NaN for a NaN. On complex numbers it is x / |x|, of modulus 1, and x itself at 0; at an infinite part, the direction its parts point to: sign (inf + 2i) = 1 + 0i; within 3 u as Powers, exponentials and trigonometry measures it. Every dtype but booleans.
Scalar forms
f_s x c is f x (scalar_like x c) and rf_s c x is f (scalar_like x c) x, c stored in x's dtype as Dtype.of_float says for a float. c is written into the program of one operation over x alone, as full writes its value. Each raises as its function does, naming itself, and Invalid_argument if c is an int outside x's dtype's range.
Operators
Infix holds the operators over values, so that open Nx keeps OCaml's operators. Each operator is the function its doc names, and raises as that function does. A local open applies them to one expression, as Nx.Infix.(a * b +$ 1.), and let open Nx.Infix in to the rest of a block. Inside Nx.( … ), Infix.( … ) adds them, as in Nx.(Infix.(sin x * x +$ 1.)). Within an open of Infix, OCaml's operators are reached through Stdlib, as Stdlib.( + ). mod_, whose result takes the dividend's sign, has no operator.
Powers, exponentials and trigonometry
Floats and complex numbers, where a function does not say floats. A stated range is the mathematical one, which rounding can leave at an end: in float8_e4m3fn, π rounds to 3.25.
On floats the error of a result is its distance in floats of the dtype from the exact value rounded to the dtype: a neighbour is one away, and -0. and +0. are one apart. Each function states a bound on it at float32 and float64. Whatever the bound, a result is exact at the special values of C99's Annex F (ISO/IEC 9899, F.10): exp (-inf) is +0., log of ±0. is -inf, sin of an infinity is NaN and atan inf is π/2 rounded. Where the exact value rounds to an infinity or to zero, the result is that infinity or zero, of the exact value's sign: exp 710. is inf and pow (-2.) (-1101.) is -0.. Another zero result has the sign Annex F gives it: sin (-0.) is -0.. A NaN operand gives a NaN, but where Annex F gives a number for every value of that operand: pow nan 0. and pow 1. nan are 1.. Which NaN, its sign and its payload are unspecified.
The formats narrower than 32 bits compute at float32 and round once (Arithmetic). Where a bound is in ulps, their result is within one ulp of the exact value rounded to their dtype. A format without infinities or NaN stores such a value as cast does: log of a float8_e4m3fn zero is -448..
nx.cpu keeps subnormal numbers (Nx_cpu). A device whose kernel library flushes them to zero reads a subnormal operand as zero and writes a subnormal result as zero; there each bound, of this section and of Special functions, holds where the operands and the result are normal.
On complex numbers each is the principal value C99 defines (ISO/IEC 9899, 7.3), with its special values at infinities, NaNs and signed zeros (Annex G). Where a result's sign comes from a NaN part, as the imaginary ∞ of sqrt (-∞ + NaN i) does, that sign is unspecified, as Annex G allows. A function whose inverse is many-valued has branch cuts on an axis, and the sign of a zero part names the side it continues from: sqrt (-4 + 0i) is 2i and sqrt (-4 - 0i) is -2i. A result f' is within k u |f| of the value f, where |f| is a normal float: u is the parts' unit roundoff, 2{^-24} for complex64 and 2{^-53} for complex128, and k is 3 for sqrt, 4 for exp, exp2, log and log2, 5 for asin, acos, atan, asinh, acosh and atanh, and 6 for sin, cos, tan, sinh, cosh and tanh. On the real axis, at a non-zero real part where the float function is defined, a function of one operand but asinh, acosh and atanh gives the float function's result as its real part; pow does at a positive base and a real exponent.
These bounds, and those the functions below state, hold where a set's kernels compute. Under a compiler such as Rune.jit, the compiler's bounds hold.
sqrt x is the square root, NaN below -0., correctly rounded on floats. On complex numbers its real part is never negative, and its branch cut is the negative real axis.
hypot x y is sqrt (x{^2} + y{^2}) with no intermediate overflow: finite wherever the result is. It is inf where x or y is an infinity, a NaN beside it included, and NaN where either is a NaN otherwise. Within 3 ulps at float32 and float64. Floats.
expm1 x is e{^x} - 1, accurate near 0: -0. at -0. and -1. at -inf. Within 1 ulp at float32 and 2 at float64. Floats.
log x is the natural logarithm: -inf at 0., NaN below. Within 1 ulp at float32 and 2 at float64. On complex numbers it is log |x| + i angle x, its imaginary part in [-π, π] and its branch cut the negative real axis.
log2 x is the base-2 logarithm. Within 1 ulp at float32 and 2 at float64.
log1p x is log (1 + x), accurate near 0: -0. at -0., -inf at -1., NaN below. Within 1 ulp at float32 and 2 at float64. Floats.
sin x is the sine of x radians. Within 2 ulps at float32 and float64.
cos x is the cosine of x radians. Within 2 ulps at float32 and float64.
tan x is the tangent of x radians. Within 4 ulps at float32 and 2 at float64.
asin x is the arcsine, in [-π/2, π/2]: on complex numbers its real part, with branch cuts on the real axis beyond -1 and 1. Within 2 ulps at float32 and float64.
acos x is the arccosine, in [0, π]: on complex numbers its real part, with asin's branch cuts. Within 2 ulps at float32 and float64.
atan x is the arctangent, in [-π/2, π/2]: on complex numbers its real part, with branch cuts on the imaginary axis beyond -i and i. Within 2 ulps at float32 and float64.
atan2 y x is the angle of the point (x, y), in [-π, π]: -π for y = -0. and x < 0. Within 3 ulps at float32 and 2 at float64. Floats.
sinh x is the hyperbolic sine. Within 3 ulps at float32 and 2 at float64.
cosh x is the hyperbolic cosine. Within 2 ulps at float32 and float64.
tanh x is the hyperbolic tangent. Within 2 ulps at float32 and float64.
asinh x is the inverse hyperbolic sine, odd, x itself near 0.: within 3 ulps at float32 and float64. On complex numbers its imaginary part is in [-π/2, π/2], with branch cuts on the imaginary axis beyond -i and i.
acosh x is the inverse hyperbolic cosine: 0. at 1., NaN below 1., within 3 ulps at float32 and float64. On complex numbers its real part is not negative and its imaginary part is in [-π, π], with a branch cut on the real axis below 1.
atanh x is the inverse hyperbolic tangent: inf at 1., -inf at -1., NaN beyond them, odd, x itself near 0.; within 3 ulps at float32 and float64. On complex numbers its imaginary part is in [-π/2, π/2], with branch cuts on the real axis beyond -1 and 1.
erf x is the error function 2/√π ∫₀ˣ e{^-t²} dt: -1. at -inf and 1. at inf. Within 2 ulps at float32 and float64.
Special functions
The laws of the distributions Rng samples. Arguments broadcast, parameters first and the variable last: gammainc a is the distribution function of the gamma distribution of shape a. Outside a function's domain, and at a NaN argument, the value is NaN; at a pole it is the function's limit there.
Each function states a bound at float32 and float64 on its error (Powers, exponentials and trigonometry). A bound of k ulps absolute near zeros also holds where the exact value is below 1. in magnitude and the result within kε of it, ε being the distance from 1. to the next float of the dtype. An inverse's bound adds κ times its forward function's bound, where κ = |p / (x F'(x))| at its result x, for F the forward function, is its condition number. A subnormal result is held to its bound in units of the least subnormal, plus half of one. These bounds hold where a set's kernels compute. Under a compiler such as Rune.jit, the compiler's bounds hold.
The formats narrower than 32 bits follow Powers, exponentials and trigonometry; where a bound is absolute, their result is within that bound at float32 plus half an ulp of their dtype.
erfc x is 1 - erf x, to its own relative precision where erf x rounds to 1.: erfc 10. is 2.088e-45 at float64. It is 2. at -inf and 0. at inf. Within 8 ulps.
erfinv p is the x with erf x = p, on [-1., 1.]: -inf at -1., -0. at -0. and inf at 1.. Within 4 + 8κ ulps, where κ = |p √π e{^x²} / (2x)| is 1 at 0. and grows as p nears ±1..
ndtr x is the standard normal distribution function Φ(x) = erfc (-x/√2) / 2. Its lower tail keeps its relative precision down to the least subnormal, at x about -38.5 at float64 and -14. at float32. It is 0. at -inf and 1. at inf. Within 16 ulps.
log_ndtr x is log (ndtr x), finite where ndtr x underflows while x²/2 is finite: down to x about -1.9e154 at float64 and -2.6e19 at float32, and -inf below. It is -inf at -inf and -0. at inf. Within 32 ulps.
ndtri p is the x with ndtr x = p, on [0., 1.]: -inf at 0. and inf at 1.. A p above 1/2 is read as one minus the upper tail, so the result keeps the precision of p near either end: ndtri 1e-300 is -37.047. Within 4 + 16κ ulps, where κ = |p / (x Φ'(x))|.
lgamma x is log |Γ(x)|, on every float: inf at 0., the negative integers and inf, NaN at -inf. For x > 0 within 16 ulps, absolute near its zeros at 1. and 2.; below 0., within 16 ulps or 16ε (1 + lgamma (1 - x)) absolutely, the precision its reflection log π - log |sin πx| - lgamma (1 - x) keeps.
digamma x is ψ(x) = Γ'(x) / Γ(x), the mean of log X for X a gamma draw of shape x, on every float: -inf at +0., inf at -0. and inf, NaN at the negative integers and -inf. For x > 0 within 16 ulps, absolute near its root at 1.4616; below 0., within 16 ulps or 16ε (1 + |π cot πx|) absolutely.
lbeta a b is log B(a, b) = lgamma a + lgamma b - lgamma (a + b) for a, b ≥ 0, computed without that sum's cancellation: lbeta 1e10 1. is -log 1e10. It is inf where either is 0. and the other finite, -inf where either is inf and the other positive, and NaN where one is 0. and the other inf. Within 256 ulps, or 512ε absolutely where it is below 1. in magnitude.
gammainc a x is the regularised lower incomplete gamma function P(a, x) = (1/Γ(a)) ∫₀ˣ t{^a-1} e{^-t} dt, the distribution function at x of the gamma distribution of shape a and unit rate, for a, x ≥ 0. It is 0. at x = 0. and where a is inf for finite x, 1. at x = inf and, its limit, at a = 0. for x > 0; NaN at a = x = 0. and a = x = inf, where the limits disagree. For 0 < a ≤ 2{^20} within 32 (1 + |log P|) + 4 ulps; above, its error grows with a.
log_gammainc a x is log P(a, x) (gammainc), finite for a, x > 0 where P underflows: log_gammainc 1000. 1. is -5913.13. It is -inf at x = 0. and where a is inf, 0. at x = inf, and -0. at a = 0. for x > 0, where P reaches 1. from below. For 0 < a ≤ 2{^20} within 16 ulps, absolute near zeros.
log_gammaincc a x is log Q(a, x) (gammaincc), finite for a > 0 wherever x is. It is 0. at x = 0., -0. where a is inf for x > 0, where Q reaches 1. from below, and -inf at x = inf and at a = 0.. Within the bound of log_gammainc.
gammaincinv a p is the x with gammainc a x = p, the gamma distribution's quantile, for a ≥ 0 and p in [0., 1.]. A p above 1/2 is read as one minus the upper tail, so the result keeps its precision near either end. It is 0. at p = 0., inf at p = 1. and where a is inf for p > 0, and at a = 0., where the mass is at 0., 0. for p < 1. For 0 < a ≤ 2{^20} within 4 + κ (32 (1 + |log m|) + 4) ulps, where m = min (p, 1 - p) is the smaller tail and κ = |m / (x ∂ₓP(a, x))|.
gammainccinv a q is the x with gammaincc a x = q, as gammaincinv from the upper tail: inf at q = 0., 0. at q = 1., and 0. at a = 0. for q > 0. Within the bound of gammaincinv, with m = min (q, 1 - q).
betainc a b x is the regularised incomplete beta function I_x(a, b) = (1/B(a, b)) ∫₀ˣ t{^a-1} (1 - t){^b-1} dt, the distribution function at x of the beta distribution of shapes a and b, for a, b ≥ 0 and x in [0., 1.]. It is 0. at x = 0. and 1. at x = 1.; its limits, 1. at a = 0. and where b is inf, and 0. at b = 0. and where a is inf; NaN where two of these disagree. For 0 < a, b ≤ 2{^20} within 64 (1 + |log I|) + 4 ulps; above, its error grows with a and b.
betaincinv a b p is the x with betainc a b x = p, the beta distribution's quantile, for a, b ≥ 0 and p in [0., 1.]. A p above 1/2 is read as one minus the upper tail, so the result keeps its precision near either end. It is 0. at p = 0. and 1. at p = 1.; where all the mass is at 0., at a = 0. or where b is inf, 0. for p < 1, and where it is at 1., at b = 0. or where a is inf, 1. for p > 0; NaN where both hold. For 0 < a, b ≤ 2{^20} within 4 + κ (64 (1 + |log m|) + 4) ulps, where m = min (p, 1 - p) is the smaller tail and κ = |m / (x ∂ₓI_x(a, b))| at the exact quantile x. Where that bound at x = 1/2 reaches 1, κ (64 (1 + |log m|) + 4) ε ≥ 1 with κ there, as at shapes far below 1, the tail is flat between masses near 0. and 1., and where p is within 64 (1 + |log m|) + 4 ulps of m of I_{1/2}(a, b) the result may be any x between them.
betainccinv a b q is the x with betaincc a b x = q, as betaincinv from the upper tail: 1. at q = 0. and 0. at q = 1.. Within the bound of betaincinv, with m = min (q, 1 - q), and its flat tail's carve-out with 1 - I_{1/2}(a, b).
log_betainc a b x is log I_x(a, b) (betainc), finite for finite a, b > 0 and x in (0, 1], where betainc underflows included. It is -inf where betainc is 0., 0. at x = 1., and -0. at a = 0. and where b is inf for x in (0, 1), where the limit is reached from below. For 0 < a, b ≤ 2{^20} within 32 ulps, absolute near zeros.
val log_betaincc :
(float, 's, 'd) t ->
(float, 's, 'd) t ->
(float, 's, 'd) t ->
(float, 's, 'd) tlog_betaincc a b x is log (1 - I_x(a, b)) (betaincc), finite for finite a, b > 0 and x in [0, 1). It is -inf where betaincc is 0., 0. at x = 0., and -0. at b = 0. and where a is inf for x in (0, 1). Within the bound of log_betainc.
i0e x is e{^-|x|} I₀(x), I₀ the modified Bessel function of the first kind of order 0: even, 1. at 0. and 0. at ±inf. The von Mises normaliser 2π I₀(k) has the logarithm k + log (2π i0e k), finite where I₀(k) overflows, past k about 714 at float64 and 92 at float32. Within 8 ulps.
i1e x is e{^-|x|} I₁(x), I₁ the modified Bessel function of the first kind of order 1: odd, so -0. at -0. and at -inf, and 0. at inf. i1e k / i0e k is the mean of cos θ under the von Mises distribution of concentration k. Within 8 ulps.
Rounding
Floats and integers; the identity on integers.
round x rounds to the nearest integer, halves away from zero.
Complex numbers
A complex value's parts are floats of half its width: float32 for complex64, float64 for complex128. real, imag, magnitude and angle take the dtype of their result first.
real dt z is z's real parts, stored in dt by the conversion rule (cast): exact where dt is as wide as the parts.
imag dt z is z's imaginary parts, as real.
magnitude dt z is z's modulus, hypot of its parts computed in the wider of dt and the parts' format, then stored in dt.
val complex :
(Stdlib.Complex.t, 'c) dtype ->
re:(float, 'a, 'd) t ->
im:(float, 'a, 'd) t ->
(Stdlib.Complex.t, 'c, 'd) tcomplex dt ~re ~im is the complex value of parts re and im, broadcast together, each stored in dt's part format by the conversion rule: infinities, NaN and signed zeros stay, and one part never affects the other. complex dt ~re:(real f z) ~im:(imag f z) is z for f the parts' format.
Raises Invalid_argument if the shapes do not broadcast.
conjugate x is x with each imaginary part negated, 0. to -0. included; x itself for a dtype that is not complex.
Comparisons and bits
Every dtype has one order: false < true; integers by value, unsigned dtypes unsigned; floats by value, -0. below +0.; complex numbers by real part, then imaginary part. A comparison is true where it holds; it compares floats by value, so -0. equals +0., and it is false where an element is a NaN, or a complex number with a NaN part, but for not_equal, which is true there. Comparisons take every dtype.
greater_equal a b is less_equal b a.
array_equal a b is the 0-d true iff a and b have one shape and equal holds at every index: false where an element is a NaN, and true for two values without elements. Values of two shapes give false and do not broadcast.
bitwise_and a b is the bitwise and of integers, the logical and of booleans. Integers and booleans.
bitwise_or a b is the bitwise or, as bitwise_and.
bitwise_xor a b is the bitwise exclusive or, as bitwise_and.
bitwise_not x is every bit of x flipped, the logical not of booleans. Integers and booleans.
lshift x n is x shifted n bits toward its high end: x 2{^n} modulo 2{^w} for a dtype of w bits, 0 once n reaches w. Integers.
Raises Invalid_argument if n < 0.
rshift x n is x shifted n bits toward its low end: x / 2{^n} rounded toward negative infinity, so a signed x keeps its sign; 0, or -1 for a negative x, once n reaches the width. Integers.
Raises Invalid_argument if n < 0.
logical_and a b is one where a and b are both not zero, zero elsewhere, in their dtype. A NaN is not zero. Every dtype.
logical_or a b is one where a or b is not zero, as logical_and.
logical_xor a b is one where exactly one of a and b is not zero, as logical_and.
logical_not x is one where x is zero, zero elsewhere, as logical_and.
isnan x is true where x is a NaN, or a complex number with a NaN part. Every dtype: false for the integers and booleans.
isinf x is true where x is an infinity, or a complex number with an infinite part. Every dtype: false for the integers, the booleans and the float formats without infinities.
isfinite x is true where x is neither an infinity nor a NaN, and for a complex number where both parts are. Every dtype: true for the integers and booleans.
clamp ~min ~max x is minimum (maximum x min) max with each bound a constant, a bound left out not applied: max where min > max, and a NaN stays a NaN. x itself without either. Every dtype.
Raises Invalid_argument if a bound is an int outside x's dtype's range.
where c x y is x's element where c is true and y's elsewhere.
Raises Invalid_argument if the shapes do not broadcast.
Reductions and scans
A reduction folds x's elements along axes, every axis by default, a negative one counted from the end, and drops them, or keeps each with extent 1 where keepdims. On one device it gives the same bits on every run of the same computation, for the same values, shapes, dtypes, placement and devices (Nx_kernel); on nx.cpu (Sums and products) its order of association depends on the shape alone. Over axes cut across devices, each device reduces the part its window holds, and only partial results move instead of x's elements: one per output and window holding terms, stacked in C order of the windows' first indices and reduced over the stack by the kernels; for argmax and argmin, each device's extreme and its position. Each device reduces its share of the result over the stack piece by piece, each piece of at most 256 KiB of the partials of every device, its own included. So a sum, prod, max or min whose result of r bytes is whole on N devices, each holding partials of r bytes, holds on each device at most r + r / N bytes and 256 (1 + 1 / N) KiB more beside its operand. Integer and boolean reductions, extremes and their positions keep the bits they have on one device. A float or complex sum, prod, mean, var, std, logsumexp or logmeanexp over cut axes has the kernels' association over each window's terms and over the stacked partials. It may differ from the result on one device within the bounds below, and is the same on every run for a given placement and set of devices. Barring overflow and underflow, a float sum of n terms stays within γ(n - 1) Σ|x| of the exact sum and a product within a relative γ(n - 1), as on one device (Error bounds); a complex product within a normwise relative γ(2n - 2). A mean is such a sum divided by n. var and std are within the bound var states. A logsumexp is within ε + ε{_t} of the exact one, ε the largest bound the kernels' library states for the logsumexp of a window's terms and ε{_t} its bound for one of t terms, t the windows holding terms, and logmeanexp is that less log n.
A scan along an axis cut across devices scans each device's window alone. The windows' totals, one row each, move to every device that holds a window of the same rows, and scan once in the windows' order; each window but the first combines each of its results with the totals' scan of the windows before it. Over integers and booleans, and for cummax and cummin, it keeps the bits it has on one device; cumsum and cumprod of floats and complex numbers keep the bounds of a reduction of each result's terms. ewma and logcumsumexp read their axis whole on each device, so a value cut along it raises (Routing).
A float result that a NaN reaches is a NaN; which NaN, its sign and its payload are unspecified. A complex number with a NaN part is a NaN to prod, max, min, their positions and scans; sum and cumsum add the parts apart. The float formats narrower than 32 bits accumulate at float32 and round once.
Each raises Invalid_argument naming itself for an axis outside x's rank or repeated, or a dtype it does not take. The host computes every dtype stated. Where a device's kernels do not compute one, as a GPU's may not compute complex numbers, the function raises naming itself.
sum ~axes x adds the elements from +0: a sum of -0. terms is +0., and a sum of none is +0.. Integers wrap. Every dtype but booleans.
prod ~axes x multiplies the elements from the first, as sum, and a product of none is 1. Floats, integers and complex numbers.
max ~axes x is the greatest element, as maximum orders them: -0. below +0., any on booleans. Every dtype.
Raises Invalid_argument where an axis it reduces is empty and the result has an element, which has then nothing to choose from.
min ~axes x is the least element, as max; all on booleans.
mean ~axes x is the sum divided by the number of elements. Over floats and complex numbers it rounds once: NaN for none. Over integers it is the exact mean truncated toward zero, as div rounds: it lies between the least and greatest element, so it never wraps. Every dtype but booleans.
Raises Invalid_argument over integers where an axis it reduces is empty and the result has an element, since the mean of nothing has no integer value, or where the axes hold more elements than an integer mean computes exactly: 2{^31} or more at 64 bits, 2{^(64 - b)} or more at b bits.
any ~axes x is true where some element is non-zero, as all. Of no element it is false.
argmax ~axis x is the position along axis of the first element greatest as max orders them: the first NaN where there is one, and 1 for [-0.; 0.]. Without axis, the position in C order among all of x's elements, read as reshape to one axis reads them, of rank 0, or every axis of extent 1 where keepdims. Every dtype.
Raises Invalid_argument naming itself for an axis outside x's rank, or where the result has an element with no element to choose from: x is empty without axis, or axis is empty and the result is not.
var ~axes ~ddof x is the variance Σ (x - m)² / (n - ddof) of the n elements, m their mean; ddof defaults to 0, the population variance, and 1 gives the sample variance. NaN of no element, or where an element is NaN or infinite. It is +∞ only where the variance passes the dtype's greatest value.
On one device, the population variance is the kernels' moments, within the bound their library states. Over axes cut across devices, it is the mean, then the deviations d from it and their squares, each summed as sum sums, and (Σ d² - Σ d (Σ d / n)) / n, +0 where that is negative: on nx.cpu's kernels, within nx.cpu's bound for its moments (Logsumexp and Moments) with L + D + L{_t} + D{_t} in place of its L + D, L{_t} and D{_t} nx.cpu's L and D for t terms, t the windows holding terms. A ddof scales it by a relative 2u more, then one rounding to the dtype. A variance the dtype holds as a subnormal, where the squares underflow before the sum, can err by n / (n - ddof) + 1 of the least subnormal more, 0 included.
Its derivative, as rune takes one, is the variance's wherever the variance is finite, and unspecified where it is +∞.
Raises Invalid_argument naming itself for a negative ddof, or one of at least n where n > 0.
std ~axes ~ddof x is the standard deviation σ: the square root of the variance var states, rounded once to the dtype. It is finite wherever σ is at most the dtype's greatest value, even where var is +∞ or rounds to 0, and its error bound is the root of var's, the subnormal clause aside: a σ below the least normal rounds as a subnormal does. As var otherwise.
Its derivative, as rune takes one, is σ's where M² / σ is at most the dtype's greatest value, M the elements' greatest magnitude, and unspecified elsewhere: at near-constant lanes near the top.
logsumexp ~axes x is log Σ exp x, computed as m + log Σ exp (x - m) for the elements' greatest m, so no term overflows: -∞ of no element or of elements all -∞, +∞ where an element is +∞ and none is NaN. On one device, its error is the bound the kernels' library states; over axes cut across devices, the bound of Reductions and scans.
logmeanexp ~axes x is log (Σ exp x / n): logsumexp less log n, taken before the one rounding to the dtype. NaN of no element.
cumsum ~axis x is, at each index, the sum of x's elements along axis up to that index, inclusive; without axis, of its elements in C order up to that one, in x's shape. Every dtype but booleans.
On one device, or along an axis no device cuts, the result at an index is a function of the elements up to it alone: the cumsum of a prefix of the axis is the prefix of the cumsum, bit for bit. Its last element need not equal sum's of the same elements.
Raises Invalid_argument naming itself for an axis outside x's rank or a dtype it does not take.
logcumsumexp ~axis x is, at each index i along axis, log Σ_{k ≤ i} exp x_k; without axis, over its elements in C order up to that one, in x's shape. It carries the elements' running maximum m and the sum of exp (x_k - m), at most i + 1, so it stays finite where log (cumsum (exp x)) overflows once an exp x_k does. Floats.
From the first NaN element on, every result is a NaN. With no NaN, a result is +∞ from a +∞ element on, and -∞ while every element is -∞.
It is the kernels' Logsumexp scan (Scans), within the bound their library states. The result at an index is a function of the elements up to it alone: the logcumsumexp of a prefix of the axis is the prefix of the logcumsumexp, bit for bit. On nx.cpu (Logsumexp and Moments scans), where the elements up to i have a finite maximum, the result before its rounding to the dtype is within (7 D + 6 ln (i + 1)) u + u|r| of the exact r, to first order, with D = max 0 (2⌈log₂ (i + 1)⌉ - 1) and u the unit roundoff of float32 for the floats narrower than it, else of the dtype.
Raises Invalid_argument naming itself for an axis outside x's rank.
ewma ~alpha x is the exponentially weighted moving average of x along axis: y₀ = x₀, then yᵢ = (1 - alpha) yᵢ₋₁ + alpha xᵢ; without axis, of its elements in C order, in x's shape.
The average of a prefix of x is the prefix of its average, bit for bit. Each device reads axis whole. Element i passes through at most max(0, 2⌈log₂(i+1)⌉ - 1) compositions of the recurrence's affine maps y ↦ a y + b, in O(n) work for n elements. A NaN reaches every later element, and an infinity as the recurrence carries it; a finite term whose weight underflows to 0 drops out.
Raises Invalid_argument naming itself unless 0 < alpha <= 1, or for an axis outside x's rank.
Activations
Floats only, each computed in float32 for the formats narrower than 32 bits and rounded once.
sigmoid x is 1 / (1 + exp (-x)), in [0, 1]: +0. and 1. only where the exact value rounds there, a subnormal where it is one. Within 3 ulps at float32 and float64.
softplus x is log (1 + exp x): exp x to the format's precision far below 0., x far above it, never overflowing. Within 3 ulps at float32 and float64.
val softmax :
?axes:int list ->
?scale:float ->
?where:'d bool_t ->
(float, 's, 'd) t ->
(float, 's, 'd) tsoftmax ~axes ~scale ~where x is exp (scale x) / Σ exp (scale x) along axes, [-1] by default; scale defaults to 1.. The exponents are shifted by the greatest scale x of each lane, so none overflows. A lane holding a NaN, or whose greatest scale x is infinite, is NaN.
where, broadcast to x's shape, selects the elements each lane holds, all of them by default: an element outside it takes no part in the lane's greatest element or its sum, whatever its value, NaN included, and is 0., in a lane holding a NaN too. A lane that holds none is 0. throughout.
Raises Invalid_argument naming itself for an axis outside x's rank or repeated, or a where that does not broadcast to x's shape.
rms_norm ~axes ~eps x is x / sqrt (mean x² + eps), the mean along axes, [-1] by default; eps defaults to 1e-6. It computes x · (1 / sqrt (Σ x² / n + eps)) for the n elements of each lane: the squares summed as sum sums them, each later step rounded once. At float32 and float64 these are the bits of mul x (rsqrt (add_s (mean ~axes ~keepdims:true (square x)) eps)).
Barring overflow and underflow, and for k u < 1/4, it is within k u / (1 - 4 k u) of the exact value, relatively, where k = (n + 8) / 2 and u is the unit roundoff of the format it computes in, before a narrow float's rounding: the squares and their sum take n roundings, the mean and eps two more, which the root halves, and the root, the reciprocal and the product one each.
A lane whose sum of squares overflows is x · 0: zeros, NaN at an infinity. With ~eps:0., a lane whose squares sum to 0 is x · ∞: [nan; nan] of [0.; 0.], and [inf] of a float32 [1e-30], whose square underflows. A lane holding a NaN is NaN.
Raises Invalid_argument naming itself for an axis outside x's rank or repeated, or if eps is negative or NaN.
Conversion
cast dt x is x's elements stored in dt; x itself where dt is x's dtype.
A float rounds once to the nearest value of a float format, ties to even. Past the format's largest finite value it is an infinity in float64, float32, float16, bfloat16 and float8_e5m2, and saturates in the formats with none, float8_e4m3fn and float4_e2m1fn. NaN stays NaN, but in float4_e2m1fn, where it is +0.. Into an integer, a float truncates toward zero and saturates to the range, NaN giving 0; an integer is kept modulo the width; a boolean is 0 or 1. Into a boolean, any element is true where it is not zero, NaN included. An integer into a float rounds once from its exact value. Into a complex dtype these rules give the real part, and the imaginary part is 0.; a complex number stores part by part into a complex dtype, as true into a boolean if a part is not zero, and by its real part into any other dtype.
bitcast dt x is x's bits read as elements of dt, NaN payloads included: x itself where dt is x's dtype. At one width the shape is x's. A dt k times narrower appends an axis of k, its elements the pieces of x's element, in the host's byte order; one k times wider reads x's last axis, of k elements, as one element, and drops it.
Raises Invalid_argument where dt is wider and x's last axis does not have k elements, and where dt is narrower and x has the greatest rank already.
Contraction
A contraction multiplies two operands along their paired axes and sums the products over the summed ones. On one device it gives the same bits on every run of the same computation, as a reduction does (Reductions and scans); on nx.cpu its order of association depends on the shapes alone. A NaN result's sign and payload are unspecified.
Over operands cut across devices, each device computes its window of the result from its windows of the operands. Where both operands' paired summed axes are cut alike, the devices that hold one window of the result compute it together: each contracts the products its windows hold, the first windows' from init and the others' from -0, and only these partial sums move, in the accumulator, stacked in C order of the windows' first indices and summed over the stack by the kernels piece by piece, as a reduction's partials are (Reductions and scans), then rounded once. Where the first windows hold every product, as an axis of fewer elements than its cuts can leave, each window of the result has the bits of that window's contraction on one device. An operand whole on every device is read at the other's cut; summed axes cut otherwise raise (Routing). Nx_quant.apply_experts, which multiplies each row by its expert's matrix, computes so too with the experts or the inputs cut across devices: each device sums its own experts' rows, every row over its window of the inputs, or, over a mesh that cuts both, its own experts' rows over its window of the inputs. Integer contractions keep the bits they have on one device. A float or complex one may differ from it within the bound contract states, and is the same on every run for a given placement and set of devices: float32 [1e7; -1e7; 0.1; 0] by ones gives 0. on one nx.cpu device and 0.1 cut two ways along its summed axis.
A product (contract, einsum, matmul, dot, vdot, tensordot, inner, vecdot, Nx_quant.apply and Nx_quant.apply_experts) given ~sum:true keeps the sum its cut summed axes make, where its terms lie, and moves nothing (Placement.partial): each device contracts its windows' products into the accumulator, the first of its group's from init and the others from -0, so that init joins once, and rounds that term once into the result's dtype. The terms are numbered in C order of the windows' first indices. place closes the sum later, after the linear work a model does on it, as a tensor-parallel layer closes once after its residual add (Routing). A product whose summed axes no device cuts is the same with ~sum:true as without. Closed, a sum of t terms is within contract's bound plus γ(t, u) Σ|term|, u the result dtype's unit roundoff: each term rounds once into the dtype, then t - 1 additions do (Higham, Accuracy and Stability of Numerical Algorithms, §4.2 and Lemma 3.3). Where the dtype is the accumulator's, the terms round nothing, the bound is γ(t - 1, u) Σ|term|, and the sum closes to the bits of the product without ~sum:true.
val contract :
?sizes:(string * int) list ->
?init:('v, 's, 'd) t ->
?sum:bool ->
('v, 's) dtype ->
Pattern.t ->
('a, 'b, 'd) t ->
('c, 'e, 'd) t ->
('v, 's, 'd) tcontract dt p a b is round_dt (init + Σ a · b) over p's summed names, for each index of p's result. sizes gives the extents of names inside groups that neither operand's shape gives alone.
Each operand's dtype must be exact into dt (Dtype.exact), so contract dt p a b is contract dt p (cast dt a) (cast dt b). A real operand of a complex result is a complex number of imaginary part +0, whose products follow complex multiplication: an infinity times it gives a NaN imaginary part. A result narrower than an operand is a cast of a wider contraction's.
The sum's accumulator is dt's carrier (Nx_kernel.carrier): float32 for a float narrower than it, int8 and uint8 for int4 and uint4, else dt. Integers wrap. It starts from init, or +0 without it, and associates in an order each kernel library states, never a function of the order a pattern lists its names. Barring overflow and underflow, before rounding it is within γ(2K + 2, u) (|init| + Σ|a||b|) of the exact sum of its K products, wherever its operands lie, where γ(n, v) = n v / (1 - n v) and u is the accumulator's unit roundoff. With ~sum:true (default false) over cut summed names, it is the devices' terms, which close within the bound Contraction adds.
A donated init receives the result in its memory where donate lets the consumer write there and the result's names are the batch names, then a's others, then b's, each in its operand's order, as in "n i, n o -> i o | n"; otherwise the result is fresh.
Raises Invalid_argument if dt is a boolean; an operand's dtype is not exact into dt; p has other than two operands; an operand repeats a name, or a summed name is in one operand alone; an operand's rank is not the number of axes p gives it, or fewer with ...; ... covers different numbers of axes in the two operands; paired extents differ; a group's extents do not multiply to its axis, or it leaves two extents unknown; a unit axis's extent is not 1; a name in sizes is not in p or is there twice, or its size is negative; or init's shape is not the result's.
val einsum :
?sizes:(string * int) list ->
?sum:bool ->
Pattern.t ->
('v, 's, 'd) t list ->
('v, 's, 'd) teinsum ~sizes p [x1; …; xn] is round (Σ x1 ⋯ xn) over p's summed names, for each index of p's result, at the operands' dtype, each xi read where its names take their indices, so that one that repeats a name reads its diagonal. It is a tree of contracts of two operands, each summing in the dtype's carrier, as contract does, and leaving its value there, so that the result rounds once. sizes is as contract's. A sum in a wider dtype is the einsum of the operands cast into it.
The summed names one operand alone holds are summed in it first, in the carrier. One operand that sums nothing gives a copy, as "i j -> j i" gives the transpose.
Of the trees whose values have at most Nx_array.Layout.max_rank axes, the tree is the one whose contractions do the fewest multiply-adds in all, a contraction doing the product of the extents of the names its two parts hold. A part's value holds the names that the result or an operand outside it needs, and each of its elements costs at least one multiply-add, so the work bounds the memory too. Of trees doing as many, each contraction's left part holds the earliest of its operands; of such parts, the larger, then the one holding the earliest operand the other lacks: equal work gives ((x1 x2) x3) x4. The tree is a function of p and the shapes alone, and so, on nx.cpu, is the result's order of association. Choosing it takes time in 3^n, so p has at most 11 operands.
(* For square matrices, a (b v): j·k multiply-adds, then i·j, where (a b) v does i·j·k, then i·k. *) let mmv = Nx.Pattern.v "i j, j k, k -> i | j k" let y = Nx.einsum mmv [ a; b; v ]
Before rounding the result is within γ(N, 2u) Σ |x1| ⋯ |xn| of the exact sum, where N = Σ (K + 1) over the contractions and the sums within one operand, each summing K products or elements per element of its result, and γ and u are as contract states.
Raises Invalid_argument if p has more than 11 operands or other than n; the dtype is a boolean; an operand's rank is not the number of axes p gives it, or fewer with ...; ... covers different numbers of axes in two operands; a name's extents differ; every tree holds a value of more than Nx_array.Layout.max_rank axes; a group's extents do not multiply to its axis, or it leaves two extents unknown; a unit axis's extent is not 1; or a name in sizes is not in p or is there twice, or its size is negative.
matmul a b is the matrix product over the last two axes, leading axes broadcast, at a's dtype and accumulated as contract does. A 1-d a is a row and a 1-d b a column, and their unit axis is dropped from the result: two 1-d operands give their 0-d inner product.
Raises Invalid_argument if an operand is 0-d, the inner extents differ, the leading axes do not broadcast, or the dtype is a boolean.
dot a b is the sum of the products of a's last axis with b's only axis, for a 1-d b, or its second to last otherwise. The result's axes are a's others, then b's: two vectors give their 0-d inner product, two matrices their product. It does not broadcast, and accumulates as matmul.
Raises Invalid_argument if an operand is 0-d, the summed extents differ, or the dtype is a boolean.
vdot a b is the 0-d sum of the products of conjugate a and b, both flattened, accumulated as matmul.
Raises Invalid_argument if a and b have different numbers of elements, or the dtype is a boolean.
cross ~axis a b is the cross product of the 3-vectors of a and b along axis (default -1) of their broadcast shape: component i is a(i+1) b(i+2) - a(i+2) b(i+1), indices modulo 3. A float format of at most 32 bits gives that difference within one ulp, finite wherever it fits the dtype; float64 within 2 ulps where no product overflows or underflows, and IEEE 754's difference of an infinite product. A zero difference has IEEE 754's sign. Integers wrap.
Raises Invalid_argument if axis is not an axis of the broadcast shape, if a or b does not have 3 elements along it, or for booleans.
val conv :
?stride:int array ->
?padding:(int * int) array ->
?dilation:int array ->
?groups:int ->
('v, 's, 'd) t ->
('v, 's, 'd) t ->
('v, 's, 'd) tconv x w slides the kernel w, [o; k…; c / groups], over x, [n; s…; c], channels last, for K spatial axes s… and kernel extents k…: the result, [n; t…; o], is at (b, p…, j) the sum over i and the taps q… of x[b, p · stride + q · dilation - lo…, i] times w[j, q…, i], an element outside x reading zero. The kernel is not flipped. x is padded along each spatial axis by padding's (lo, hi) (default none), a negative one cutting; stride and dilation default to ones; and t = (s + lo + hi - e) / stride + 1, none where the padded axis is shorter than e, the dilated kernel's extent dilation (k - 1) + 1. A kernel extent of 0 sums no term: e is 0 and each element is +0. Under groups (default 1) the channels split into groups runs, output channels g · o / groups to (g + 1) · o / groups - 1 reading input run g alone.
Each output sums its K products, one per tap and channel of its group, from +0, in the dtype's carrier, as contract does, and rounds once: the float formats narrower than 32 bits sum at float32. Integers wrap. Before rounding it is within γ(2K + 2, u) Σ|x||w| of the exact sum, γ and u as contract states. On one device it gives the same bits on every run of the same computation (Reductions and scans), computing as follows:
- With other than one input channel a group,
c' = c / groups, it is onecontractofx's windows withw, batched over the groups: an output's products are numberedq · c' + i,qthe tap's place in C order of the taps' indices andithe channel among thec'its group reads, and summed ascontractsums them.xpadded is a copy ofx, orxitself where nothing pads it, and its windows a view of it. - With one, as a depthwise convolution has, it is one reduction over
x's padded windows, never stored, of the products numberedi · Q + q,Qthe taps, summed assumdoes.
Raises Invalid_argument if x has fewer than three axes; w has another rank; stride, padding or dilation do not have K entries; a stride or a dilation is below 1; groups is below 1 or does not divide o; x's channels are not groups times w's last extent; or the dtype is a boolean.
val conv_transpose :
?stride:int array ->
?padding:(int * int) array ->
?dilation:int array ->
?groups:int ->
('v, 's, 'd) t ->
('v, 's, 'd) t ->
('v, 's, 'd) tconv_transpose x w is conv's transpose with respect to its input: for the z of the least spatial extents whose conv z w, under the same arguments, has x's shape [n; t…; o], it is the [n; s…; c] value with sum (mul (conv z w) x) = sum (mul z (conv_transpose x w)) for every such z: s = (t - 1) stride + e - lo - hi, e as conv states, or 0 where that is negative or t = 0. It is conv at stride one of x with stride - 1 zeros between its elements, by w reversed along its spatial axes and with its channels exchanged, and sums as conv states with o / groups channels a group: one contract over x spread and padded, a copy where zeros spread or pad it, or, with one channel a group, one reduction that never stores it.
Raises Invalid_argument as conv, reading o from x's last axis, or if no input has a conv of x's shape.
Other products
Products of tensors, vectors and chains of matrices, each a contract, a chain of them or an elementwise mul. Each takes every dtype but booleans, for which it raises Invalid_argument.
val tensordot :
?axes:(int list * int list) ->
?sum:bool ->
('v, 's, 'd) t ->
('v, 's, 'd) t ->
('v, 's, 'd) ttensordot ~axes:(la, lb) a b is the sum of the products of a and b over the axes la of a, paired in order with the axes lb of b (default a's last two with b's first two). The result's axes are a's others, then b's, each in order: ~axes:([], []) gives the outer product. It accumulates as matmul.
Raises Invalid_argument if la and lb differ in length, an axis is not one of its operand's or repeats, or paired extents differ.
outer a b is of shape [|numel a; numel b|], its element (i, j) the product of a's ith element and b's jth, in C order.
vecdot ~axis a b is the sum of the products of conjugate a and b along each operand's axis axis (default -1); their other axes broadcast to the result's shape. It accumulates as matmul.
Raises Invalid_argument if axis is not an axis of both, their extents along it differ, or their other axes do not broadcast.
kron a b is the Kronecker product. With the operand of fewer axes given leading axes of extent 1, each axis has a's extent times b's, and the element at i sb + j along each axis, sb b's extent there, is a's element at the is times b's at the js.
Raises Invalid_argument if the axes of extent other than 1 of a and of b number more than Nx_array.Layout.max_rank together.
multi_dot [x1; …; xn] is the matrix product x1 x2 ⋯ xn of n >= 2 operands. The middle ones are matrices; a 1-d x1 is a row and a 1-d xn a column, and the result drops their unit axis. It multiplies in the order of fewest multiply-adds, an m × k by a k × p doing m k p; of orders doing as many, the one that splits the chain earliest, within each part too, so the order is a function of the shapes alone. Each product sums in the dtype's carrier, as contract does, and leaves its value there, so the result rounds once, within the bound einsum states for its tree.
Raises Invalid_argument if n < 2, an operand has another rank, or adjacent inner extents differ.
matrix_power a n is each square matrix of the batch to the power n: the identity for n = 0, and inv a to the power -n for a negative n. It squares repeatedly, ⌊log₂ |n|⌋ squarings and one product per further set bit of |n|, each a matmul rounded to the dtype.
Raises Invalid_argument if a has fewer than two axes or is not square, and, for a negative n, for a dtype inv does not take.
Attention
Attention weighs values by how well their keys match a query: for each query row, a softmax over the row's scores against the keys, times the values. Masks and score rewrites are plain Nx code over the coordinates of one query-key pair.
type ('v, 's, 'd) kv = Keys and values, by key head hk and key tk.
The 0-d coordinates of one query-key pair: the query row, the key column, the row's visible length, its entry of lengths within [0, tk], and the number of query rows.
causal p holds where p.k <= p.q + (p.len - p.tq): the last query row sees every key before the row's length, and each row before it one key fewer.
val attention :
?scale:float ->
?lengths:'d int64_t ->
?mask:('d pair -> 'd bool_t) ->
?score:('d pair -> (float, 'a, 'd) t -> (float, 'a, 'd) t) ->
acc:(float, 'a) dtype ->
kv:('v, 's, 'd) kv ->
('v, 's, 'd) t ->
('v, 's, 'd) t * (float, 'a, 'd) tattention ~acc ~kv q is (o, lse) for q [lead…; hq; tq; dk], hk dividing hq (zero heads divide zero), query head h reading key head h / (hq / hk). o, [lead…; hq; tq; dv], is the softmax over each row's visible keys of scale · q kᵀ (default 1 / √dk), rewritten by score, times v, in q's dtype; lse, [lead…; hq; tq], is the row's log-sum-exp in acc. A pair is visible where mask holds, its key is before its lead index's entry of lengths ([lead…], default tk), and a paged key's table entry is in [0, pages). A row with no visible key, or only -∞ scores, gives o = 0 and lse = -∞.
A hidden pair's score never changes o or lse, and a table entry outside [0, pages) reads no page. A finite value at a hidden key never changes o: its weight is exactly zero. Values past a row's length are never read. nx.cpu never lets a value at a key hidden from a row reach its o, a NaN or an infinity included. Other interpretations may make the row's o NaN for a NaN or an infinity at a hidden key within its length, as Rune.jit and rune's derivatives do. A visible key whose weight underflows to zero still counts: a NaN in its value makes the row's o NaN. A sum of zeros is +0.
mask and score are plain Nx code over 0-d values, applied once to stand-ins and kept as programs that run at each pair. Their results, and the arrays they capture, broadcast to [lead…; hq; tq; tk]; a captured array is read at each pair. On the pair's values they apply elementwise functions and broadcasts; another operation on them, or a read of one, raises naming Nx.attention.
let window p = Nx.(logical_and (causal p) (less p.q (add_s p.k 128L))) let alibi slopes p s = Nx.(Infix.(s - cast float32 (p.q - p.k) * slopes)) (* slopes [hq; 1; 1] *) let o, lse = Nx.attention ~mask:window ~score:(alibi slopes) ~acc:Nx.float32 ~kv q
q's dtype is float16, bfloat16, float32 or float64, and acc is float32 or float64, not narrower than q's dtype. Scores, their maxima and sums, the weights and lse compute in acc. Where v is float16 or bfloat16, the value product may take each weight through one rounding to v's dtype. o rounds once to q's dtype.
The bound below holds at a row with a visible key, for finite q, k and v, barring overflow and underflow. Let u be acc's unit roundoff and u{_q} that of q's dtype. Let η bound the error of each of the row's visible scores, as score rewrites them, and b bound their magnitudes. Each element of o is within u{_q} |o| + (1 + u{_q}) (e{^w} - 1 + γ(tk + 1, 2u) e{^w}) M of the exact one, and lse within η + ε, where w = (1 + u) (2η + ε) + (2b + ln tk) u + 5u/(1 - u). Here M is the element's Σ p |v| over the row's visible keys at their exact weights p, at most its greatest |v|, ε is the bound the kernels' library states for attention's lse of tk keys, and γ is as contract defines it. Where the value product takes the weights rounded to v's dtype, the bound adds g (u{_v} e{^w} M + 2{^-25} Σ |v|), where g = (1 + u{_q}) (1 + γ(tk + 1, 2u)), u{_v} is v's unit roundoff, and the sum, for float16 alone, is over the visible keys whose exact weight is below 2{^-14} e{^w}. Without score, η is ((1 + u)² (1 + γ(dk + 1, 2u)) - 1) a, about (2dk + 4) u a, and b is at most a, where a is the greatest |scale| Σ |q k| of a visible key. The bound holds where a set's kernels compute, as those of Powers, exponentials and trigonometry do. Under a compiler such as Rune.jit, the compiler's bounds compose the same way.
Over keys cut across devices into n parts, a page table's pages for Paged, each device computes (o, lse) over its part in acc as one device does, its mask and score reading the pair's coordinates in the whole; lse is the parts' log-sum-exp, a Logsumexp of n terms, and o the sum over the parts in their order of each one's o times exp (lse_part - lse), each product fused, rounded once to q's dtype. Its bits depend on the placement alone. The bound above then holds with η doubled, ε three times the greatest part's plus the library's bound for a Logsumexp of n terms plus (2b + ln tk + 5) u, and γ(tk + n + 1, 2u) in place of γ(tk + 1, 2u). A paged call's loads are read whole.
It is one operation, Prim.t.Attention, whose mask and score are one program of two outputs over a pair. The set's kernel library computes it (Nx_kernel.S.attention), reading paged keys and values in place; nx.cpu stores no array of [lead…; hq; tq; tk]. An interpretation receives it whole.
Raises Invalid_argument if q's dtype is another, acc is not float32 or float64 or is narrower than q's dtype, q has fewer than three axes, extents disagree, hk does not divide hq, lengths is not of shape [lead…], dk is 0 without scale, or a result or capture of mask or score does not broadcast to [lead…; hq; tq; tk].
Matrices
A value of two axes or more is a batch of matrices over its last two: rows, then columns.
matrix_transpose x is x with its last two axes exchanged; x itself below two axes. A view.
tril ~k x is x where column - row <= k (default 0) and zero above, in each matrix of the batch.
Raises Invalid_argument if x has fewer than two axes.
triu ~k x is x where column - row >= k (default 0) and zero below, as tril.
diagonal ~offset ~axis1 ~axis2 x is the elements of x at index i along axis1 and i + offset along axis2 (default the last two axes, and 0), for each i where both lie in their axes; that axis replaces the two, last, the others kept in order. A view where x's two axes merge without a copy.
Raises Invalid_argument if x has fewer than two axes, or axis1 and axis2 are not two distinct axes of x.
diag ~k v is, for a vector v of n elements, the square of n + |k| whose kth diagonal (default 0, above the main one where positive) holds v and whose other elements are zero; for a matrix, its diagonal diagonal ~offset:k v.
Raises Invalid_argument if v has neither one nor two axes.
Linear algebra
Factorisations, solves and norms of the matrices of a batch, laid out as Matrices says, m rows, n columns and k = min m n. They compute float32, float64, complex64 and complex128: another dtype raises Invalid_argument. vector_norm, matrix_norm, matrix_rank and cond also take the float formats narrower than 32 bits. A dtype's precision is float64's for float64 and complex128, and float32's for the other floats and complex64. aᴴ is the conjugate transpose, the transpose of a real a.
A result has its operand's dtype, except a real result (svd's singular values, svdvals' and eigvalsh's values, eigh's eigenvalues, slogdet's logarithm, lstsq's residuals and singular values, the norms vector_norm and matrix_norm, and cond) and eig's and eigvals' complex results, each of which takes its dtype first, of a real or a complex operand. A norm or condition number is computed at the wider of its operand's precision and its dtype's, every other result at its operand's precision. Each is then rounded once to its dtype by the conversion rule, as real stores: a dtype without NaN or infinities holds what cast gives for the NaN and infinities each function states.
Each matrix of a batch is computed on its own, and data never raises: a matrix whose factorisation fails, as each function states, is NaN in its own results alone. For a finite a, all ~axes:[ -2; -1 ] (isfinite (cholesky a)) tests which matrices of a batch are positive definite.
Each kernel library states its error bounds. A factorisation leaves some choices open: the order of eig's eigenvalues and of eigvals'; qr's factors, unique up to a unit factor per column of q and row of r where r's diagonal has no zero; the vectors of svd, eigh and eig, unique up to a unit factor where their value is simple and up to a unitary basis of its space where it repeats; and the columns Complete factors add.
The type for a square matrix's triangles, each with its diagonal: the elements on and below it, or on and above it.
The type for how much of a matrix's unitary factors a factorisation gives: k columns of the left one and k rows of the right one, or the whole squares, of m and of n.
cholesky a is the factor of the Hermitian positive definite matrix that a's triangle (default Lower) and the real parts of its diagonal name, which alone are read: L, lower triangular with a real and positive diagonal and a = L Lᴴ, or under ~triangle:Upper U = Lᴴ, upper triangular with a = Uᴴ U. The other triangle is +0. A matrix whose factorisation meets a pivot that is not positive, or NaN, is NaN.
Raises Invalid_argument if a has fewer than two axes or is not square, or for its dtype.
qr a is (q, r) with a = q r, q's columns orthonormal and r upper triangular: q is […; m; k] and r […; k; n], or under ~factors:Complete […; m; m] and […; m; n].
Raises Invalid_argument if a has fewer than two axes, or for its dtype.
lu a is (perm, l, u) where row i of matmul l u is row perm.{i} of a, l is […; m; k], lower triangular with ones on its diagonal, and u is […; k; n], upper triangular; perm is […; m].
The factorisation pivots partially: at each column it moves up the row at or below the diagonal whose magnitude (|re| + |im| for a complex) is greatest, the first such row on a tie, so each element of a real l has a magnitude of at most 1. A NaN is never greater, and nothing is greater than a NaN. A singular a factors as any other, its zero pivots on u's diagonal.
Raises Invalid_argument if a has fewer than two axes, or for its dtype.
val svd :
?factors:factors ->
(float, 'r) dtype ->
('v, 's, 'd) t ->
('v, 's, 'd) t * (float, 'r, 'd) t * ('v, 's, 'd) tsvd dt a is (u, s, vh), a the product of u, the diagonal matrix of s and vh, s the […; k] singular values in dt, descending and not negative, a zero one +0; u and vh have orthonormal columns and rows: […; m; k] and […; k; n], or under ~factors:Complete […; m; m] and […; n; n]. A matrix that holds NaN or an infinity, or on which the iteration does not converge, is NaN in all three.
Raises Invalid_argument if a has fewer than two axes, or for its dtype.
svdvals dt a is svd's s of a, computed without its vectors by an iteration that may round otherwise: its elements lie within the kernel library's bounds of s's, not always bit for bit. A matrix that holds NaN or an infinity, or on which the iteration does not converge, is NaN.
Raises Invalid_argument if a has fewer than two axes, or for its dtype.
val eig :
(Stdlib.Complex.t, 'c) dtype ->
('v, 's, 'd) t ->
(Stdlib.Complex.t, 'c, 'd) t * (Stdlib.Complex.t, 'c, 'd) teig dt a is (w, v) in dt, the […; n] eigenvalues of the real or complex a and columns of unit norm v, matmul a v being v with each column scaled by its eigenvalue. NaN as for svd.
Raises Invalid_argument if a has fewer than two axes or is not square, or for its dtype.
val eigh :
?triangle:triangle ->
(float, 'r) dtype ->
('v, 's, 'd) t ->
(float, 'r, 'd) t * ('v, 's, 'd) teigh dt a is (w, v), the […; n] eigenvalues in dt of the Hermitian a, ascending, and orthonormal columns v, matmul a v being v with each column scaled by its eigenvalue. a is the Hermitian matrix its triangle (default Lower) and the real parts of its diagonal name, which alone are read. A matrix whose read part holds NaN or an infinity, or on which the iteration does not converge, is NaN in both.
Raises Invalid_argument if a has fewer than two axes or is not square, or for its dtype.
det a is the determinant of each matrix of a, the product of the diagonal of lu's u negated once per row interchange. A zero determinant is +0, and a matrix with no row has determinant 1. The product is taken in the dtype and overflows or underflows where a partial product does. slogdet holds determinants beyond the dtype's range.
Raises Invalid_argument if a has fewer than two axes or is not square, or for its dtype.
slogdet dt a is (sign, logabs) with det a = sign · exp logabs, from lu's pivots: logabs in dt is the sum of their magnitudes' logarithms, and sign is 1. or -1. for a real a and a complex number of magnitude 1. for a complex one. A singular matrix, one with a zero pivot, has sign +0 and logabs neg_infinity whatever its other pivots.
Raises Invalid_argument as det.
The type for the matrix a solve applies: aᵀ, or aᴴ. For a real a they are one.
val solve_triangular :
?triangle:triangle ->
?transpose:transpose ->
?unit_diag:bool ->
('v, 's, 'd) t ->
('v, 's, 'd) t ->
('v, 's, 'd) tsolve_triangular a b is x with matmul a x = b, or under transpose with matmul aᵀ x = b or matmul aᴴ x = b, reading a's triangle (default Lower) alone, without its diagonal under ~unit_diag:true (default false), whose ones it takes instead. A b with one axis fewer than a is a vector, […; n], and x has its shape; otherwise b is […; n; r], r right-hand sides. a's and b's leading axes broadcast. A zero on a diagonal it reads makes that matrix's x NaN.
Raises Invalid_argument if a has fewer than two axes or is not square, for its dtype, if b has neither a's rank nor one axis fewer, if b's rows are not n, or if the leading axes do not broadcast.
solve a b is x with matmul a x = b, for a square a, from lu's factors and two triangular solves; b is as solve_triangular takes it. A matrix with a pivot of exactly zero makes its x NaN. A matrix singular in exact arithmetic whose pivot rounds to a tiny nonzero value gives large, infinite or NaN elements instead.
Raises Invalid_argument as solve_triangular.
inv a is solve a of the identity: NaN for a matrix solve finds singular.
Raises Invalid_argument if a has fewer than two axes or is not square, or for its dtype.
val lstsq :
?rtol:float ->
(float, 'r) dtype ->
('v, 's, 'd) t ->
('v, 's, 'd) t ->
('v, 's, 'd) t * (float, 'r, 'd) t * 'd int64_t * (float, 'r, 'd) tlstsq dt a b is (x, residuals, rank, s): x the least-squares solution of matmul a x ≈ b of least norm, whatever a's shape and rank, from s, svd's singular values of a in dt. A singular value at or below rtol times its matrix's greatest counts as zero; rtol defaults to max m n times a's machine epsilon. rank is the number of the others, -1 for a matrix on which svd fails, whose x is NaN. residuals is the squared norm of each column of b - matmul a x in dt: b's shape without its rows. b is as solve_triangular takes it, with m rows.
Raises Invalid_argument if a has fewer than two axes, for its dtype, if b has neither a's rank nor one axis fewer, if b's rows are not m, if the leading axes do not broadcast, or if rtol is negative or NaN.
pinv a is the […; n; m] Moore-Penrose pseudoinverse of a, from svd, or from eigh of the Hermitian a its lower triangle names under ~hermitian:true (default false). A singular value, or an eigenvalue's magnitude, at or below rtol times its matrix's greatest counts as zero; rtol defaults to max m n times the dtype's machine epsilon. A matrix on which that factorisation fails is NaN.
Raises Invalid_argument if a has fewer than two axes, is not square under ~hermitian:true, for its dtype, or if rtol is negative or NaN.
val vector_norm :
(float, 'r) dtype ->
?ord:float ->
?axes:int list ->
?keepdims:bool ->
('v, 's, 'd) t ->
(float, 'r, 'd) tvector_norm dt x is, in dt, the norm of order ord (default 2.) of the vector of the elements of x along axes (default all of them), dropping those axes unless keepdims. Of the magnitudes |x|, the moduli of a complex x, it is (Σ |x|^ord)^(1/ord) for a nonzero ord, the count of nonzero elements for 0., and the greatest or least magnitude for infinity or neg_infinity. A negative ord over no element is infinity.
The sums of powers are scaled by the greatest magnitude, or the least for a negative ord, so that a norm the dtype holds never overflows or underflows on the way: the norm of [1e200; 1e200] is 1.414e200. An element whose magnitude is NaN makes its norm NaN.
Raises Invalid_argument if an axis is outside x's rank or repeats, if ord is NaN, or if ord is infinity or neg_infinity and a vector has no element.
val matrix_norm :
(float, 'r) dtype ->
?ord:[ `Fro | `Nuc | `One | `Two | `Inf | `NegOne | `NegTwo | `NegInf ] ->
?keepdims:bool ->
('v, 's, 'd) t ->
(float, 'r, 'd) tmatrix_norm dt x is, in dt, the norm of order ord (default `Fro) of each matrix of x, dropping its last two axes unless keepdims. Of a matrix:
`Froissqrt (Σ |x|²), scaled asvector_norm's, and`Nucthe sum of its singular values.`Twois its greatest singular value and`NegTwoits least.`Oneis the greatest sum of a column's magnitudes and`NegOnethe least;`Infand`NegInfare the same of its rows.
Raises Invalid_argument if x has fewer than two axes, or if ord takes a greatest or least singular value, column or row of a matrix that has none.
matrix_rank a is the rank of each matrix of a, of a's batch shape: the number of its singular values above rtol times the greatest, as lstsq counts them, rtol defaulting to max m n times the dtype's machine epsilon, float32's for a narrower float. A matrix of no element has rank 0. A matrix on which svdvals is NaN, one that holds NaN or an infinity or on which the iteration does not converge, has rank -1: its rank is undefined.
Raises Invalid_argument if a has fewer than two axes, for its dtype, or if rtol is negative or NaN.
cond dt a is, in dt, the condition number of each matrix of a in the norm p (default `Two), dropping its last two axes: under `Two its greatest singular value over its least, under `One and `Inf its matrix_norm of order p times that of its inv. A singular matrix, whose least singular value is zero or whose inv is NaN, has an infinite condition number; one singular only to rounding has one near 1 / ε or above, ε the machine epsilon of the precision it is computed at. A matrix that holds NaN or an infinity, or on which svdvals fails, has NaN.
Raises Invalid_argument if a has fewer than two axes or a matrix of no element, if dt has no infinities, or if p is `One or `Inf and a is not square.
Fourier transforms
Along an axis of n points, the forward transform is y[k] = Σ_j x[j] e^(-2πi jk/n) and the inverse the same sum with e^(+2πi jk/n). A transform along several axes is the transform along each in turn. A transform of no point is the empty sum, zero.
axes defaults to the last axis. s gives the number of points along each of axes, in order: the operand is cut to it, or extended with zeros. A transform of N points in all is scaled by norm: under Backward (default) the inverse divides by N, under Forward the forward one does, and under Ortho each divides by √N.
Each raises Invalid_argument if the operand is 0-d, if axes is empty, if an axis is outside the operand's rank or repeats, if s does not give one length per axis, or if a length is negative.
The type for the scaling of a transform and its inverse.
val fft :
?axes:int list ->
?s:int list ->
?norm:fft_norm ->
(Stdlib.Complex.t, 's, 'd) t ->
(Stdlib.Complex.t, 's, 'd) tfft x is the forward transform of x along axes.
val ifft :
?axes:int list ->
?s:int list ->
?norm:fft_norm ->
(Stdlib.Complex.t, 's, 'd) t ->
(Stdlib.Complex.t, 's, 'd) tifft x is the inverse transform of x along axes: under one norm, ifft (fft x) is x to rounding.
val rfft :
(Stdlib.Complex.t, 'c) dtype ->
?axes:int list ->
?s:int list ->
?norm:fft_norm ->
(float, 's, 'd) t ->
(Stdlib.Complex.t, 'c, 'd) trfft dt x is the forward transform of the real x along axes, the last of them keeping bins 0 to n/2 of its n points, the others being redundant: one bin for n = 0. It computes at the wider of x's precision and dt's, the formats narrower than 32 bits at float32, and rounds the scaled result once to dt.
val irfft :
(float, 'r) dtype ->
?axes:int list ->
?s:int list ->
?norm:fft_norm ->
(Stdlib.Complex.t, 's, 'd) t ->
(float, 'r, 'd) tirfft dt x is the real inverse of rfft along axes, of n points along the last of them from its first n/2 + 1 bins, which are cut or extended with zeros to that many. s gives the points of the result; without it, the other axes keep their extents and n is 2 (m - 1) for m bins. Along the last axis, bin k above n/2 reads as the conjugate of bin n - k, and the imaginary parts of bin 0 and, for an even n, of bin n/2 are not read: it is this linear map for every operand, Hermitian or not. It computes and rounds as rfft.
Raises Invalid_argument also if the last axis has no bin and s does not give its points.
val hfft :
(float, 'r) dtype ->
?axes:int list ->
?s:int list ->
?norm:fft_norm ->
(Stdlib.Complex.t, 's, 'd) t ->
(float, 'r, 'd) thfft dt z is the forward transform over axes of the Hermitian signal h, with h(-j) = conj h(j) for every index j over axes taken modulo their lengths, whose first n/2 + 1 samples along the last of axes are z, read as irfft reads its bins: its spectrum is real. Under one norm it is irfft dt (conjugate z) scaled as a forward transform, so Backward does not scale it. It takes axes and s, computes and rounds as irfft, and raises as it does.
val ihfft :
(Stdlib.Complex.t, 'c) dtype ->
?axes:int list ->
?s:int list ->
?norm:fft_norm ->
(float, 's, 'd) t ->
(Stdlib.Complex.t, 'c, 'd) tihfft dt x is the first n/2 + 1 samples, along the last of axes, of the Hermitian signal whose transform over axes is the real x: conjugate (rfft dt x) scaled as an inverse transform. Under one norm, hfft ~s (ihfft x) is x to rounding, s the lengths of x's axes. It takes axes and s, computes and rounds as rfft, and raises as it does.
The type for the four discrete cosine and sine transforms, I to IV.
val dct :
?type_:trig_type ->
?axes:int list ->
?norm:fft_norm ->
(float, 's, 'd) t ->
(float, 's, 'd) tdct x is the discrete cosine transform of type type_ (default Type_2) of x along each of axes: of n points x[j], type II is y[k] = 2 Σ_j x[j] cos (πk(2j + 1) / 2n), type I y[k] = x[0] + (-1)^k x[n-1] + 2 Σ_(0<j<n-1) x[j] cos (πjk / (n-1)), type III y[k] = x[0] + 2 Σ_(j>0) x[j] cos (πj(2k + 1) / 2n) and type IV y[k] = 2 Σ_j x[j] cos (π(2j + 1)(2k + 1) / 4n). Under Forward it divides by the transform's logical length, 2 (n - 1) for type I and 2n otherwise, and under Ortho it is the orthonormal transform. A copy of x's shape and dtype. It computes the formats narrower than 32 bits at float32 and rounds the result once.
Raises Invalid_argument also if type I is asked of an axis of one element.
val dst :
?type_:trig_type ->
?axes:int list ->
?norm:fft_norm ->
(float, 's, 'd) t ->
(float, 's, 'd) tdst x is the discrete sine transform of type type_ (default Type_2) along each of axes: type II is y[k] = 2 Σ_j x[j] sin (π(k + 1)(2j + 1) / 2n), type I y[k] = 2 Σ_j x[j] sin (π(j + 1)(k + 1) / (n + 1)), type III y[k] = (-1)^k x[n-1] + 2 Σ_(j<n-1) x[j] sin (π(j + 1)(2k + 1) / 2n) and type IV y[k] = 2 Σ_j x[j] sin (π(2j + 1)(2k + 1) / 4n). Its logical length is 2 (n + 1) for type I and 2n otherwise, and norm scales and narrow formats compute as dct's.
val idst :
?type_:trig_type ->
?axes:int list ->
?norm:fft_norm ->
(float, 's, 'd) t ->
(float, 's, 'd) tidst x is the inverse of dst under the same type_ and norm.
fftfreq dt ~d n is the host value of the frequency of each of fft's n bins for samples d apart (default 1.), in cycles per unit of d: i / (d n) for i in [0, (n - 1) / 2], then (i - n) / (d n) for the rest, the negative frequencies. Each is computed at float64 from d n rounded once, then stored in dt: fftfreq float64 4 is 0, 0.25, -0.5 and -0.25.
Raises Invalid_argument if n < 1 or d is zero or not finite.
val fftfreq_at :
'd Placement.t ->
(float, 's) dtype ->
?d:float ->
int ->
(float, 's, 'd) tval rfftfreq_at :
'd Placement.t ->
(float, 's) dtype ->
?d:float ->
int ->
(float, 's, 'd) tfftshift x is x rolled along each of axes (default the last) by n / 2 places toward the end, n its extent there: the zero frequency of a transform moves to the middle, after the negative frequencies. fftshift (fftfreq float64 5) is -0.4, -0.2, 0, 0.2 and 0.4. A copy.
Raises Invalid_argument if x is 0-d, axes is empty, or an axis is outside x's rank or repeats.
hann dt n is the host value of the periodic Hann window of n points, 0.5 - 0.5 cos (2πi / n), computed at float64 and rounded once: the point that would open the next period is dropped, so shifted copies sum to a constant.
Raises Invalid_argument if n is negative.
val hann_at : 'd Placement.t -> (float, 's) dtype -> int -> (float, 's, 'd) tval stft :
(Stdlib.Complex.t, 'c) dtype ->
window:int ->
?step:int ->
?win:(float, 's, 'd) t ->
(float, 's, 'd) t ->
(Stdlib.Complex.t, 'c, 'd) tstft dt ~window x is the short-time transform of x along its last axis: frames of window samples, one every step (default window / 4, at least 1), each multiplied by win and transformed by rfft. […; n] gives […; frames; window / 2 + 1], time first, with frames = (n - window) / step + 1; samples past the last whole frame are dropped. win defaults to hann (dtype x) window, and ones (dtype x) [| window |] frames without a taper.
Raises Invalid_argument if x is 0-d, window or step is below 1, window exceeds the last axis, or win is not [window].
val istft :
(float, 'r) dtype ->
window:int ->
?step:int ->
?win:(float, 'r, 'd) t ->
?length:int ->
(Stdlib.Complex.t, 'c, 'd) t ->
(float, 'r, 'd) tistft dt ~window z is the signal whose stft under the same window, step and win is z: each frame inverted by irfft, multiplied by win, summed back where it was taken, and divided by the sum of win's squares landing there. […; frames; window / 2 + 1] gives […; (frames - 1) step + window], or length samples, cut or extended with zeros. A sample every frame's win holds at zero carries nothing and is 0.: under the default taper, sample 0.
Raises Invalid_argument if z has fewer than two axes, window is below 1, step is outside [1, window], the last axis is not window / 2 + 1, win is not [window], or length < 1.
Order and counted lengths
Sorting, selection, searching and grouping order elements by one order, the sort order:
- floats as
-∞ < … < -0. < +0. < … < +∞ < NaN, every NaN equal to every other whatever its sign and payload; false < true, and integers by value, unsigned ones as unsigned;- complex numbers by real part, then imaginary part, each as a float; one with a NaN part equals every other such and is above every other complex number.
A descending sort orders by its exact reverse: NaN first, +0. before -0.. Keys are elements, or rows of a 2-d value compared column 0 first: lexsort sorts them, searchsorted searches them and unique groups them. Complex numbers have no keys.
sort ~descending ~axis x is x's elements along axis (default -1) in the sort order, or its reverse where descending (default false), and the positions they come from. The sort is stable in both directions: equal elements keep their order, NaNs included. The values are take_along_axis ~axis positions x, bit for bit.
Raises Invalid_argument if axis is not an axis of x.
argsort ~descending ~axis x is snd (sort ~descending ~axis x).
top_k ~k ~axis x is the first k entries along axis (default -1) of sort ~descending:true ~axis x: the greatest first, equal ones lowest position first, a NaN before every number. One sort that keeps k.
Raises Invalid_argument if axis is not an axis of x or k is outside [0, n], n the axis' extent.
order_key kd x is, for each element of x, an unsigned integer of kd whose order is the sort order: two keys compare, read unsigned, as their elements sort, and are equal iff the elements are equal in the sort order. kd is uint8, uint16, uint32 or uint64, at least as wide as x's elements; uint64 keys every dtype but the complex ones.
The key of an element is:
- for a signed integer, its value in the signed integer of
kd's width with the sign bit flipped; - for an unsigned integer, its value, and for a boolean
0or1; - for a float, the bits of its value in the float of
kd's width, all of them flipped where the sign bit is set and only the sign bit elsewhere, so-0.keys just below+0.; every NaN keys to all ones. The float of a width isx's own dtype at its width, bfloat16 for bfloat16 at 16 bits, float8_e4m3fn for float4_e2m1fn at 8 bits, and otherwise float16, float32 or float64.
An exact cast to a dtype no wider than kd keeps the key wherever the two dtypes key through the same float: order_key uint64 (cast int64 x) is order_key uint64 x for every signed integer x. A key of several columns is a row of order keys made with stack ~axis:1; a column that sorts descending is bitwise_not of its key.
Raises Invalid_argument if x is complex, or if kd is not one of the four or is narrower than x's elements.
lexsort keys is the stable permutation that sorts the keys of keys: n elements of a 1-d keys, or n rows of a 2-d one, compared column 0 first, each in the sort order. take ~axis:0 (lexsort keys) keys is in order and equal keys keep their order; a 2-d keys with no column leaves every row in place. It sorts one column at a time, the last first: w sorts for w columns.
Raises Invalid_argument if keys is neither 1-d nor 2-d, or is complex.
searchsorted ~side s v is, for each key of v, the number of keys of s before it (`Left), or before or equal to it (`Right). Keys compare in the sort order, except that -0. equals +0., as less compares numbers.
- A 1-d
sholdsmkeys of one element;vholds keys of one element in any shape, and the result hasv's shape. - A 2-d
sof shape[|m; w|]holdsmrows;vhas shape[|n; w|], and the result shape[|n|].
It is a binary search of ⌈log₂ (m + 1)⌉ rounds, each a gather of one key of s per key of v. Where s is not in order, each result is still in [0, m].
Raises Invalid_argument if s is neither 1-d nor 2-d, is complex, or is 2-d and v is not 2-d with as many columns.
Lengths that are data
A length that is data is read once, on the host, as the function's one read: a placed operand's set synchronises once. A traced value cannot be read, so there the function raises naming itself, unless size fixes the length, in which case nothing is read.
positions c is each index i of the 1-d c repeated c.{i} times, in increasing order. A boolean counts 1 where it holds, so positions c is the indices where c holds; a negative count counts zero, as does a uint64 count past int64's range. Without size, the result's length is the counts' sum, read on the host unless c is empty. With size, the result has size entries and reads nothing: the first positions, then dim 0 c. Along an axis of c's length that pad is out of range, so take reads zero there and scatter writes nothing; along a longer axis it is a real position.
Raises Invalid_argument if c is not 1-d, is a float or a complex, if size is negative, or if, without size, the counts' sum passes max_int.
argwhere t is the coordinates of t's elements other than zero, a row each in C order: of shape [k; ndim t] for k such elements, whose row i is the coordinates in t's shape of the ith entry of positions (flatten (not_equal t (zeros_like t))). A NaN is not zero, -0. is, and a 0-d t other than zero has one row of no coordinate. Without size, k is read on the host. With size, the result has size rows and reads nothing: the first rows, then rows of t's extents, out of range along every axis.
Raises Invalid_argument if size is negative.
nonzero ~size t is the coordinates of t's elements other than zero, one 1-d value per axis: the jth is column j of argwhere ~size t, a view. With size, a pad entry is the axis' extent, outside it.
Raises Invalid_argument if t is 0-d or size is negative.
extract c x is the elements of x where c is not zero, in C order, c and x flattened: a NaN is not zero, -0. is. Without size, its length is read on the host. With size, it has size entries and reads nothing: the first such elements, then fill (default zero).
Raises Invalid_argument if c and x differ in their number of elements, size is negative, or fill is outside x's dtype.
type 'd groups = {- #
ids : 'd int64_t;(*Each key's group,
*)nentries. - #
first : 'd int64_t;(*Each group's first key, ascending.
*) - #
counts : 'd int64_t;(*Each group's number of keys.
*) - #
count : 'd int64_t;(*
*)k, 0-d.
}The type for n keys in k groups of equal keys, numbered in order of first appearance.
unique keys groups the keys of keys (elements of a 1-d keys, rows of a 2-d one) that are equal in the sort order: every NaN is one group, -0. and +0. two. For g = unique keys, g.ids.{0} is 0, take ~axis:0 g.first keys is the distinct keys in order of first appearance, and key i's group starts at g.first.{g.ids.{i}}, at most i.
Without size, first and counts have k entries, k read on the host. With size they have size entries and nothing is read: past k, first holds n and counts 0; below k, the later groups are cut from them, and ids still numbers those groups from size.
It numbers the keys in one call of the kernel Nx_kernel.S.unique, over their order keys (order_key), or their bits for int64 and uint64 keys. Without size it asks for first and counts of min n (max 4096 (n / 16)) entries; where k is more, each key's first row and count are a scatter of the rows and one of ones at the ids.
Raises Invalid_argument if keys is neither 1-d nor 2-d, is complex, or if size is negative.
quantile ~axis qs x is the quantiles qs of x along axis, or of all its elements without axis. The result leads with an axis of one entry per probability, followed by x's axes without axis.
The quantile q of n elements interpolates linearly between the order statistics around h = q (n - 1) in the sort order: with a and b the elements at floor h and floor h + 1, it is a + (h - floor h) (b - a), and a exactly where h is an integer or a equals b. NaN sorts last, so it reaches the upper quantiles. The narrow floats interpolate in float32 and round once. One sort serves every probability.
Raises Invalid_argument if a probability is outside [0, 1] or NaN, if axis is not an axis of x, or if there is no element along it.
histogram [ (e1, x1); …; (ed, xd) ] counts points into the cells of the edges e1 to ed: point i is (x1.{i}, …, xd.{i}), one per element of the coordinates, which share one shape. The result has one axis per dimension, of dim 0 ek - 1 bins; cell (j1, …, jd) counts the points whose coordinate xk is in bin jk of ek for every k, exactly, as count does.
- Bin
jholds the coordinates fromek.{j}included toek.{j+1}excluded; the last bin includes its right edge too. Coordinates compare aslessdoes, so-0.is+0.. - A point outside the edges along a dimension, NaN included, is in no cell.
Edges must ascend; along edges that do not, a point lands in some cell or in none.
Raises Invalid_argument if no dimension is given, if an edge vector is not 1-d of two or more edges, or if the coordinates' shapes differ.
val weighted_histogram :
weights:('w, 'r, 'd) t ->
((float, 's, 'd) t * (float, 's, 'd) t) list ->
('w, 'r, 'd) tweighted_histogram ~weights dims is histogram dims's cells, each holding the sum of its points' weights in weights' dtype. weights has the coordinates' shape, one weight a point. Each cell sums its points' weights in C order into 0, as scatter's Add does, so a narrow float sums in float32 and rounds once. A cell no point lands in is +0.
Raises Invalid_argument as histogram does, if weights has another shape than the coordinates, or if it is boolean.
Windows and segments
A function here reads its operand in groups of elements: classes, windows along its last axes, or rows grouped by positions held in data. Each computes where its operands lie.
The spatial axes are an operand's last K axes, and the axes before them are batch axes, each computed alone. Along a spatial axis of n elements, padded with lo zeros before and hi after, a window of k elements with stride s and dilation d holds the padded axis' elements w s + j d, j < k, for window w. There are (n + lo + hi - d (k - 1) - 1) / s + 1 windows, and none where d (k - 1) + 1 exceeds n + lo + hi. Windows across several spatial axes are every combination of one window per axis, in C order, and so are their taps j. Except for extract_patches, whose result they are, these functions read the windows where they lie and store none.
one_hot ~num_classes i is i's shape with an axis of num_classes appended, holding 1 at the class each element of i names and 0 at the others: an element outside [0, num_classes) gives a row of zeros. Integers only.
Raises Invalid_argument if i is not of an integer dtype or num_classes is negative.
val extract_patches :
kernel_size:int array ->
stride:int array ->
dilation:int array ->
padding:(int * int) array ->
('v, 's, 'd) t ->
('v, 's, 'd) textract_patches ~kernel_size ~stride ~dilation ~padding x is the windows of x over its last K = Array.length kernel_size axes, padded with zeros by padding's (lo, hi) pairs. A value of shape [lead…; spatial…] gives [lead…; P; L], P the product of kernel_size and L the number of windows: its element at (…, p, l) is window l's tap p. A copy.
Raises Invalid_argument if kernel_size is empty; stride, dilation and padding do not have one entry per kernel axis; a size, stride or dilation is below 1 or a padding negative; or x has fewer axes than kernel_size.
val combine_patches :
output_size:int array ->
kernel_size:int array ->
stride:int array ->
dilation:int array ->
padding:(int * int) array ->
('v, 's, 'd) t ->
('v, 's, 'd) tcombine_patches ~output_size ~kernel_size ~stride ~dilation ~padding p is the adjoint of extract_patches from spatial axes output_size: p, of shape [lead…; P; L], is summed into [lead…; output_size…], each of its elements added at the position that extract_patches reads it from, and one that lands in the padding dropped. An element no window covers is +0. With one geometry, the sum of extract_patches x * y is the sum of x * combine_patches y.
The sums add from +0 in an order the kernel library states, the float formats narrower than 32 bits at float32, rounded once. A float sum keeps the bound of a sum of its terms (Reductions and scans). Integers wrap. Every dtype but booleans.
Raises Invalid_argument as extract_patches does on the geometry, if output_size does not have one entry per kernel axis or has a negative one, if p has fewer than two axes or its last two are not P and L, or if p is boolean.
val correlate :
?padding:[ `Full | `Same | `Valid ] ->
('v, 's, 'd) t ->
('v, 's, 'd) t ->
('v, 's, 'd) tcorrelate ~padding x kernel is the cross-correlation of x with kernel over the last K = ndim kernel axes of x, each sum of products taken without flipping kernel. padding defaults to `Valid.
Along each spatial axis, of n elements in x and k in kernel, value i correlates kernel with the elements of x from i - k + 1, those outside x being zero. padding keeps a run of consecutive values. With m = min n k:
`Fullkeeps then + k - 1from value0;`Validkeeps themax n k - m + 1where the shorter ofxandkernellies within the longer, from valuem - 1. An emptyxlies withinkernelatk + 1positions, so it givesk + 1zeros;`Samekeepsmax n k, from value(m - 1) / 2ifk <= nand from valuem / 2ifk > n. Withk <= n, valueicorrelateskernelwith the elements ofxfromi - k / 2toi + (k - 1) / 2: an odd kernel is centred on elementi.
For x = [|1.; 2.; 3.; 4.; 5.; 6.; 7.|] and kernel = [|1.; 10.; 100.; 1000.|], `Same gives [|2100.; 3210.; 4321.; 5432.; 6543.; 7654.; 765.|].
Each value sums one product per element of kernel, from +0, in an order fixed by the shapes, as sum adds. The float formats narrower than 32 bits multiply and sum at float32 and round once. Integers wrap. Every dtype but booleans.
Raises Invalid_argument if x has fewer axes than kernel, kernel has an axis of no element, or the dtype is boolean.
Segments and ranges
A row of x is its value at one position of its first axis. A segment is the rows whose id, held in data, is one number. A range is the rows from a bound lo, included, to a bound hi, excluded, held in data. An id or a bound never raises. An id outside the segments drops its row, and a bound clips into [0, n] for n rows, so a range may reach past either end of x.
reduce_segments and reduce_ranges fold each segment's or range's rows, in row order, by a combine:
Addsums from+0, so a sum of-0.rows is+0.; the float formats narrower than 32 bits sum atfloat32and round once, and integers wrap. Every dtype but booleans.MaxandMinkeep each element's greatest or least across the rows asmaximumorders them, or a NaN where one is, asmaxdoes. Every dtype.Setkeeps the last row.
A segment or range of no row holds the identity: Add's and Set's zero, or Max's least and Min's greatest value in the dtype's order. Those are neg_infinity and infinity, -448. and 448. in float8_e4m3fn and -6. and 6. in float4_e2m1fn, which have no infinity, the least and greatest integers, false and true, and for complex numbers both parts so.
reduce_segments c ~segments ids x is segments rows, row s the rows i of x with ids.{i} = s folded by c. ids holds one id per row, and the id -1 puts a row in no segment. It is scatter ~combine:c ~axis:0 of x's rows into segments rows of the identity, so its sums have scatter's bits. The result lies as x does, with its rows whole on each device. Over ids or x cut along the rows, the devices fold and close as scatter does. ids that lie elsewhere than the devices read them move there, as a scatter's positions do (Routing). With no row or no segment, it reads neither ids nor x and holds the identity where the result lies.
let ids = Nx.create Nx.int64 [| 5 |] [| 0L; 2L; 0L; -1L; 2L |] let x = Nx.create Nx.float64 [| 5 |] [| 1.; 2.; 3.; 4.; 5. |] (* [|4.; 0.; 7.|] and [|3.; neg_infinity; 5.|] *) let sums = Nx.reduce_segments Add ~segments:3 ids x let maxima = Nx.reduce_segments Max ~segments:3 ids x
Counts are Add of ones, a mean is Add divided by the counts, and each segment's first row is Min of the row positions, then a take.
Raises Invalid_argument if segments is negative, x is 0-d, ids is not 1-d with one id per row of x, c is Add and x is boolean, or ids or x lies where it cannot be read (Routing).
val map_segments :
segments:int ->
'd int64_t ->
('d int64_t -> ('v, 's, 'd) t -> ('w, 'r, 'd) t) ->
('v, 's, 'd) t ->
('w, 'r, 'd) tmap_segments ~segments ids f x is f applied to each position alone, the per-position map: at each position p of ids, the row f gives p's row of x with owner ids.{p} where ids.{p} is in [0, segments), and zeros otherwise, whatever x holds at p. f is called once on the rows grouped by id, so what f reads by owner is read once per block, where a per-position map reads it once per position.
Shapes. Positions are the indices of ids, of shape [| s… |]. The first ndim ids axes of x broadcast to [| s… |], and p's row is x at p after that broadcast, of shape [| r… |]. The result has shape [| s…; r'… |]. Ids may repeat. Results agree with the per-position map up to how f rounds rows it computes together.
What f receives. f owners rows is called exactly once. rows is [| g; c; r… |] at x's dtype, g blocks of c rows, and owners is [| g |], every owner in range. Every row of block b is the row of an in-range position whose id is owners.{b}: each such position fills one row, and the block's other rows repeat rows it already holds. A block no position fills takes the owner and row of an in-range position of its group. A position whose id is out of range never reaches f. In a group with no in-range position, every row is zeros. The order of blocks, and of rows in a block, is unspecified. f returns [| g; c; r'… |], with r' independent of g and c, and computes each row of its result from that row and its block's owner alone.
Selection. Results are selected, never multiplied by a mask: what f gives a repeated row never reaches the result, and a position out of range reaches neither f nor the result, whatever its row holds.
Groups and blocks. Positions are grouped per device. g and c depend on shapes, segments and placement, never on the ids' values, so the shapes f sees never depend on the routing. c is 1 when a group holds few positions per segment, as one token over many experts: then each position is its own block and nothing is sorted. With segments zero or no positions, g is 0 and the result is zeros. ids and x may be split over devices along their first axis only, identically or one of them whole on each device; rows, owners and f's result are split along their first axis as the positions are. Where the windows of that axis differ in extent, as five ids over two devices, each index is a group of its own, so ids then holds one position per index: one axis, or further axes of extent 1. Each block is then one row, and f reads each owner's data once per position. An operation in f that refuses a value derived from owners or rows raises naming ids or x, and the place of it that reads the value where the operation needs it (Routing).
let ids = Nx.create Nx.int64 [| 4 |] [| 1L; -1L; 0L; 1L |] let x = Nx.create Nx.float64 [| 4; 1 |] [| 1.; 2.; 3.; 4. |] let scale = Nx.create Nx.float64 [| 2; 1; 1 |] [| 10.; 100. |] (* [|100.; 0.; 30.; 400.|] *) let y = Nx.map_segments ~segments:2 ids (fun owners rows -> Nx.mul rows (Nx.take ~axis:0 owners scale)) x
Raises Invalid_argument if segments is negative, x has fewer axes than ids or its leading axes do not broadcast to ids', ids or x is split along another axis or in another number of windows, ids holds several positions per index in windows of unequal extents, naming the place that puts ids and x whole, or f's result does not have shape [| g; c; … |]. What f raises reaches the caller as raised, and the operations f makes are named by the functions it calls.
val matmul_segments :
?sum:bool ->
'd int64_t ->
('v, 's, 'd) t ->
('v, 's, 'd) t ->
('v, 's, 'd) tmatmul_segments ids x w is each position's row of x times the matrix of w its id names: at each position p of ids, p's row of x times w's matrix ids.{p}, or zeros where ids.{p} is not in [0, e), whatever that row holds. w is [| e; k; n |], each matrix laid out as matmul's b (a packed weight, rows of k values, is Nx_quant.apply_experts' [| e; n; k |]), ids [| s… |] and x [| s'…; k |], s'… broadcasting to s…; the result is [| s…; n |]. It accumulates as matmul does, in one contraction whose rows are grouped by id, and a row's bits are a function of the shapes, the placements, its row and its matrix: they do not depend on the other positions' ids. ids and x's rows may be cut along the positions together, into windows of one extent, each device computing its own; w cut along its matrices gives each device's rows of its own matrices, which close by partials, or stay the devices' terms with ~sum:true (Contraction).
(* ids : [| tokens; 4 |], x : [| tokens; 1; k |], w : [| e; k; n |] *) let y = Nx.matmul_segments ids x w (* [| tokens; 4; n |] *)
Raises Invalid_argument if w is not 3-d, x does not have one axis more than ids or its leading axes do not broadcast to ids', x's last axis is not k, or the dtype is a boolean.
Ranges
moments_ranges, and reduce_ranges but for Set, combine a range of k > 1 rows in row order by a tree of depth at most 1 + ⌈log₂ (k - 1)⌉ that its clipped bounds fix. A range's bits are a function of its clipped bounds and its rows alone, so a large row outside a range never cancels into its sum. They take O((n + m) log n) work and O(n + m) memory for n rows and m ranges.
So that a range's bits do not depend on where x lies, they read x's rows whole on each device, where a reduction of a cut axis would compute by partials (Routing). x cut along its first axis raises Invalid_argument naming the place that puts its rows whole on each device and keeps its other cuts. Set reads the rows where they lie, as take does. With its rows whole, x cut along other axes is folded by each device over its own columns, and the result lies as x does. lo and hi lie whole on x's devices or cut along their axis, which cuts the result's first axis, over devices other than those that cut x's other axes, such as another axis of a mesh. A bound cut otherwise raises Invalid_argument naming x or a bound and the place of it that reads it beside the other (Routing). That place may put x whole, off its cut of the columns, where the bounds join the operations first.
reduce_ranges c ~lo ~hi x is one row per range, row i the rows of x from lo.{i} to hi.{i} folded by c. A range whose clipped lo is at or after its clipped hi holds the identity. The ranges from 0 to i + 1 compute a running fold.
let x = Nx.create Nx.float64 [| 5 |] [| 1.; 2.; 3.; 4.; 5. |] let lo = Nx.create Nx.int64 [| 4 |] [| -2L; 0L; 3L; 4L |] let hi = Nx.create Nx.int64 [| 4 |] [| 1L; 3L; 3L; 9L |] (* [|1.; 6.; 0.; 5.|] and [|1.; 3.; neg_infinity; 5.|] *) let sums = Nx.reduce_ranges Add ~lo ~hi x let maxima = Nx.reduce_ranges Max ~lo ~hi x
Raises Invalid_argument if x is 0-d, lo and hi are not 1-d of one length, c is Add and x is boolean, c is not Set and x is cut along its first axis, or a bound is cut where it cannot be read.
val moments_ranges :
lo:'d int64_t ->
hi:'d int64_t ->
(float, 's, 'd) t ->
(float, 's, 'd) t * (float, 's, 'd) tmoments_ranges ~lo ~hi x is (mean, var), one row of each per range. Row i holds the mean μ and the variance Σ (t - μ)² / k of the k rows t from lo.{i} to hi.{i}, element by element.
A range of no row has NaN for both. A NaN or both infinities in a range make its mean NaN, and infinities of one sign make it that infinity. The variance is NaN where the mean is not finite.
It computes in the carrier (Nx_kernel.carrier) and stores each result as cast does, so a NaN is +0. in float4_e2m1fn and a value past the greatest saturates in float8_e4m3fn and float4_e2m1fn. Let R be the rows' greatest less their least, V their variance, u the carrier's unit roundoff and D the tree's depth bound. For 2 ≤ k ≤ 1/u, to first order in u and barring underflow, the mean in the carrier is within 7 D u R + u|μ| of the exact one and the variance within (2 D + 5) u V + (7 D² - 3 D) u R √V. The rounding into the dtype adds its own. One row is exact, except that a -0. row's mean is +0.. Within that bound, finite rows have a finite mean and a variance that is +∞ only past the dtype's greatest value.
A derivative of a result, as rune takes one, is that of this computation, and over a range of finite rows depends on those rows alone. The variance's derivative is unspecified where the rows' squared deviations sum past the carrier's greatest value.
Raises Invalid_argument if x is 0-d, lo and hi are not 1-d of one length, x is cut along its first axis, or a bound is cut where it cannot be read.
Structures
A structure is an OCaml type whose values hold arrays: a model's parameters, an optimiser's state, a cache. Ptree describes one by the function that walks it, and check and associative_scan take its description.
check s ok data fail raises fail i d if an element of ok is false, i the index of the first in C order and d the structure data with each array replaced by its element at i, a 0-d value whole on each device of that array's placement (scalar_like). Each array of data broadcasts to ok's shape; an empty ok passes. A check with no data passes Ptree.unit and ().
It is a check: on the host it raises at the call, and on another set at the next wait on that set. It computes on ok's set. What fail raises passes on, there, with the backtrace of its raise.
s's walk must rebuild data with each array replaced by a 0-d one, as fail's d is built; a walk that checks its arrays' shapes, as Rng.walk does, may refuse. check tries that rebuild at the call over stand-ins that hold no bytes, so that trying computes nothing on the devices. Each stand-in is 0-d, of its array's dtype, and whole on each device of that array's placement. A walk may read their shapes, dtypes and placements. An operation on a stand-in raises "a value traced by Nx.check reached eager execution", and a read of its elements raises too.
Raises Invalid_argument naming Nx.check at the call if an array of data does not broadcast to ok's shape, or if s's walk raises it over the stand-ins, with the walk's message.
associative_scan s f x is the inclusive scan of f along axis over the structure x: its element i combines x's elements 0 to i by f, each earlier run as f's first operand, an element being the structure of every array's slice at one index of its axis. axis defaults to 0 and counts from each array's last axis when negative.
f must be associative and elementwise: f (f a b) c equals f a (f b c), and an element of f a b depends on a's and b's elements at its own index alone. The combinations follow Brent and Kung's circuit: of the elements x₀, …, xₙ₋₁, the pairs pₖ = f x₂ₖ x₂ₖ₊₁ are scanned so into q, then y₀ = x₀, y₂ₖ₊₁ = qₖ and y₂ₖ = f qₖ₋₁ x₂ₖ. Element i is thus a tree of f whose shape depends on i alone, of depth at most max(0, 2⌈log₂(i+1)⌉ - 1): the scan of a prefix of x is the prefix of the scan of x, bit for bit, where f computes each element from its operands' elements at its index alone, whatever their slices' extents. For a float f these are not the left fold's bits. f is called at most 2⌈log₂ n⌉ times for n elements, each time over slices of every array, and combines fewer than 2n pairs of elements in all; f needs no identity. The circuit also combines elements past i in runs that element i does not read, so a derivative of element i, as rune takes one, can be NaN where f's partial derivatives at such a run are not finite.
Other scans are lifts of f: an exclusive scan is the inclusive one shifted by one along axis, f's identity first; a reversed scan is the scan of x flipped along axis, flipped back; a segmented scan is the scan of pairs (start, v), start true at each segment's first element, by fun (s1, v1) (s2, v2) -> (logical_or s1 s2, where s2 v2 (f v1 v2)).
Raises Invalid_argument naming Nx.associative_scan if axis is not an axis of an array of x, if two arrays of x have different extents along it, or if f returns a structure whose visits, dtypes or shapes differ from its operands'.
Random numbers
Printing
pp ppf x prints x's dtype and shape, on its set off the host, then its elements as nested rows, as float32 [3] [1, 2, 3] or, 0-d, float32 [] 1.5. From two axes on the elements start a line of their own, one entry of each outer axis per line:
float32 [2; 3] [[1, 2, 3], [4, 5, 6]]
A vector starts the line after its header where both do not fit, and a row wider than the margin wraps after its bracket.
Elements print as Dtype.pp_value does: a float in the shortest form that reads back to it in its format, an unsigned integer unsigned. Past 1000 elements an axis longer than 6 shows its first and last 3 around .... A value with elements is read as to_array reads it, so it raises a failed check pending on its set (Waiting and checks).
A traced value prints traced by its interpretation; a dead one the function it was donated to; one on a lost device the loss. None of these raises.
val pp_shape : Stdlib.Format.formatter -> int array -> unitpp_shape ppf s prints s as nx's messages do: [2; 3].
Errors
Every misuse raises Invalid_argument whose message is the function the user called, a colon, the reason and, where they matter, the operands as dtype shape on set, as in Nx.add: shapes [2; 3] and [4] do not broadcast. The function is the one the user called whatever functions it computes through. A function the user gives, as init's f or an interpretation's rule, keeps its own errors: what it raises reaches the caller as raised. A lost device raises Rig.Lost, and a device's memory that runs out Rig.Out_of_memory. Host memory that runs out raises OCaml's Out_of_memory.
Arrays
The crossing to the array layer, for libraries that read or make bytes: formats, C bindings, compilers. Layouts and buffers are visible here and nowhere else in this module. It shares memory, copying nothing: a value and the arrays crossing with it read the same bytes, so a caller who writes them breaks "nothing changes a value". copy gives a value memory of its own.