docs menu

on this page

Module Expr.Str

Text.

Text counts and slices Unicode scalar values. Categorical values are text.

#type pattern

The type for patterns: what text is matched against. A pattern holds no state, and may be shared. Every pattern reads as a regular expression (Regular expressions): literal is its text with quote applied, prefix s and suffix s put \\A and \\z around it, and pieces joins with (?s:.*?).

#val literal : string -> pattern

literal s matches s anywhere in the text.

Raises Invalid_argument if s is empty or not valid UTF-8, as prefix and suffix do.

#val prefix : string -> pattern

prefix s matches s at the start of the text.

#val suffix : string -> pattern

suffix s matches s at the end of the text.

#val pieces : string list -> pattern

pieces ss matches the strings ss in order without overlap: matches (pieces [ "special"; "requests" ]) is SQL's LIKE '%special%requests%'.

Raises Invalid_argument if ss is empty or holds an empty or non-UTF-8 string.

#val length : (string, 's) t -> (int, 's) t

length a is the number of Unicode scalar values of a, as int64.

#val slice : offset:int -> length:int -> (string, 's) t -> (string, 's) t

slice ~offset ~length a is the at most length scalar values of a from offset, counted from the end when offset is negative, as Query.slice counts rows.

Raises Invalid_argument if length < 0.

#val regex : string -> pattern

regex r matches the regular expression r anywhere in the text; see Regular expressions for the language. It compiles here, once.

Raises Invalid_argument, naming the byte, if r is not valid UTF-8 or is not a pattern of the language, among them backreferences, lookaround and character-class subtraction or intersection.

#val quote : string -> string

quote s is a regular expression that matches exactly s. It builds patterns from data: regex ("^" ^ quote p).

#val matches : pattern -> (string, 's) t -> (bool, 's) t

matches p a is true iff p matches a.

#val extract : ?group:int -> pattern -> (string, 's) t -> (string, 's) t

extract ~group p a is group group of the first match of p in a, 0, the whole match, by default. It is null where a has no match or the group did not participate, and "" where the group matched the empty text. extract ~group:1 (regex "id=(\\d+)") a.

Raises Invalid_argument if p has no group group.

#val find_all : pattern -> (string, 's) t -> (string array, 's) t

find_all p a is the matches of p in a, in order, as a list[string], found as split finds them: the empty list where a has no match.

#val split : pattern -> (string, 's) t -> (string array, 's) t

split p a is the pieces of a between the matches of p, found left to right without overlap, as a list[string]: k matches give k + 1 pieces, so a leading, trailing or repeated p gives an empty piece, and no match gives one piece, a itself. split (literal "_") s of "sub-01_ses-02" is ["sub-01"; "ses-02"]. An empty match may split between scalar values (Regular expressions): split (regex "x*") of "ab" is [""; "a"; "b"; ""].

#val replace : pattern -> by:string -> (string, 's) t -> (string, 's) t

replace p ~by a is a with each match of p, found as split finds them, replaced by by: replace (literal "aa") ~by:"b" of "aaa" is "ba", and a row with no match is unchanged.

With a regex p, by is a template: $n and ${n} are group n, $0 the whole match, $name and ${name} the group of that name, and $$ a dollar. A digit run is a group number, so $1_y is group 1 then _y. A group that did not participate writes nothing. With any other p, by is literal text.

Raises Invalid_argument if by is not valid UTF-8, or, with a regex p, if by holds a $ that starts none of the above or names a group p does not have.

#val trim : (string, 's) t -> (string, 's) t

trim a is a without the white space at its start and end. White space is the scalar values that have the White_Space property in Unicode 18.0.0's PropList.txt: U+0009 to U+000D, U+0020, U+0085, U+00A0, U+1680, U+2000 to U+200A, U+2028, U+2029, U+202F, U+205F and U+3000. U+200B and U+FEFF, which have no width, are kept, and text of white space alone trims to the empty text.

#val lower : (string, 's) t -> (string, 's) t

lower a is a in lowercase by Unicode's Default Case Conversion (Unicode 18.0.0, chapter 3, 3.13): each scalar value maps to its full lowercase mapping, from SpecialCasing.txt where it has an unconditional entry and from UnicodeData.txt otherwise, so the text can change byte length (U+0130 lowers to i and U+0307, three bytes for two). U+03A3 lowers to U+03C2, the final form, where a Cased scalar precedes it and no Cased scalar follows it, Case_Ignorable scalars being passed over on both sides as Table 3-17's Final_Sigma states, and to U+03C3 otherwise. No language's tailoring is applied: Turkish, Azeri and Lithuanian text lowers as any other.

#val upper : (string, 's) t -> (string, 's) t

upper a is a in uppercase by the same rules, from the full uppercase mappings: U+00DF becomes SS, and U+FB03 FFI. Uppercase mapping depends on no context, and no language's tailoring is applied.

#val concat : string -> (string, 's) t list -> (string, 's) t

concat sep es is the texts of es joined by the literal sep, as String.concat sep joins strings: concat "_" [ session; unit ] of "s01" and "u3" is "s01_u3". It is null where one of es is null, and coalesce [ e; string "" ] reads a null e as the empty text. concat sep [] is the empty text.

Raises Invalid_argument if sep is not valid UTF-8.

#val parse : 'a Type.t -> (string, 's) t -> ('a, 's) t

parse ty a is the value of type ty that the text a writes, in the forms that talon.csv reads: true and false; decimal integers with an optional sign; decimal or scientific floats, inf, -inf and nan; ISO 8601 dates; and ISO 8601 datetimes, with an offset or Z for an instant and without one for a wall clock. Any other text, and a value that ty does not hold, is a data error. ty is a bool, integer, float, date or datetime type, or a categorical, which takes only the strings of its dictionary.

Regular expressions

A regex is written in the syntax of RE2 and Perl and matches with UTS #18 Level 1 semantics for Unicode 18.0.0. A match is always a whole number of scalar values.

Matching

A match is the leftmost one. At that byte, alternatives are tried left to right and a repetition takes as many iterations as it can, or as few if lazy: regex "a|ab" of "ab" matches "a". Where the body can match the empty text, the first iteration of the loop of x*, x+ or the end of x{n,} may match it and end the loop, and a later iteration of the loop never matches it. Each counted copy of x{n} and x{n,m} is its own iteration and may match it.

A pattern is matched against the row's whole text, so ^, \\A and \\b see the start of the row wherever a search resumes. Successive matches run left to right without overlap. An empty match that abuts the previous match is discarded, and after an empty match the search resumes one scalar value further on.

Matching a row of n bytes takes time at most proportional to m * n, for a pattern of m instructions, and a kernel's matching memory is bounded, whatever the rows' length. x{n} compiles to n copies of x and nested counts multiply, so a pattern's size and compile time follow from what it writes. A pattern built from untrusted text must be bounded by the program that builds it.

Backreferences and lookaround cannot be matched in that time and are refused, as are possessive quantifiers and atomic groups. Level 2, the \\X grapheme escape and character-class subtraction and intersection are outside the language.

The language

Construct                Meaning
any non-metacharacter    that scalar value
.                        any scalar but U+000A; with (?s), any scalar
[abc] [^abc] [a-z]       classes of scalars; a first ] or - is literal
\d \D \w \W \s \S      Annex C: Nd; Alphabetic, Mark, Nd, Pc,
                         Join_Control; White_Space, the set Str.trim
                         trims
\pN \p{Name} \P{Name}   a General_Category, Script or binary property;
\p{Name=Value}          names as UAX #44 loose matching reads them
\xHH \x{H..}            a code point in hex
\n \t \r \f \v \a \e   the character; \ before punctuation is that
                         punctuation
^ $                      the start and end of the row; with (?m), also
                         after and before U+000A
\A \z                   the start and end of the row in every mode
\b \B                   between a \w and a non-\w scalar, or not
xy x|y                   sequence; the first branch wins
x* x+ x? x{n} x{n,} x{n,m}  repetition, greedy; with a following ?, lazy
(x) (?:x)                capture and non-capture
(?<name>x) (?P<name>x)   a named capture
(?flags) (?flags:x)      flags i m s, and negation with -, to the end of
                         the group or within x

\\d is Unicode's Decimal_Number; \\w and \\s are the Standard Recommendation columns of Annex C. (?i) applies simple case folding, closing a class over it before any negation, so (?i)[^a] matches neither a nor A. Line boundaries are U+000A only, \\b follows \\w and does not skip marks, and conformance is at UTS #18 Level 1 except for the rules above and the set operations of RL1.3, which [^\\P{L}\\p{Greek}] and the like write as complements.