VRL expression reference

Syntax

VRL programs can be constructed with the following syntax rules.

Comment

A comment serves as program documentation and is identified with #. Each line must be preceeded with a # character. VRL currently does not allow for block comments.

Examples

# comment
# multi-line
# comment

Keywords

Keywords are reserved words that are used for primitive language features, such as if, and cannot be used as variable assignments or other custom directives. The following words are reserved:

  • abort
  • as
  • break
  • continue
  • else
  • false
  • for
  • if
  • impl
  • in
  • let
  • loop
  • null
  • return
  • self
  • std
  • then
  • this
  • true
  • type
  • until
  • use
  • while

Whitespace

Whitespace is any non-empty string as defined by the Unicode White_Space property.

VRL is a “free-form” language, meaning that all forms of whitespace serve only to separate tokens in the grammar, and have no semantic significance.

Literal expressions

As in most other languages, literals in VRL are values written exactly as they are meant to be interpreted. Literals include things like strings, Booleans, and integers.

Array

An array literal is a comma-delimited set of expressions that represents a contiguous growable array type.

Examples

[]
["first", "second", "third"]
["mixed", 1, 1.0, true, false, {"foo": "bar"}]
["first-level", ["second-level", ["third-level"]]
[.field1, .field2, to_int!("2"), variable_1]
[
  "expressions",
  1 + 2,
  2 == 5,
  true || false
]

Boolean

A Boolean literal represents a binary value which can only be either true or false.

Examples

true
false

Float

A float literal is a decimal representation of a 64-bit floating-point type (specifically, the “binary64” type defined in IEEE 754-2008).

A decimal floating-point literal consists of an integer part (decimal digits), a decimal point, a fractional part (decimal digits).

Examples

1_000_000.01
1000000.01
1.001

Characteristics

Limits
Floats in VRL can range from -1.7976931348623157E+308f64 to 1.7976931348623157E+308f64. Floats outside that range are wrapped.
Underscores
Floats can use underscore (_) characters instead of , to make them human readable. For example, 1_000_000.

Integer

An integer literal is a sequence of digits representing a 64-bit signed integer type.

Examples

1_000_000
1000000

Characteristics

Limits
Integers in VRL can range from -9223372036854775807 to 9223372036854775807. Integers outside that range are wrapped.
Underscore
Integers can use underscore (_) characters instead of , to make them human readable. For example, 1_000_000.

Null

A null literal is the absence of a defined value.

Examples

null

Object

An object literal is a growable key/value structure that is syntactically equivalent to a JSON object.

A well-formed JSON document is a valid VRL object.

Examples

{
	"field1": "value1",
	"field2": [ "value2", "value3", "value4" ],
	"field3": { "field4": "value5" }
}
{
	"field1": .some_path,
	"field2": some_variable,
	"field3": { "subfield": "some value" }
}

Characteristics

Ordering
Object fields are ordered alphabetically by the key in ascending order. Therefore, operations like encoding into JSON produce a string with keys that are in ascending alphabetical order.

Regular Expression

A regular expression literal represents a Regular Expression used for string matching and parsing.

Regular expressions are defined by the r sigil and wrapped with single quotes (r'...'). The value between the quotes uses the Rust regex syntax.

Examples

r'^Hello, World!$'
r'(?i)^Hello, World!$'
r'^\d{4}-\d{2}-\d{2}$'
r'(?P<y>\d{4})-(?P<m>\d{2})-(?P<d>\d{2})'

Characteristics

Flags

Regular expressions allow for flags. Flags can be combined, as in r'(?ixm)pattern', r'(?im)pattern', etc.

To learn more about regular expressions in Rust—and by extension in VRL—we strongly recommend the in-browser Rustexp expression editor and tester.

Enum options
OptionDescription
USwap the meaning of x* and x*?
iCase insensitive
mMulti-line mode
sAllow . to match
uUnicode support (enabled by default)
xIgnore whitespace
Named Captures

Regular expressions support named capture groups, allowing extractions to be associated with keys. Named captures should be preceded with a ?P<name> declaration. This regex, for example…

r'(?P<y>\d{4})-(?P<m>\d{2})-(?P<d>\d{2})'

…extracts captures with the y, m, and d keys.

String

A string literal is a UTF-8–encoded string. String literals can be raw or interpreted.

Raw string literals are composed of the uninterpreted (implicitly UTF-8-encoded) characters between single quotes identified with the s sigil and wrapped with single quotes (s'...'); in particular, backslashes have no special meaning and the string may contain newlines.

Interpreted string literals are character sequences between double quotes ("..."). Within the quotes, any character may appear except unescaped newline and unescaped double quote. The text between the quotes forms the result of the literal, with backslash escapes interpreted as defined below. Strings can be templated by enclosing variables in {{..}}. The value of the variables are inserted into the string at that position.

Examples

"Hello, world! 🌎"
"Hello, world! \u1F30E"
"Hello, \
 world!"
"Hello, {{ planet }}!"
s'Hello, world!'
s'{ "foo": "bar" }'

Characteristics

Backslash escapes
Special characters, such as newlines, can be expressed with a backslash escape.
Enum options
OptionDescription
\"Double quote
\'Single quote
\0Null
\\Backslash
\nNewline
\rCarriage return
\tTab
\u{7FFF}24-bit Unicode character code (up to 6 digits)
\{Brace
Concatenation
Strings can be concatenated with the + operator.
Invalid Characters
Invalid UTF-8 sequences are replaced with the � character.
Multiline strings
Long strings can be split over multiple lines by adding a backslash just before the newline. The newline and any whitespace at the start of the ensuing line is not included in the string.
Templates
Strings can be templated by enclosing a variable name with {{..}}. The value of the variable is inserted into the string at this position at runtime. Currently, the variable has to be a string. Only variables are supported, if you want to insert a path from the event you must assign it to a variable first. To insert a {{ into the string it can be escaped with a \ escape: \{{..\}}. We plan to expand this in future to allow paths and format strings to enable non string variables.

Timestamp

A timestamp literal defines a native timestamp expressed in the RFC 3339 format with a nanosecond precision.

Timestamp literals are defined by the t sigil and wrapped with single quotes (t'2021-02-11T10:32:50.553955473Z').

Examples

t'2021-02-11T10:32:50.553955473Z'
t'2021-02-11T10:32:50.553Z'
t'2021-02-11T10:32:50.553-04:00'

Characteristics

Timezones
As defined in RFC 3339 format, timestamp literals support UTC and local offsets.

Dynamic expressions

VRL is an expression-oriented language. A VRL program consists entirely of expressions and every expression returns a value.

Abort

An abort expression causes the VRL program to terminate, aborting any modifications made to the event.

Grammar

abort ~ message?
ArgumentMeaning
messagemessage is an optional debug message that can be used for diagnostic purposes and is included in a remap transform’s dropped event metadata.

Examples

Ignoring invalid events
Vector event
{
  "log": {
    "message": "hello world"
  }
}
VRL program
if contains(string!(.message), "hello") {
	abort
}
.message = "not hello world"
Resulting event
{
  "message": "hello world"
}

Arithmetic

An arithmetic expression performs an operation on two expressions (operands) as defined by the operator.

Although arithmetic is commonly applied to numbers, you can use it with other types as well, such as strings.

Grammar

expression ~ operator ~ expression
ArgumentMeaning
expressionThe expression can be any expression that returns a valid type as defined by the operator.
operatorThe operator defines the operation performed on the left-hand- and right-hand-side operands.

Examples

Sum (int)
VRL program
1 + 1
Resulting event
2
Sum (float)
VRL program
1.0 + 1.0
Resulting event
2
Sum (numeric)
VRL program
1 + 1.0
Resulting event
2
Sum (string)
VRL program
"Hello" + ", " + "World!"
Resulting event
Hello, World!
Difference (int)
VRL program
2 - 1
Resulting event
1
Difference (float)
VRL program
2.0 - 1.0
Resulting event
1
Difference (numeric)
VRL program
2.0 - 1
Resulting event
1
Multiplication (int)
VRL program
2 * 1
Resulting event
2
Multiplication (float)
VRL program
2.0 * 1.0
Resulting event
2
Multiplication (numeric)
VRL program
2.0 * 1
Resulting event
2
Float division (int)
VRL program
2 / 1
Resulting event
2
Float division (float)
VRL program
2.0 / 1.0
Resulting event
2
Float division (numeric)
VRL program
2.0 / 1
Resulting event
2
Remainder
VRL program
mod(3, 2)
Resulting event
1

Assignment

An assignment expression assigns the result of the right-hand-side expression to the left-hand-side target (path or variable).

Grammar

target ~ ("," ~ error)? ~ operator ~ expression
ArgumentMeaning
errorThe error allows for optional assignment to errors when the right-hand-side expression is fallible. This is commonly used when invoking fallible functions.
expression

If the target is a variable, the expression can be any expression.

If the target is a path, the expression can be any expression that returns a supported object value type (i.e. not a regular expression).

operatorThe operator delimits the target and expression and defines assignment conditions.
targetThe target must be a path, with an optional second variable for error handling if the right-hand side is fallible.

Examples

Path assignment
VRL program
.message = "Hello, World!"
Nested path assignment
VRL program
.parent.child = "Hello, World!"
Double assignment
VRL program
.first = .second = "Hello, World!"
Array element assignment
VRL program
.array[1] = "Hello, World!"
Variable assignment
VRL program
my_variable = "Hello, World!"
.my_field = my_variable
Resulting event
Hello, World!
Object merge assignment
VRL program
my_variable = {"message": "Hello, World!"}
my_variable |= {"level": "info"}
Resulting event
{
  "level": "info",
  "message": "Hello, World!"
}
Fallible assignment (success)
VRL program
.parsed, .err = parse_json("{\"Hello\": \"World!\"}")
Fallible assignment (error)
VRL program
.parsed, .err = parse_json("malformed")

Block

A block expression is a sequence of one or more expressions within matching brace brackets.

Blocks can’t be empty. Instead, empty blocks ({}) are treated as blank objects.

Grammar

"{" ~ NEWLINE* ~ expressions ~ NEWLINE* ~ "}"
ArgumentMeaning
expressionsOne or more expressions.

Examples

Simple block
VRL program
{
	message = "{\"Hello\": \"World!\"}"
	parse_json!(message)
}
Resulting event
{
  "Hello": "World!"
}
Assignment block
VRL program
.structured = {
	message = "{\"Hello\": \"World!\"}"
	parse_json!(message)
}

Coalesce

A coalesce expression is composed of multiple expressions (operands) delimited by a coalesce operator, short-circuiting on the first expression that doesn’t violate the operator condition.

Grammar

expression ~ (operator ~ expression)+
ArgumentMeaning
expressionThe expression (operand) can be any expression.
operatorThe operator delimits two or more expressions.

Examples

Error coalescing
VRL program
parse_syslog("not syslog") ?? parse_common_log("not common") ?? "malformed"
Resulting event
malformed

Comparison

A comparison expression compares two expressions (operands) and produces a Boolean as defined by the operator. Please refer to the match function for matching a string against a regex.

Grammar

expression ~ operator ~ expression
ArgumentMeaning
expressionThe expression (operand) can be any expression that returns a valid type as defined by the operator.
operatorThe operator defines the operation performed on the left-hand and right-hand side operations.

Examples

Equal integers
VRL program
1 == 1
Resulting event
true
Equal integer and float
VRL program
1 == 1.0
Resulting event
true
Not equal
VRL program
1 != 2
Resulting event
true
Equal string
VRL program
x = "foo"
x == "foo"
Resulting event
true
Not equal strings
VRL program
"foo" != "bar"
Resulting event
true
Greater than or equal
VRL program
2 >= 2
Resulting event
true
Greater than
VRL program
2 > 1
Resulting event
true
Less than or equal
VRL program
2 <= 2
Resulting event
true
Less than
VRL program
1 < 2
Resulting event
true
Less than timestamps
VRL program
t'2024-04-04T22:22:22.234142+01:00' < t'2024-04-04T22:22:22.234142+04:00'

Function call

A function call expression invokes built-in VRL functions.

Grammar

function ~ abort? ~ "(" ~ arguments? ~ ")" ~ closure?
ArgumentMeaning
abort

abort represents a literal ! that can optionally be used with fallible functions to abort the program when the function fails:

result = f!()

Otherwise, errors must be handled:

result, err = f()

Failure to handle errors from fallible functions results in compile-time errors. See the error reference for more info.

argumentsThe arguments are comma-delimited expressions that can optionally be prefixed with the documented name.
closure

The closure is an optional piece of code resolved by the function call. It is primarily used in functions that iterate over collections. Its syntax is as follows:

for_each([]) -> |index, value| { ... }
functionfunction represents the name of the built-in function.

Examples

Positional function invocation
VRL program
split("hello, world!", ", ")
Resulting event
["hello","world!"]
Named function invocation (ordered)
VRL program
split("hello, world!", pattern: ", ")
Resulting event
["hello","world!"]
Named function invocation (unordered)
VRL program
split(pattern: ", ", value: "hello, world!")
Resulting event
["hello","world!"]
Infallible function invocation
VRL program
split("apples and pears and bananas", " and ")
Resulting event
["apples","pears","bananas"]
Fallible function invocation
Vector event
{
  "log": {
    "message": "apples and pears and bananas"
  }
}
VRL program
# The compiler cannot determine the argument type thus we need to do error handling.
split!(.message, " and ")
Resulting event
["apples","pears","bananas"]

Characteristics

Deprecation

VRL functions can be marked as “deprecated”. When a function is deprecated, a warning will be shown at runtime.

Suggestions on how to update the VRL program can usually be found in the actual warning and the function documentation.

Function Fallibility

VRL functions can be marked as “fallible” or “infallible”. When a function is defined as fallible, it can fail at runtime, requiring the error to be handled before the program can be compiled.

If a function is defined as infallible, it means that given the correct function arguments, the function can never fail at runtime, and thus no error handling is needed.

Note that even if a function is defined as infallible, if any of its arguments can fail at runtime, the function is considered to be fallible, and thus the error case needs to be handled in this case.

The VRL compiler ensures all potential errors in a program are handled, so there’s no need to worry about missing any potential runtime failures.

Purity
VRL functions can be marked as “pure” or “impure”. When a function is pure, it is idempotent and has no side-effects. Otherwise, it is impure.

If

An if expression specifies the conditional execution of two branches according to the value of a Boolean expression. If the Boolean expression evaluates to true, the “if” branch is executed, otherwise the “else” branch is executed (if present).

Grammar

"if" ~ predicate ~ block ~ ("else if" ~ predicate ~ block)* ~ ("else" ~ block)?
ArgumentMeaning
predicateThe predicate must be an expression that resolves to a Boolean. If a Boolean isn’t returned, a compile-time error is raised. The predicate can contain multiple expressions. Multiple expression predicates must be wrapped in parentheses. The expressions need to be separated by either a semicolon (;) or a new line.

Examples

True if expression
VRL program
if true {
	"Hello, World!"
}
Resulting event
Hello, World!
False if expression
VRL program
if false {
	# not evaluated
	null
}
If/else expression
VRL program
if false {
	# not evaluated
	null
} else {
	"Hello, World!"
}
Resulting event
Hello, World!
If/else if/else expression
VRL program
if false {
	# not evaluated
	null
} else if false {
	# not evaluated
	null
} else {
	"Hello, World!"
}
Resulting event
Hello, World!
Multiline expression
VRL program
x = 3
if (x = x + 1; x == 5) {
	# not evaluated
	null
} else if (
	x = x + 1
	x == 5
) {
	"Hello, World!"
}
Resulting event
Hello, World!

Index

An index expression denotes an element of an array. Array indices in VRL start at zero.

Grammar

"[" ~ index ~ "]"
ArgumentMeaning
indexThe index represents the zero-based position of the element.

Examples

Array index expression
Vector event
{
  "log": {
    "array": [
      "first",
      "second"
    ]
  }
}
VRL program
.array[0]
Resulting event
first

Logical

A logical expression compares two expressions (operands), short-circuiting on the last expression evaluated as defined by the operator.

Grammar

expression ~ operator ~ expression
ArgumentMeaning
expressionThe expression (operand) can be any expression that returns a valid type as defined by the operator.
operatorThe operator defines the operation performed on the left-hand- and right-hand-side operations.

Examples

AND
VRL program
true && true
Resulting event
true
OR (boolean)
VRL program
false || "foo"
Resulting event
foo
OR (null)
VRL program
null || "foo"
Resulting event
foo
NOT
VRL program
!false
Resulting event
true

Path

A path expression is a sequence of period-delimited segments that represent the location of a value within an object. A leading “.” means the path points to the event. A leading “%” means the path points to the event metadata.

Grammar

"." ~ path_segments
ArgumentMeaning
"%"The % character represents the root of the event metadata.
"."The . character represents the root of the event. All paths must begin with . or %
path_segmentspath_segments denote a segment of a nested path. Each segment must be delimited by a . character and only contain alpha-numeric characters and _ (a-zA-Z0-9_). Segments that contain characters outside of this range must be quoted.

Examples

Root event path
Vector event
{
  "log": {
    "message": "Hello, World!"
  }
}
VRL program
.
Resulting event
{
  "message": "Hello, World!"
}
Root metadata path
Vector event
{
  "log": {
    "message": "Hello, World!"
  }
}
VRL program
%
Top-level path
Vector event
{
  "log": {
    "message": "Hello, World!"
  }
}
VRL program
.message
Resulting event
Hello, World!
Nested path
Vector event
{
  "log": {
    "parent": {
      "child": "Hello, World!"
    }
  }
}
VRL program
.parent.child
Resulting event
Hello, World!
Array element path (first)
Vector event
{
  "log": {
    "array": [
      "first",
      "second"
    ]
  }
}
VRL program
.array[0]
Resulting event
first
Array element path (second)
Vector event
{
  "log": {
    "array": [
      "first",
      "second"
    ]
  }
}
VRL program
.array[1]
Resulting event
second
Quoted path
Vector event
{
  "log": {
    "parent.key.with.special characters": {
      "child": "Hello, World!"
    }
  }
}
VRL program
."parent.key.with.special characters".child
Resulting event
Hello, World!

Variable

A variable expression names variables. A variable is a sequence of one or more letters and digits. The first character in a variable must be a letter.

Grammar

first ~ (trailing)*
ArgumentMeaning
firstThe first character can only be an alpha-numeric character (a-zA-Z0-9).
trailingThe trailing characters must only contain ASCII alpha-numeric and underscore characters (a-zA-Z0-9_).

Examples

Simple variable
VRL program
my_variable = 1
my_variable == 1
Resulting event
true
Variable with path
VRL program
my_object = { "one": 1 }
my_object.one
Resulting event
1