VRL Expression Reference

Here you'll find a comprehensive list of all VRL expressions and the syntax required to achieve them. VRL is an expression-oriented language where expressions make up the language.

Syntax

VRL programs can be constructed with the following syntax rules.

Comment

A comment serves as program documentation and is identified with #. Each line must be preceeded with a # character. VRL currently does not allow for block comments.

  • Examples

    # comment
    # multi-line
    # comment

Keywords

Keywords are reserved words that are used for primitive language features, such as if, and cannot be used as variable assignments or other custom directives. The following words are reserved:

  • abort
  • as
  • break
  • continue
  • else
  • false
  • for
  • if
  • impl
  • in
  • let
  • loop
  • null
  • return
  • self
  • std
  • then
  • this
  • true
  • type
  • until
  • use
  • while

    Whitespace

    Whitespace is any non-empty string as defined by the Unicode White_Space property.

    VRL is a "free-form" language, meaning that all forms of whitespace serve only to separate tokens in the grammar, and have no semantic significance.

      Literal Expressions

      As in most other languages, literals in VRL are values written exactly as they are meant to be interpreted. Literals include things like strings, Booleans, and integers.

      Array

      An array literal is a comma-delimited set of expressions that represents a contiguous growable array type.

      • Examples

        []
        ["first", "second", "third"]
        ["mixed", 1, 1.0, true, false, {"foo": "bar"}]
        ["first-level", ["second-level", ["third-level"]]
        [.field1, .field2, to_int!("2"), variable_1]
        [
        "expressions",
        1 + 2,
        2 == 5,
        true || false
        ]

      Boolean

      A Boolean literal represents a binary value which can only be either true or false.

      • Examples

        true
        false

      Float

      A float literal is a decimal representation of a 64-bit floating-point type (specifically, the "binary64" type defined in IEEE 754-2008).

      A decimal floating-point literal consists of an integer part (decimal digits), a decimal point, a fractional part (decimal digits).

      • Characteristics

        • Limits

          Floats in VRL can range from -1.7976931348623157E+308f64 to 1.7976931348623157E+308f64. Floats outside that range are wrapped.

        • Underscores

          Floats can use underscore (_) characters instead of , to make them human readable. For example, 1_000_000.

      • Examples

        1_000_000.01
        1000000.01
        1.001

      Integer

      An integer literal is a sequence of digits representing a 64-bit signed integer type.

      • Characteristics

        • Limits

          Integers in VRL can range from -9223372036854775807 to 9223372036854775807. Integers outside that range are wrapped.

        • Underscore

          Integers can use underscore (_) characters instead of , to make them human readable. For example, 1_000_000.

      • Examples

        1_000_000
        1000000

      Null

      A null literal is the absence of a defined value.

      • Examples

        null

      Object

      An object literal is a growable key/value structure that is syntactically equivalent to a JSON object.

      A well-formed JSON document is a valid VRL object.

      • Characteristics

        • Ordering

          Object fields are ordered alphabetically by the key in ascending order. Therefore, operations like encoding into JSON produce a string with keys that are in ascending alphabetical order.

      • Examples

        {
        "field1": "value1",
        "field2": [ "value2", "value3", "value4" ],
        "field3": { "field4": "value5" }
        }
        {
        "field1": .some_path,
        "field2": some_variable,
        "field3": { "subfield": "some value" }
        }

      Regular Expression

      A regular expression literal represents a Regular Expression used for string matching and parsing.

      Regular expressions are defined by the r sigil and wrapped with single quotes (r'...'). The value between the quotes uses the Rust regex syntax.

      • Characteristics

        • Flags

          Regular expressions allow for flags. Flags can be combined, as in r'(?ixm)pattern', r'(?im)pattern', etc.

          To learn more about regular expressions in Rust—and by extension in VRL—we strongly recommend the in-browser Rustexp expression editor and tester.

        • Named Captures

          Regular expressions support named capture groups, allowing extractions to be associated with keys. Named captures should be preceded with a ?P<name> declaraction. This regex, for example...

          r'(?P<y>\d{4})-(?P<m>\d{2})-(?P<d>\d{2})'

          ...extracts captures with the y, m, and d keys.

      • Examples

        r'^Hello, World!$'
        r'^Hello, World!$'i
        r'^\d{4}-\d{2}-\d{2}$'
        r'(?P<y>\d{4})-(?P<m>\d{2})-(?P<d>\d{2})'

      String

      A string literal is a UTF-8–encoded string. String literals can be raw or interpreted.

      Raw string literals are composed of the uninterpreted (implicitly UTF-8-encoded) characters between single quotes identified with the s sigil and wrapped with single quotes (s'...'); in particular, backslashes have no special meaning and the string may contain newlines.

      Interpreted string literals are character sequences between double quotes ("..."). Within the quotes, any character may appear except newline and unescaped double quote. The text between the quotes forms the result of the literal, with backslash escapes interpreted as defined below.

      • Characteristics

        • Backslash escapes

          Special characters, such as newlines, can be expressed with a backslash escape.

        • Concatenation

          Strings can be concatenated with the + operator.

        • Invalid Characters

          Invalid UTF-8 sequences are replaced with the character.

      • Examples

        "Hello, world! 🌎"
        "Hello, world! \\u1F30E"
        s'Hello, world!'
        s'{ "foo": "bar" }'

      Timestamp

      A timestamp literal defines a native timestamp expressed in the RFC 3339 format with a nanosecond precision.

      Timestamp literals are defined by the t sigil and wrapped with single quotes (t'2021-02-11T10:32:50.553955473Z').

      • Characteristics

        • Timezones

          As defined in RFC 3339 format, timestamp literals support UTC and local offsets.

      • Examples

        t'2021-02-11T10:32:50.553955473Z'
        t'2021-02-11T10:32:50.553Z'
        t'2021-02-11T10:32:50.553-04:00'

      Dynamic Expressions

      VRL is an expression-oriented language. A VRL program consists entirely of expressions, with every expression returning a value.

      Arithmetic

      An arithmetic expression performs an operation on two expressions (operands) as defined by the operator.

      Although arithmetic is commonly applied to numbers, you can use it with other types as well, such as strings.

      • Grammar

        expression ~ operator ~ expression
        • expression

          The expression can be any expression that returns a valid type as defined by the operator.

        • operator

          The operator defines the operation performed on the left-hand- and right-hand-side operands.

          • Enum!

      • Examples

        Sum (int)
        1 + 1
        2
        Try it yourself with the `vector vrl` subcommand.

      Assignment

      An assignment expression assigns the result of the right-hand-side expression to the left-hand-side target (path or variable).

      • Grammar

        target ~ ("," ~ error)? ~ operator ~ expression
        • target

          The target must be a path, with an optional second variable for error handling if the right-hand side is fallible.

        • error

          The error allows for optional assignment to errors when the right-hand-side expression is fallible. This is commonly used when invoking fallible functions.

        • operator

          The operator delimits the target and expression and defines assignment conditions.

          • Enum!

        • expression

          If the target is a variable, the expression can be any expression.

          If the target is a path, the expression can be any expression that returns a supported object value type (i.e. not a regular expression).

      • Examples

        Path assignment
        VRL program
        .message = "Hello, World!"
        Vector event (after)
        {
        "message": "Hello, World!"
        }

      Block

      A block expression is a sequence of one or more expressions within matching brace brackets.

      Blocks can't be empty. Instead, empty blocks ({}) are treated as blank objects.

      • Grammar

        "{" ~ NEWLINE* ~ expressions ~ NEWLINE* ~ "}"
        • expressions

          One or more expresions.

      • Examples

        Simple block
        {
        message = "{\"Hello\": \"World!\"}"
        parse_json!(message)
        }
        {
        "Hello": "World!"
        }
        Try it yourself with the `vector vrl` subcommand.

      Coalesce

      A coalesce expression is composed of multiple expressions (operands) delimited by a coalesce operator, short-circuiting on the first expression that doesn't violate the operator condition.

      • Grammar

        expression ~ (operator ~ expression)+
        • expression

          The expression (operand) can be any expression.

        • operator

          The operator delimits two or more expressions.

          • Enum!

      • Examples

        parse_syslog("not syslog") ?? parse_common_log("not common") ?? "malformed"
        "malformed"
        Try it yourself with the `vector vrl` subcommand.

      Comparison

      A comparison expression compares two expressions (operands) and produces a Boolean as defined by the operator.

      • Grammar

        expression ~ operator ~ expression
        • expression

          The expression (operand) can be any expression that returns a valid type as defined by the operator.

        • operator

          The operator defines the operation performed on the left-hand and right-hand side operations.

          • Enum!

      • Examples

        Equal
        1 == 1
        true
        Try it yourself with the `vector vrl` subcommand.

      Function call

      A function call expression invokes built-in VRL functions.

      • Grammar

        function ~ abort? ~ "(" ~ arguments? ~ ")"
        • function

          function represents the name of the built-in function.

        • abort

          abort represents a literal ! that can optionally be used with fallible functions to abort the program when the function fails:

          result = f!()

          Otherwise, errors must be handled:

          result, err = f()

          Failure to handle errors from fallible functions results in compile-time errors. See the error reference for more info.

        • arguments

          The arguments are comma-delimited expressions that can optionally be prefixed with the documented name.

          • Named arguments

            All function arguments in VRL are assigned names, including required leading arguments. Named arguments are suffixed with a colon (:), with the value proceeding the name:

            argument_name: "value"
            argument_name: (1 + 2)

            The value is treated as another expression.

          • Positional arguments

            Function calls support nameless positional arguments. Arguments must be supplied in the order they are documented:

            f(1, 2)
          • Argument type safety

            Function arguments enforce type safety when the type of the value supplied is known:

            round("not a number") # fails at compile time

            If the type of the value is not known, you need to handle the potential argument error:

            number = int(.message) ?? 0
            round(number)

            See the errors reference for more info.

      • Examples

        Positional function invocation
        split("hello, world!", ", ")
        [
        "hello",
        "world!"
        ]
        Try it yourself with the `vector vrl` subcommand.

      If

      An if expression specifies the conditional execution of two branches according to the value of a Boolean expression. If the Boolean expression evaluates to true, the "if" branch is executed, otherwise the "else" branch is executed (if present).

      • Grammar

        "if" ~ predicate ~ block ~ ("else if" ~ predicate ~ block)* ~ ("else" ~ block)?
        • predicate

          The predicate must be an expression that resolves to a Boolean. If a Boolean isn't returned, a compile-time error is raised.

      • Examples

        True if expression
        if true {
        "Hello, World!"
        }
        "Hello, World!"
        Try it yourself with the `vector vrl` subcommand.

      Index

      An index expression denotes an element of an array. Array indices in VRL start at zero.

      • Grammar

        "[" ~ index ~ "]"
        • index

          The index represents the zero-based position of the element.

          • Zero-based indices

            Indexes are zero-based where 0 represents the first array element.

      • Examples

        Vector event
        {
        "array": [
        "first",
        "second"
        ]
        }
        VRL program
        .array[0]
        "first"
        Try it yourself with the `vector vrl` subcommand.

      Logical

      A logical expression compares two expressions (operands), short-circuiting on the last expression evaluated as defined by the operator.

      • Grammar

        expression ~ operator ~ expression
        • expression

          The expression (operand) can be any expression that returns a valid type as defined by the operator.

        • operator

          The operator defines the operation performed on the left-hand- and right-hand-side operations.

          • Enum!

      • Examples

        AND
        true && true
        true
        Try it yourself with the `vector vrl` subcommand.

      Path

      A path expression is a sequence of period-delimited segments that represent the location of a value within an object.

      • Grammar

        "." ~ path_segments
        • "."

          The "." character represents the root of the event. Therefore, all paths must begin with the . character, and . alone is a valid path.

        • path_segments

          path_segments denote a segment of a nested path. Each segment must be delimited by a . character and only contain alpha-numeric characters and _ (a-zA-Z0-9_). Segments that contain characters outside of this range must be quoted.

          • Array element paths

            Array elements can be accessed by their index:

            .array[0]
          • Path segment coalecing

            Path segments can be coalesced, allowing for the first non-null value to be used. This is particularly useful when working with externally tagged data:

            .grand_parent.(parent1 | parent2).child
          • Dynamic paths

            Dynamic paths are currently not supported.

          • Nested object paths

            Nested object values are accessed by delimiting each ancestor path with .:

            .parent.child
          • Non-existent paths

            Non-existent paths resolve to null.

          • Path quoting

            Path segments can be quoted to include special characters, such as spaces, periods, and others:

            ."parent.key.with.special \"characters\"".child
          • Valid path characters

            Path segments only allow for underscores and ASCII alpha-numeric characters ([a-zA-Z0-9_]). Segments must be delimited with periods (.). If a segment contains characters outside of this list it must be quoted.

      • Examples

        Root path
        Vector event
        {
        "message": "Hello, World!"
        }
        VRL program
        .
        {
        "message": "Hello, World!"
        }
        Try it yourself with the `vector vrl` subcommand.

      Variable

      A variable expression names variables. A variable is a sequence of one or more letters and digits. The first character in a variable must be a letter.

      • Grammar

        first ~ (trailing)*
        • first

          The first character can only be an alpha-numeric character (a-zA-Z0-9).

        • trailing

          The trailing characters must only contain ASCII alpha-numeric and underscore characters (a-zA-Z0-9_).

      • Examples

        Simple variable
        my_variable = 1
        1
        Try it yourself with the `vector vrl` subcommand.