VRL Function Reference
Here you'll find a comprehensive list of all built-in VRL functions. Functions are categorized by their purpose and
sorted alphabetically for easy discovery. To use these functions in Vector, please see the
function call expression and the
Vector remap
transform.
Array functions
append
append(value, items)
Appends each item in the items
array to the end of the value
array.
Spec
append(value: <array>, items: <array>) :: <array>- required
value
- The initial array - required
items
- The items to append return
- a<array>
Examples
append([1, 2], [3, 4])[1,2,3,4]Try it yourself with the `vector vrl` subcommand.
push
push(value, item)
Adds the item
to the end of the value
array.
Spec
push(value: <array>, item: <any>) :: <array>- required
value
- The target array. - required
item
- The item to push. return
- a<array>
Returns a new array. The
value
is not modified in place.
Examples
push([1, 2], 3)[1,2,3]Try it yourself with the `vector vrl` subcommand.
Codec functions
decode_base64
decode_base64(value, charset: "standard")
Decodes the value
(a Base64 string) into its original string.
Spec
decode_base64(value: <string>, [charset: <string>]) :: <string>, <error>- required
value
- The Base64 data to decode. - optional
charset
- The character set to use when decoding the data. Must be one of the following values:standard
- Standard Base64 format.url_safe
- Modified Base64 for URL variants.
return
- a<string>
Errors
This function is fallible, error handling is required.
Examples
Decode Base64 data (default)decode_base64!("eW91IGhhdmUgc3VjY2Vzc2Z1bGx5IGRlY29kZWQgbWU=")"you have successfully decoded me"Try it yourself with the `vector vrl` subcommand.
encode_base64
encode_base64(value, padding: true, charset: "standard")
Encodes the value
to Base64.
Spec
encode_base64(value: <string>, [padding: <boolean>], [charset: <string>]) :: <string>- required
value
- The string to encode. - optional
padding
- Whether the Base64 output is padded. - optional
charset
- The character set to use when encoding the data. Must be one of the following values:standard
- Standard Base64 format.url_safe
- Modified Base64 for URL variants.
return
- a<string>
Examples
Encode to Base64 (default)encode_base64("please encode me")"cGxlYXNlIGVuY29kZSBtZQ=="Try it yourself with the `vector vrl` subcommand.
encode_json
encode_json(value)
Encodes the value
to JSON.
Spec
encode_json(value: <any>) :: <string>- required
value
- The value to convert to a JSON string. return
- a<string>
Examples
.payload = encode_json({"hello": "world"})"{\"hello\":\"world\"}"Try it yourself with the `vector vrl` subcommand.
Coerce functions
to_bool
to_bool(value)
Coerces the value
into a boolean.
Spec
to_bool(value: <boolean | integer | float | null | string>) :: <boolean>, <error>- required
value
- The value to convert to a Boolean. return
- a<boolean>
If
value
is"true"
,"t"
,"yes"
, or"y"
,true
is returned.If
value
is"false"
,"f"
,"no"
,"n"
, or"0"
,false
is returned.If
value
is0.0
,false
is returned, otherwisetrue
is returned.If
value
is0
,false
is returned, otherwisetrue
is returned.If
value
isnull
,false
is returned.If
value
is a Boolean, it's returned unchanged.
Errors
This function is fallible, error handling is required.
Examples
Coerce to a Boolean (string)to_bool!("yes")trueTry it yourself with the `vector vrl` subcommand.
to_float
to_float(value)
Coerces the value
into a float.
Spec
to_float(value: <float | integer | boolean | string>) :: <float>, <error>- required
value
- The value to convert to a float. Must be convertible to a float, otherwise an error is raised. return
- a<float>
If
value
is a string, it must be the string representation of an float or else an error is raised.If
value
is a boolean,0.0
is returned forfalse
and1.0
is returned fortrue
.
Errors
This function is fallible, error handling is required.
Examples
to_float!("3.145")3.145Try it yourself with the `vector vrl` subcommand.
to_int
to_int(value)
Coerces the value
into an integer.
Spec
to_int(value: <integer | float | boolean | string | timestamp>) :: <integer>, <error>- required
value
- The value to convert to an integer. return
- a<integer>
If
value
is a string, it must be the string representation of an integer or else an error is raised.If
value
is a boolean,0
is returned forfalse
and1
is returned fortrue
.If
value
is a timestamp, a Unix timestamp (in seconds) is returned.
Errors
This function is fallible, error handling is required.
Examples
Coerce to an int (string)to_int!("2")2Try it yourself with the `vector vrl` subcommand.
to_string
to_string(value)
Coerces the value
into a string.
Spec
to_string(value: <integer | float | boolean | string | timestamp | null>) :: <string>, <error>- required
value
- The value to convert to a string. return
- a<string>
If
value
is an integer or float, returns the string representation.If
value
is a boolean, returns"true"
or"false"
.If
value
is a timestamp, returns an RFC 3339) representation.If
value
is a null, returns""
.
Errors
This function is fallible, error handling is required.
Examples
Coerce to a string (Boolean)to_string(true)"true"Try it yourself with the `vector vrl` subcommand.
to_timestamp
to_timestamp(value)
Coerces the value
into a timestamp.
Spec
to_timestamp(value: <string | integer | timestamp>) :: <timestamp>, <error>- required
value
- The value that is to be converted to a timestamp. If a string, must be a valid representation of atimestamp
, and nodefault
exists, anArgumentError
will be raised. return
- a<timestamp>
If
value
is astring
, the timestamp is parsed in these formats.If
value
is aninteger
, it assumed to be a Unix representation of the timestamp (the number of seconds after January 1st, 1970).
Errors
This function is fallible, error handling is required.
Examples
to_timestamp!("2020-10-21T16:00:00Z")"2020-10-21T16:00:00Z"Try it yourself with the `vector vrl` subcommand.
Convert functions
to_syslog_facility
to_syslog_facility(value)
Converts the value
, a Syslog facility code, into its corresponding
Syslog keyword. i.e. 0 into "kern"
, 1 into "user"
, etc.
Spec
to_syslog_facility(value: <integer>) :: <string>, <error>- required
value
- The facility code. return
- a<string>
Errors
This function is fallible, error handling is required.
Examples
to_syslog_facility!(4)"auth"Try it yourself with the `vector vrl` subcommand.
to_syslog_level
to_syslog_level(value)
Converts the value
, a Syslog severity level, into its corresponding keyword,
i.e. 0 into "emerg"
, 1 into `"alert", etc.
Spec
to_syslog_level(value: <integer>) :: <string>, <error>- required
value
- The severity level. return
- a<string>
Errors
This function is fallible, error handling is required.
Examples
to_syslog_level!(5)"notice"Try it yourself with the `vector vrl` subcommand.
to_syslog_severity
to_syslog_severity(value)
Converts the value
, a Syslog log level keyword, into a Syslog integer
severity level (0
to 7
).
Spec
to_syslog_severity(value: <string>) :: <integer>, <error>- required
value
- The Syslog level keyword to convert. return
- a<integer>
The now-deprecated keywords
panic
,error
, andwarn
are converted to0
,3
, and4
respectively.
Errors
This function is fallible, error handling is required.
Examples
to_syslog_severity!("alert")1Try it yourself with the `vector vrl` subcommand.
to_unix_timestamp
to_unix_timestamp(value, unit: "seconds")
Converts the value
timestamp into a Unix timestamp.
Returns the number of seconds since the Unix epoch by default, but milliseconds or nanoseconds can also be
specified by unit
.
Spec
to_unix_timestamp(value: <timestamp>, [unit: <string>]) :: <integer>- required
value
- The timestamp to convert to Unix. - optional
unit
- The time unit. Must be one of the following values:seconds
- Express Unix time in secondsmilliseconds
- Express Unix time in millisecondsnanoseconds
- Express Unix time in nanoseconds
return
- a<integer>
Examples
Convert to a Unix timestamp (seconds)to_unix_timestamp(t'2021-01-01T00:00:00+00:00')1609459200Try it yourself with the `vector vrl` subcommand.
Debug functions
assert
assert(condition, message)
Asserts the condition
, which must be a Boolean expression. The program is aborted with the message
if the
condition evaluates to false
.
Spec
assert(condition: <boolean>, message: <string>) :: <null>, <error>- required
condition
- The condition to check. - required
message
- The failure message that's reported ifcondition
evalues tofalse
. return
- a<null>
Notices
This function has special behavior you should be aware of.
Errors
This function is fallible, error handling is required.
Examples
Assertion (true)ok, err = assert("foo" == "foo", message: "Foo must be foo!")trueTry it yourself with the `vector vrl` subcommand.
log
log(value, level: "info", rate_limit_secs: 1)
Logs the value
to Vector's stdout at the specified level
.
Spec
log(value: <any>, [level: <string>], [rate_limit_secs: <integer>]) :: <null>- required
value
- The value to log. - optional
level
- The log level. Must be one of the following values:trace
- Log at thetrace
level.debug
- Log at thedebug
level.info
- Log at theinfo
level.warn
- Log at thewarn
level.error
- Log at theerror
level.
- optional
rate_limit_secs
- Specifies that the log message is output no more than once per the given number of seconds. Use a value of0
to turn rate limiting off. return
- a<null>
Examples
Log a messagelog("Hello, World!", level: "info", rate_limit_secs: 60)nullTry it yourself with the `vector vrl` subcommand.
Enumerate functions
compact
compact(value, recursive: true, null: true, string: true, object: true, array: true, nullish: false)
Compacts the value
by removing "empty" values, where emptiness is defined using the
available parameters.
Spec
compact(value: <array | object>, [recursive: <boolean>], [null: <boolean>], [string: <boolean>], [object: <boolean>], [array: <boolean>], [nullish: <boolean>]) :: <array | object>- required
value
- The object or array to compact. - optional
recursive
- Whether the compaction be recursive. - optional
null
- Whether null should be treated as an empty value. - optional
string
- Whether an empty string should be treated as an empty value. - optional
object
- Whether an empty object should be treated as an empty value. - optional
array
- Whether an empty array should be treated as an empty value. - optional
nullish
- Tests whether the value is "nullish" as defined by theis_nullish
function. return
- a<array | object>
The return type matches the
value
type.
Examples
Compact an arraycompact(["foo", "bar", "", null, [], "buzz"], string: true, array: true, null: true)["foo","bar","buzz"]Try it yourself with the `vector vrl` subcommand.
flatten
flatten(value)
Flattens the value
into a single-level representation.
Spec
flatten(value: <array | object>) :: <array | object>- required
value
- The array or object to flatten. return
- a<array | object>
The return type matches the
value
type.
Examples
Flatten arrayflatten([1, [2, 3, 4], [5, [6, 7], 8], 9])[1,2,3,4,5,6,7,8,9]Try it yourself with the `vector vrl` subcommand.
includes
includes(value, item)
Determines whether the value
array includes the specified item
.
Spec
includes(value: <array>, item: <any>) :: <boolean>- required
value
- The array. - required
item
- The item to check. return
- a<boolean>
Examples
includes(["apple", "orange", "banana"], "banana")trueTry it yourself with the `vector vrl` subcommand.
length
length(value)
Returns the length of the value
.
Spec
length(value: <array | object | string>) :: <integer>- required
value
- The array or object return
- a<integer>
Returns the size of the array if
value
is an array.Returns the size of the string if
value
is a string.Returns the number of map keys if
value
is a map (nested keys are ignored)
Examples
Length (object)length({"portland": "Trail Blazers","seattle": "Supersonics"})2Try it yourself with the `vector vrl` subcommand.
Event functions
del
del(path)
Removes the field specified by the path
from the current event object.
Spec
del(path: <string>) :: <any>- required
path
- The path of the field to delete. return
- a<any>
Returns the value of the field being deleted. Returns
null
if the field doesn't exist.
Notices
This function has special behavior you should be aware of.
Examples
Delete a fieldVector event (before){"field1": 1,"field2": 2}VRL programdel(.field1)Vector event (after){"field2": 2}
exists
exists(path)
Checks whether the path
exists for the current event.
Spec
exists(path: <path>) :: <boolean>- required
path
- The path of the field to check. return
- a<boolean>
Examples
Exists (field)Vector event{"field": 1}VRL programexists(.field)trueTry it yourself with the `vector vrl` subcommand.
Hash functions
md5
md5(value)
Calculates an md5 hash of the value
.
Spec
md5(value: <string>) :: <string>- required
value
- The string to calculate the hash for. return
- a<string>
Examples
md5("foo")"acbd18db4cc2f85cedef654fccc4a4d8"Try it yourself with the `vector vrl` subcommand.
sha1
sha1(value)
Calculates a SHA-1 hash of the value
.
Spec
sha1(value: <string>) :: <string>- required
value
- The string to calculate the hash for. return
- a<string>
Examples
sha1("foo")"0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33"Try it yourself with the `vector vrl` subcommand.
sha2
sha2(value, variant: "SHA-512/256")
Calculates a SHA-2 hash of the value
.
Spec
sha2(value: <string>, [variant: <string>]) :: <string>- required
value
- The string to calculate the hash for. - optional
variant
- The variant of the algorithm to use. Must be one of the following values:SHA-224
- SHA-224 algorithmSHA-256
- SHA-256 algorithmSHA-384
- SHA-384 algorithmSHA-512
- SHA-512 algorithmSHA-512/224
- SHA-512/224 algorithmSHA-512/256
- SHA-512/256 algorithm
return
- a<string>
Examples
sha2("foo", variant: "SHA-512/224")"d68f258d37d670cfc1ec1001a0394784233f88f056994f9a7e5e99be"Try it yourself with the `vector vrl` subcommand.
sha3
sha3(value, variant: "SHA3-512")
Calculates a SHA-3 hash of the value
.
Spec
sha3(value: <string>, [variant: <string>]) :: <string>- required
value
- The string to calculate the hash for. - optional
variant
- The variant of the algorithm to use. Must be one of the following values:SHA3-224
- SHA3-224 algorithmSHA3-256
- SHA3-256 algorithmSHA3-384
- SHA3-384 algorithmSHA3-512
- SHA3-512 algorithm
return
- a<string>
Examples
sha3("foo", variant: "SHA3-224")"f4f6779e153c391bbd29c95e72b0708e39d9166c7cea51d1f10ef58a"Try it yourself with the `vector vrl` subcommand.
IP functions
ip_cidr_contains
ip_cidr_contains(cidr, ip)
Determines whether the ip
is contained in the block referenced by the cidr
.
Spec
ip_cidr_contains(cidr: <string>, ip: <string>) :: <boolean>, <error>- required
cidr
- The CIDR mask (v4 or v6). - required
ip
- The IP address (v4 or v6). return
- a<boolean>
Errors
This function is fallible, error handling is required.
Examples
IPv4 contains CIDRip_cidr_contains!("192.168.0.0/16", "192.168.10.32")trueTry it yourself with the `vector vrl` subcommand.
ip_subnet
ip_subnet(ip, subnet)
Extracts the subnet address from the ip
using the supplied subnet
.
Spec
ip_subnet(ip: <string>, subnet: <string>) :: <string>, <error>- required
ip
- The IP address (v4 or v6). - required
subnet
- The subnet to extract from the IP address. This can be either a prefix length like/8
or a net mask like255.255.0.0
. The net mask can be either an IPv4 or IPv6 address. return
- a<string>
Notices
This function has special behavior you should be aware of.
Errors
This function is fallible, error handling is required.
Examples
IPv4 subnetip_subnet!("192.168.10.32", "255.255.255.0")"192.168.10.0"Try it yourself with the `vector vrl` subcommand.
ip_to_ipv6
ip_to_ipv6(ip)
Converts the ip
to an IPv6 address.
Spec
ip_to_ipv6(ip: <string>) :: <string>, <error>Errors
This function is fallible, error handling is required.
Examples
ip_to_ipv6!("192.168.10.32")"::ffff:192.168.10.32"Try it yourself with the `vector vrl` subcommand.
ipv6_to_ipv4
ipv6_to_ipv4(ip)
Converts the ip
to an IPv4 address. ip
is returned unchanged if it's already an IPv4 address. If ip
is
currently an IPv6 address then it needs to be IPv4 compatible, otherwise an error is thrown.
Spec
ipv6_to_ipv4(ip: <string>) :: <string>, <error>- required
ip
- The IPv4-mapped IPv6 address to convert. return
- a<string>
The
ip
is returned unchanged if it's already an IPv4 address. If it's an IPv6 address it must be IPv4 compatible, otherwise an error is thrown.
Errors
This function is fallible, error handling is required.
Examples
ipv6_to_ipv4!("::ffff:192.168.0.1")"192.168.0.1"Try it yourself with the `vector vrl` subcommand.
Number functions
ceil
ceil(value, precision: 0)
Rounds the value
up to the specified precision
.
Spec
ceil(value: <integer | float>, [precision: <integer>]) :: <integer | float>- required
value
- The number to round up. - optional
precision
- The number of decimal places to round to. return
- a<integer | float>
Returns an integer if
precision
is0
(this is the default). Returns a float otherwise.
Examples
Round a number up (without precision)ceil(4.345)5.0Try it yourself with the `vector vrl` subcommand.
floor
floor(value, precision: 0)
Rounds the value
down to the specified precision
.
Spec
floor(value: <integer | float>, [precision: <integer>]) :: <integer | float>- required
value
- The number to round down. - optional
precision
- The number of decimal places to round to. return
- a<integer | float>
Returns an integer if
precision
is0
(this is the default). Returns a float otherwise.
Examples
Round a number down (without precision)floor(4.345)4.0Try it yourself with the `vector vrl` subcommand.
format_number
format_number(value, scale: null, decimal_separator: ".", grouping_separator: ",")
Formats the value
into a string representation of the number.
Spec
format_number(value: <integer | float>, [scale: <integer>], [decimal_separator: <string>], [grouping_separator: <string>]) :: <string>- required
value
- The number to format as a string. - optional
scale
- The number of decimal places to display. - optional
decimal_separator
- The character to use between the whole and decimal parts of the number. - optional
grouping_separator
- The character to use between each thousands part of the number. return
- a<string>
Examples
format_number(1234567.89, 3, decimal_separator: ".", grouping_separator: ",")"1,234,567.890"Try it yourself with the `vector vrl` subcommand.
round
round(value, precision: 0)
Rounds the value
to the specified precision
.
Spec
round(value: <integer | float>, [precision: <integer>]) :: <integer | float>- required
value
- The number to round. - optional
precision
- The number of decimal places to round to. return
- a<integer | float>
If
precision
is0
, then an integer is returned, otherwise a float is returned.
Examples
Round a number (without precision)round(4.345)4.0Try it yourself with the `vector vrl` subcommand.
Object functions
merge
merge(to, from, deep: false)
Merges the from
object into the to
object.
Spec
merge(to: <string>, from: <object>, [deep: <boolean>]) :: <object>- required
to
- The object to merge into. - required
from
- The object to merge from. - optional
deep
- A deep merge is performed iftrue
, otherwise only top-level fields are merged. return
- a<object>
The field from the
from
object is chosen if a key exists in both objects.Objects are merged recursively if
deep
is specified, a key exists in both objects, and both of those fields are also objects.
Examples
Object merge (shallow)merge({"parent1": {"child1": 1,"child2": 2},"parent2": {"child3": 3}},{"parent1": {"child2": 4,"child5": 5}}){"parent1": {"child2": 4,"child5": 5},"parent2": {"child3": 3}}Try it yourself with the `vector vrl` subcommand.
Parse functions
parse_apache_log
parse_apache_log(value, timestamp_format: "%d/%b/%Y:%T %z", format)
Parses Apache access and error log lines. Lines can be in common
,
combined
, or default error
format.
Spec
parse_apache_log(value: <string>, [timestamp_format: <string>], format: <string>) :: <object>, <error>- required
value
- The string to parse. - optional
timestamp_format
- The date/time format to use for encoding the timestamp. The time is parsed in local time if the timestamp doesn't specify a timezone. - required
format
- The format to use for parsing the log. Must be one of the following values:common
- Common formatcombined
- Apache combined formaterror
- Default Apache error format
return
- a<object>
Notices
This function has special behavior you should be aware of.
Errors
This function is fallible, error handling is required.
Examples
Parse via Apache log format (common)parse_apache_log!("127.0.0.1 bob frank [10/Oct/2000:13:55:36 -0700] \"GET /apache_pb.gif HTTP/1.0\" 200 2326", format: "common"){"host": "127.0.0.1","identity": "bob","user": "frank","timestamp": "2000-10-10T20:55:36Z","message": "GET /apache_pb.gif HTTP/1.0","method": "GET","path": "/apache_pb.gif","protocol": "HTTP/1.0","status": 200,"size": 2326}Try it yourself with the `vector vrl` subcommand.
parse_aws_alb_log
parse_aws_alb_log(value)
Parses value
in the Elastic Load Balancer Access format.
Spec
parse_aws_alb_log(value: <string>) :: <object>, <error>- required
value
- Access log of the Application Load Balancer. return
- a<object>
Errors
This function is fallible, error handling is required.
Examples
parse_aws_alb_log!("http 2018-11-30T22:23:00.186641Z app/my-loadbalancer/50dc6c495c0c9188 192.168.131.39:2817 - 0.000 0.001 0.000 200 200 34 366 \"GET http://www.example.com:80/ HTTP/1.1\" \"curl/7.46.0\" - - arn:aws:elasticloadbalancing:us-east-2:123456789012:targetgroup/my-targets/73e2d6bc24d8a067 \"Root=1-58337364-23a8c76965a2ef7629b185e3\" \"-\" \"-\" 0 2018-11-30T22:22:48.364000Z \"forward\" \"-\" \"-\" \"-\" \"-\" \"-\" \"-\""){"type": "http","timestamp": "2018-11-30T22:23:00.186641Z","elb": "app/my-loadbalancer/50dc6c495c0c9188","client_host": "192.168.131.39:2817","target_host": null,"request_processing_time": 0.0,"target_processing_time": 0.001,"response_processing_time": 0.0,"elb_status_code": "200","target_status_code": "200","received_bytes": 34,"sent_bytes": 366,"request_method": "GET","request_url": "http://www.example.com:80/","request_protocol": "HTTP/1.1","user_agent": "curl/7.46.0","ssl_cipher": null,"ssl_protocol": null,"target_group_arn": "arn:aws:elasticloadbalancing:us-east-2:123456789012:targetgroup/my-targets/73e2d6bc24d8a067","trace_id": "Root=1-58337364-23a8c76965a2ef7629b185e3","domain_name": null,"chosen_cert_arn": null,"matched_rule_priority": "0","request_creation_time": "2018-11-30T22:22:48.364000Z","actions_executed": "forward","redirect_url": null,"error_reason": null,"target_port_list": [],"target_status_code_list": [],"classification": null,"classification_reason": null}Try it yourself with the `vector vrl` subcommand.
parse_aws_cloudwatch_log_subscription_message
parse_aws_cloudwatch_log_subscription_message(value)
Parses AWS CloudWatch Logs events (configured through AWS Cloudwatch subscriptions) from the
aws_kinesis_firehose
source.
Spec
parse_aws_cloudwatch_log_subscription_message(value: <string>) :: <object>, <error>- required
value
- The string representation of the message to parse. return
- a<object>
Errors
This function is fallible, error handling is required.
Examples
Vector event{"message": "{\n \"messageType\": \"DATA_MESSAGE\",\n \"owner\": \"111111111111\",\n \"logGroup\": \"test\",\n \"logStream\": \"test\",\n \"subscriptionFilters\": [\n\t\"Destination\"\n ],\n \"logEvents\": [\n\t{\n\t \"id\": \"35683658089614582423604394983260738922885519999578275840\",\n\t \"timestamp\": 1600110569039,\n\t \"message\": \"{\\\"bytes\\\":26780,\\\"datetime\\\":\\\"14/Sep/2020:11:45:41 -0400\\\",\\\"host\\\":\\\"157.130.216.193\\\",\\\"method\\\":\\\"PUT\\\",\\\"protocol\\\":\\\"HTTP/1.0\\\",\\\"referer\\\":\\\"https://www.principalcross-platform.io/markets/ubiquitous\\\",\\\"request\\\":\\\"/expedite/convergence\\\",\\\"source_type\\\":\\\"stdin\\\",\\\"status\\\":301,\\\"user-identifier\\\":\\\"-\\\"}\"\n\t}\n ]\n}"}VRL programparse_aws_cloudwatch_log_subscription_message!(.message){"owner": "111111111111","message_type": "DATA_MESSAGE","log_group": "test","log_stream": "test","subscription_filters": ["Destination"],"log_events": [{"id": "35683658089614582423604394983260738922885519999578275840","message": "{\"bytes\":26780,\"datetime\":\"14/Sep/2020:11:45:41 -0400\",\"host\":\"157.130.216.193\",\"method\":\"PUT\",\"protocol\":\"HTTP/1.0\",\"referer\":\"https://www.principalcross-platform.io/markets/ubiquitous\",\"request\":\"/expedite/convergence\",\"source_type\":\"stdin\",\"status\":301,\"user-identifier\":\"-\"}","timestamp": "2020-09-14T19:09:29.039Z"}]}Try it yourself with the `vector vrl` subcommand.
parse_aws_vpc_flow_log
parse_aws_vpc_flow_log(value, format: null)
Parses value
in the VPC Flow Logs format.
Spec
parse_aws_vpc_flow_log(value: <string>, [format: <string>]) :: <object>, <error>- required
value
- VPC Flow Log. - optional
format
- VPC Flow Log format. return
- a<object>
Errors
This function is fallible, error handling is required.
Examples
Parse AWS VPC Flow log (default format)parse_aws_vpc_flow_log!("2 123456789010 eni-1235b8ca123456789 - - - - - - - 1431280876 1431280934 - NODATA"){"version": 2,"account_id": 123456789010,"interface_id": "eni-1235b8ca123456789","srcaddr": null,"dstaddr": null,"srcport": null,"dstport": null,"protocol": null,"packets": null,"bytes": null,"start": 1431280876,"end": 1431280934,"action": null,"log_status": "NODATA"}Try it yourself with the `vector vrl` subcommand.
parse_common_log
parse_common_log(value, timestamp_format: "%d/%b/%Y:%T %z")
Parses the value
using the Common Log Format (CLF).
Spec
parse_common_log(value: <string>, [timestamp_format: <string>]) :: <object>, <error>- required
value
- The string to parse. - optional
timestamp_format
- The date/time format to use for encoding the timestamp. return
- a<object>
Notices
This function has special behavior you should be aware of.
Errors
This function is fallible, error handling is required.
Examples
Parse via Common Log Format (with default timestamp format)parse_common_log!("127.0.0.1 bob frank [10/Oct/2000:13:55:36 -0700] \"GET /apache_pb.gif HTTP/1.0\" 200 2326"){"host": "127.0.0.1","identity": "bob","user": "frank","timestamp": "2000-10-10T20:55:36Z","message": "GET /apache_pb.gif HTTP/1.0","method": "GET","path": "/apache_pb.gif","protocol": "HTTP/1.0","status": 200,"size": 2326}Try it yourself with the `vector vrl` subcommand.
parse_duration
parse_duration(value, unit)
Parses the value
into a human-readable duration format specified by unit
.
Spec
parse_duration(value: <string>, unit: <string>) :: <float>, <error>- required
value
- The string of the duration. - required
unit
- The output units for the duration. Must be one of the following values:ns
- Nanoseconds (1 billion nanoseconds in a second)us
- Microseconds (1 million microseconds in a second)µs
- Microseconds (1 million microseconds in a second)ms
- Milliseconds (1 thousand microseconds in a second)cs
- Centiseconds (100 centiseconds in a second)ds
- Deciseconds (10 deciseconds in a second)s
- Secondsm
- Minutes (60 seconds in a minute)h
- Hours (60 minutes in an hour)d
- Days (24 hours in a day)
return
- a<float>
Errors
This function is fallible, error handling is required.
Examples
parse_duration!("1005ms", unit: "s")1.005Try it yourself with the `vector vrl` subcommand.
parse_glog
parse_glog(value)
Parses the value
using the glog (Google Logging Library) format.
Spec
parse_glog(value: <string>) :: <object>, <error>- required
value
- The string to parse. return
- a<object>
Errors
This function is fallible, error handling is required.
Examples
parse_glog!("I20210131 14:48:54.411655 15520 main.c++:9] Hello world!"){"level": "info","timestamp": "2021-01-31T14:48:54.411655Z","id": 15520,"file": "main.c++","line": 9,"message": "Hello world!"}Try it yourself with the `vector vrl` subcommand.
parse_grok
parse_grok(value, pattern, remove_empty: true)
Parses the value
using the grok
format. All patterns listed here
are supported.
Spec
parse_grok(value: <string>, pattern: <string>, [remove_empty: <boolean>]) :: <object>, <error>- required
value
- The string to parse. - required
pattern
- The Grok pattern. - optional
remove_empty
- If set totrue
, any patterns that resolve to an empty value are removed from the result. return
- a<object>
Notices
This function has special behavior you should be aware of.
Errors
This function is fallible, error handling is required.
Examples
parse_grok!("2020-10-02T23:22:12.223222Z info Hello world","%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:level} %{GREEDYDATA:message}"){"timestamp": "2020-10-02T23:22:12.223222Z","level": "info","message": "Hello world"}Try it yourself with the `vector vrl` subcommand.
parse_json
parse_json(value)
Parses the value
as JSON.
Spec
parse_json(value: <string>) :: <boolean | integer | float | string | object | array | null>, <error>- required
value
- The string representation of the JSON to parse. return
- a<boolean | integer | float | string | object | array | null>
Notices
This function has special behavior you should be aware of.
Errors
This function is fallible, error handling is required.
Examples
parse_json!("{\"key\": \"val\"}"){"key": "val"}Try it yourself with the `vector vrl` subcommand.
parse_key_value
parse_key_value(value, key_value_delimiter: "=", field_delimiter: " ")
Parses the value
in key/value format. Also known as logfmt).
- Keys and values can be wrapped with
"
. "
characters can be escaped using\
.
Spec
parse_key_value(value: <string>, [key_value_delimiter: <string>], [field_delimiter: <string>]) :: <object>, <error>- required
value
- The string to parse. - optional
key_value_delimiter
- The string that separates the key from the value. - optional
field_delimiter
- The string that separates each key/value pair. return
- a<object>
Notices
This function has special behavior you should be aware of.
Errors
This function is fallible, error handling is required.
Examples
Parse logfmt logparse_key_value!("@timestamp=\"Sun Jan 10 16:47:39 EST 2021\" level=info msg=\"Stopping all fetchers\" tag#production=stopping_fetchers id=ConsumerFetcherManager-1382721708341 module=kafka.consumer.ConsumerFetcherManager"){"@timestamp": "Sun Jan 10 16:47:39 EST 2021","level": "info","msg": "Stopping all fetchers","tag#production": "stopping_fetchers","id": "ConsumerFetcherManager-1382721708341","module": "kafka.consumer.ConsumerFetcherManager"}Try it yourself with the `vector vrl` subcommand.
parse_logfmt
parse_logfmt(value)
Parses the value
in logfmt.
- Keys and values can be wrapped with
"
. "
characters can be escaped by\
.
Spec
parse_logfmt(value: <string>) :: <object>, <error>- required
value
- The string to parse. return
- a<object>
Notices
This function has special behavior you should be aware of.
Errors
This function is fallible, error handling is required.
Examples
parse_logfmt!("@timestamp=\"Sun Jan 10 16:47:39 EST 2021\" level=info msg=\"Stopping all fetchers\" tag#production=stopping_fetchers id=ConsumerFetcherManager-1382721708341 module=kafka.consumer.ConsumerFetcherManager"){"@timestamp": "Sun Jan 10 16:47:39 EST 2021","level": "info","msg": "Stopping all fetchers","tag#production": "stopping_fetchers","id": "ConsumerFetcherManager-1382721708341","module": "kafka.consumer.ConsumerFetcherManager"}Try it yourself with the `vector vrl` subcommand.
parse_regex
parse_regex(value, pattern)
Parses the value
via the provided Regex pattern
.
This function differs from the parse_regex_all
function in that it returns only the first match.
Spec
parse_regex(value: <string>, pattern: <regex>) :: <object>, <error>- required
value
- The string to search. - required
pattern
- The regular expression pattern to search against. return
- a<object>
Matches return all capture groups corresponding to the leftmost matches in the text.
Raises an error if no match is found.
Notices
This function has special behavior you should be aware of.
Errors
This function is fallible, error handling is required.
Examples
Parse using Regex (with capture groups)parse_regex!("first group and second group.", r'(?P<number>.*?) group'){"number": "first","0": "first group","1": "first"}Try it yourself with the `vector vrl` subcommand.
parse_regex_all
parse_regex_all(value, pattern)
Parses the value
via the provided Regex pattern
.
This function differs from the parse_regex
function in that it returns all matches, not just the first.
Spec
parse_regex_all(value: <string>, pattern: <regex>) :: <array>, <error>- required
value
- The string to search. - required
pattern
- The regular expression pattern to search against. return
- a<array>
Matches return all capture groups corresponding to the leftmost matches in the text.
Raises an error if no match is found.
Notices
This function has special behavior you should be aware of.
Errors
This function is fallible, error handling is required.
Examples
parse_regex_all!("first group and second group.", r'(?P<number>\w+) group')[{"number": "first","0": "first group","1": "first"},{"number": "second","0": "second group","1": "second"}]Try it yourself with the `vector vrl` subcommand.
parse_syslog
parse_syslog(value)
Parses the value
in Syslog format.
Spec
parse_syslog(value: <string>) :: <object>, <error>- required
value
- The text containing the Syslog message to parse. return
- a<object>
Notices
This function has special behavior you should be aware of.
Errors
This function is fallible, error handling is required.
Examples
parse_syslog!(s'<13>1 2020-03-13T20:45:38.119Z dynamicwireless.name non 2426 ID931 [exampleSDID@32473 iut="3" eventSource= "Application" eventID="1011"] Try to override the THX port, maybe it will reboot the neural interface!'){"severity": "notice","facility": "user","timestamp": "2020-03-13T20:45:38.119Z","hostname": "dynamicwireless.name","appname": "non","procid": 2426,"msgid": "ID931","message": "Try to override the THX port, maybe it will reboot the neural interface!","exampleSDID@32473.eventID": "1011","exampleSDID@32473.eventSource": "Application","exampleSDID@32473.iut": "3","version": 1}Try it yourself with the `vector vrl` subcommand.
parse_timestamp
parse_timestamp(value, format)
Parses the value
in strptime format
.
Spec
parse_timestamp(value: <string>, format: <string>) :: <timestamp>, <error>- required
value
- The text of the timestamp. - required
format
- The strptime format. return
- a<timestamp>
Errors
This function is fallible, error handling is required.
Examples
parse_timestamp!("10-Oct-2020 16:00+00:00", format: "%v %R %:z")"2020-10-10T16:00:00Z"Try it yourself with the `vector vrl` subcommand.
parse_tokens
parse_tokens(value)
Parses the value
in "token" format. A token is considered to be one of the following:
- A word surrounded by whitespace.
- Text delimited by double quotes:
".."
. Quotes can be included in the token if they are escaped by a backslash (\
). - Text delimited by square brackets:
[..]
. Closing square brackets can be included in the token if they are escaped by a backslash (\
).
Spec
parse_tokens(value: <string>) :: <array>, <error>- required
value
- The string to tokenize. return
- a<array>
Notices
This function has special behavior you should be aware of.
Errors
This function is fallible, error handling is required.
Examples
parse_tokens("A sentence \"with \\\"a\\\" sentence inside\" and [some brackets]")["A","sentence","with \\\"a\\\" sentence inside","and","some brackets"]Try it yourself with the `vector vrl` subcommand.
parse_url
parse_url(value)
Parses the value
in URL format.
Spec
parse_url(value: <string>) :: <object>, <error>- required
value
- The text of the URL. return
- a<object>
Errors
This function is fallible, error handling is required.
Examples
parse_url!("ftp://foo:bar@vector.dev:4343/foobar?hello=world#123"){"scheme": "ftp","username": "foo","password": "bar","host": "vector.dev","port": 4343,"path": "/foobar","query": {"hello": "world"},"fragment": "123"}Try it yourself with the `vector vrl` subcommand.
Random functions
uuid_v4
uuid_v4
Generates a random UUIDv4 string.
Spec
uuid_v4() :: <string>return
- a<string>
Examples
uuid_v4()"1d262f4f-199b-458d-879f-05fd0a5f0683"Try it yourself with the `vector vrl` subcommand.
String functions
contains
contains(value, substring, case_sensitive: true)
Determines whether the value
string contains the specified substring
.
Spec
contains(value: <string>, substring: <string>, [case_sensitive: <boolean>]) :: <boolean>- required
value
- The text to search. - required
substring
- The substring to search for invalue
. - optional
case_sensitive
- Whether the match should be case sensitive. return
- a<boolean>
Examples
String contains (case sensitive)contains("The Needle In The Haystack", "Needle")trueTry it yourself with the `vector vrl` subcommand.
downcase
downcase(value)
Downcases the value
string, where "downcase" is defined according to the terms of the
Unicode Derived Core Property Lowercase.
Spec
downcase(value: <string>) :: <string>- required
value
- The string to convert to lowercase. return
- a<string>
Examples
downcase("Hello, World!")"hello, world!"Try it yourself with the `vector vrl` subcommand.
ends_with
ends_with(value, substring, case_sensitive: true)
Determines whether the value
string ends with the specified substring
.
Spec
ends_with(value: <string>, substring: <string>, [case_sensitive: <boolean>]) :: <boolean>- required
value
- The string to search. - required
substring
- The substring with whichvalue
must end. - optional
case_sensitive
- Whether the match should be case sensitive. return
- a<boolean>
Examples
String ends with (case sensitive)ends_with("The Needle In The Haystack", "The Haystack")trueTry it yourself with the `vector vrl` subcommand.
join
join(value, separator: null)
Joins each string in the value
array into a single string, with items optionally separated from one another
by a separator
.
Spec
join(value: <array>, [separator: <string>]) :: <string>- required
value
- The array of strings to join together. - optional
separator
- The string separating each original element when joined. return
- a<string>
Examples
Join array (no separator)join!(["bring", "us", "together"])"bringustogether"Try it yourself with the `vector vrl` subcommand.
match
match(value, pattern)
Determines whether the value
matches the pattern
.
Spec
match(value: <string>, pattern: <regex>) :: <boolean>- required
value
- The value to match. - required
pattern
- The regular expression pattern to match against. return
- a<boolean>
Examples
match("I'm a little teapot", r'teapot')trueTry it yourself with the `vector vrl` subcommand.
replace
replace(value, pattern, with, count: -1)
Replaces all matching instances of pattern
in the value
.
Spec
replace(value: <string>, pattern: <regex | string>, with: <string>, [count: <integer>]) :: <string>- required
value
- The original string. - required
pattern
- Replace all matches of this pattern. Can be a static string or a regular expression. - required
with
- The string that the matches are replaced with. - optional
count
- The maximum number of replacements to perform. -1 means replace all matches. return
- a<string>
Examples
Replace literal textreplace("Apples and Bananas", "and", "not")"Apples not Bananas"Try it yourself with the `vector vrl` subcommand.
slice
slice(value, start, end: "String length")
Returns a slice of the value
between the start
and end
positions.
If the start
and end
parameters are negative, they refer to positions counting from the right of the
string or array. If end
refers to a position that is greater than the length of the string or array
a slice up to the end of the string or array is returned.
Spec
slice(value: <array | string>, start: <integer>, [end: <integer>]) :: <string>- required
value
- The string or array to slice. - required
start
- The inclusive start position. A zero-based index that can be negative. - optional
end
- The inclusive end position. A zero-based index that can be negative. return
- a<string>
Examples
Slice a string (positve index)slice!("Supercalifragilisticexpialidocious", start: 5, end: 13)"califrag"Try it yourself with the `vector vrl` subcommand.
split
split(value, pattern, limit: null)
Splits the value
string via the pattern
.
Spec
split(value: <string>, pattern: <string | regex>, [limit: <integer>]) :: <string>- required
value
- The string to split. - required
pattern
- The string is split whenever this pattern is matched. - optional
limit
- The maximum number of substrings to return. return
- a<string>
If
limit
is specified, the remainder of the string is returned unsplit afterlimit
has been reached.
Examples
Split a string (no limit)split("apples and pears and bananas", " and ")["apples","pears","bananas"]Try it yourself with the `vector vrl` subcommand.
starts_with
starts_with(value, substring, case_sensitive: true)
Determines whether the value
begins with the substring
.
Spec
starts_with(value: <string>, substring: <string>, [case_sensitive: <boolean>]) :: <boolean>- required
value
- The string to search. - required
substring
- The substring that thevalue
must start with. - optional
case_sensitive
- Whether the match should be case sensitive. return
- a<boolean>
Examples
String starts with (case sensitive)starts_with("The Needle In The Haystack", "The Needle")trueTry it yourself with the `vector vrl` subcommand.
strip_ansi_escape_codes
strip_ansi_escape_codes(value)
Strips ANSI escape codes from the value
.
Spec
strip_ansi_escape_codes(value: <string>) :: <string>- required
value
- The string to strip. return
- a<string>
Examples
strip_ansi_escape_codes("\e[46mfoo\e[0m bar")"foo bar"Try it yourself with the `vector vrl` subcommand.
strip_whitespace
strip_whitespace(value)
Strips whitespace from the start and end of the value
, where whitespace is defined by the Unicode
White_Space
property
Spec
strip_whitespace(value: <string>) :: <string>- required
value
- The string to trim. return
- a<string>
Examples
strip_whitespace(" A sentence. ")"A sentence."Try it yourself with the `vector vrl` subcommand.
truncate
truncate(value, limit, ellipsis)
Truncates the value
string up to the limit
number of characters.
Spec
truncate(value: <string>, limit: <integer | float>, ellipsis: <boolean>) :: <string>- required
value
- The string to truncate. - required
limit
- The number of characters to truncate the string after. - required
ellipsis
- An ellipsis (...
) is appended if this is set totrue
and thevalue
string ends up being truncated because it's exceeded thelimit
. return
- a<string>
The string is returned unchanged its length is less than
limit
.If
ellipsis
istrue
, then an ellipsis (...
) is appended to the string (beyond the specifiedlimit
).
Examples
truncate("A rather long sentence.", limit: 11, ellipsis: true)"A rather lo..."Try it yourself with the `vector vrl` subcommand.
upcase
upcase(value)
Upcases the value
, where "upcase" is defined according to the terms of the Unicode Derived Core Property
Uppercase.
Spec
upcase(value: <string>) :: <string>- required
value
- The string to convert to uppercase. return
- a<string>
Examples
upcase("Hello, World!")"HELLO, WORLD!"Try it yourself with the `vector vrl` subcommand.
System functions
get_env_var
get_env_var(name)
Returns the value of the environment variable specifed by name
.
Spec
get_env_var(name: <string>) :: <string>, <error>- required
name
- The name of the environment variable. return
- a<string>
Errors
This function is fallible, error handling is required.
Examples
get_env_var!("HOME")"/root"Try it yourself with the `vector vrl` subcommand.
get_hostname
get_hostname
Returns the local system's hostname.
Spec
get_hostname() :: <string>, <error>return
- a<string>
Errors
This function is fallible, error handling is required.
Examples
Vector event (before){}VRL program.hostname = get_hostname!()Vector event (after){"hostname": "localhost.localdomain"}
Timestamp functions
format_timestamp
format_timestamp(value, format)
Formats the value
into a string representation of the timestamp.
Spec
format_timestamp(value: <timestamp>, format: <string>) :: <string>- required
value
- The timestamp to format as text. - required
format
- The format string as decribed by the Chrono library. return
- a<string>
Examples
Format a timestamp (ISO8601/RFC 3339)format_timestamp!(t'2020-10-21T16:00:00Z', format: "%+")"2020-10-21T16:00:00+00:00"Try it yourself with the `vector vrl` subcommand.
now
now
Returns the current timestamp in the UTC timezone with nanosecond precision.
Spec
now() :: <timestamp>return
- a<timestamp>
Examples
now()"2021-03-04T10:51:15.928937Z"Try it yourself with the `vector vrl` subcommand.
Type functions
array
array(value)
Returns the value
if it's an array and errors otherwise. This enables the type checker to guarantee that the
returned value is an array and can be used in any function that expects one.
Spec
array(value: <any>) :: <array>, <error>- required
value
- The value that you need to ensure is an array. return
- a<array>
Returns the
value
if it's an array.Raises an error if not an array.
Errors
This function is fallible, error handling is required.
Examples
Vector event{"value": [1,2,3]}VRL programarray!(.value)[1,2,3]Try it yourself with the `vector vrl` subcommand.
bool
bool(value)
Returns the value
if it's a Boolean and errors otherwise. This enables the type checker to guarantee that the
returned value is a Boolean and can be used in any function that expects one.
Spec
bool(value: <any>) :: <boolean>, <error>- required
value
- The value that you need to ensure is a Boolean. return
- a<boolean>
Returns
value
if it's a Boolean.Raises an error if not a Boolean.
Errors
This function is fallible, error handling is required.
Examples
Vector event{"value": false}VRL programbool!(.value)falseTry it yourself with the `vector vrl` subcommand.
float
float(value)
Returns the value
if it's a float and errors otherwise. This enables the type checker to guarantee that the
returned value is a float and can be used in any function that expects one.
Spec
float(value: <any>) :: <float>, <error>- required
value
- The value that you need to ensure is a float. return
- a<float>
Returns the
value
if it's a float.Raises an error if not a float.
Errors
This function is fallible, error handling is required.
Examples
Vector event{"value": 42.0}VRL programfloat!(.value)42.0Try it yourself with the `vector vrl` subcommand.
int
int(value)
Returns the value
if it's an integer and errors otherwise. This enables the type checker to guarantee that the
returned value is an integer and can be used in any function that expects one.
Spec
int(value: <any>) :: <integer>, <error>- required
value
- The value that you need to ensure is an integer. return
- a<integer>
Returns the
value
if it's an integer.Raises an error if not an integer.
Errors
This function is fallible, error handling is required.
Examples
Vector event{"value": 42}VRL programint!(.value)42Try it yourself with the `vector vrl` subcommand.
is_nullish
is_nullish(value)
Determines whether the value
is "nullish," where nullish denotes the absence of a
meaningful value.
Spec
is_nullish(value: <any>) :: <boolean>- required
value
- The value to check for "nullishness," i.e. a useless value. return
- a<boolean>
Returns
true
ifvalue
isnull
.Returns
true
ifvalue
is"-"
.Returns
true
ifvalue
is whitespace as defined by UnicodeWhite_Space
property.Returns
false
ifvalue
is anything else.
Examples
Null detection (blank string)is_nullish("")trueTry it yourself with the `vector vrl` subcommand.
object
object(value)
Returns the value
if it's an object and errors otherwise. This enables the type checker to guarantee that the
returned value is an object and can be used in any function that expects one.
Spec
object(value: <any>) :: <object>, <error>- required
value
- The value that you need to ensure is an object. return
- a<object>
Returns the
value
if it's an object.Raises an error if not an object.
Errors
This function is fallible, error handling is required.
Examples
Vector event{"value": {"field1": "value1","field2": "value2"}}VRL programobject!(.value){"field1": "value1","field2": "value2"}Try it yourself with the `vector vrl` subcommand.
string
string(value)
Returns the value
if it's a string and errors otherwise. This enables the type checker to guarantee that the
returned value is a string and can be used in any function that expects one.
Spec
string(value: <any>) :: <string>, <error>- required
value
- The value that you need to ensure is a string. return
- a<string>
Returns the
value
if it's a string.Raises an error if not a string.
Errors
This function is fallible, error handling is required.
Examples
Vector event{"message": "{\"field\": \"value\"}"}VRL programstring!(.message)"{\"field\": \"value\"}"Try it yourself with the `vector vrl` subcommand.
tag_types_externally
tag_types_externally(value)
Adds type information to all (nested) scalar values in the provided value
.
The type information is added externally, meaning that value
has the shape of "type": value
after this
transformation.
Spec
tag_types_externally(value: <any>) :: <object | array | null>- required
value
- The value to tag with types. return
- a<object | array | null>
Examples
Tag types externally (scalar)tag_types_externally(123){"integer": 123}Try it yourself with the `vector vrl` subcommand.
timestamp
timestamp(value)
Returns the value
if it's a timestamp and errors otherwise. This enables the type checker to guarantee that
the returned value is a timestamp and can be used in any function that expects one.
Spec
timestamp(value: <any>) :: <timestamp>, <error>- required
value
- The value that you need to ensure is a timestamp. return
- a<timestamp>
Returns the
value
if it's a timestamp.Raises an error if not a timestamp.
Errors
This function is fallible, error handling is required.
Examples
Vector event{"timestamp": "2020-10-10T16:00:00Z"}VRL programok, err = timestamp(.timestamp)"function call error for \"timestamp\" at (10:31): expected \"timestamp\", got \"string\""Try it yourself with the `vector vrl` subcommand.