/ Apr 21st, 2021 /

Adding the `abort` keyword to VRL

Aborting processing of events in `remap`

Vector 0.13 introduces a new abort expression to the Vector Remap Language (VRL). This expression can be used to abort processing in a remap transform. This is useful for aborting processing of invalid events.

By default, events that encounter an abort expression will be dropped by the remap transform, but this behavior can be modified to simply pass along the unmodified event to the next transform by setting drop_on_abort to false.

Example

As an example, the abort expression can be used if you want to do some validation on the event before processing it, discarding any invalid events.

Given a config of:

[sources.in]
type = "generator"
format = "shuffle"
interval = 1.0
lines = ['{ "message": "valid message", "type": "ok"}', '{ "message": "invalid message", "type": "unknown"}']
[transforms.remap]
type = "remap"
inputs = ["in"]
source = """
. |= object!(parse_json!(string!(.message)))
if .type != "ok" {
abort # unknown type
}
"""
[sinks.out]
type = "console"
inputs = ["remap"]
encoding.codec = "json"

You would expect to see something like:

{"message":"valid message","timestamp":"2021-04-20T20:26:05.214875Z","type":"ok"}
{"message":"valid message","timestamp":"2021-04-20T20:26:06.213879Z","type":"ok"}
{"message":"valid message","timestamp":"2021-04-20T20:26:07.214254Z","type":"ok"}
{"message":"valid message","timestamp":"2021-04-20T20:26:08.215219Z","type":"ok"}

That is, that all of the events that don't have a type of ok, would be dropped.

This change was made available in 0.13.0. You can get this change by downloading the latest stable release.

Like What You See?