Route
Split a stream of events into multiple sub-streams based on user-supplied conditions
Splits a stream of events into multiple sub-streams based on a set of conditions.
Also, see the Exclusive Route transform for routing an event to a single stream.
Configuration
Example configurations
{
"transforms": {
"my_transform_id": {
"type": "route",
"inputs": [
"my-source-or-transform-id"
]
}
}
}
[transforms.my_transform_id]
type = "route"
inputs = [ "my-source-or-transform-id" ]
transforms:
my_transform_id:
type: route
inputs:
- my-source-or-transform-id
{
"transforms": {
"my_transform_id": {
"type": "route",
"inputs": [
"my-source-or-transform-id"
],
"reroute_unmatched": true,
"route": {
"foo-does-not-exist": {
"source": "!exists(.foo)",
"type": "vrl"
},
"foo-exists": {
"source": "exists(.foo)",
"type": "vrl"
}
}
}
}
}
[transforms.my_transform_id]
type = "route"
inputs = [ "my-source-or-transform-id" ]
reroute_unmatched = true
[transforms.my_transform_id.route.foo-does-not-exist]
source = "!exists(.foo)"
type = "vrl"
[transforms.my_transform_id.route.foo-exists]
source = "exists(.foo)"
type = "vrl"
transforms:
my_transform_id:
type: route
inputs:
- my-source-or-transform-id
reroute_unmatched: true
route:
foo-does-not-exist:
source: "!exists(.foo)"
type: vrl
foo-exists:
source: exists(.foo)
type: vrl
graph
optional objectExtra graph configuration
Configure output for component when generated with graph command
graph.node_attributes
optional objectNode attributes to add to this component’s node in resulting graph
They are added to the node as provided
graph.node_attributes.*
required string literalinputs
required [string]A list of upstream source or transform IDs.
Wildcards (*
) are supported.
See configuration for more info.
reroute_unmatched
optional boolReroutes unmatched events to a named output instead of silently discarding them.
Normally, if an event doesn’t match any defined route, it is sent to the <transform_name>._unmatched
output for further processing. In some cases, you may want to simply discard unmatched events and not
process them any further.
In these cases, reroute_unmatched
can be set to false
to disable the <transform_name>._unmatched
output and instead silently discard any unmatched events.
true
route
optional objectA map from route identifiers to logical conditions. Each condition represents a filter which is applied to each event.
The following identifiers are reserved output names and thus cannot be used as route IDs:
_unmatched
_default
Each route can then be referenced as an input by other components with the name
<transform_name>.<route_id>
. If an event doesn’t match any route, and if reroute_unmatched
is set to true
(the default), it is sent to the <transform_name>._unmatched
output.
Otherwise, the unmatched event is instead silently discarded.
route.*
required conditionAvailable syntaxes
Syntax | Description | Example |
---|---|---|
vrl | A Vector Remap Language (VRL) Boolean expression. | .status_code != 200 && !includes(["info", "debug"], .severity) |
datadog_search | A Datadog Search query string. | *stack |
is_log | Whether the incoming event is a log. |
|
is_metric | Whether the incoming event is a metric. |
|
is_trace | Whether the incoming event is a trace. |
|
Shorthand for VRL
If you opt for the vrl
syntax for this condition, you can set the condition
as a string via the condition
parameter, without needing to specify both a source
and a type
. The
table below shows some examples:
Config format | Example |
---|---|
YAML | condition: .status == 200 |
TOML | condition = ".status == 200" |
JSON | "condition": ".status == 200" |
Condition config examples
Standard VRL
*:
type: "vrl"
source: ".status == 500"
* = { type = "vrl", source = ".status == 500" }
"*": {
"type": "vrl",
"source": ".status == 500"
}
Outputs
<route_id>
<transform_name>.<route_id>
.Telemetry
Metrics
linkcomponent_discarded_events_total
counterfilter
transform, or false if due to an error.component_errors_total
countercomponent_received_event_bytes_total
countercomponent_received_events_count
histogramA histogram of the number of events passed in each internal batch in Vector’s internal topology.
Note that this is separate than sink-level batching. It is mostly useful for low level debugging performance issues in Vector due to small internal batches.
component_received_events_total
countercomponent_sent_event_bytes_total
countercomponent_sent_events_total
counterutilization
gaugeExamples
Split by log level
Given this event...{
"log": {
"level": "info"
}
}
transforms:
my_transform_id:
type: route
inputs:
- my-source-or-transform-id
route:
debug: .level == "debug"
info: .level == "info"
warn: .level == "warn"
error: .level == "error"
[transforms.my_transform_id]
type = "route"
inputs = [ "my-source-or-transform-id" ]
[transforms.my_transform_id.route]
debug = '.level == "debug"'
info = '.level == "info"'
warn = '.level == "warn"'
error = '.level == "error"'
{
"transforms": {
"my_transform_id": {
"type": "route",
"inputs": [
"my-source-or-transform-id"
],
"route": {
"debug": ".level == \"debug\"",
"info": ".level == \"info\"",
"warn": ".level == \"warn\"",
"error": ".level == \"error\""
}
}
}
}
{
"level": "info"
}
Split by metric namespace
Given this event...{
"metric": {
"counter": {
"value": 10000
},
"kind": "absolute",
"name": "memory_available_bytes",
"namespace": "host",
"tags": {}
}
}
transforms:
my_transform_id:
type: route
inputs:
- my-source-or-transform-id
route:
app: .namespace == "app"
host: .namespace == "host"
[transforms.my_transform_id]
type = "route"
inputs = [ "my-source-or-transform-id" ]
[transforms.my_transform_id.route]
app = '.namespace == "app"'
host = '.namespace == "host"'
{
"transforms": {
"my_transform_id": {
"type": "route",
"inputs": [
"my-source-or-transform-id"
],
"route": {
"app": ".namespace == \"app\"",
"host": ".namespace == \"host\""
}
}
}
}
{
"counter": {
"value": 10000
},
"kind": "absolute",
"name": "memory_available_bytes",
"namespace": "host",
"tags": {}
}
How it works
Routing to multiple components
The following is an example of how you can create two routes that feed three downstream components.
It is worth noting that a single route can feed multiple downstream components.
transforms:
my-routes:
inputs: [ some_source ]
type: route
route:
foo-exists: 'exists(.foo)'
foo-doesnt-exist: '!exists(.foo)'
remap-route-1:
type: remap
inputs:
- my-routes.foo-exists
source: |
.route = "route 1"
remap-route-2:
type: remap
inputs:
- my-routes.foo-doesnt-exist
source: |
.route = "route 2"
remap-route-3:
type: remap
inputs:
- my-routes.foo-exists
source: |
.route = "route 3"
tests:
- name: case-1
inputs:
- type: log
insert_at: my-routes
log_fields:
foo: X
outputs:
- extract_from: remap-route-1
conditions:
- type: vrl
source: |
assert!(exists(.foo))
assert_eq!(.route, "route 1")
- extract_from: remap-route-3
conditions:
- type: vrl
source: |
assert!(exists(.foo))
assert_eq!(.route, "route 3")
- name: case-2
inputs:
- type: log
insert_at: my-routes
log_fields:
bar: X
outputs:
- extract_from: remap-route-2
conditions:
- type: vrl
source: |
assert!(!exists(.foo))
assert_eq!(.route, "route 2")