Filter Transform

The Vector filter transform filters events based on a set of conditions.

Configuration

[transforms.my_transform_id]
type = "filter" # required
inputs = ["my-source-or-transform-id", "prefix-*"] # required
condition = '.status_code != 200 && !includes(["info", "debug"], .severity)' # required

Telemetry

This component provides the following metrics that can be retrieved through the internal_metrics source. See the metrics section in the monitoring page for more info.

  • counter

    events_discarded_total

    The total number of events discarded by this component. This metric includes the following tags:

    • instance - The Vector instance identified by host and port.

    • job - The name of the job producing Vector metrics.

  • counter

    events_in_total

    The total number of events accepted by this component. This metric includes the following tags:

    • component_kind - The Vector component kind.

    • component_name - The Vector component ID.

    • component_type - The Vector component type.

    • instance - The Vector instance identified by host and port.

    • job - The name of the job producing Vector metrics.

  • counter

    processed_events_total

    The total number of events processed by this component. This metric includes the following tags:

    • component_kind - The Vector component kind.

    • component_name - The Vector component ID.

    • component_type - The Vector component type.

    • file - The file that produced the error

    • instance - The Vector instance identified by host and port.

    • job - The name of the job producing Vector metrics.

  • counter

    events_out_total

    The total number of events emitted by this component. This metric includes the following tags:

    • component_kind - The Vector component kind.

    • component_name - The Vector component ID.

    • component_type - The Vector component type.

    • instance - The Vector instance identified by host and port.

    • job - The name of the job producing Vector metrics.

  • counter

    processed_bytes_total

    The total number of bytes processed by the component. This metric includes the following tags:

    • component_kind - The Vector component kind.

    • component_name - The Vector component ID.

    • component_type - The Vector component type.

    • instance - The Vector instance identified by host and port.

    • job - The name of the job producing Vector metrics.

Examples

Given the following Vector log event:

[
{
"level": "debug",
"message": "I'm a noisy debug log"
},
{
"level": "info",
"message": "I'm a normal info log"
}
]

And the following configuration:

vector.toml
[transforms.filter]
type = "filter"
condition = '.level == "debug"'

The following Vector log event will be output:

[
{
"level": "info",
"message": "I'm a normal info log"
}
]

How It Works

State

This component is stateless, meaning its behavior is consistent across each input.