LOG

grok_parser transform

The grok_parser transform accepts log events and allows you to parse a log field value with Grok.

Configuration

vector.toml
[transforms.my_transform_id]
# REQUIRED - General
type = "grok_parser" # example, must be: "grok_parser"
inputs = ["my-source-id"] # example
pattern = "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:level} %{GREEDYDATA:message}" # example
# OPTIONAL - General
drop_field = true # default
field = "message" # default
# OPTIONAL - Types
[transforms.my_transform_id.types]
status = "int"

Options

4 items
commonbooloptional

drop_field

If true will drop the specifiedfield after parsing.

Default: true
View examples
commonstringoptional

field

The log field to execute thepattern against. Must be a string value.

Default: "message"
View examples
commonstringrequired

pattern

The Grok pattern

No default
View examples
commontableoptional

types

Key/Value pairs representing mapped log field types.

commonstringenumrequired

[field-name]

A definition of log field type conversions. They key is the log field name and the value is the type. strptime specifiers are supported for the timestamp type.

No default
Enum, must be one of: "bool" "float" "int" "string" "timestamp"
View examples

How It Works

Available Patterns

Vector uses the Rust grok library. All patterns listed here are supported. It is recommended to use maintained patterns when possible since they can be improved over time by the community.

Debugging

We recommend the Grok debugger for Grok testing.

Environment Variables

Environment variables are supported through all of Vector's configuration. Simply add ${MY_ENV_VAR} in your Vector configuration file and the variable will be replaced before being evaluated.

You can learn more in the Environment Variables section.

Performance

Grok is approximately 50% slower than the regex_parser transform. We plan to add a performance test for this in the future. While this is still plenty fast for most use cases we recommend using the regex_parser transform if you are experiencing performance issues.