Coercer Transform

The Vector coercer transform accepts and outputs log events allowing you to coerce log fields into fixed types.

Configuration

  • Common
  • Advanced
vector.toml
[transforms.my_transform_id]
# REQUIRED - General
type = "coercer" # must be: "coercer"
inputs = ["my-source-id"] # example
# OPTIONAL - Types
[transforms.my_transform_id.types]
status = "int" # example
duration = "float" # example
success = "bool" # example
timestamp = "timestamp|%F" # example
timestamp = "timestamp|%a %b %e %T %Y" # example

Options

2 items
booloptional

drop_unspecified

Set to true to drop all fields that are not specified in the types table. Make sure both message and timestamp are specified in the types table as their absense will cause the original message data to be dropped along with other extraneous fields.

No default
View examples
tablecommonoptional

types

Key/Value pairs representing mapped log field types.

stringenumcommonoptional

[field-name]

A definition of log field type conversions. They key is the log field name and the value is the type. strptime specifiers are supported for the timestamp type.

No default
Enum, must be one of: "bool" "float" "int" "string" "timestamp"
View examples

Output

The coercer transform accepts and outputs log events allowing you to coerce log fields into fixed types. For example:

Given the following input:

{
// ... existing fields
"bytes_in": "5667",
"bytes_out": "20574",
"host": "5.86.210.12",
"message": "GET /embrace/supply-chains/dynamic/vertical",
"status": "201",
"timestamp": "19/06/2019:17:20:49 -0400",
"user_id": "zieme4647"
}

And the following configuration:

vector.toml
[transforms.<transform-id>]
type = "coercer"
[transforms.<transform-id>.types]
bytes_in = "int"
bytes_out = "int"
timestamp = "timestamp|%d/%m/%Y:%H:%M:%S %z"
status = "int"

A log event will be output with the following structure:

{
// ... existing fields
"bytes_in": 5667,
"bytes_out": 20574,
"host": "5.86.210.12",
"message": "GET /embrace/supply-chains/dynamic/vertical",
"status": 201,
"timestamp": <19/06/2019:17:20:49 -0400>,
"user_id": "zieme4647"
}

How It Works

Environment Variables

Environment variables are supported through all of Vector's configuration. Simply add ${MY_ENV_VAR} in your Vector configuration file and the variable will be replaced before being evaluated.

You can learn more in the Environment Variables section.