Coercer Transform

The Vector coercer transform shapes logs

Configuration

[transforms.my_transform_id]
# General
type = "coercer" # required
inputs = ["my-source-or-transform-id"] # required
# Types
types.status = "int" # example
types.duration = "float" # example
types.success = "bool" # example
types.timestamp_iso8601 = "timestamp|%F" # example
types.timestamp_custom = "timestamp|%a %b %e %T %Y" # example
types.parent.child = "int" # example
  • optionalbool

    drop_unspecified

    Set to true to drop all fields that are not specified in the types table. Make sure both message and timestamp are specified in the types table as their absense will cause the original message data to be dropped along with other extraneous fields.

    • Default: false
    • View examples
  • commonoptionaltable

    types

    Key/value pairs representing mapped log field names and types. This is used to coerce log fields into their proper types.

Output

Telemetry

This component provides the following metrics that can be retrieved through the internal_metrics source. See the metrics section in the monitoring page for more info.

  • counter

    processed_events_total

    The total number of events processed by this component. This metric includes the following tags:

    • component_kind - The Vector component kind.

    • component_name - The Vector component ID.

    • component_type - The Vector component type.

    • file - The file that produced the error

    • instance - The Vector instance identified by host and port.

    • job - The name of the job producing Vector metrics.

  • counter

    processed_bytes_total

    The total number of bytes processed by the component. This metric includes the following tags:

    • component_kind - The Vector component kind.

    • component_name - The Vector component ID.

    • component_type - The Vector component type.

    • instance - The Vector instance identified by host and port.

    • job - The name of the job producing Vector metrics.

Examples

Given the following Vector event:

{
"log": {
"bytes_in": "5667",
"bytes_out": "20574",
"host": "5.86.210.12",
"message": "GET /embrace/supply-chains/dynamic/vertical",
"status": "201",
"timestamp": "19/06/2019:17:20:49 -0400",
"user_id": "zieme4647"
}
}

And the following configuration:

[transforms.coercer]
type = "coercer"
types.bytes_in = "int"
types.bytes_out = "int"
types.status = "int"
types.timestamp = "timestamp|%d/%m/%Y:%H:%M:%S %z"

The following Vector log event will be output:

{
"bytes_in": 5667,
"bytes_out": 20574,
"host": "5.86.210.12",
"message": "GET /embrace/supply-chains/dynamic/vertical",
"status": 201,
"timestamp": "19/06/2019:17:20:49 -0400",
"user_id": "zieme4647"
}

How It Works