Add Fields Transform

The Vector add_fields transform accepts and outputs log events allowing you to add one or more log fields.

Configuration

vector.toml

[transforms.my_transform_id]
  # REQUIRED - General
  type = "add_fields" # must be: "add_fields"
  inputs = ["my-source-id"] # example

  # REQUIRED - Fields
  [transforms.my_transform_id.fields]
    string_field = "string value" # example
    env_var_field = "${ENV_VAR}" # example
    int_field = 1 # example
    float_field = 1.2 # example
    bool_field = true # example
    timestamp_field = 1979-05-27T00:32:00Z # example
    parent = {child = "child_value"} # example
    list_field = ["first", "second", "third"] # example

Options

tablecommonrequired

fields

A table of key/value pairs representing the keys to be added to the event.

*commonrequired

[field-name]

The name of the field to add. Accepts all supported types. Use . for adding nested fields.

No default

View examples

Output

The add_fields transform accepts and outputs log events allowing you to add one or more log fields. For example:

Given the following configuration:

vector.toml

[transforms.my_transform]
  type = "add_fields"
  inputs = [...]

  [transforms.my_transform.fields]
    field1 = "string value"
    field2 = 1
    field3 = 2.0
    field4 = true
    field5 = 2019-05-27T07:32:00Z
    field6 = ["item 1", "item 2"]
    field7.nested = "nested value",
    field8 = "#{HOSTNAME}"
    field9 = "{{field1}} {{field2}}"

A log event will be output with the following structure:

{
  // ... existing fields
  "field1": "string value",
  "field2": 1,
  "field3": 2.0,
  "field4": true,
  "field5": <timestamp:2019-05-27T07:32:00Z>,
  "field6.0": "item1",
  "field6.1": "item2",
  "field7.nested": "nested value",
  "field8": "my.hostname.com"
}

While unrealistic, this example demonstrates the various accepted types and how they're repsented in Vector's internal log structure.

How It Works

Arrays

The add_fields transform will support TOML arrays. Keep in mind that the values must be simple type (not tables), and each value must the same type. You cannot mix types:

[transforms.<transform-id>]
  # ...

  [transforms.<transform-id>.fields]
    my_array = ["first", "second", "third"]

Results in:

{
  "my_array.0": "first",
  "my_array.1": "second",
  "my_array.2": "third"
}

Learn more about how log events are structured.

Complex Transforming

The add_fields transform is designed for simple key additions. If you need more complex transforming then we recommend using a more versatile transform like the lua transform.

Environment Variables

Environment variables are supported through all of Vector's configuration. Simply add ${MY_ENV_VAR} in your Vector configuration file and the variable will be replaced before being evaluated.

You can learn more in the Environment Variables section.

Key Conflicts

Keys specified in this transform will replace existing keys.

Nested Fields

The add_fields transform will support dotted keys or TOML tables. We recommend the dotted key syntax since it is less verbose for this usecase:

[transforms.<transform-id>]
  # ...

  [transforms.<transform-id>.fields]
    parent.child.grandchild = "my_value"

Results in:

{
  "parent.child.grandchild": "my_value"
}

Learn more about how log events are structured.

Removing Fields

See the remove_fields transform.

Types

All supported configuration value types are accepted. This includes primitivate types (string, int, float, boolean) and special types, such as arrays and nested fields.

Value Templating

It is possible to use template values in the field. If an input lacks one of the keys needed for a templated field it will drop that field from the output.

[transforms.<transform-id>]
  # ...

  [transforms.<transform-id>.fields]
    my_field = "{{timestamp}} {{message}}"