Templates

Vector supports a template syntax for select configuration options. This allows you to use event fields when configuring Vector:

field = "application_id={{ application_id }}/date=%F/"

Example

For example, let's partition data on AWS S3 by application_id and date. We can accomplish this with the key_prefix option in the aws_s3 sink:

[sinks.backup]
  type = "aws_s3"
  bucket = "all_application_logs"
  key_prefix = "application_id={{ application_id }}/date=%F/"

Notice that Vector allows direct field references as well as strftime specifiers. If we were to run the following log event though Vector:

{
  "timestamp": "2020-02-14T01:22:23.223Z",
  "application_id": 1,
  "message": "Hello world"
}

The value of the key_prefix option would equal:

application_id=1/date=2020-02-14

This effectively enables dynamic partitioning, something fundamental to storing log data.

Syntax

Event Fields

Individual log event fields can be access with the {{<field-path-notation>}} syntax:

option = "{{ field_path_notation }}"

Vector's field notation is simple. It uses . to target nested fields and [<index>] to target array values. Learn more in the field notation docs.

Strftime Specifiers

In addition to directly accessing fields, Vector offers a shortcut for injecting strftime specifiers:

options = "year=%Y/month=%m/day=%d/"

The value is based off of the timestamp field, and the name of this field can be changed via the global timestamp_key option.

Escaping

You can escape this syntax by prefixing the character with a \. For example, you can escape the event field syntax like so:

option = "\{{ field_name }}"

And strptime specified like so:

options = "year=\%Y/month=\%m/day=\%d/"

Each value above will be treated literally.

How It Works

Array Values

Array values can be accessed using Vector's field notation syntax:

option = "{{ parent.child[0] }}"

Fallback Values

Vector currently does not support fallback values. Issue 1692 is open to add this functionality. In the interim, you can use the lua transform to set a default value:

[transforms.set_defaults]
  # REQUIRED
  type = "lua"
  inputs = ["my-source-id"]
  source = """
if event["my_field"] == nil then
  event["my_field"] = "default"
end
"""

Missing Fields

If a field is missing, a blank string will be inserted in its place. Vector will not error, drop the event, or log anything.

Nested Fields

Nested fields can be accessed using Vector's field notation syntax:

option = "{{ parent.child }}"