Global options reference

Global configuration parameters

acknowledgements

common optional object

Controls how acknowledgements are handled by all sources. These settings may be overridden in individual sources.

Controls if sources will wait for destination sinks to deliver the events, or persist them to a disk buffer, before acknowledging receipt. If set to true, all capable sources will have acknowledgements enabled.

default: false

data_dir

optional string literal

The directory used for persisting Vector state, such as on-disk buffers, file checkpoints, and more. Please make sure the Vector project has write permissions to this directory.

Examples

"/var/lib/vector"

"/var/local/lib/vector/"

"/home/user/vector/"

default: /var/lib/vector/

enrichment_tables

optional object

Configuration options for an enrichment table to be used in a remap transform. Currently supported are:

CSV files
MaxMind databases

For the lookup in the enrichment tables to be as performant as possible, the data is indexed according to the fields that are used in the search. Note that indices can only be created for fields for which an exact match is used in the condition. For range searches, an index isn’t used and the enrichment table drops back to a sequential scan of the data. A sequential scan shouldn’t impact performance significantly provided that there are only a few possible rows returned by the exact matches in the condition. We don’t recommend using a condition that uses only date range searches.

enrichment_tables.file

required object

Configuration options for the file that provides the enrichment table.

enrichment_tables.file.encoding

required object

Configuration options for the encoding of the enrichment table’s file.

enrichment_tables.file.encoding.delimiter

optional string literal

The delimiter used to separate fields in each row of the CSV file.

Examples

":"

default: ,

enrichment_tables.file.encoding.include_headers

optional bool

Set include_headers to true if the first row of the CSV file contains the headers for each column. This is the default behavior.

If you set it to false, there are no headers and the columns are referred to by their numerical index.

default: true

enrichment_tables.file.encoding.type

common optional string literal

The encoding of the file. Currently, only CSV is supported.

default: csv

enrichment_tables.file.path

required string literal

The path of the enrichment table file. Currently, only CSV files are supported.

Examples

"/data/info.csv"

"./info.csv"

enrichment_tables.file.schema

common optional object

Key/value pairs representing mapped log field names and types. This is used to coerce log fields from strings into their proper types. The available types are listed in the Types list below.

Timestamp coercions need to be prefaced with timestamp|, for example "timestamp|%F". Timestamp specifiers can use either of the following:

One of the built-in-formats listed in the Timestamp Formats table below.
The time format specifiers from Rust’s chrono library.

Types

bool
string
float
integer
date
timestamp (see the table below for formats)

Timestamp Formats

Format	Description	Example
`%F %T`	`YYYY-MM-DD HH:MM:SS`	`2020-12-01 02:37:54`
`%v %T`	`DD-Mmm-YYYY HH:MM:SS`	`01-Dec-2020 02:37:54`
`%FT%T`	ISO 8601[RFC 3339](https://tools.ietf.org/html/rfc3339) format without time zone	`2020-12-01T02:37:54`
`%a, %d %b %Y %T`	RFC 822/2822 without time zone	`Tue, 01 Dec 2020 02:37:54`
`%a %d %b %T %Y`	`date` command output without time zone	`Tue 01 Dec 02:37:54 2020`
`%a %b %e %T %Y`	ctime format	`Tue Dec 1 02:37:54 2020`
`%s`	UNIX timestamp	`1606790274`
`%FT%TZ`	ISO 8601/RFC 3339 UTC	`2020-12-01T09:37:54Z`
`%+`	ISO 8601/RFC 3339 UTC with time zone	`2020-12-01T02:37:54-07:00`
`%a %d %b %T %Z %Y`	`date` command output with time zone	`Tue 01 Dec 02:37:54 PST 2020`
`%a %d %b %T %z %Y`	`date` command output with numeric time zone	`Tue 01 Dec 02:37:54 -0700 2020`
`%a %d %b %T %#z %Y`	`date` command output with numeric time zone (minutes can be missing or present)	`Tue 01 Dec 02:37:54 -07 2020`

Note: the examples in this table are for 54 seconds after 2:37 am on December 1st, 2020 in Pacific Standard Time.

Examples

{
  "duration": "float",
  "status": "int",
  "success": "bool",
  "timestamp_custom": "timestamp|%a %b %e %T %Y",
  "timestamp_iso8601": "timestamp|%F",
  "timestamp_unix": "timestamp|%F %T"
}

enrichment_tables.geoip

required object

Configuration options for MaxMind databases.

The following MaxMind databases are currently supported:

GeoLite2-ASN.mmdb (free) — Determine the autonomous system number and organization associated with an IP address.
GeoLite2-City.mmdb (free) — Determine the country, subdivisions, city, and postal code associated with IPv4 and IPv6 addresses worldwide.
GeoIP2-City.mmdb (paid) — Determine the country, subdivisions, city, and postal code associated with IPv4 and IPv6 addresses worldwide.
GeoIP2-ISP.mmdb (paid) — Determine the Internet Service Provider (ISP), organization name, and autonomous system organization and number associated with an IP address.

The database file should be in the MaxMind DB file format.

This enrichment table only supports lookup with IP address.

enrichment_tables.geoip.locale

optional string literal

The locale to use to lookup the country name and region name for the city database. See Locations Files

Examples

"de"

"en"

"es"

"fr"

"ja"

"pt-BR"

"ru"

"zh-CN"

default: en

enrichment_tables.geoip.path

required string literal

Path to the MaxMind GeoIP2 or GeoLite2 binary city database file (GeoLite2-City.mmdb). Other databases, such as the country database, are not supported.

Examples

"/path/to/GeoLite2-City.mmdb"

"/path/to/GeoLite2-ISP.mmdb"

expire_metrics

optional object

If set, Vector will configure the internal metrics system to automatically remove all metrics that have not been updated in the given time. This value must be positive.

expire_metrics.nsecs

common optional uint

The fractional number of seconds after which to expire metrics.

Examples

expire_metrics.secs

common optional uint

The whole number of seconds after which to expire metrics.

Examples

expire_metrics_secs

optional float

If set, Vector will configure the internal metrics system to automatically remove all metrics that have not been updated in the given number of seconds. This value must be positive.

Note that internal counters that are expired but are later updated will have their values reset to zero. Be careful to set this value high enough to avoid expiring critical but infrequently updated internal counters.

Examples

healthchecks

optional object

Configures health checks for all sinks.

healthchecks.enabled

common optional bool

Disables all health checks if false, otherwise sink specific option overrides it.

default: true

healthchecks.require_healthy

optional bool

Exit on startup if any sinks' health check fails. Overridden by --require-healthy command line flag.

default: false

log_schema

optional object

Configures default log schema for all events. This is used by Vector source components to assign the fields on incoming events.

log_schema.host_key

common optional string literal

Sets the event key to use for the event host field.

Examples

"host"

"@host"

default: host

log_schema.message_key

common optional string literal

Sets the event key to use for the event message field.

Examples

"message"

"@message"

default: message

log_schema.metadata_key

common optional string literal

Sets the event key to use for event metadata field (e.g. error or abort annotations in the remap transform).

Examples

"@metadata"

"meta"

default: metadata

log_schema.source_type_key

common optional string literal

Sets the event key to use for the event source type field that is set by some sources.

Examples

"source_type"

"@source_type"

default: source_type

log_schema.timestamp_key

common optional string literal

Sets the event key to use for the event timestamp field.

Examples

"timestamp"

"@timestamp"

default: timestamp

proxy

optional object

Configures an HTTP(S) proxy for Vector to use.

proxy.enabled

optional bool

If false the proxy will be disabled.

default: true

proxy.http

optional string literal

The URL to proxy HTTP requests through.

Examples

"http://foo.bar:3128"

proxy.https

optional string literal

The URL to proxy HTTPS requests through.

Examples

"http://foo.bar:3128"

proxy.no_proxy

optional [string]

A list of hosts to avoid proxying. Allowed patterns here include:

Pattern	Example match
Domain names	`example.com` matches requests to `example.com`
Wildcard domains	`.example.com` matches requests to `example.com` and its subdomains
IP addresses	`127.0.0.1` matches requests to 127.0.0.1
CIDR blocks	`192.168.0.0./16` matches requests to any IP addresses in this range
Splat	`*` matches all hosts

secret

optional object

Configuration options to retrieve secrets from external backend in order to avoid storing secrets in plaintext in Vector config. Currently, only the exec backend is supported. Multiple backends can be configured. To signify Vector that it should look for a secret to retrieve use the SECRET[<backend_name>.<secret_key>]. This placeholder will then be replaced by the secret retrieved from the relevant backend.

secret.exec

required object

Run a local command to retrieve secrets.

The provided command will be run and provided a list of secrets to fetch, determined from the configuration file, on stdin as JSON in the format:

{"version": "1.0", "secrets": ["secret1", "secret2"]}

The executable is expected to respond with the values of these secrets on stdout, also as JSON, in the format:

{
	"secret1": {"value": "secret_value", "error": null},
	"secret2": {"value": null, "error": "could not fetch the secret"}
}

If an error is returned for any secrets, or if the command exits with a non-zero status code, Vector will log the errors and exit.

Secrets will be loaded when Vector starts or if Vector receives a SIGHUP signal triggering its configuration reload process.

secret.exec.command

required [string]

The command to be run, plus any arguments required.

Examples

[
  "/path/to/get-secret",
  "-s"
]

[
  "/path/to/vault-wrappper"
]

secret.exec.timeout

optional uint

The amount of time Vector will wait for the command to complete.

default: 5 (seconds)

timezone

optional string literal

The name of the time zone to apply to timestamp conversions that do not contain an explicit time zone. The time zone name may be any name in the TZ database, or local to indicate system local time.

Examples

"local"

"America/NewYork"

"EST5EDT"

default: local