Global options reference

Global configuration parameters

acknowledgements

common optional object

Controls how acknowledgements are handled for all sinks by default.

See End-to-end Acknowledgements for more information on how Vector handles event acknowledgement.

Whether or not end-to-end acknowledgements are enabled.

When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.

Enabling or disabling acknowledgements at the sink level takes precedence over any global acknowledgements configuration.

data_dir

optional string literal

The directory used for persisting Vector state data.

This is the directory where Vector will store any state data, such as disk buffers, file checkpoints, and more.

Vector must have write permissions to this directory.

default: /var/lib/vector/

enrichment_tables

optional object

Configuration options for an enrichment table to be used in a remap transform. Currently supported are:

For the lookup in the enrichment tables to be as performant as possible, the data is indexed according to the fields that are used in the search. Note that indices can only be created for fields for which an exact match is used in the condition. For range searches, an index isn’t used and the enrichment table drops back to a sequential scan of the data. A sequential scan shouldn’t impact performance significantly provided that there are only a few possible rows returned by the exact matches in the condition. We don’t recommend using a condition that uses only date range searches.

enrichment_tables.file

required object
File-specific settings.
Relevant when: type = "file"
File encoding configuration.
The delimiter used to separate fields in each row of the CSV file.
default: ,

Whether or not the file contains column headers.

When set to true, the first row of the CSV file will be read as the header row, and the values will be used for the names of each column. This is the default behavior.

When set to false, columns are referred to by their numerical index.

default: true
enrichment_tables.file.encoding.type
required string literal enum
File encoding type.
Enum options
OptionDescription
csvDecodes the file as a CSV (comma-separated values) file.
Examples
"csv"
enrichment_tables.file.path
required string literal

The path of the enrichment table file.

Currently, only CSV files are supported.

The interval used for making writes visible in the table. Longer intervals might get better performance, but there is a longer delay before the data is visible in the table. Since every TTL scan makes its changes visible, only use this value if it is shorter than the scan_interval.

By default, all writes are made visible immediately.

Relevant when: type = "memory"
Configuration of internal metrics
Relevant when: type = "memory"

Determines whether to include the key tag on internal metrics.

This is useful for distinguishing between different keys while monitoring. However, the tag’s cardinality is unbounded.

default: false

enrichment_tables.locale

optional string literal

The locale to use when querying the database.

MaxMind includes localized versions of some of the fields within their database, such as country name. This setting can control which of those localized versions are returned by the transform.

More information on which portions of the geolocation data are localized, and what languages are available, can be found here.

Relevant when: type = "geoip"
default: en

Maximum size of the table in bytes. All insertions that make this table bigger than the maximum size are rejected.

By default, there is no size limit.

Relevant when: type = "memory"

enrichment_tables.path

required string literal

Path to the MaxMind GeoIP2 or GeoLite2 binary city database file (GeoLite2-City.mmdb).

Other databases, such as the country database, are not supported. mmdb enrichment table can be used for other databases.

Relevant when: type = "geoip" or type = "mmdb"
The scan interval used to look for expired records. This is provided as an optimization to ensure that TTL is updated, but without doing too many cache scans.
Relevant when: type = "memory"
default: 30

Key/value pairs representing mapped log field names and types.

This is used to coerce log fields from strings into their proper types. The available types are listed in the Types list below.

Timestamp coercions need to be prefaced with timestamp|, for example "timestamp|%F". Timestamp specifiers can use either of the following:

  1. One of the built-in-formats listed in the Timestamp Formats table below.
  2. The time format specifiers from Rust’s chrono library.

Types

  • bool
  • string
  • float
  • integer
  • date
  • timestamp (see the table below for formats)

Timestamp Formats

FormatDescriptionExample
%F %TYYYY-MM-DD HH:MM:SS2020-12-01 02:37:54
%v %TDD-Mmm-YYYY HH:MM:SS01-Dec-2020 02:37:54
%FT%TISO 8601/RFC 3339, without time zone2020-12-01T02:37:54
%FT%TZISO 8601/RFC 3339, UTC2020-12-01T09:37:54Z
%+ISO 8601/RFC 3339, UTC, with time zone2020-12-01T02:37:54-07:00
%a, %d %b %Y %TRFC 822/RFC 2822, without time zoneTue, 01 Dec 2020 02:37:54
%a %b %e %T %Yctime formatTue Dec 1 02:37:54 2020
%sUNIX timestamp1606790274
%a %d %b %T %Ydate command, without time zoneTue 01 Dec 02:37:54 2020
%a %d %b %T %Z %Ydate command, with time zoneTue 01 Dec 02:37:54 PST 2020
%a %d %b %T %z %Ydate command, with numeric time zoneTue 01 Dec 02:37:54 -0700 2020
%a %d %b %T %#z %Ydate command, with numeric time zone (minutes can be missing or present)Tue 01 Dec 02:37:54 -07 2020
Relevant when: type = "file"
enrichment_tables.schema.*
required string literal
Represents mapped log field names and types.
TTL (time-to-live in seconds) is used to limit the lifetime of data stored in the cache. When TTL expires, data behind a specific key in the cache is removed. TTL is reset when the key is replaced.
Relevant when: type = "memory"
default: 600

enrichment_tables.type

required string literal enum
enrichment table type
Enum options
OptionDescription
fileExposes data from a static file as an enrichment table.
geoipExposes data from a MaxMind GeoIP2 database as an enrichment table.
memoryExposes data from a memory cache as an enrichment table. The cache can be written to using a sink.
mmdbExposes data from a MaxMind database as an enrichment table.
Examples
"file"
"memory"
"geoip"
"mmdb"

expire_metrics

optional object

If set, Vector will configure the internal metrics system to automatically remove all metrics that have not been updated in the given time.

If set to a negative value expiration is disabled.

Warning

Deprecated, please use expire_metrics_secs instead.

expire_metrics.nsecs

common optional uint
The fractional number of seconds after which to expire metrics.
Examples
0

expire_metrics.secs

common optional uint
The whole number of seconds after which to expire metrics.
Examples
60

expire_metrics_secs

optional float

The amount of time, in seconds, that internal metrics will persist after having not been updated before they expire and are removed.

Set this to a value larger than your internal_metrics scrape interval (default 5 minutes) so metrics live long enough to be emitted and captured.

log_schema

optional object

Default log schema for all events.

This is used if a component does not have its own specific log schema. All events use a log schema, whether or not the default is used, to assign event fields on incoming events.

log_schema.host_key

optional string literal

The name of the event field to treat as the host which sent the message.

This field will generally represent a real host, or container, that generated the message, but is somewhat source-dependent.

default: .host

log_schema.message_key

optional string literal

The name of the event field to treat as the event message.

This would be the field that holds the raw message, such as a raw log line.

default: .message

log_schema.metadata_key

optional string literal

The name of the event field to set the event metadata in.

Generally, this field will be set by Vector to hold event-specific metadata, such as annotations by the remap transform when an error or abort is encountered.

default: .metadata

log_schema.source_type_key

optional string literal

The name of the event field to set the source identifier in.

This field will be set by the Vector source that the event was created in.

default: .source_type

log_schema.timestamp_key

optional string literal
The name of the event field to treat as the event timestamp.
default: .timestamp

proxy

optional object

Proxy configuration.

Configure to proxy traffic through an HTTP(S) proxy when making external requests.

Similar to common proxy configuration convention, you can set different proxies to use based on the type of traffic being proxied. You can also set specific hosts that should not be proxied.

proxy.enabled

optional bool
Enables proxying support.
default: true

proxy.http

optional string literal

Proxy endpoint to use when proxying HTTP traffic.

Must be a valid URI string.

Examples
"http://foo.bar:3128"

proxy.https

optional string literal

Proxy endpoint to use when proxying HTTPS traffic.

Must be a valid URI string.

Examples
"http://foo.bar:3128"

proxy.no_proxy

optional [string]

A list of hosts to avoid proxying.

Multiple patterns are allowed:

PatternExample match
Domain namesexample.com matches requests to example.com
Wildcard domains.example.com matches requests to example.com and its subdomains
IP addresses127.0.0.1 matches requests to 127.0.0.1
CIDR blocks192.168.0.0/16 matches requests to any IP addresses in this range
Splat* matches all hosts

secret

optional object

Configuration options to retrieve secrets from external backend in order to avoid storing secrets in plaintext in Vector config. Multiple backends can be configured. Use SECRET[<backend_name>.<secret_key>] to tell Vector to retrieve the secret. This placeholder is replaced by the secret retrieved from the relevant backend.

When type is exec, the provided command will be run and provided a list of secrets to fetch, determined from the configuration file, on stdin as JSON in the format:

{"version": "1.0", "secrets": ["secret1", "secret2"]}

The executable is expected to respond with the values of these secrets on stdout, also as JSON, in the format:

{
    "secret1": {"value": "secret_value", "error": null},
    "secret2": {"value": null, "error": "could not fetch the secret"}
}

If an error is returned for any secrets, or if the command exits with a non-zero status code, Vector will log the errors and exit.

Otherwise, the secret must be a JSON text string with key/value pairs. For example:

{
    "username": "test",
    "password": "example-password"
}

If an error occurred while reading the file or retrieving the secrets, Vector logs the error and exits.

Secrets are loaded when Vector starts or if Vector receives a SIGHUP signal triggering its configuration reload process.

secret.auth

optional object
Configuration of the authentication strategy for interacting with AWS services.
Relevant when: type = "aws_secrets_manager"
secret.auth.access_key_id
required string literal
The AWS access key ID.
Examples
"AKIAIOSFODNN7EXAMPLE"
secret.auth.assume_role
required string literal
The ARN of an IAM role to assume.
Examples
"arn:aws:iam::123456789098:role/my_role"
secret.auth.credentials_file
required string literal
Path to the credentials file.
Examples
"/my/aws/credentials"
secret.auth.external_id
optional string literal
The optional unique external ID in conjunction with role to assume.
Examples
"randomEXAMPLEidString"
secret.auth.imds
optional object
Configuration for authenticating with AWS through IMDS.
Connect timeout for IMDS.
default: 1 (seconds)
Number of IMDS retries for fetching tokens and metadata.
default: 4
Read timeout for IMDS.
default: 1 (seconds)

Timeout for successfully loading any credentials, in seconds.

Relevant when the default credentials chain or assume_role is used.

Examples
30
secret.auth.profile
optional string literal

The credentials profile to use.

Used to select AWS credentials from a provided credentials file.

Examples
"develop"
default: default
secret.auth.region
optional string literal

The AWS region to send STS requests to.

If not set, this defaults to the configured region for the service itself.

Examples
"us-west-2"
secret.auth.secret_access_key
required string literal
The AWS secret access key.
Examples
"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
secret.auth.session_name
optional string literal

The optional RoleSessionName is a unique session identifier for your assumed role.

Should be unique per principal or reason. If not set, session name will be autogenerated like assume-role-provider-1736428351340

Examples
"vector-indexer-role"

secret.command

required [string]

Command arguments to execute.

The path to the script or binary must be the first argument.

Relevant when: type = "exec"

secret.endpoint

optional string literal
Custom endpoint for use with AWS-compatible services.
Relevant when: type = "aws_secrets_manager"
Examples
"http://127.0.0.0:5000/path/to/service"

secret.path

required string literal
File path to read secrets from.
Relevant when: type = "file" or type = "directory"

secret.region

optional string literal
The AWS region of the target service.
Relevant when: type = "aws_secrets_manager"
Examples
"us-east-1"
Remove trailing whitespace from file contents.
Relevant when: type = "directory"
default: false

secret.secret_id

required string literal
ID of the secret to resolve.
Relevant when: type = "aws_secrets_manager"

secret.timeout

optional uint
The timeout, in seconds, to wait for the command to complete.
Relevant when: type = "exec"
default: 5

secret.tls

optional object
TLS configuration.
Relevant when: type = "aws_secrets_manager"
secret.tls.alpn_protocols
optional [string]

Sets the list of supported ALPN protocols.

Declare the supported ALPN protocols, which are used during negotiation with a peer. They are prioritized in the order that they are defined.

secret.tls.ca_file
optional string literal

Absolute path to an additional CA certificate file.

The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format.

Examples
"/path/to/certificate_authority.crt"
secret.tls.crt_file
optional string literal

Absolute path to a certificate file used to identify this server.

The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format.

If this is set and is not a PKCS#12 archive, key_file must also be set.

Examples
"/path/to/host_certificate.crt"
secret.tls.key_file
optional string literal

Absolute path to a private key file used to identify this server.

The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format.

Examples
"/path/to/host_certificate.key"
secret.tls.key_pass
optional string literal

Passphrase used to unlock the encrypted key file.

This has no effect unless key_file is set.

Examples
"${KEY_PASS_ENV_VAR}"
"PassWord1"
secret.tls.server_name
optional string literal

Server name to use when using Server Name Indication (SNI).

Only relevant for outgoing connections.

Examples
"www.example.com"

Enables certificate verification. For components that create a server, this requires that the client connections have a valid client certificate. For components that initiate requests, this validates that the upstream has a valid certificate.

If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on, until the verification process reaches a root certificate.

Do NOT set this to false unless you understand the risks of not verifying the validity of certificates.

Enables hostname verification.

If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension.

Only relevant for outgoing connections.

Do NOT set this to false unless you understand the risks of not verifying the remote hostname.

secret.type

required string literal enum
secret type
Enum options
OptionDescription
aws_secrets_managerAWS Secrets Manager.
directoryDirectory.
execExec.
fileFile.
Examples
"file"
"directory"
"exec"
"aws_secrets_manager"

telemetry

optional object

Telemetry options.

Determines whether source and service tags should be emitted with the component_sent_* and component_received_* events.

telemetry.tags

optional object
Configures whether to emit certain tags
True if the service tag should be emitted in the component_received_* and component_sent_* telemetry.
default: false
True if the source tag should be emitted in the component_received_* and component_sent_* telemetry.
default: false

timezone

optional string literal

The name of the time zone to apply to timestamp conversions that do not contain an explicit time zone.

The time zone name may be any name in the TZ database or local to indicate system local time.

Note that in Vector/VRL all timestamps are represented in UTC.

Examples
"local"
"America/New_York"
"EST5EDT"