Global options reference
Global configuration parameters
acknowledgements
common optional objectControls how acknowledgements are handled for all sinks by default.
See End-to-end Acknowledgements for more information on how Vector handles event acknowledgement.
acknowledgements.enabled
optional boolWhether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements
configuration.
data_dir
optional string literalThe directory used for persisting Vector state data.
This is the directory where Vector will store any state data, such as disk buffers, file checkpoints, and more.
Vector must have write permissions to this directory.
/var/lib/vector/
enrichment_tables
optional objectConfiguration options for an enrichment table to be used in a
remap
transform. Currently supported are:
For the lookup in the enrichment tables to be as performant as possible, the data is indexed according to the fields that are used in the search. Note that indices can only be created for fields for which an exact match is used in the condition. For range searches, an index isn’t used and the enrichment table drops back to a sequential scan of the data. A sequential scan shouldn’t impact performance significantly provided that there are only a few possible rows returned by the exact matches in the condition. We don’t recommend using a condition that uses only date range searches.
enrichment_tables.file
required objecttype = "file"
enrichment_tables.file.encoding
required objectenrichment_tables.file.encoding.delimiter
optional string literal,
enrichment_tables.file.encoding.include_headers
optional boolWhether or not the file contains column headers.
When set to true
, the first row of the CSV file will be read as the header row, and
the values will be used for the names of each column. This is the default behavior.
When set to false
, columns are referred to by their numerical index.
true
enrichment_tables.file.encoding.type
required string literal enumOption | Description |
---|---|
csv | Decodes the file as a CSV (comma-separated values) file. |
enrichment_tables.file.path
required string literalThe path of the enrichment table file.
Currently, only CSV files are supported.
enrichment_tables.flush_interval
optional uintThe interval used for making writes visible in the table.
Longer intervals might get better performance,
but there is a longer delay before the data is visible in the table.
Since every TTL scan makes its changes visible, only use this value
if it is shorter than the scan_interval
.
By default, all writes are made visible immediately.
type = "memory"
enrichment_tables.internal_metrics
optional objecttype = "memory"
enrichment_tables.internal_metrics.include_key_tag
optional boolDetermines whether to include the key tag on internal metrics.
This is useful for distinguishing between different keys while monitoring. However, the tag’s cardinality is unbounded.
false
enrichment_tables.locale
optional string literalThe locale to use when querying the database.
MaxMind includes localized versions of some of the fields within their database, such as country name. This setting can control which of those localized versions are returned by the transform.
More information on which portions of the geolocation data are localized, and what languages are available, can be found here.
type = "geoip"
en
enrichment_tables.max_byte_size
optional uintMaximum size of the table in bytes. All insertions that make this table bigger than the maximum size are rejected.
By default, there is no size limit.
type = "memory"
enrichment_tables.path
required string literalPath to the MaxMind GeoIP2 or GeoLite2 binary city database file (GeoLite2-City.mmdb).
Other databases, such as the country database, are not supported.
mmdb
enrichment table can be used for other databases.
type = "geoip" or type = "mmdb"
enrichment_tables.scan_interval
optional uinttype = "memory"
30
enrichment_tables.schema
optional objectKey/value pairs representing mapped log field names and types.
This is used to coerce log fields from strings into their proper types. The available types are listed in the Types
list below.
Timestamp coercions need to be prefaced with timestamp|
, for example "timestamp|%F"
. Timestamp specifiers can use either of the following:
- One of the built-in-formats listed in the
Timestamp Formats
table below. - The time format specifiers from Rust’s
chrono
library.
Types
bool
string
float
integer
date
timestamp
(see the table below for formats)
Timestamp Formats
Format | Description | Example |
---|---|---|
%F %T | YYYY-MM-DD HH:MM:SS | 2020-12-01 02:37:54 |
%v %T | DD-Mmm-YYYY HH:MM:SS | 01-Dec-2020 02:37:54 |
%FT%T | ISO 8601/RFC 3339, without time zone | 2020-12-01T02:37:54 |
%FT%TZ | ISO 8601/RFC 3339, UTC | 2020-12-01T09:37:54Z |
%+ | ISO 8601/RFC 3339, UTC, with time zone | 2020-12-01T02:37:54-07:00 |
%a, %d %b %Y %T | RFC 822/RFC 2822, without time zone | Tue, 01 Dec 2020 02:37:54 |
%a %b %e %T %Y | ctime format | Tue Dec 1 02:37:54 2020 |
%s | UNIX timestamp | 1606790274 |
%a %d %b %T %Y | date command, without time zone | Tue 01 Dec 02:37:54 2020 |
%a %d %b %T %Z %Y | date command, with time zone | Tue 01 Dec 02:37:54 PST 2020 |
%a %d %b %T %z %Y | date command, with numeric time zone | Tue 01 Dec 02:37:54 -0700 2020 |
%a %d %b %T %#z %Y | date command, with numeric time zone (minutes can be missing or present) | Tue 01 Dec 02:37:54 -07 2020 |
type = "file"
enrichment_tables.schema.*
required string literalenrichment_tables.ttl
optional uinttype = "memory"
600
enrichment_tables.type
required string literal enumOption | Description |
---|---|
file | Exposes data from a static file as an enrichment table. |
geoip | Exposes data from a MaxMind GeoIP2 database as an enrichment table. |
memory | Exposes data from a memory cache as an enrichment table. The cache can be written to using a sink. |
mmdb | Exposes data from a MaxMind database as an enrichment table. |
expire_metrics
optional objectIf set, Vector will configure the internal metrics system to automatically remove all metrics that have not been updated in the given time.
If set to a negative value expiration is disabled.
expire_metrics.nsecs
common optional uintexpire_metrics.secs
common optional uintexpire_metrics_secs
optional floatThe amount of time, in seconds, that internal metrics will persist after having not been updated before they expire and are removed.
Set this to a value larger than your internal_metrics
scrape interval (default 5 minutes)
so metrics live long enough to be emitted and captured.
log_schema
optional objectDefault log schema for all events.
This is used if a component does not have its own specific log schema. All events use a log schema, whether or not the default is used, to assign event fields on incoming events.
log_schema.host_key
optional string literalThe name of the event field to treat as the host which sent the message.
This field will generally represent a real host, or container, that generated the message, but is somewhat source-dependent.
.host
log_schema.message_key
optional string literalThe name of the event field to treat as the event message.
This would be the field that holds the raw message, such as a raw log line.
.message
log_schema.metadata_key
optional string literalThe name of the event field to set the event metadata in.
Generally, this field will be set by Vector to hold event-specific metadata, such as
annotations by the remap
transform when an error or abort is encountered.
.metadata
log_schema.source_type_key
optional string literalThe name of the event field to set the source identifier in.
This field will be set by the Vector source that the event was created in.
.source_type
log_schema.timestamp_key
optional string literal.timestamp
proxy
optional objectProxy configuration.
Configure to proxy traffic through an HTTP(S) proxy when making external requests.
Similar to common proxy configuration convention, you can set different proxies to use based on the type of traffic being proxied. You can also set specific hosts that should not be proxied.
proxy.http
optional string literalProxy endpoint to use when proxying HTTP traffic.
Must be a valid URI string.
proxy.https
optional string literalProxy endpoint to use when proxying HTTPS traffic.
Must be a valid URI string.
proxy.no_proxy
optional [string]A list of hosts to avoid proxying.
Multiple patterns are allowed:
Pattern | Example match |
---|---|
Domain names | example.com matches requests to example.com |
Wildcard domains | .example.com matches requests to example.com and its subdomains |
IP addresses | 127.0.0.1 matches requests to 127.0.0.1 |
CIDR blocks | 192.168.0.0/16 matches requests to any IP addresses in this range |
Splat | * matches all hosts |
secret
optional objectConfiguration options to retrieve secrets from external backend in order to avoid storing secrets in plaintext
in Vector config. Multiple backends can be configured. Use SECRET[<backend_name>.<secret_key>]
to tell Vector to retrieve the secret. This placeholder is replaced by the secret
retrieved from the relevant backend.
When type
is exec
, the provided command will be run and provided a list of
secrets to fetch, determined from the configuration file, on stdin as JSON in the format:
{"version": "1.0", "secrets": ["secret1", "secret2"]}
The executable is expected to respond with the values of these secrets on stdout, also as JSON, in the format:
{
"secret1": {"value": "secret_value", "error": null},
"secret2": {"value": null, "error": "could not fetch the secret"}
}
If an error
is returned for any secrets, or if the command exits with a non-zero status code,
Vector will log the errors and exit.
Otherwise, the secret must be a JSON text string with key/value pairs. For example:
{
"username": "test",
"password": "example-password"
}
If an error occurred while reading the file or retrieving the secrets, Vector logs the error and exits.
Secrets are loaded when Vector starts or if Vector receives a SIGHUP
signal triggering its
configuration reload process.
secret.auth
optional objecttype = "aws_secrets_manager"
secret.auth.access_key_id
required string literalsecret.auth.assume_role
required string literalsecret.auth.credentials_file
required string literalsecret.auth.external_id
optional string literalsecret.auth.imds
optional objectsecret.auth.imds.max_attempts
optional uint4
secret.auth.load_timeout_secs
optional uintTimeout for successfully loading any credentials, in seconds.
Relevant when the default credentials chain or assume_role
is used.
secret.auth.profile
optional string literalThe credentials profile to use.
Used to select AWS credentials from a provided credentials file.
default
secret.auth.region
optional string literalThe AWS region to send STS requests to.
If not set, this defaults to the configured region for the service itself.
secret.auth.secret_access_key
required string literalsecret.auth.session_name
optional string literalThe optional RoleSessionName is a unique session identifier for your assumed role.
Should be unique per principal or reason. If not set, session name will be autogenerated like assume-role-provider-1736428351340
secret.command
required [string]Command arguments to execute.
The path to the script or binary must be the first argument.
type = "exec"
secret.endpoint
optional string literaltype = "aws_secrets_manager"
secret.path
required string literaltype = "file" or type = "directory"
secret.region
optional string literaltype = "aws_secrets_manager"
secret.remove_trailing_whitespace
optional booltype = "directory"
false
secret.secret_id
required string literaltype = "aws_secrets_manager"
secret.timeout
optional uinttype = "exec"
5
secret.tls
optional objecttype = "aws_secrets_manager"
secret.tls.alpn_protocols
optional [string]Sets the list of supported ALPN protocols.
Declare the supported ALPN protocols, which are used during negotiation with a peer. They are prioritized in the order that they are defined.
secret.tls.ca_file
optional string literalAbsolute path to an additional CA certificate file.
The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format.
secret.tls.crt_file
optional string literalAbsolute path to a certificate file used to identify this server.
The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format.
If this is set and is not a PKCS#12 archive, key_file
must also be set.
secret.tls.key_file
optional string literalAbsolute path to a private key file used to identify this server.
The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format.
secret.tls.key_pass
optional string literalPassphrase used to unlock the encrypted key file.
This has no effect unless key_file
is set.
secret.tls.server_name
optional string literalServer name to use when using Server Name Indication (SNI).
Only relevant for outgoing connections.
secret.tls.verify_certificate
optional boolEnables certificate verification. For components that create a server, this requires that the client connections have a valid client certificate. For components that initiate requests, this validates that the upstream has a valid certificate.
If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on, until the verification process reaches a root certificate.
Do NOT set this to false
unless you understand the risks of not verifying the validity of certificates.
secret.tls.verify_hostname
optional boolEnables hostname verification.
If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension.
Only relevant for outgoing connections.
Do NOT set this to false
unless you understand the risks of not verifying the remote hostname.
secret.type
required string literal enumOption | Description |
---|---|
aws_secrets_manager | AWS Secrets Manager. |
directory | Directory. |
exec | Exec. |
file | File. |
telemetry
optional objectTelemetry options.
Determines whether source
and service
tags should be emitted with the
component_sent_*
and component_received_*
events.
telemetry.tags
optional objecttelemetry.tags.emit_service
optional boolservice
tag should be emitted
in the component_received_*
and component_sent_*
telemetry.false
telemetry.tags.emit_source
optional boolsource
tag should be emitted
in the component_received_*
and component_sent_*
telemetry.false
timezone
optional string literalThe name of the time zone to apply to timestamp conversions that do not contain an explicit time zone.
The time zone name may be any name in the TZ database or local
to indicate system
local time.
Note that in Vector/VRL all timestamps are represented in UTC.