Global options reference
Global configuration parameters
acknowledgements
common optional objectacknowledgements.enabled
common optional booltrue
, all capable sources will have acknowledgements enabled.false
data_dir
optional string literal/var/lib/vector/
enrichment_tables
optional objectConfiguration options for an enrichment table to be used in a
remap
transform. Currently supported are:
For the lookup in the enrichment tables to be as performant as possible, the data is indexed according to the fields that are used in the search. Note that indices can only be created for fields for which an exact match is used in the condition. For range searches, an index isn’t used and the enrichment table drops back to a sequential scan of the data. A sequential scan shouldn’t impact performance significantly provided that there are only a few possible rows returned by the exact matches in the condition. We don’t recommend using a condition that uses only date range searches.
enrichment_tables.file
required objectenrichment_tables.file.encoding
required objectenrichment_tables.file.encoding.delimiter
optional string literal,
enrichment_tables.file.encoding.include_headers
optional boolSet include_headers
to true
if the first row of the CSV file contains the
headers for each column. This is the default behavior.
If you set it to false
, there are no headers and the columns are referred to
by their numerical index.
true
enrichment_tables.file.encoding.type
common optional string literalcsv
enrichment_tables.file.path
required string literalenrichment_tables.file.schema
common optional objectKey/value pairs representing mapped log field names and types. This is used to coerce log fields from strings into their proper types. The available types are listed in the Types list below.
Timestamp coercions need to be prefaced with timestamp|
, for example "timestamp|%F"
.
Timestamp specifiers can use either of the following:
- One of the built-in-formats listed in the Timestamp Formats table below.
- The time format specifiers from Rust’s
chrono
library.
Types
bool
string
float
integer
date
timestamp
(see the table below for formats)
Timestamp Formats
Format | Description | Example |
---|---|---|
%F %T | YYYY-MM-DD HH:MM:SS | 2020-12-01 02:37:54 |
%v %T | DD-Mmm-YYYY HH:MM:SS | 01-Dec-2020 02:37:54 |
%FT%T | ISO 8601/RFC 3339 format without time zone | 2020-12-01T02:37:54 |
%a, %d %b %Y %T | RFC 822/2822 without time zone | Tue, 01 Dec 2020 02:37:54 |
%a %d %b %T %Y | date command output without time zone | Tue 01 Dec 02:37:54 2020 |
%a %b %e %T %Y | ctime format | Tue Dec 1 02:37:54 2020 |
%s | UNIX timestamp | 1606790274 |
%FT%TZ | ISO 8601/RFC 3339 UTC | 2020-12-01T09:37:54Z |
%+ | ISO 8601/RFC 3339 UTC with time zone | 2020-12-01T02:37:54-07:00 |
%a %d %b %T %Z %Y | date command output with time zone | Tue 01 Dec 02:37:54 PST 2020 |
%a %d %b %T %z %Y | date command output with numeric time zone | Tue 01 Dec 02:37:54 -0700 2020 |
%a %d %b %T %#z %Y | date command output with numeric time zone (minutes can be missing or present) | Tue 01 Dec 02:37:54 -07 2020 |
Note: the examples in this table are for 54 seconds after 2:37 am on December 1st, 2020 in Pacific Standard Time.
enrichment_tables.geoip
required objectConfiguration options for MaxMind databases.
The following MaxMind databases are currently supported:
- GeoLite2-ASN.mmdb (free) — Determine the autonomous system number and organization associated with an IP address.
- GeoLite2-City.mmdb (free) — Determine the country, subdivisions, city, and postal code associated with IPv4 and IPv6 addresses worldwide.
- GeoIP2-City.mmdb (paid) — Determine the country, subdivisions, city, and postal code associated with IPv4 and IPv6 addresses worldwide.
- GeoIP2-ISP.mmdb (paid) — Determine the Internet Service Provider (ISP), organization name, and autonomous system organization and number associated with an IP address.
- GeoIP2-Anonymous-IP.mmdb (paid) — Determine proxy, VPN, hosting, and other anonymous IP addresses.
The database file should be in the MaxMind DB file format.
This enrichment table only supports lookup with IP address.
enrichment_tables.geoip.locale
optional string literalen
enrichment_tables.geoip.path
required string literalenrichment_tables.mmdb
required objectConfiguration options for generic MaxMind databases.
The database file should be in the MaxMind DB file format.
This enrichment table only supports lookup with IP address.
enrichment_tables.mmdb.path
required string literalenrichment_tables.type
required string literal enumexpire_metrics
optional objectexpire_metrics.nsecs
common optional uintexpire_metrics.secs
common optional uintexpire_metrics_secs
optional floatIf set, Vector will configure the internal metrics system to automatically remove all metrics that have not been updated in the given number of seconds. This value must be positive.
Note that internal counters that are expired but are later updated will have their values reset to zero. Be careful to set this value high enough to avoid expiring critical but infrequently updated internal counters.
healthchecks
optional objecthealthchecks.enabled
common optional booltrue
healthchecks.require_healthy
optional bool--require-healthy
command line flag.false
log_schema
optional objectlog_schema.host_key
common optional string literalhost
log_schema.message_key
common optional string literalmessage
log_schema.metadata_key
common optional string literalremap
transform).metadata
log_schema.source_type_key
common optional string literalsource_type
log_schema.timestamp_key
common optional string literaltimestamp
proxy
optional objectproxy.http
optional string literalproxy.https
optional string literalproxy.no_proxy
optional [string]A list of hosts to avoid proxying. Allowed patterns here include:
Pattern | Example match |
---|---|
Domain names | example.com matches requests to example.com |
Wildcard domains | .example.com matches requests to example.com and its subdomains |
IP addresses | 127.0.0.1 matches requests to 127.0.0.1 |
CIDR blocks | 192.168.0.0./16 matches requests to any IP addresses in this range |
Splat | * matches all hosts |
schema
optional objectschema.log_namespace
optional boolfalse
secret
optional objectSECRET[<backend_name>.<secret_key>]
to tell Vector to retrieve the secret. This placeholder is replaced by the secret
retrieved from the relevant backend.secret.aws_secrets_manager
required objectRetrieve secrets from AWS Secrets Manager.
The secret must be a JSON text string with key/value pairs. For example:
{
"username": "test",
"password": "example-password"
}
If an error occurred retrieving the secrets, Vector logs the error and exits.
Secrets are loaded when Vector starts or if Vector receives a SIGHUP
signal triggering its
configuration reload process.
secret.aws_secrets_manager.secret_id
required string literalsecret.exec
required objectRun a local command to retrieve secrets.
The provided command will be run and provided a list of secrets to fetch, determined from the configuration file, on stdin as JSON in the format:
{"version": "1.0", "secrets": ["secret1", "secret2"]}
The executable is expected to respond with the values of these secrets on stdout, also as JSON, in the format:
{
"secret1": {"value": "secret_value", "error": null},
"secret2": {"value": null, "error": "could not fetch the secret"}
}
If an error
is returned for any secrets, or if the command exits with a non-zero status code,
Vector will log the errors and exit.
Secrets are loaded when Vector starts or if Vector receives a SIGHUP
signal triggering its
configuration reload process.
secret.exec.command
required [string]secret.exec.timeout
optional uint5
(seconds)telemetry
optional objecttelemetry.tags
optional objectvector_component_sent_events_total
and
vector_component_sent_event_bytes_total
metrics.telemetry.tags.emit_service
optional boolAdds a service
tag with the service component the event was received from.
For logs this is the field that has been determined to mean service
. Each source may
define different fields for this. For example, with syslog
events the appname
field
is used.
Metric events will use the tag named service
.
If no service is available a -
is emitted for this tag.
false
telemetry.tags.emit_source
common optional boolAdd a source
tag with the source component the event was received from.
If there is no source component, for example if the event was generated by
the lua
transform a -
is emitted for this tag.
false
timezone
optional string literallocal
to indicate system local time.local