0.35 Upgrade Guide

Vector’s 0.35.0 release includes breaking changes:

1. The Throttle transform’s events_discarded_total internal metric is now opt-in
2. The file internal metric tag is now opt-in for file-based components
3. Datadog sinks now default the API key and Site to the value of environment variables
4. Removal of requests_completed_total, request_duration_seconds, and requests_received_total Internal Metrics

and deprecations:

1. Deprecation of undefined environment variable warnings

and potentially impactful changes:

1. HTTP server-based sources now include a keepalive.max_connection_age_secs config option that defaults to 5 minutes
2. Update component_sent_bytes_total to correctly report uncompressed bytes for all sinks
3. Update component_received_bytes_total to correctly report decompressed bytes for all sources
4. Update default values for the request.retry_max_duration_secs and request.rate_limit_num sink configuration options

We cover them below to help you upgrade quickly:

Upgrade guide

Breaking Changes

The Throttle transform’s events_discarded_total internal metric is now opt-in

The Throttle transform’s events_discarded_total internal metric, which includes the key tag, is now only emitted on an opt-in basis. Users can opt-in to emit this metric by setting internal_metrics.emit_events_discarded_per_key to true in the corresponding Throttle transform component config. This change is motivated by the fact that the key metric tag has potentially unbounded cardinality.

To view events discarded without the key tag, use the component_discarded_events_total internal metric.

The file internal metric tag is now opt-in for file-based components

File-based components (file source, Kubernetes logs source, file sink) now include a internal_metrics.include_file_tag config option that determines whether the file tag is included on the component’s corresponding internal metrics. This config option defaults to false, as this tag is likely to be of high cardinality.

Removal of requests_completed_total, request_duration_seconds, and requests_received_total Internal Metrics

The requests_completed_total, request_duration_seconds, and requests_received_total internal metrics were removed in the 0.35.0 release.

Deprecations

Deprecation of undefined environment variable warnings

Vector has the ability to interpolate environment variables into its configuration as a means of templating configuration. Currently, if an environment variable is undefined, unless the ? interpolation syntax is used, Vector outputs a warning and replaces the environment variable with an empty string. In a future release , this will change from a warning to a hard error. This behavior can be opted into earlier by using the --strict-env-vars flag (or VECTOR_STRICT_ENV_VARS environment variable) when running Vector.

The reasoning for this change is that users often miss the undefined variable warning and are confused by Vector not behaving as the expected. In particular, this happens when using $1 in regex capture groups in VRL without realizing they need to be escaped as $$1 to avoid interpolation.

Potentially impactful changes

HTTP server-based sources now include a keepalive.max_connection_age_secs config option that defaults to 5 minutes

HTTP server-based sources include a new keepalive.max_connection_age_secs configuration option, which defaults to 5 minutes (300 seconds). When enabled, this closes incoming TCP connections that reach the maximum age by sending a Connection: close header in the response. While this parameter is crucial for managing the lifespan of persistent, incoming connections to Vector and for effective load balancing, it can be disabled by setting keepalive.max_connection_age_secs to a very large number like 100000000 (ideally you could set it to null, but Vector currently requires all values to be valid TOML values, of which null is not).

Update component_sent_bytes_total to correctly report uncompressed bytes for all sinks

The AppSignal, Datadog Metrics, GreptimeDB, GCP Cloud Monitoring, Honeycomb, and HTTP sinks now correctly report uncompressed bytes, rather than compressed bytes, for the component_sent_bytes_total internal metric.

Update component_received_bytes_total to correctly report decompressed bytes for all sources

The Heroku Logs, HTTP Server, Prometheus Remote Write, and Splunk HEC sources now correctly report decompressed bytes, rather than compressed bytes, for the component_received_bytes_total internal metric.

Update default values for the request.retry_max_duration_secs and request.rate_limit_num sink configuration options

The request.retry_max_duration_secs config option previously defaulted to 3600 seconds. It now defaults to 30 seconds.

Also, a bug was fixed that prevented component-level default values from being applied. In particular, this updates the default value for request.rate_limit_num for the following sinks:

Note that all changes described above are reflected in the component reference documentation.

Datadog sinks now default the API key and Site to the value of environment variables

The Datadog Logs, Metrics, and Traces sinks now default the values of the default_api_key and site configuration options to the values of environment variables DD_API_KEY and DD_SITE, respectively.