Vector v0.46.0 release notes

Upgrading Vector
When upgrading, we recommend stepping through minor versions as these can each contain breaking changes while Vector is pre-1.0. These breaking changes are noted in their respective upgrade guides.
Known issues
  • Several AWS integrations such as aws_kinesis_firehose sink, aws_kinesis_streams sink and aws_s3 source stopped working. This was reported in issue 22840.

The Vector team is pleased to announce version 0.46.0!

Release highlights:

  • A new postgres sink is now available and it supports logs, metrics and traces!
  • vector top now supports filtering out components by their component ID.
  • A new global option expire_metrics_per_metric_set is now available and it enables more fine-grained control over individual metric sets.

Vector Changelog

8 new features

  • vector top now supports filtering out components by their component ID using glob patterns with a new --components option. This is very similar to vector tap --outputs-of and --inputs-of options. This can be useful in cases where you have a lot of components and they don’t fit in the terminal (as scrolling is not supported yet in vector top). By default, all components are shown with a glob pattern of *.

    The glob pattern semantics can be found in the glob crate documentation.

    Example usage: vector top --components "demo*" will only show the components that match the glob pattern demo*.


    Thanks to jorgehermo9 for contributing this change!
  • Add a new postgres sink which allows to send log, metric and trace events to a postgres database.
    Thanks to jorgehermo9 for contributing this change!
  • Added a new optional global option expire_metrics_per_metric_set, enabling configuration of metrics expiration, similar to expire_metrics_secs, but enables defining different values per metric set, defined with a name and/or set of labels. expire_metrics_secs is used as a global default for sets not matched by this.
    Thanks to esensar, Quad9DNS for contributing this change!
  • Add the ability to run memory enrichment_table as a source, periodically dumping all the stored data and optionally removing it from the table.
    Thanks to esensar, Quad9DNS for contributing this change!
  • Add message buffering feature to websocket_server sink, that enables replaying missed messages to newly connected clients.
    Thanks to esensar, Quad9DNS for contributing this change!
  • websocket_server server sink component can now have customizable additional tags on metrics that it generates. They can hold fixed values, headers, client IPs and more.
    Thanks to esensar, Quad9DNS for contributing this change!
  • Adds a force_path_style option to the aws_s3 source, matching support added in the aws_s3 sink previously, that allows users to configure usage of virtual host-style bucket addressing. The value defaults to true to maintain existing (path-based addressing) behavior.
    Thanks to sbalmos for contributing this change!
  • The sample transform now accepts a ratio via a new ratio configuration parameter. This allow expressing the rate of forwarded events as a percentage.
    Thanks to graphcareful for contributing this change!

3 enhancements

  • The TLS crt_file and key_file from http sinks are now watched when --watch_config is enabled and therefore changes to those files will trigger a config reload without the need to restart Vector.
    Thanks to gllb for contributing this change!
  • Add support for static labels in gcp_stackdriver_logs sink. This enhancement enables users to define static labels directly in the gcp_stackdriver_logs sink configuration. Static labels are key-value pairs that are consistently applied to all log entries sent to Google Cloud Logging, improving log organization and filtering capabilities.
    Thanks to stackempty for contributing this change!
  • Add support for dynamic labels in gcp_stackdriver_logs sink via labels_key. This enhancement allows Vector to automatically map fields from structured log entries to Google Cloud LogEntry labels. When a structured log contains fields matching the configured labels_key, Vector will populate the corresponding labels in the Google Cloud LogEntry, enabling better log organization and filtering in Google Cloud Logging.
    Thanks to stackempty for contributing this change!

8 bug fixes

  • Prevent overflow when calculating the next refresh interval for Google Cloud Storage tokens.
    Thanks to graphcareful for contributing this change!
  • Prevent panic when an enrichment table has the same name as one of components.
    Thanks to esensar, Quad9DNS for contributing this change!
  • Fix an issue where using length_delimited encoder in framing.method, the last or only message is not correctly framed.
    Thanks to callum-ryan for contributing this change!
  • Fixed a bug in the aws_kinesis_firehose source, where the store_access_key option did not correctly store the access key.
    Thanks to noibu-gregory for contributing this change!
  • Fixed a bug in the aws_s3 sink, where the endpoint_url field in AWS_CONFIG_FILE was not respected by Vector.
    Thanks to ganelo for contributing this change!
  • Fixes a vector top bug was introduced in version 0.45 which prevented connections from being established.
    Thanks to JakubOnderka for contributing this change!
  • Fix a bug in the host_metrics source which caused the process_cpu_usage metric to always stay 0.
    Thanks to pront for contributing this change!
  • Fix potential panic in the host_metrics source when collecting TCP metrics.
    Thanks to pront for contributing this change!

VRL Changelog

VRL was updated to v0.23.0. This includes the following changes:

Breaking Changes

  • The ip_cidr_contains function now validates the cidr argument during the compilation phase if it is a constant string or array. Previously, invalid constant CIDR values would only trigger an error during execution.

Previously, if an invalid CIDR was passed as a constant, an error was thrown at runtime:

error[E000]: function call error for "ip_cidr_contains" at (0:45): unable to parse CIDR: couldn't parse address in network: invalid IP address syntax
┌─ :1:1
│
1 │ ip_cidr_contains!("INVALID", "192.168.10.32")
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ unable to parse CIDR: couldn't parse address in network: invalid IP address syntax
│
= see language documentation at https://vrl.dev
= try your code in the VRL REPL, learn more at https://vrl.dev/examples

Now, we see a compilation error:

error[E610]: function compilation error: error[E403] invalid argument
┌─ :1:1
│
1 │ ip_cidr_contains!("INVALID", "192.168.10.32")
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
│ │
│ invalid argument "ip_cidr_contains"
│ error: "cidr" must be valid cidr
│ received: "INVALID"
│
= learn more about error code 403 at https://errors.vrl.dev/403
= see language documentation at https://vrl.dev
= try your code in the VRL REPL, learn more at https://vrl.dev/examples

This change improves error detection by identifying invalid CIDR values earlier, reducing unexpected failures at runtime and provides better performance.

New Features

  • Support for encoding and decoding lz4 block compression with the encode_lz4 and decode_lz4 functions.

Enhancements

  • The encode_proto function was enhanced to automatically convert integer, float, and boolean values when passed to string proto fields. (https://github.com/vectordotdev/vrl/pull/1304)
  • The parse_user_agent method now uses the ua-parser library which is much faster than the previous library. The method’s output remains unchanged.
  • Added support for excluded_boundaries in the snakecase() function. This allows users to leverage the same function snakecase() that they’re already leveraging but tune it to handle specific scenarios where default boundaries are not desired.

For example,

snakecase("s3BucketDetails", excluded_boundaries: ["digit_lower", "lower_digit", "upper_digit"])
// Output: s3_bucket_details

Fixes

  • The parse_nginx_log function can now parse delaying requests error messages.

Download Version 0.46.0

Linux
tar.gz
ARM64 ARMv7 x86_64
Linux (deb)
deb
ARM64 ARMv7 x86_64
Linux (rpm)
rpm
ARM64 ARMv7 x86_64
macOS
tar.gz
x86_64 ARM64
Windows
zip
x86_64
Windows (MSI)
msi
x86_64
browse all files...