Vector v0.27.0 release notes

The Vector team is pleased to announce version 0.27.0!

This release includes the usual litany of smaller enhancements and bug fixes as well as:

  • Support for a new metric tag data model that supports a wider range of tags than the simple key/value pairs previously allowed including tags that are not key/value as well as tag keys that appear multiple times, with different values. See the release highlight for more details about this feature and how to enable it.
  • Support for tracing memory allocations within Vector to aid in troubleshooting Vector’s memory use. See the announcement blog post for more details about this feature and how to enable it.

Be sure to check out the upgrade guide for breaking changes in this release.

Upgrading Vector
When upgrading, we recommend stepping through minor versions as these can each contain breaking changes while Vector is pre-1.0. These breaking changes are noted in their respective upgrade guides.

Highlights

Known issues

  • Vector sources do not correctly tag the component_events_in_total and component_events_out_total internal metrics with their component tags (component_id, component_kind, and component_type). This affects reporting in vector top. Fixed in v0.27.1.
  • The log_schema.timestamp_key cannot be set to "" to suppress adding a timestamp. Fixed in 0.28.2.
  • TCP-based sources like socket, logstash, and fluent would sometimes panic when back-pressure logic calculated a lower limit on the number of incoming connections than 2, which is intended to be the minimum limit. Fixed in 0.28.2.

Changelog

14 enhancements

  • A new path option was added to loki sink to override the default URL path of /loki/api/v1/push. Thanks to Sh4d1 for contributing this change!

  • VRL no longer rejects querying an object returned by the merge function. This fixes programs that look like:

    .object.key = "some value"
. |= parse_key_value!(del(.message))
.object.key = "some other value" # previously would error
  • Supported enrichment table types are now output by vector list. Thanks to w4 for contributing this change!
  • The pulsar source now supports configuring a producer name to use via the new producer_name option. Thanks to zamazan4ik for contributing this change!
  • A new abs function was added to VRL to calculate the absolute value of a numeric value. Thanks to zamazan4ik for contributing this change!
  • The parse_cef now has a transform_custom_fields parameter that can be set to extract custom key/value fields from the CEF message. Thanks to ktff for contributing this change!
  • AWS components now support configuring custom IMDS request parameters that are used during authentication: auth.max_attempts, auth.connect_timeout, and auth.read_timeout. Thanks to kevinpark1217 for contributing this change!
  • The pulsar sink now supports compression via the standard compression option. Thanks to zamazan4ik for contributing this change!
  • The kubernetes_logs source now more efficiently parses CRI logs. Thanks to Ilmarii for contributing this change!
  • The nats source now allows adding the subject key to the event as metadata by setting subject_key_field to the name of the field you’d like to store the subject key in. Thanks to makarchuk for contributing this change!
  • The datadog_logs sink now allows configuring custom HTTP request headers. This is not typically needed when sending logs directly to Datadog.
  • The loki sink is now more efficient by sending events with multiple label sets in the same request rather than separating them. Thanks to atodekangae for contributing this change!
  • The kubernetes_logs source now supports the read_from and ignore_older_secs options that exist on the file sink which offer further control how Vector reads from the pod log files. Thanks to zamazan4ik for contributing this change!
  • The aws_kinesis_firehose source now supports configuring multiple valid access keys. This is useful during key rotation. Thanks to dizlv for contributing this change!

3 new features

  • Support for a new metric tag data model that supports a wider range of tags than the simple key/value pairs previously allowed including tags that are not key/value as well as tag keys that appear multiple times, with different values. See the release highlight for more details about this feature and how to enable it.

  • Vector has added support for tracking component memory allocations to help users understand which components may be allocating more memory than expected. This is an opt-in feature, due to ~20% performance overhead, available by passing --allocation-tracing when running Vector. When enabled, the internal_metrics source will publish new component_allocated_bytes, component_allocated_bytes_total, and component_deallocated_bytes_total metrics. These are will then also be viewable in vector top.

    There is a caveat to the reported metrics in that memory passed between components is not tracked as allocations. For example, if a source allocates memory to store incoming events, those events are passed along to a downstream component, the memory will still appear as though owned by the source component that allocated it. We hope to improve this in the future to track memory passed between component boundaries.

    See the announcement blog post for more details!

  • New decode_base16 and encode_base16 functions were added to VRL for interacting with base16 encoding. Thanks to WilliamApted-Org for contributing this change!

13 bug fixes

  • The kafka source and sink now accepts inline PEM-encoded certificates for tls.crt_file to match other sources and sinks (and the documented behavior). Thanks to nabokihms for contributing this change!
  • Vector avoids panicking when reloading under load. Thanks to Zettroke for contributing this change!
  • The kubernetes_logs source again disabling annotation of namespace labels by setting namespace_labels to "" again. This was regression in v0.26.0.
  • Vector now gracefully shutsdown if the Vector API cannot bind to the configured port. Thanks to zamazan4ik for contributing this change!
  • Vector now gracefully shutsdown if the a configured prometheus_exporter cannot bind to the configured port. Thanks to zamazan4ik for contributing this change!
  • Vector now gracefully shutsdown if the a configured http cannot bind to the configured port. Thanks to zamazan4ik for contributing this change!
  • The elasticsearch sink now accepts reading compressed responses. It uses the compression option to set an Accept-Encoding option in requests to Elasticsearch.
  • The aws_ec2_metadata transform is now capable of fetching instance tags by specifying the tags to fetch in the new tags option. Thanks to blefevre for contributing this change!
  • The aws_cloudwatch_metrics now supports sending 30 dimensions, rather than 10, as the AWS API now accepts this number of dimensions.
  • Use of the filter function in the VRL REPL no longer panics.
  • The http sink now allows overriding the Content-Type header via request.headers. Thanks to EdMcBane for contributing this change!
  • Vector now supports being run on operating systems with an older version of GNU libc (>=2.17) for the x86_64 architecture. This includes OSes such as CentOS 7, Amazon Linux 1, and Ubuntu 14.04. Thanks to apollo13 for contributing this change!
  • The remap transform now falls back to using the globally configured timezone if one is not set at the remap transform level.

Download Version 0.27.0

Linux
tar.gz
ARM64 ARMv7 x86_64
Linux (deb)
deb
ARM64 ARMv7 x86_64
Linux (rpm)
rpm
ARM64 ARMv7 x86_64
macOS
tar.gz
x86_64
Windows
zip
x86_64
Windows (MSI)
msi
x86_64
browse all files...