Vector v0.28.0 release notes

The Vector team is pleased to announce version 0.28.0!

This is a smaller maintenance release primarily including bug fixes and small enhancements, while we do some background work to enable upcoming new features.

With this release we also completed an initiative to generate Vector’s reference documentation from the configuration structures in the code which will result in less inaccuracies in published documentation.

Be sure to check out the upgrade guide for breaking changes in this release.

Upgrading Vector
When upgrading, we recommend stepping through minor versions as these can each contain breaking changes while Vector is pre-1.0. These breaking changes are noted in their respective upgrade guides.

Highlights

Known issues

  • AWS components, except the aws_s3 sink, are not functional due to issues with request signing. This is fixed in v0.28.1.
  • The framing.*.max_length configuration options cannot be used on the socket source as Vector returns an error about them conflicting with the deprecated top-level max_length configuration option. This is fixed in v0.28.1.
  • The http_server source incorrectly defaults to GET rather than POST for method. Fixed in 0.28.2.
  • The elasticsearch sink panics when starting if bulk.index is unspecified and the default mode of bulk is used. Fixed in 0.28.2.
  • The syslog source incorrectly inserted the source IP of incoming messages as source_id rather than source_ip. Fixed in 0.28.2.

Vector Changelog

3 new features

  • encode_gzip and decode_gzip functions were added to VRL to interact with gzip’d data.
    Thanks to zamazan4ik for contributing this change!
  • encode_zlib and decode_zlib functions were added to VRL to interact with zlib compressed data.
    Thanks to zamazan4ik for contributing this change!
  • encode_zstd and decode_zstd functions were added to VRL to interact with zstd compressed data.
    Thanks to zamazan4ik for contributing this change!

12 enhancements

  • The azure_blob sink now allows setting a custom endpoint for use with alternative Azure clouds like USGov and China.
    Thanks to archoversight for contributing this change!
  • The clickhouse sink now supports a date_time_best_effort config option to have ClickHouse parse a greater variety of timestamps (like RFC3339).
    Thanks to DarkWanderer for contributing this change!
  • The http sink now supports payload_prefix and payload_suffix options to prepend and append text in the HTTP bodies it is sending. This happens after batches are encoded and so can be used, for example, to wrap the batches in a JSON envelope.
    Thanks to jdiebold for contributing this change!
  • The aws_kinesis_firehose now has a store_access_key, similar to the splunk_hec and datadog_agent sources, to store the token that the incoming request was sent with in the event secrets. This can be read later in VRL to drive behavior.
    Thanks to dizlv for contributing this change!
  • The reduce transform now has a max_events option that can be used to limit the total number of events in a reduced batch.
    Thanks to jches for contributing this change!
  • The kafka source now tries to commit offsets during shutdown to avoid duplicate processing on start-up.
    Thanks to aholmberg for contributing this change!
  • Vector no longer panics when attempting to create more than 254 allocation groups for memory allocation tracking. This would happen when there were more than 254 components in a config or during unit tests where a lot of components are spun up independently.
  • The reduce transform performance improved by only flushing when events were ready to be flushed and avoiding repeated checks for stale events.
    Thanks to dbcfd for contributing this change!
  • A seahash function was added to VRL for a non-cryptographic fast hash.
    Thanks to psemeniuk for contributing this change!
  • The pulsar sink now supports batching via the added batch.max_events configuration option.
    Thanks to zamazan4ik for contributing this change!
  • The kafka source is now capable of making consumer lag metrics available via the internal_metrics source. This can be enabled by setting metrics.topic_lag_metric to true. Note that this can result in high cardinality metrics given they are tagged with the topic and partition id.
    Thanks to zamazan4ik for contributing this change!
  • The redis source now retries failed requests with an exponential back-off rather than immediately (which can cause high resource usage).
    Thanks to hargut for contributing this change!

7 bug fixes

  • Vector reload better detects which components have changed, no longer reloading components that have configuration options that are maps where none of the keys or values changed.
    Thanks to aholmberg for contributing this change!
  • Loading of secrets in configuration now allows the same secret to be used multiple times. Previously this would result in Vector raising an error that the secret was undefined on the second or later uses.
  • Changing max_line_bytes on the file source no longer typically invalidates all previous checksums. It still will invalidate the checksum if the value is set to lower than the length of the line used that checksum but this should be much less common.
    Thanks to Ilmarii for contributing this change!
  • Vector GCP sinks now correctly refresh authentication tokens when healthchecks are disabled.
    Thanks to punkerpunker for contributing this change!
  • VRL now correctly calculates the type definitions of boolean expressions that short-circuit. This should avoid panics and the inability to write what should be valid VRL programs in rare situations.
  • The gcp_stackdriver_metrics now correctly encodes metrics to send to GCP.
    Thanks to jasonahills for contributing this change!
  • The axiom sink now always sets the timestamp-field header, which tells Axiom where to find the timestamp, to @timestamp rather than the configured log_schema.timestamp_key since Vector was always sending it as @timestamp.the upgrade guide for details.

5 chore

  • The deprecated disk_v1 buffer type was removed. See the upgrade guide for details and how to migrate if you are using this buffer type.
  • The deprecated units of the journald source was removed. include_units should be used instead. See the upgrade guide for more details.
  • Vector’s AWS components now use OpenSSL for the TLS implementation rather than rustls. This shouldn’t have any observable effects to end-users but allows for the use of these components on FIPS-compliant systems. See the upgrade guide for more details. Let us know if you see any issue related to this change.

  • The deprecated VRL functions for accessing metadata were removed:

    • get_metadata_field
    • set_metadata_field
    • delete_metadata_field

    In favor of the new metadata syntax of %<field name> to interact with these fields. See the upgrade guide for more details.

  • The apex sink was dropped given that the service no longer exists. See the upgrade guide for details.

Download Version 0.28.0

Linux
tar.gz
ARM64 ARMv7 x86_64
Linux (deb)
deb
ARM64 ARMv7 x86_64
Linux (rpm)
rpm
ARM64 ARMv7 x86_64
macOS
tar.gz
x86_64
Windows
zip
x86_64
Windows (MSI)
msi
x86_64
browse all files...