Vector v0.29.0 release notes

The Vector team is pleased to announce version 0.29.0!

In addition to the usual smaller enhancements and bug fixes, this release also includes:

A new appsignal sink to send logs and metrics to AppSignal
A new databend sink to send logs to Databend
A new webhdfs sink to send log data to HDFS via the WebHDFS interface.
A new csv encoding option for sinks to encode events as CSV

Be sure to check out the upgrade guide for breaking changes in this release.

Special thanks to Xuanwo for collaborating with us to introduce the OpenDAL library into Vector, starting with the webhdfs sink, to prove out the ability to use it to more easily support additional Vector sinks in the future.

Additionally, for Vector developers, two new tutorials were added for developing Vector sinks: docs. We hope this makes it easier to add Vector sinks in the future.

Upgrading Vector

When upgrading, we recommend stepping through minor versions as these can each contain breaking changes while Vector is pre-1.0. These breaking changes are noted in their respective upgrade guides.

Highlights

0.29 Upgrade Guide

April 11, 2023

type: breaking change

Known issues

Certain configurations containing a chain of at least two remap transforms no longer result in a panic at startup. Fixed in 0.29.1.

Changelog

8 enhancements

The http_server source now supports receiving requests using zstd compression. Thanks to zamazan4ik for contributing this change!
The socket source now has a max_connection_duration_secs option that will terminate connections after the specified number of seconds. This is useful to force clients to reconnect when connecting through a load balancer. Thanks to joscha-alisch for contributing this change!
A new webhdfs sink was added to send log data to HDFS via the WebHDFS interface.
This sink is also the first to use the new OpenDAL library which may enable us to more quickly add support for other sinks in the future as that library expands storage support.
Thanks to Xuanwo for contributing this change!
The loki sink now supports sending a field containing a map as labels with configuration like:
```
labels:
	"*": "{{ labels }}"
```
to send the key/value map stored at labels as the set of labels to Loki.
Thanks to hargut for contributing this change!
AWS components can now also assume-role whenever static credentials are provided in the configuration file through the auth.assume_role option. Thanks to sbalmos for contributing this change!
The deprecated api_key option was removed from Datadog sinks. It had been replaced by default_api_key.
Vector now logs signals that it receives, and handles, at the INFO level. Unhandled signals are ignored. Thanks to zamazan4ik for contributing this change!
An hmac function was added to VRL to calculate HMACs. Thanks to sbalmos for contributing this change!

7 new features

A new databend sink was added to forward logs to Databend. Thanks to everpcpc for contributing this change!
A new csv encoding option was added for sinks. Example configuration:
```
encoding:
	codec: csv
	csv:
		fields:
		- field1
		- field2
```
This will encode the event as CSV using the field1 and field2 fields, in order.
Note that CSV headers are not added. This requires batch header support that will be added in the future.
Thanks to everpcpc for contributing this change!
The kubernetes_logs source now adds container_image_id as metadata to log events. Thanks to nabokihms for contributing this change!
A few functions to generate random values were added to VRL:
- random_bool: generate a random boolean
- random_int: generate a random integer
- random_float: generate a random float
The aws_ecs_metrics source now collects precpu_online_cpus stats. Thanks to asingh072318 for contributing this change!
The file source now handles acknowledgements when Vector is shutting down and sinks are flushing events. Previously the file source would ignore all acknowledgements once Vector starts to shut down. Thanks to jches for contributing this change!
A new appsignal sink was added to forward logs and metrics to AppSignal. Thanks to tombruijn for contributing this change!

10 bug fixes

Vector components can now bind to the same port on different IP addresses including different IP address families. Previously Vector would report that the addresses conflicted. Thanks to ruhi-stripe for contributing this change!
The amqp source and sink now support TLS connections.
Support for AWS credential files was re-added for all AWS components. This was dropped during the migration of the SDK we were using from rusoto to AWS’s official Rust SDK. Thanks to protochron for contributing this change!
The aws_s3 source no longer deadlocks under low client_concurrency when acknowledgements are enabled. Thanks to mdeltito for contributing this change!
The statsd sink now supports encoding aggregate histograms. Not supported, yet, is encoding sketches as histograms. Thanks to pedro-stanaka for contributing this change!
Encoding of timestamps by the gelf codec was fixed to correctly encode as an integer rather than an an RFC3339 string. Thanks to scMarkus for contributing this change!
The distributed SystemD unit file now has Restart set to Always to match expected user behavior when Vector exits. Thanks to KannarFr for contributing this change!
Comparisons of float fields in Vector was fixed to completely compare the value. Previously it was only comparing the integer portion. This mostly came up in VRL when doing field comparisons like .foo == .bar where .foo and .bar were float values.
The prometheus_scrape source now handles larger counts for summary metrics (up to uint64 instead of uint32). Thanks to uralmetal for contributing this change!
The kubernetes_logs source now backs-off when retrying failed requests to the Kubernetes API. events. Thanks to nabokihms for contributing this change!

Vector v0.29.0 release notes

Highlights

Known issues

Changelog

8 enhancements

7 new features

10 bug fixes

Download Version 0.29.0