0.27 Upgrade Guide

Vector’s 0.27.0 release includes breaking changes:

1. statsd components now support enhanced metric tags
2. datadog_agent source now supports enhanced metric tags
3. Changes to native and native_json codecs

and potentially impactful changes:

1. Increase in possible tags sent with aws_cloudwatch_metrics sink

and deprecations:

1. aws_kinesis_firehose source’s access_key deprecated in favor of access_keys

We cover them below to help you upgrade quickly:

Upgrade guide

Breaking changes

statsd components now support enhanced metric tags

Prior to this release, the statsd source parsed incoming empty tags (tags with a key but no value), as having a boolean value “true”. For example, in the tagset “atag,btag:bvalue” , the tag “atag” would be parsed to a value of “true”. With this release, that behavior is now changed and the statsd source parses empty tags as empty tags.

Similarly, prior to this release, the statsd sink encoded metric tags with a boolean value of “true”, to being an empty tag. With this release, the statsd sink now correctly encodes a value of “true” to “true”, and empty tags as empty tags.

datadog_agent source sink now supports enhanced metric tags

Prior to this release, the datadog_agent source parsed incoming empty tags (tags with a key but no value) as empty strings. With this release, empty tags are properly parsed as empty tags.

Changes to native and native_json codecs

The native and native_json codecs have been modified to include support for enhanced tags (bare tags or multi-valued tags) on metrics by optionally encoding tag values as arrays of values. This will cause Vector, when using these codecs and a metric source or transform that generates enhanced tags, to emit metrics that are not backwards compatible with previous versions. Vector can continue to load in events emitted by older versions.

This has two potential implications that you should consider:

1. Disk buffers should be backed up if you want to be able to roll back to an older Vector version since new disk buffer entries may not be readable by older Vector versions. The disk buffers location can be found under the Vector data directory.

2. When upgrading Vector to Vector communication using the vector source and sink or the native or native_json codecs, make sure you upgrade the consumers first followed by the producers to ensure newer versions of Vector aren’t sending data to older versions, which may not be able to be read.

Potentially impactful changes

Increase in possible tags sent with aws_cloudwatch_metrics sink

Before this release, the aws_cloudwatch_metrics sink would only send up to ten tags to AWS as metric dimensions. This limit has been increased to thirty based on the current documentation.

This could increase the cardinality of your metrics by including previously dropped tags. The following VRL program would allow you to drop tags when there are more than ten, maintaining the original behavior.

count = 10
tags = {}
for_each(object!(.tags)) ->|key, value| {
    if count > 0 {
        count = count - 1
        tags = set!(tags, [key], value)
    }
}
.tags = tags

Deprecation Notices

aws_kinesis_firehose source’s access_key deprecated in favor of access_keys

Before this release, the aws_kinesis_firehose source would only allow matching against single configured access keys. Now multiple access keys can be configured via access_keys configuration option. For now, both access_key and access_keys are supported, via merging them to the single output parameter. Afterwards, only access_keys will be supported.