The Vector team is excited to announce version 0.49.0
!
Please refer to the upgrade guide for breaking changes in this release.
Release highlights:
websocket
source was introduced. A WebSocket source in Vector enables ingestion of real-time data from services that expose WebSocket APIs.http
sink’s uri
and request.headers
config fields now support templating, enabling dynamic construction based on event data.--watch-config
flag now also watches for changes in enrichment table files.vector_buffer_byte_size
and vector_buffer_events
gauges.prometheus_remote_write
sink now offers a expire_metrics_secs
config option. This fixes an issue where incremental metrics were preserved for the lifetime of Vector’s runtime causing indefinite memory growth.http
sink’s uri
config is now templateable, allowing for dynamic URI building based on event fields.query_settings
and async_insert_settings
option in the clickhouse
sink, which allows users to configure asynchronous inserts.--watch-config
flag now also watches for changes in enrichment table files.time_settings
configuration to the dedupe
transform, allowing the max_age
of items in the deduplication cache to be set. This helps distinguish between true duplicates and expected repetition in data over longer periods of time.max_packet_size
option to set max packet size for the mqtt
source and sink.websocket
source.max_size
configuration option for memory buffers.request_retry_partial
behavior for the elasticsearch
sink was changed. Now only the failed retriable requests in a bulk is retried (instead of all requests).expire_metrics_secs
configuration to the Prometheus remote write sink, leveraging the new TTL-based cache.redis
sink.redis
sink.aws_s3
sink by allowing users to configure which requests to retry.http
sink request.headers
configuration now supports templated values.The nats
sink now supports message headers when publishing to JetStream.
It introduces a configurable, templated Nats-Msg-Id header that ensures a unique ID for each message. This enables broker-level deduplication, resulting in stronger delivery guarantees and exactly-once semantics when combined with idempotent consumers.
protobuf
codecs now support all telemetry data types (logs, metrics, traces).UnsignedIntTemplate
now supports strftime
formatting. For example, this %Y%m%d%H
template evaluates timestamps to a number.utilization
metric is now properly published periodically, even when there are no events flowing through the components.elasticsearch
sink that caused URI credentials to be ignored. They are now correctly used.postgres
sink healthcheck was not implemented correctly. Now Vector can start when healthcheck.enabled
is set to false
.nats
sink now does not return an error when an unresolvable or unavailable URL is provided.Note: If --require-healthy
is set, Vector stops on startup.interval_ms
field. This field was previously writeable but not readable.metrics-tag-values
was set to full
.dnstap
and tcp
sources where throughput could drop significantly when the number of connections exceeded the number of available cores.vector_buffer_byte_size
and vector_buffer_events
gauges.DD_API_KEY
value contains newline characters. Such values are now handled gracefully.log_to_metric
configuration bug where the all_metrics
field could not be used without also specifying metrics
. It can now be set independently.log_to_metric
transforms now emits a pair expansion error. This error was previously silently ignored.heroku_logs
and demo_logs
sinks could output logs, metrics and traces depending on the decoding. Now they can only output logs.
This behavior was unintuitive and undocumented.The parse_cef
now trims unnecessary whitespace around escaped values in both headers and extension fields, improving accuracy and reliability when dealing with messy input strings.
authors: yjagdale (https://github.com/vectordotdev/vrl/pull/1430)
The parse_syslog
function now treats RFC 3164 structured data items with no parameters (e.g., [exampleSDID@32473]
) as part of the main
message, rather than parsing them as structured data. Items with parameters (e.g., [exampleSDID@32473 field="value"]
) continue to be
parsed as structured data. (https://github.com/vectordotdev/vrl/pull/1435)
encode_lz4
no longer prepends the uncompressed size by default, improving compatibility with standard LZ4 tools. A new prepend_size
flag restores the old behavior if needed. Also, decode_lz4
now also accepts prepend_size
and a buf_size
option (default: 1MB).
authors: jlambatl (https://github.com/vectordotdev/vrl/pull/1447)
Added haversine
function for calculating haversine distance and bearing.
authors: esensar Quad9DNS (https://github.com/vectordotdev/vrl/pull/1442)
Add validate_json_schema
function for validating JSON payloads against JSON schema files. An optional configuration parameter ignore_unknown_formats
is provided to change how custom formats are handled by the validator. Unknown formats can be silently ignored by setting this to true
and validation continues without failing due to those fields.
authors: jlambatl (https://github.com/vectordotdev/vrl/pull/1443)
Sign up to receive emails on the latest Vector content and new releases
Thank you for joining our Updates Newsletter