Vector v0.10.0 release notes

We’re doing this integration right. We’ve been alpha testing with a number of large clusters and we’ve honed in on a very high-quality approach that reflects the UX Vector is going after. You can learn more in our Kubernetes Integration RFC. Do you want to beta test? Tell us!

WASM plugins 🔌

This release included an experimental implementation of a WASM transform behind a wasm build flag. We plan to stabilize WASM transforms, as well as add WASM sinks and sources, enabling you to extend Vector in the language of your choice with little performance penalty. ❤️ WASM. Check out the WASM Plugin RFC for more info.

Dynamic HTTP Rate-Limiting (AIMD)

Fiddling with rate-limits is a frustrating endeavor. If you set them too high you risk overwhelming the service; too low and you’re unnecessarily limiting throughput. What happens if you deploy a few more Vector instances? Vector is planning to solve this by automatically detecting the optimal rate limits, taking inspiration from TCP congestion control algorithms. Check out the Dynamic HTTP Rate Limiting RFC.

Changelog

26 enhancements

splunk_hec sink Add sourcetype

2318 topology Shutdown Vector if all sources finish

2533 filter transform swimlanes transform Add support for negating conditions in check_fields

2514 elasticsearch sink Add explicit AWS region

2532 journald source Add support for include/exclude units

2540 regex_parser transform Add RegexSet support to regex

2493 json_parser transform log parse errors as warnings

2593 http sink add bearer auth strategy

2607 security Add support for loading multiple CAs

2616 networking Add support for inline TLS keys

2724 humio sink new relic sink Add compression to humio and new_relic

2682 elasticsearch sink add pipeline config

2734 cli Consolidate and beautify validate

2622 gcp_stackdriver_logs sink Add severity key

2732 sinks Use to_raw_value instead of to_string in JsonArrayBuffer

2860 aws_s3 sink Add content options

2848 elasticsearch sink Show client errors as warning logs

2825 elasticsearch sink add assume_role to aws auth

2895 kafka source kafka sink Add SASL to kafka

2897 sinks Batch buffer rework

2866 journald source Allow remapping priority numbers to words

2751 file source Option to remove file after some time of reaching eof

2908 sinks add compression option for aws sinks

2953 kafka source Extract timestamp from event

2956 operations Add musl and glibc support to install.sh

2969 syslog source Accept octet-counting encoded messages

2955

12 new features

new source New generator source

2286 operations Optimize CI test workflow with slim builds

2440 kafka sink Add support for templates in the topic config

2598 internal_metrics source Instrument few more components with metrics

2620 new sink Initial influxdb_logs sink implementation

2474 platforms Add dev workflow simplification rfc

2685 new transform Introduce WASM Plugins

2006 operations Introduce Dev Workflow improvements

2723 filter transform Adding list of string as filter option

2745 operations More usable event RFC

2692 filter transform Adding a condition to filter using IP CIDR

2831 new transform add reduce transform

2870

44 bug fixes

kafka sink Run entire healthcheck lazily

2501 socket sink Properly detect shutdown

2429 loki sink replace OrigID with OrgID

2536 operations Iterate over target dirs to avoid missing mapall command on macOS

2539 datadog_metrics sink Use post requests

2547 splunk_hec sink Send time in . format

2570 humio_logs sink remove superfluous warning

2583 elasticsearch sink remove id key from body

2581 journald source Fix units/include_units switch

2567 socket sink Fix possible infinite loop decoding too-long lines

2587 aws_s3 sink Allow removed keys to be used for partitioning

2591 datadog_logs sink Enable tls by default

2527 tests Fixup loki tests

2613 tests Remove race between sink and source

2619 file source Fix the bug with channel closing at file server

2652 kafka sink kafa source Fix the Kafka TLS key parameter handling

2658 topology Check if transform/sink is present

2617 sources Shutdown starting from tcp and unix sockets

2618 influxdb_logs sink Fix buffer compression

2679 kafka sink Three KafkaCompression typos in tests

2694 splunk_hec sink Remove extra slash on Uri join

2711 pulsar sink Add new EncodingConfig

2671 sinks increment request id

2765 config Handle templates with invalid strftime elements

2737 aws_cloudwatch_sink Make throttling errors retryable

2752 aws_cloudwatch_logs sink Drop excessive large events

2770 sinks upgrade tower-limit to fix rate limiting

2779 kafka sink Fix typo integration tests

2832 sinks address potential deadlock

2838 influxdb_metrics sink Use i for unsigned ints on Influx v1

2868 splunk_hec source fix wrap path

2873 elasticsearch sink refresh aws creds when required

2859 aws_cloudwatch_logs sink Handle and enforce batch.max_bytes

2916 operations Fix the cmark-gfm installation at builder image

2996 operations Use perl rename at builder

2999 gcp provider sync token request

2993 loki sink Set a proper default for the batch buffer initialization

3016 deps update crates in lock file after merges

3030 cli Cleanup generate command

3046 docs adjust log event in coercer transform

3072 reduce transform improve array handling

3076 loki sink Drop the batch byte size limit

3025 file sink append instead of overwriting files

3084 elasticsearch sink stop logging every received event

3085