Vector v0.8.0 Release Notes
A wholesome platter of features,
February 20th, 2020
The easiest way to stay up-to-date. One email on the 1st of every month. No spam, ever.
The features for this release have been pouring in. Clearly the
Vector team need better hobbies.
We're excited to release Vector v0.8.0! Vector follows
semantic versioning, and this is an initial dev release. This release brings 14 new features, 24 enhancements, 11 bug fixes. Checkout the highlights for notable features and, as always, let us know what you think! Highlights
Automatically Merge Split Docker Events!
Anyone that was worked with Docker logs knows how frustrating this problem
can be. Docker, by default, splits log messages that exceed 16kb. While 16kb
seems like a lot, it can easily be exceeded if you're logging rich structured
events. This can be a very difficult and frustrating problem to solve with
other tools (we speak from experience). In this release Vector solves this
automatically with a new `auto_partial_merge` option in the `docker` source.
We love assimilation and look forward to a future where our individualistic
human personalities can also be merged into a societal hive mind.
Improved Control Flow With Swimlanes
makes it much easier to configure conditional branches of
transforms and sinks. For example, you can easily create
Remember to occasionally let your branches mingle so that they don't completely
Config Unit Test Improvements
Unit testing your configuration files has been supported by Vector for a while
now. Based on feedback we've received and some healthy dog-fooding we've
expanded their functionality and made a lot of improvements to how they work.
If you aren't yet using unit tests in your configs we strongly recommend you try
it out. You can find out more in
Custom Field Schemas
Not everyone enjoys Marmite, we get it, some people have
wrong different tastes.
Similarly, not everyone is going to want to follow our default field schema
(`host`, `message`, `timestamp`). Rather than making you work around this with
transforms we've added some handy global options to set these universally:
# OPTIONAL - Log schema
host_key = "host" # default
message_key = "message" # default
timestamp_key = "timestamp" # default
Support for Multiple Config Files
Vector is now able to execute a topology spread across multiple config files,
which allows you to break large pipelines down into bite size, easier managed,
chunks. Running them is as simple as:
vector -c ./configs/first.toml -c ./configs/second.toml -c ./more/*.toml
Subscribe to our newsletter and you'll be notified when we learn how to do this
with human emotions.
14 New Features
cli Support multiple configuration files. config Option to reload config on file change config Global default log schemas kubernetes source Add more `include_container_names`, `include_pod_uids`, and `include_namespaces` filters new sink Initial `gcp_stackdriver_logging` sink implementation new sink Initial `logdna` sink implementation new sink Initial `humio_logs` sink implementation new sink Initial `loki` sink new sink Initial `influxdb_metrics` sink implementation new sink Initial `gcp_cloud_storage` sink implementation new transform New `merge` transform new transform Initial `rename_fields` transform implementation new transform Add `swimlanes` transform socket sink Add unix domain socket support to the `socket` sink
add_fields transform Support templating for dynamic values aws provider Add support for `assume_role` to all AWS sinks aws_cloudwatch_logs sink Enable more logging by default aws_s3 sink Add several S3 object-level options (`grant_full_control`, `server_side_encryption`, and more) cli Improve `vector generate` syntax and formatting cli Test sub command improvements clickhouse sink Add `timestamp_format` field coercer transform Add new `drop_unspecified` option to drop unspecified fields config Clarify topology warnings config Allow multiple unit test inputs config Add `no_outputs_from` field to unit tests docker source Add `auto_partial_merge` to automatically merge partial events file source Better multi-line support kafka sink Support advanced `librdkafka` options kafka source Support advanced `librdkafka` options kafka source Add support for TLS log data model Support `Array` and `Map` kinds in `Value` log data model Support null values in log events observability Warn log for non retryable requests observability Add proper non retry messages observability Enable `rdkafka` by default syslog source Improve best effort support for Syslog formats (5424, 3164, nginx style, etc) topology Improve cyclic deps error message
11 Bug Fixes
aws_kinesis_firehose sink Use `describe_delivery_stream` AWS API method for healthcheck aws_s3 sink Fix a bug with `retry_limit` is set to 1 docker source Always check containers for self journald source Fix the journalctl start date to work across all timezones json_parser transform only drop source field if JSON parse succeeds kubernetes source Use checksum fingerprinting lua transform Respect Lua types when converting back to events (#857) rpm platform Create `vector` user when installing RPM package rpm platform Fix group creation in the RPM package sematext sink Fix EU endpoint and docs socket sink Allow for non-IP address host names