Vector v0.7.0 Release Notes
You can't make an omelet without cracking some eggs. We have our eyes set on version 1 and a backlog of breaking changes we wanted to merge before we reach it. Version 0.7.0 is our attempt to cluster as many breaking changes as we can into a single release.
We're excited to release Vector v0.7.0! Vector follows semantic versioning, and this is an initial dev release. This release brings 13 new features, 10 enhancements, 12 bug fixes, 1 perf improvement. Checkout the highlights for notable features and, as always, let us know what you think!
Breaking Changes
The `tcp` and `udp` sources & sinks have been merged into `socket`
The `tcp` and `udp` sources no longer exist and have been merged into a new `socket` type. Likewise, the `tcp` sink has been merged into the `socket` sink. Migration is straight forward, simply change the `type` to `socket` and add the field `mode` to match the socket type (`tcp` or `udp`):
[sources.my_tcp_source] - type = "tcp" + type = "socket" address = "0.0.0.0:9000" + mode = "tcp" [sources.my_tcp_sink] - type = "tcp" + type = "socket" address = "92.12.333.224:5000" + mode = "tcp"
Request, batching and auth fields refactored for sink components
Request based sinks have had their request fields nested under the table `request` and no longer use fixed retry intervals, instead using a fibonacci sequence for backing off retry attempts.
Batching fields are now nested under the table `batch`, with the field `size` replaced with either `max_events` or `max_size` in order to clarify its purpose (capping discrete events or bytes respectively).
Finally, authentication fields have been moved from the table `basic_auth` into a general purpose `auth` table complemented with a `strategy` field.
These changes effect the following sinks:
- `aws_cloudwatch_logs`
- `aws_kinesis_firehose`
- `aws_kinesis_streams`
- `aws_s3`
- `clickhouse`
- `datadog_metrics`
- `elasticsearch`
- `gcp_pubsub`
- `http`
- `new_relic_logs`
- `splunk_hec`
In order to migrate all fields prefixed with `request_` must be placed within a `request` table with the prefix removed. The config field `retry_backoff_secs` must also be replaced with two new fields `retry_initial_backoff_secs` and `retry_max_duration_secs`. Fields prefixed with `batch_` must be placed within a `batch` table with the prefix removed. Instances of `batch_size` should be renamed `max_size` or `max_events` (refer to the relevant sink docs for the correct variant). Finally, the table `basic_auth` should renamed `auth` with a field `strategy` added:
[sinks.my_sink_id]
type = "http"
inputs = ["my-source-id"]
uri = "https://10.22.212.22:9000/endpoint"
- batch_size = 1049000
+ [sinks.my_sink_id.batch]
+ max_size = 1049000
- [sinks.my_sink_id.basic_auth]
+ [sinks.my_sink_id.auth]
+ strategy = "basic"
user = "${USERNAME_ENV_VAR}"
password = "${PASSWORD_ENV_VAR}"
- request_in_flight_limit = 5
- request_retry_backoff_secs = 1
+ [sinks.my_sink_id.request]
+ in_flight_limit = 5
+ retry_initial_backoff_secs = 1
+ retry_max_duration_secs = 10
The buffer field `num_items` has been renamed to `max_events`
The buffer field `num_items` has been renamed to `max_events` in order to clarify its purpose. Migration involves simply changing all occurrences of `buffer.num_items` to `buffer.max_events`:
[sinks.my_sink_id]
type = "aws_s3"
inputs = ["my-source-id"]
bucket = "my-bucket"
[sinks.my_sink_id.buffer]
type = "memory"
- num_items = 500
+ max_events = 500
The `kafka` sink now takes a comma separated list of bootstrap servers
The `kafka` sink field `bootstrap_servers` has been changed from an array to a string, expecting a comma separated list of bootstrap servers similar to the `kafka` source:
[sinks.my_sink_id] type = "kafka" inputs = ["my-source-id"] - bootstrap_servers = ["10.14.22.123:9092", "10.14.23.332:9092"] + bootstrap_servers = "10.14.22.123:9092,10.14.23.332:9092"
The `journald` source now pipes from journalctl
The `journald` source originally used libsystemd in order to consume logs from Journald. Since this was causing issues for some distributions of Vector it has now been changed to pipe logs from journalctl.
Therefore, when using a `journald` source users must now ensure that the `journalctl` binary is available. The field `local_only` has also been removed as it is no longer applicable.
The `splunk_hec` sink does not index any fields by default
There is no longer a distinction within Vector between explicit and implicit event fields. All fields are now implicit and therefore the `splunk_hec` sink will not index any fields by default.
In order to mark desired fields as indexed you can use the optional configuration option `indexed_fields`:
[sinks.my_sink_id] type = "splunk_hec" inputs = ["my-source-id"] + indexed_fields = ["foo", "bar"]