Vector v0.22.2 release notes
The Vector team is pleased to announce version 0.22.2!
Note this release does include one potentially impactful behavior change to fix a bug in the internal_logs
source. The internal_logs source now, as was intended, applies the same rate limiting logic applied to logs
that Vector would write to stderr. This means, rather than seeing each log individually, duplicate logs in
small time windows are rolled up and logged like:
{"host":"consigliere","message":"Internal log [function call error for "parse_json" at (14:34): unable to parse json: trailing characters at line 1 column 8] has been rate limited 282948 times.","metadata":{"kind":"event","level":"ERROR","module_path":"vrl_stdlib::log","target":"vrl_stdlib::log"},"pid":2064943,"source_type":"internal_logs","timestamp":"2022-06-14T22:52:50.329188616Z","vector":{"component_id":"transform0","component_kind":"transform","component_name":"transform0","component_type":"remap"}}
{"host":"consigliere","internal_log_rate_secs":1,"message":"function call error for "parse_json" at (14:34): unable to parse json: trailing characters at line 1 column 8","metadata":{"kind":"event","level":"ERROR","module_path":"vrl_stdlib::log","target":"vrl_stdlib::log"},"pid":2064943,"source_type":"internal_logs","timestamp":"2022-06-14T22:52:50.329201656Z","vector":{"component_id":"transform0","component_kind":"transform","component_name":"transform0","component_type":"remap"},"vrl_position":54}
{"host":"consigliere","message":"Internal log [function call error for "parse_json" at (14:34): unable to parse json: trailing characters at line 1 column 8] is being rate limited.","metadata":{"kind":"event","level":"ERROR","module_path":"vrl_stdlib::log","target":"vrl_stdlib::log"},"pid":2064943,"source_type":"internal_logs","timestamp":"2022-06-14T22:52:50.329212676Z","vector":{"component_id":"transform0","component_kind":"transform","component_name":"transform0","component_type":"remap"}}
This change was made to bring internal_logs in line with Vector’s logging and avoid unexpected high bills from
ingestion in downstream systems when a catastrophic failure occurs.
Note: Please see the release notes for v0.22.0 for additional changes if upgrading from
v0.21.X. In particular, the upgrade guide for breaking changes.
Vector Changelog
2 bug fixes
- The
diskbuffer max record size was increased to match the max data file size of 128 MB. This makes it much less likely that apanicwill be hit due to large record batches coming in. We have work planned to make this more robust by splitting up batches larger than the data file size. - The
internal_logssource now correctly applies the same log rate limiting that is applied to logs Vector logs tostderr.