0.49 Upgrade Guide

An upgrade guide that addresses breaking changes in 0.49.0

There is one Vector behavior change.

docker_logs source behavior change

Previous Behavior

A configuration using docker_logs would start even if the Docker socket (or alternate configuration method) was not available.

New Behavior

If the Docker socket (or alternate configuration method) is not available, Vector will fail to start.

VRL version 0.26.0 included a couple of breaking changes.

encode_lz4 and decode_lz4 function changes

Previous Behavior

encode_lz4 used to prepend the uncompressed size by default.

New Behavior

encode_lz4 no longer prepends the uncompressed size by default, improving compatibility with standard LZ4 tools. A new prepend_size flag restores the old behavior if needed. Also, decode_lz4 now also accepts prepend_size and a buf_size option (default: 1MB).

Action needed

Existing users of encode_lz4 and decode_lz4 will need to update their functions to include the argument prepend_size: true to maintain existing compatibility.

parse_cef function changes

The parse_cef now trims unnecessary whitespace around escaped values in both headers and extension fields, improving accuracy and reliability when dealing with messy input strings.

Scenario: parse_cef with whitespace post cef fields

Input 

CEF:1|Security|threatmanager|1.0|100|worm successfully stopped|10| dst=2.1.2.2 msg=Detected a threat. No action needed spt=1232

Previous Behavior: Runtime Error

If an input for parse_cef included spaces ( ), the line couldn’t be parsed correctly and resulted in a runtime error.

error[E000]: function call error for "parse_cef" at (0:20): Could not parse whole line successfully
┌─ :1:1
│.message = "CEF:1|Security|threatmanager|1.0|100|worm successfully stopped|10| dst=2.1.2.2 msg=Detected a threat. No action needed spt=1232"
1 │ parse_cef!(.message)
│ ^^^^^^^^^^^^^^^^^^^^ Could not parse whole line successfully
│
= see language documentation at https://vrl.dev
= try your code in the VRL REPL, learn more at https://vrl.dev/examples

New Behavior: parses data correctly 

{
  "cefVersion": "1",
  "deviceEventClassId": "100",
  "deviceProduct": "threatmanager",
  "deviceVendor": "Security",
  "deviceVersion": "1.0",
  "dst": "2.1.2.2",
  "msg": "Detected a threat. No action needed",
  "name": "worm successfully stopped",
  "severity": "10",
  "spt": "1232"
}

Scenario: parse_cef with whitespace in cef fields

Input 

CEF:1|Security|threatmanager|1.0|100|worm successfully stopped|10| dst=2.1.2.2 msg=Detected a threat. No action needed  spt=1232

Previous Behavior: Trailing whitespace 

"msg": "Detected a threat. No action needed   "

New Behavior: No trailing whitespace 

"msg": "Detected a threat. No action needed"`

parse_syslog function changes

The parse_syslog function now treats RFC 3164 structured data items with no parameters (e.g., [exampleSDID@32473]) as part of the main message, rather than parsing them as structured data. Items with parameters (e.g., [exampleSDID@32473 field="value"]) continue to be parsed as structured data.