Remap support for the
reduce transform (multi-line logs)
Use VRL to specify conditions for reducing multi-log lines into a single log event
reduce transform for Vector enables you to reduce multiple log
events into a single event, which is useful for logs that are split across
multiple lines, such as JVM stack traces. With
reduce you can specify both a
merge strategy for the events and a condition for specifying when a group of
events either starts or ends (that group is then collapsed, i.e. reduced into
a single event).
reduce transform required you to specify conditions using
check_fields. But now you can use Vector Remap Language expressions in the
remap transform to specify those conditions. This should make using
reduce more natural and expressive.
The example configuration below shows the same
reduce transform using the old
check_fields) and the new system (
[transforms.merge_stack_trace] type = "reduce" inputs = ["jvm-logs"] merge_strategies.message = "concat_newline" # Using check_fields -starts_when.type = "check_fields" -starts_when.message.regex = "/^\\w.*/" -starts_when.severity.eq = "info" # Using remap +starts_when.type = "remap" +starts_when.source = 'match(string!(.message), r'^\\w.*') && .severity == "info"'