LOG

JSON Parser Transform

The Vector json_parser transform accepts and outputs log events allowing you to parse a log field value as JSON.

Configuration

  • Common
  • Advanced
vector.toml
[transforms.my_transform_id]
  # REQUIRED
  type = "json_parser" # must be: "json_parser"
  inputs = ["my-source-id"] # example
  drop_invalid = true # example


  # OPTIONAL
  drop_field = true # default
  field = "message" # default

Options

5 items
boolcommonrequired

drop_field

If the specified field should be dropped (removed) after parsing. If parsing fails, the field will not be removed, irrespective of this setting.

Default: true
View examples
boolcommonrequired

drop_invalid

If true events with invalid JSON will be dropped, otherwise the event will be kept and passed through. See Invalid JSON for more info.

No default
View examples
stringcommonrequired

field

The log field to decode as JSON. Must be a string value type. See Invalid JSON for more info.

Default: "message"
View examples
booloptional

overwrite_target

If target_field is set and the log contains a field of the same name as the target, it will only be overwritten if this is set to true.

No default
View examples
stringoptional

target_field

If this setting is present, the parsed JSON will be inserted into the log as a sub-object with this name. If a field with the same name already exists, the parser will fail and produce an error.

No default
View examples

Output

The json_parser transform accepts and outputs log events allowing you to parse a log field value as JSON. For example:

  • Simple
  • Wrapped

Given the following log event:

{
  "message": "{"key": "value"}"
}

You can parse the JSON with:

[transforms.json]
  inputs = ["<source_id>"]
  type   = "json_parser"
  field  = "message"

This would produce the following event as output:

{
  "key": "value"
}

By default, Vector drops fields after parsing them via the drop_field option.

How It Works

Chaining / Unwrapping

Please see the I/O section for an example of chaining and unwrapping JSON.

Correctness

The json_parser source has been involved in the following correctness tests:

Learn more in the Correctness sections.

Environment Variables

Environment variables are supported through all of Vector's configuration. Simply add ${MY_ENV_VAR} in your Vector configuration file and the variable will be replaced before being evaluated.

You can learn more in the Environment Variables section.

Invalid JSON

If the value for the specified field is not valid JSON you can control keep or discard the event with the drop_invalid option. Setting it to true will discard the event and drop it entirely. Setting it to false will keep the event and pass it through. Note that passing through the event could cause problems and violate assumptions about the structure of your event.

Key Conflicts

Any key present in the decoded JSON will override existin keys in the event.

Nested Fields

If the decoded JSON includes nested fields it will be deep merged into the event. For example, given the following event:

{
  "message": "{"parent": {"child2": "value2"}}",
  "parent": {
    "child1": "value1"
  }
}

Parsing the "message" field would result the following structure:

{
  "parent": {
    "child1": "value1",
    "child2": "value2"
  }
}
Support
Contents
Resources