parse

JSON Parser Transform

The Vector json_parser transform parses a log field value as JSON.

Configuration

[transforms.my_transform_id]
  type = "json_parser" # required
  inputs = ["my-source-or-transform-id"] # required
  drop_field = true # optional, default
  drop_invalid = true # required
  field = "message" # optional, default
5 items
  • commonoptionalbool

    drop_field

    If the specified field should be dropped (removed) after parsing. If parsing fails, the field will not be removed, irrespective of this setting.

    • Default: true
    • View examples
  • commonrequiredbool

    drop_invalid

    If true events with invalid JSON will be dropped, otherwise the event will be kept and passed through. See Invalid JSON for more info.

    • View examples
  • commonoptionalstring

    field

    The log field to decode as JSON. Must be a string value type. See Invalid JSON for more info.

    • Default: "message"
    • View examples
  • optionalbool

    overwrite_target

    If target_field is set and the log contains a field of the same name as the target, it will only be overwritten if this is set to true.

    • Default: false
    • View examples
  • optionalstring

    target_field

    If this setting is present, the parsed JSON will be inserted into the log as a sub-object with this name. If a field with the same name already exists, the parser will fail and produce an error.

    • View examples

Telemetry

This component provides the following metrics that can be retrieved through the internal_metrics source. See the metrics section in the monitoring page for more info.

  • counter

    processed_events_total

    The total number of events processed by this component. This metric includes the following tags:

    • component_kind - The Vector component kind.

    • component_name - The Vector component ID.

    • component_type - The Vector component type.

    • file - The file that produced the error

    • instance - The Vector instance identified by host and port.

    • job - The name of the job producing Vector metrics.

  • counter

    processed_bytes_total

    The total number of bytes processed by the component. This metric includes the following tags:

    • component_kind - The Vector component kind.

    • component_name - The Vector component ID.

    • component_type - The Vector component type.

    • instance - The Vector instance identified by host and port.

    • job - The name of the job producing Vector metrics.

How It Works

Invalid JSON

If the value for the specified field is not valid JSON you can control keeping or discarding the event with the drop_invalid option. Setting it to true will discard the event and drop it entirely. Setting it to false will keep the event and pass it through. Note that passing through the event could cause problems and violate assumptions about the structure of your event.

Merge Conflicts

Key Conflicts

Any key present in the decoded JSON will override existing keys in the event.

Object Conflicts

If the decoded JSON includes nested fields it will be deep merged into the event. For example, given the following event:

{
  "message": "{"parent": {"child2": "value2"}}",
  "parent": {
    "child1": "value1"
  }
}

Parsing the "message" field would result the following structure:

{
  "parent": {
    "child1": "value1",
    "child2": "value2"
  }
}

Notice that the parent.child1 key was preserved.

Support
Contents
Resources