JSON Parser Transform
The Vector json_parser
transform
parses a log field value as JSON.
Configuration
- Common
- Advanced
- vector.toml
- vector.yaml
- vector.json
[transforms.my_transform_id]type = "json_parser" # requiredinputs = ["my-source-or-transform-id"] # requireddrop_field = true # optional, defaultdrop_invalid = true # requiredfield = "message" # optional, default
- optionalbool
drop_field
If the specified
field
should be dropped (removed) after parsing. If parsing fails, the field will not be removed, irrespective of this setting.- Default:
true
- View examples
- Default:
- requiredbool
drop_invalid
If
true
events with invalid JSON will be dropped, otherwise the event will be kept and passed through. See Invalid JSON for more info.- View examples
- optionalstring
field
The log field to decode as JSON. Must be a
string
value type. See Invalid JSON for more info.- Default:
"message"
- View examples
- Default:
- optionalbool
overwrite_target
If
target_field
is set and the log contains a field of the same name as the target, it will only be overwritten if this is set totrue
.- Default:
false
- View examples
- Default:
- optionalstring
target_field
If this setting is present, the parsed JSON will be inserted into the log as a sub-object with this name. If a field with the same name already exists, the parser will fail and produce an error.
- View examples
Telemetry
This component provides the following metrics that can be retrieved through
the internal_metrics
source. See the
metrics section in the
monitoring page for more info.
- counter
processed_events_total
The total number of events processed by this component. This metric includes the following tags:
component_kind
- The Vector component kind.component_name
- The Vector component ID.component_type
- The Vector component type.file
- The file that produced the errorinstance
- The Vector instance identified by host and port.job
- The name of the job producing Vector metrics.
- counter
processed_bytes_total
The total number of bytes processed by the component. This metric includes the following tags:
component_kind
- The Vector component kind.component_name
- The Vector component ID.component_type
- The Vector component type.instance
- The Vector instance identified by host and port.job
- The name of the job producing Vector metrics.
How It Works
Invalid JSON
If the value for the specified field
is not valid JSON you can control keeping
or discarding the event with the drop_invalid
option. Setting it to true
will
discard the event and drop it entirely. Setting it to false
will keep the
event and pass it through. Note that passing through the event could cause
problems and violate assumptions about the structure of your event.
Merge Conflicts
Key Conflicts
Any key present in the decoded JSON will override existing keys in the event.
Object Conflicts
If the decoded JSON includes nested fields it will be deep merged into the event. For example, given the following event:
{"message": "{"parent": {"child2": "value2"}}","parent": {"child1": "value1"}}
Parsing the "message"
field would result the following structure:
{"parent": {"child1": "value1","child2": "value2"}}
Notice that the parent.child1
key was preserved.