Log Event

Description

A log event is a structured represention of a point-in-time event. It contains an arbitrary set of fields (key/value pairs) that describe the event.

Examples

{
"host": "my.host.com",
"message": "<13>Feb 13 20:07:26 74794bfb6795 root[8539]: i am foobar",
"timestamp": "2019-11-01T21:15:47+00:00"
}

Schema

  • stringoptional

    host

    Represents the originating host of the log. This is automatically set within select sources if the key does not exist. Change this field name via the global host_key option or the source-level host_key option for relevant sources.

    See Changing The Default Schema for more info.

    • No default
    • View examples
  • stringcommonrequired

    message

    Represents the log message. Change this field name via the global message_key option or the source-level message_key option for relevant sources.

    See Changing The Default Schema for more info.

    • No default
    • View examples
  • stringcommonrequired

    source

    The source from which the log originated. This is a specific source that is chosen and set by each Vector source. For example, this the file source sets this to the file name, and the docker source sets this to the container name. Change this field name via the global source_key option or the source-level source_key option for relevant sources.

    • No default
    • View examples
  • stringoptional

    source_type

    The official type of Vector's source component from which the log originates. Change this field name via the global source_type_key option or the source-level source_type_key option for relevant sources.

    • No default
    • View examples
  • timestampcommonrequired

    timestamp

    A normalized Rust DateTime struct in UTC. Change this field name via the global timestamp_key option or the source-level timestamp_key option for relevant sources.

    See Changing The Default Schema and Timestamp Coercion for more info.

    • No default
    • View examples
  • *optional

    [custom-key]

    In addition to the defined fields, a log event can have any number of additional fields. This includes nested fields.

    • No default
    • View examples

Components

View all log compatible components

How It Works

Changing The Default Schema

Vector is unique in that you are not required to use a specific schema. You can change the default schema. This makes integrating Vector into existing pipelines much easier. The message, host, and timestamp field names can all be changed via the global log_schema options.

Field name collisions

When you send structured data to Vector your field names always take precedence. For example, if you send a JSON object with a timestamp key, Vector will not override the value of that field. Vector will only set that field if it is not present.

Time Zones

If Vector receives a timestamp that does not contain timezone information Vector assumes the timestamp is in local time, and will convert the timestamp to UTC from the local time.

Dot Notation

Some components, such as rename_fields transform, accept name of a field as an option. In order to specify a nested field to them, use the dot notation which can be described by an example:

parent_field.child_field

The dot notation also supports accessing array fields using by placing the index between [ and ] after the array field name, for example

array[0]

The indexes start from 0, missing values are auto-filled by null values.

Timestamp Coercion

There are cases where Vector interacts with formats that do not have a formal timestamp defintion, such as JSON. In these cases, Vector will ingest the timestamp in it's primitive form (string or integer). You can then coerce the field into a timestamp using the coercer transform. If you are parsing this data out of a string, all Vector parser transforms include a types option, allowing you to extract and coerce in one step.

Types

Strings

Strings are UTF-8 compatible and are only bounded by the available system memory.

Ints

Integers are signed integers up to 64 bits.

Floats

Floats are 64-bit IEEE 754 floats.

Booleans

Booleans represent binary true/false values.

Timestamps

Timestamps are represented as DateTime Rust structs stored as UTC.

Null Values

For compatibility with JSON log events, Vector also supports null values.

Maps

Maps are associative arrays mapping string fields to values of any type.

Arrays

Array fields are sequences of values of any type.