Syslog Source

The Vector syslog source receives logs from Syslog.

Configuration

Common
Advanced

vector.toml
vector.yaml
vector.json

[sources.my_source_id]
  type = "syslog" # required
  address = "0.0.0.0:9000" # required, required when mode = `tcp` or `udp`
  max_length = 102400 # optional, default, bytes
  mode = "tcp" # required
  path = "/path/to/socket" # required, required when mode = `unix`

8 items

commonrequired

commonrequired*string
address
The address to listen for connections on, or systemd#N to use the Nth socket passed by systemd socket activation. If an address is used it must include a port.
- Syntax: literal
- Only required when: mode = `tcp` or `udp`
- View examples
optionalstring
host_key
The key name added to each event representing the current host. This can also be globally set via the global [host_key](#host_key) option.
- Syntax: literal
- Default: "host"
optionaltable
keepalive
Configures the TCP keepalive behavior for the connection to the source.
- optionaluint
  time_secs
  The time a connection needs to be idle before sending TCP keepalive probes.
commonoptionaluint
max_length
The maximum bytes size of incoming messages before they are discarded.
- Default: 102400 (bytes)
enumcommonrequiredstring
mode
The type of socket to use.
- Syntax: literal
- Enum, must be one of: "tcp" "udp" "unix_datagram" "unix_stream"
- View examples
commonrequired*string
path
The unix socket path. This should be an absolute path.
- Syntax: literal
- Only required when: mode = `unix`
- View examples
optionaluint
shutdown_timeout_secs
The timeout before a connection is forcefully closed during shutdown.
- Only relevant when: mode = `tcp``
- Default: 30 (seconds)
optionaltable
tls
Configures the TLS options for incoming connections.
- optionalstring
  ca_file
  Absolute path to an additional CA certificate file, in DER or PEM format (X.509), or an in-line CA certificate in PEM format.
  - Syntax: literal
  - View examples
- optionalstring
  crt_file
  Absolute path to a certificate file used to identify this server, in DER or PEM format (X.509) or PKCS#12, or an in-line certificate in PEM format. If this is set, and is not a PKCS#12 archive, key_file must also be set. This is required if enabled is set to true.
  - Syntax: literal
  - View examples
- optionalbool
  enabled
  Require TLS for incoming connections. If this is set, an identity certificate is also required.
  - Default: false
  - View examples
- optionalstring
  key_file
  Absolute path to a private key file used to identify this server, in DER or PEM format (PKCS#8), or an in-line private key in PEM format.
  - Syntax: literal
  - View examples
- optionalstring
  key_pass
  Pass phrase used to unlock the encrypted key file. This has no effect unless key_file is set.
  - Syntax: literal
  - View examples
- optionalbool
  verify_certificate
  If true, Vector will require a TLS certificate from the connecting host and terminate the connection if the certificate is not valid. If false (the default), Vector will not request a certificate from the client.
  - Default: false
  - View examples

Output

This component outputs log events with the following fields:

{
  "*" : "hello world",
  "appname" : "app-name",
  "facility" : "1",
  "host" : "my-host.local",
  "hostname" : "my.host.com",
  "message" : "Hello world",
  "msgid" : "ID47",
  "procid" : "8710",
  "severity" : "notice",
  "source_ip" : "127.0.0.1",
  "timestamp" : "2020-10-10T17:07:36+00:00",
  "version" : 1
}

12 items

commonrequired

commonrequiredstring
*
In addition to the defined fields, any Syslog 5424 structured fields are parsed and inserted as root level fields.
- Syntax: literal
- View examples
commonrequiredstring
appname
The appname extracted from the Syslog formatted line. If a appname is not found, then the key will not be added.
- Syntax: literal
- View examples
commonrequiredstring
facility
The facility extracted from the Syslog line. If a facility is not found, then the key will not be added.
- Syntax: literal
- View examples
commonrequiredstring
host
The local hostname, equivalent to the gethostname command.
- Syntax: literal
- View examples
commonrequiredstring
hostname
The hostname extracted from the Syslog line. (host is also this value if it exists in the log.)
- Syntax: literal
- View examples
commonrequiredstring
message
The message extracted from the Syslog line. See Parsing for more info.
- Syntax: literal
- View examples
commonrequiredstring
msgid
The msgid extracted from the Syslog line. If a msgid is not found, then the key will not be added.
- Syntax: literal
- View examples
commonrequiredstring
procid
The procid extracted from the Syslog line. If a procid is not found, then the key will not be added.
- Syntax: literal
- View examples
commonrequiredstring
severity
The severity extracted from the Syslog line. If a severity is not found, then the key will not be added.
- Syntax: literal
- View examples
commonrequiredstring
source_ip
The upstream hostname. In the case where mode = "unix" the socket path will be used. (host is also this value if hostname does not exist in the log.)
- Syntax: literal
- View examples
commonrequiredtimestamp
timestamp
The time extracted from the Syslog formatted line. If parsing fails, then the exact time the event was ingested into Vector is used.
- View examples
commonrequireduint
version
The version extracted from the Syslog line. If a version is not found, then the key will not be added.
- View examples

Telemetry

This component provides the following metrics that can be retrieved through the internal_metrics source. See the metrics section in the monitoring page for more info.

counter
events_in_total
The number of events accepted by this component either from tagged origin like file and uri, or cumulatively from other origins. This metric includes the following tags:
- component_kind - The Vector component kind.
- component_name - The Vector component ID.
- component_type - The Vector component type.
- container_name - The name of the container from which the event originates.
- file - The file from which the event originates.
- instance - The Vector instance identified by host and port.
- job - The name of the job producing Vector metrics.
- mode - The connection mode used by the component.
- peer_addr - The IP from which the event originates.
- peer_path - The pathname from which the event originates.
- pod_name - The name of the pod from which the event originates.
- uri - The sanitized uri from which the event originates.
counter
connection_read_errors_total
The total number of errors reading datagram. This metric includes the following tags:
- component_kind - The Vector component kind.
- component_name - The Vector component ID.
- component_type - The Vector component type.
- instance - The Vector instance identified by host and port.
- job - The name of the job producing Vector metrics.
- mode -
counter
processed_bytes_total
The number of bytes processed by the component. This metric includes the following tags:
- component_kind - The Vector component kind.
- component_name - The Vector component ID.
- component_type - The Vector component type.
- container_name - The name of the container from which the bytes originate.
- file - The file from which the bytes originate.
- instance - The Vector instance identified by host and port.
- job - The name of the job producing Vector metrics.
- mode - The connection mode used by the component.
- peer_addr - The IP from which the bytes originate.
- peer_path - The pathname from which the bytes originate.
- pod_name - The name of the pod from which the bytes originate.
- uri - The sanitized uri from which the bytes originate.
counter
processed_events_total
The total number of events processed by this component. This metric is deprecated in place of using events_in_total and events_out_total metrics. This metric includes the following tags:
- component_kind - The Vector component kind.
- component_name - The Vector component ID.
- component_type - The Vector component type.
- instance - The Vector instance identified by host and port.
- job - The name of the job producing Vector metrics.
counter
events_out_total
The total number of events emitted by this component. This metric includes the following tags:
- component_kind - The Vector component kind.
- component_name - The Vector component ID.
- component_type - The Vector component type.
- instance - The Vector instance identified by host and port.
- job - The name of the job producing Vector metrics.
counter
utf8_convert_errors_total
The total number of errors converting bytes to a UTF-8 string in UDP mode. This metric includes the following tags:
- component_kind - The Vector component kind.
- component_name - The Vector component ID.
- component_type - The Vector component type.
- instance - The Vector instance identified by host and port.
- job - The name of the job producing Vector metrics.
- mode - The connection mode used by the component.

Examples

Given the following input:

<13>1 2020-03-13T20:45:38.119Z dynamicwireless.name non 2426 ID931 [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] Try to override the THX port, maybe it will reboot the neural interface!

And the following configuration:

vector.toml

[sources.syslog]
type = "syslog"

The following Vector log event will be output:

{
  "severity": "notice",
  "facility": "user",
  "timestamp": "2020-03-13T20:45:38.119Z",
  "host": "my-host.local",
  "source_ip": "34.33.222.212",
  "hostname": "dynamicwireless.name",
  "appname": "non",
  "procid": "2426",
  "msgid": "ID931",
  "iut": "3",
  "eventSource": "Application",
  "eventID": "1011",
  "message": "Try to override the THX port, maybe it will reboot the neural interface!"
}

How It Works

Context

By default, the syslog source will augment events with helpful context keys as shown in the "Output" section.

Line Delimiters

Each line is read until a new line delimiter, the 0xA byte, is found.

Parsing

Vector makes a best effort to parse the various Syslog formats out in the wild. This includes RFC 6587, RFC 5424, RFC 3164, and other common variations (such as the Nginx Syslog style). It's unfortunate that the Syslog specification is not more accurately followed, but we hope Vector insulates you from these deviations.

If parsing fails, Vector will include the entire Syslog line in the message key. If you find this happening often, we recommend using the socket source combined with the regex_parser transform to implement your own ingestion and parsing scheme. Or, open an issue requesting support for your specific format.

State

This component is stateless, meaning its behavior is consistent across each input.

Transport Layer Security (TLS)

Vector uses Openssl for TLS protocols. You can adjust TLS behavior via the tls.* options.

#Configuration

#address

#host_key

#keepalive

#time_secs

#max_length

#mode

#path

#shutdown_timeout_secs

#tls

#ca_file

#crt_file

#enabled

#key_file

#key_pass

#verify_certificate

#Output

*

#appname

#facility

#host

#hostname

#message

#msgid

#procid

#severity

#source_ip

#timestamp

#version

#Telemetry

#events_in_total

#connection_read_errors_total

#processed_bytes_total

#processed_events_total

#events_out_total

#utf8_convert_errors_total

#Examples

#How It Works

#Context

#Line Delimiters

#Parsing

#State

#Transport Layer Security (TLS)

Configuration

address

host_key

keepalive

time_secs

max_length

mode

path

shutdown_timeout_secs

tls

ca_file

crt_file

enabled

key_file

key_pass

verify_certificate

Output

appname

facility

host

hostname

message

msgid

procid

severity

source_ip

timestamp

version

Telemetry

events_in_total

connection_read_errors_total

processed_bytes_total

processed_events_total

events_out_total

utf8_convert_errors_total