receive

Syslog Source

The Vector syslog source receives logs from Syslog.

Configuration

[sources.my_source_id]
  type = "syslog" # required
  address = "0.0.0.0:9000" # required, required when mode = `tcp` or `udp`
  max_length = 102400 # optional, default, bytes
  mode = "tcp" # required
  path = "/path/to/socket" # required, required when mode = `unix`
8 items
  • commonrequired*string

    address

    The address to listen for connections on, or systemd#N to use the Nth socket passed by systemd socket activation. If an address is used it must include a port.

    • Only required when: mode = `tcp` or `udp`
    • View examples
  • optionalstring

    host_key

    The key name added to each event representing the current host. This can also be globally set via the global [host_key](#host_key) option.

    • Default: "host"
  • optionaltable

    keepalive

    Configures the TCP keepalive behavior for the connection to the source.

    • optionaluint

      time_secs

      The time a connection needs to be idle before sending TCP keepalive probes.

    • commonoptionaluint

      max_length

      The maximum bytes size of incoming messages before they are discarded.

      • Default: 102400 (bytes)
    • enumcommonrequiredstring

      mode

      The type of socket to use.

      • Enum, must be one of: "tcp" "udp" "unix_datagram" "unix_stream"
      • View examples
    • commonrequired*string

      path

      The unix socket path. This should be an absolute path.

      • Only required when: mode = `unix`
      • View examples
    • optionaluint

      shutdown_timeout_secs

      The timeout before a connection is forcefully closed during shutdown.

      • Only relevant when: mode = `tcp``
      • Default: 30 (seconds)
    • optionaltable

      tls

      Configures the TLS options for incoming connections.

      • optionalstring

        ca_file

        Absolute path to an additional CA certificate file, in DER or PEM format (X.509), or an in-line CA certificate in PEM format.

        • View examples
      • optionalstring

        crt_file

        Absolute path to a certificate file used to identify this server, in DER or PEM format (X.509) or PKCS#12, or an in-line certificate in PEM format. If this is set, and is not a PKCS#12 archive, key_file must also be set. This is required if enabled is set to true.

        • View examples
      • optionalbool

        enabled

        Require TLS for incoming connections. If this is set, an identity certificate is also required.

        • Default: false
        • View examples
      • optionalstring

        key_file

        Absolute path to a private key file used to identify this server, in DER or PEM format (PKCS#8), or an in-line private key in PEM format.

        • View examples
      • optionalstring

        key_pass

        Pass phrase used to unlock the encrypted key file. This has no effect unless key_file is set.

        • View examples
      • optionalbool

        verify_certificate

        If true, Vector will require a TLS certificate from the connecting host and terminate the connection if the certificate is not valid. If false (the default), Vector will not request a certificate from the client.

        • Default: false
        • View examples

    Output

    This component outputs log events with the following fields:

    {
      "*" : "hello world",
      "appname" : "app-name",
      "facility" : "1",
      "host" : "my-host.local",
      "hostname" : "my.host.com",
      "message" : "Hello world",
      "msgid" : "ID47",
      "procid" : "8710",
      "severity" : "notice",
      "source_ip" : "127.0.0.1",
      "timestamp" : "2020-10-10T17:07:36+00:00",
      "version" : 1
    }
    12 items
    • commonrequiredstring

      *

      In addition to the defined fields, any Syslog 5424 structured fields are parsed and inserted as root level fields.

      • View examples
    • commonrequiredstring

      appname

      The appname extracted from the Syslog formatted line. If a appname is not found, then the key will not be added.

      • View examples
    • commonrequiredstring

      facility

      The facility extracted from the Syslog line. If a facility is not found, then the key will not be added.

      • View examples
    • commonrequiredstring

      host

      The local hostname, equivalent to the gethostname command.

      • View examples
    • commonrequiredstring

      hostname

      The hostname extracted from the Syslog line. (host is also this value if it exists in the log.)

      • View examples
    • commonrequiredstring

      message

      The message extracted from the Syslog line. See Parsing for more info.

      • View examples
    • commonrequiredstring

      msgid

      The msgid extracted from the Syslog line. If a msgid is not found, then the key will not be added.

      • View examples
    • commonrequiredstring

      procid

      The procid extracted from the Syslog line. If a procid is not found, then the key will not be added.

      • View examples
    • commonrequiredstring

      severity

      The severity extracted from the Syslog line. If a severity is not found, then the key will not be added.

      • View examples
    • commonrequiredstring

      source_ip

      The upstream hostname. In the case where mode = "unix" the socket path will be used. (host is also this value if hostname does not exist in the log.)

      • View examples
    • commonrequiredtimestamp

      timestamp

      The exact time the event was ingested into Vector.

      • View examples
    • commonrequireduint

      version

      The version extracted from the Syslog line. If a version is not found, then the key will not be added.

      • View examples

    Telemetry

    This component provides the following metrics that can be retrieved through the internal_metrics source. See the metrics section in the monitoring page for more info.

    • counter

      connection_read_errors_total

      The total number of errors reading datagram. This metric includes the following tags:

      • component_kind - The Vector component kind.

      • component_name - The Vector component ID.

      • component_type - The Vector component type.

      • instance - The Vector instance identified by host and port.

      • job - The name of the job producing Vector metrics.

      • mode -

    • counter

      processed_events_total

      The total number of events processed by this component. This metric includes the following tags:

      • component_kind - The Vector component kind.

      • component_name - The Vector component ID.

      • component_type - The Vector component type.

      • file - The file that produced the error

      • instance - The Vector instance identified by host and port.

      • job - The name of the job producing Vector metrics.

    • counter

      utf8_convert_errors_total

      The total number of errors converting bytes to a UTF-8 string in UDP mode. This metric includes the following tags:

      • component_kind - The Vector component kind.

      • component_name - The Vector component ID.

      • component_type - The Vector component type.

      • instance - The Vector instance identified by host and port.

      • job - The name of the job producing Vector metrics.

      • mode - The connection mode used by the component.

    • counter

      processed_bytes_total

      The total number of bytes processed by the component. This metric includes the following tags:

      • component_kind - The Vector component kind.

      • component_name - The Vector component ID.

      • component_type - The Vector component type.

      • instance - The Vector instance identified by host and port.

      • job - The name of the job producing Vector metrics.

    Examples

    Given the following input:

    <13>1 2020-03-13T20:45:38.119Z dynamicwireless.name non 2426 ID931 [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] Try to override the THX port, maybe it will reboot the neural interface!

    And the following configuration:

    [sources.syslog]
    type = "syslog"

    The following Vector log event will be output:

    {
      "severity": "notice",
      "facility": "user",
      "timestamp": "2020-03-13T20:45:38.119Z",
      "host": "my-host.local",
      "source_ip": "34.33.222.212",
      "hostname": "dynamicwireless.name",
      "appname": "non",
      "procid": "2426",
      "msgid": "ID931",
      "iut": "3",
      "eventSource": "Application",
      "eventID": "1011",
      "message": "Try to override the THX port, maybe it will reboot the neural interface!"
    }

    How It Works

    Context

    By default, the syslog source will augment events with helpful context keys as shown in the "Output" section.

    Line Delimiters

    Each line is read until a new line delimiter, the 0xA byte, is found.

    Parsing

    Vector makes a best effort to parse the various Syslog formats out in the wild. This includes RFC 6587, RFC 5424, RFC 3164, and other common variations (such as the Nginx Syslog style). It's unfortunate that the Syslog specification is not more accurately followed, but we hope Vector insulates you from these deviations.

    If parsing fails, Vector will include the entire Syslog line in the message key. If you find this happening often, we recommend using the socket source combined with the regex_parser transform to implement your own ingestion and parsing scheme. Or, open an issue requesting support for your specific format.

    Transport Layer Security (TLS)

    Vector uses Openssl for TLS protocols. You can adjust TLS behavior via the tls.* options.

    Support
    Contents
    Resources