Docker Source

The Vector docker_logs source collects logs from Docker.

Requirements

• Docker >= 1.24 is required.

Warnings

• Collecting logs directly from the Docker Engine is known to have performance problems for very large setups. If you have a large setup, please consider alternative collection methods, such as the Docker syslog or Docker journald driver drivers.
• To avoid collecting logs from itself when deployed as a container, the Docker source uses current hostname to find out which container it is inside. If a container's ID matches the hostname, that container will be excluded. If you change container's hostname, consider manually excluding Vector container using exclude_containers.

Setup

This component is part of a larger setup strategy for the Docker platform.

Docker Installation

Configuration

• Common
• Advanced

• vector.toml
• vector.yaml
• vector.json

[sources.my_source_id]
  type = "docker_logs" # required
  docker_host = "http://localhost:2375" # optional, no default
  include_containers = ["include_", "include_me_0", "ad08cc418cf9"] # optional, no default
  include_images = ["httpd", "redis"] # optional, no default
  include_labels = ["com.example.vendor=Timber Inc.", "com.example.name=Vector"] # optional, no default

10 items

common

• optionalbool

  auto_partial_merge

  Setting this to false will disable the automatic merging of partial events. See Merging Split Messages for more info.

  • Default: true
  • View examples
• commonoptionalstring

  docker_host

  The Docker host to connect to. Use an HTTPS URL to enable TLS encryption. If absent, Vector will try to use DOCKER_HOST enviroment variable. If DOCKER_HOST is also absent, Vector will use default Docker local socket (/var/run/docker.sock on Unix flatforms, //./pipe/docker_engine on Windows).

  • Syntax: literal
  • View examples
• optional[string]

  exclude_containers

  A list of container IDs or names to match against for containers you don't want to collect logs from. Prefix matches are supported, so you can supply just the first few characters of the ID or name of containers you want to exclude. This can be used in conjunction with include_containers.

  • View examples
• optionalstring

  host_key

  The key name added to each event representing the current host. This can also be globally set via the global [host_key](#host_key) option.

  • Syntax: literal
  • Default: "host"
• commonoptional[string]

  include_containers

  A list of container IDs or names to match against for containers you want to collect logs from. Prefix matches are supported, so you can supply just the first few characters of the ID or name of containers you want to include. This can be used in conjunction with exclude_containers.

  • View examples
• commonoptional[string]

  include_images

  A list of image names to match against. If not provided, all images will be included.

  • View examples
• commonoptional[string]

  include_labels

  A list of container object labels to match against when filtering running containers. This should follow the described label's syntax in docker object labels docs.

  • View examples
• optionaltable

  multiline

  Multiline parsing configuration. If not specified, multiline parsing is disabled.

  • commonrequiredstring

    condition_pattern

    Condition regex pattern to look for. Exact behavior is configured via mode.

    This field accepts a valid [Rust regular expression]urls.rustregex_syntax. Wrapping / characters are _not required or permitted.

    • Syntax: regex
    • View examples
  • enumcommonrequiredstring

    mode

    Mode of operation, specifies how the condition_pattern is interpreted.

    • Syntax: literal
    • Enum, must be one of: "continue_through" "continue_past" "halt_before" "halt_with"
    • View examples
  • commonrequiredstring

    start_pattern

    Start regex pattern to look for as a beginning of the message.

    This field accepts a valid [Rust regular expression]urls.rustregex_syntax. Wrapping / characters are _not required or permitted.

    • Syntax: regex
    • View examples
  • commonrequireduint

    timeout_ms

    The maximum time to wait for the continuation. Once this timeout is reached, the buffered message is guaranteed to be flushed, even if incomplete.

    • View examples
• optionaluint

  retry_backoff_secs

  The amount of time to wait before retrying after an error.

  • Default: 1 (seconds)
• optionaltable

  tls

  TLS options to connect to the Docker deamon. This has no effect unless docker_host is an HTTPS URL. If absent, Vector will try to use environment variable DOCKER_CERT_PATH and then DOCKER_CONFIG. If both environment variables are absent, Vector will try to read certificates in ~/.docker/.

  • commonrequiredstring

    ca_file

    Path to CA certificate file.

    • Syntax: literal
    • View examples
  • commonrequiredstring

    crt_file

    Path to TLS certificate file.

    • Syntax: literal
    • View examples
  • commonrequiredstring

    key_file

    Path to TLS key file.

    • Syntax: literal
    • View examples

Env Vars

• commonoptionalstring

  DOCKER_CERT_PATH

  Path to look for TLS certificates when tls configuration is absent. Vector will use:

  • $DOCKER_CERT_PATH/ca.pem: CA certificate.
  • $DOCKER_CERT_PATH/cert.pem: TLS certificate.
  • $DOCKER_CERT_PATH/key.pem: TLS key.
  • Syntax: literal
  • View examples
• commonoptionalstring

  DOCKER_CONFIG

  Path to look for TLS certificates when both tls configuration and DOCKER_CERT_PATH are absent.

  • Syntax: literal
  • View examples
• commonoptionalstring

  DOCKER_HOST

  The Docker host to connect to when docker_host configuration is absent.

  • Syntax: literal
  • View examples

Output

This component outputs log events with the following fields:

{
  "*" : "Started GET / for 127.0.0.1 at 2012-03-10 14:28:14 +0100",
  "container_created_at" : "2020-10-10T17:07:36+00:00",
  "container_id" : "9b6247364a03",
  "container_name" : "evil_ptolemy",
  "host" : "my-host.local",
  "image" : "ubuntu:latest",
  "message" : "Started GET / for 127.0.0.1 at 2012-03-10 14:28:14 +0100",
  "stream" : "stdout",
  "timestamp" : "2020-10-10T17:07:36+00:00"
}

9 items

commonrequired

• commonrequiredtimestamp

  container_created_at

  A UTC timestamp representing when the container was created.

  • View examples
• commonrequiredstring

  container_id

  The Docker container ID that the log was collected from.

  • Syntax: literal
  • View examples
• commonrequiredstring

  container_name

  The Docker container name that the log was collected from.

  • Syntax: literal
  • View examples
• commonrequiredstring

  host

  The local hostname, equivalent to the gethostname command.

  • Syntax: literal
  • View examples
• commonrequiredstring

  image

  The image name that the container is based on.

  • Syntax: literal
  • View examples
• commonrequiredstring

  *

  Each container label is inserted with it's exact key/value pair.

  • Syntax: literal
  • View examples
• commonrequiredstring

  message

  The raw log message.

  • Syntax: literal
  • View examples
• enumcommonrequiredstring

  stream

  The standard stream that the log was collected from.

  • Syntax: literal
  • Enum, must be one of: "stdout" "stderr"
  • View examples
• commonrequiredtimestamp

  timestamp

  The UTC timestamp extracted from the Docker log event.

  • View examples

Telemetry

This component provides the following metrics that can be retrieved through the internal_metrics source. See the metrics section in the monitoring page for more info.

• counter

  communication_errors_total

  The total number of errors stemming from communication with the Docker daemon. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  container_metadata_fetch_errors_total

  The total number of errors encountered when fetching container metadata. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  container_processed_events_total

  The total number of container events processed. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  containers_unwatched_total

  The total number of times Vector stopped watching for container logs. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  containers_watched_total

  The total number of times Vector started watching for container logs. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  logging_driver_errors_total

  The total number of logging driver errors encountered caused by not using either the jsonfile or journald driver. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  processed_bytes_total

  The total number of bytes processed by the component. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  events_out_total

  The total number of events emitted by this component. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  processed_events_total

  The total number of events processed by this component. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • file - The file that produced the error

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

Examples

Given the following input:

{
  "stream": "stdout",
  "message": "150.75.72.205 - - [03/Oct/2020:16:11:29 +0000] "HEAD /initiatives HTTP/1.1" 504 117"
}

And the following configuration:

vector.toml

[sources.docker_logs]
type = "docker_logs"
include_images = ["mingrammer/flog"]

The following Vector log event will be output:

{
  "container_created_at": "2020-10-03T16:11:29.443232Z",
  "container_id": "fecc98177eca7fb75a2b2186c418bf9a0cd3a05a1169f2e2293bf8987a9d96ab",
  "container_name": "flog",
  "image": "mingrammer/flog",
  "message": "150.75.72.205 - - [03/Oct/2020:16:11:29 +0000] \"HEAD /initiatives HTTP/1.1\" 504 117",
  "stream": "stdout",
  "host": "my-host.local"
}

How It Works

Context

By default, the docker_logs source will augment events with helpful context keys as shown in the "Output" section.

Merging Split Messages

Docker, by default, will split log messages that exceed 16kb. This can be a rather frustrating problem because it produces malformed log messages that are difficult to work with. Vector's solves this by default, automatically merging these messages into a single message. You can turn this off via the auto_partial_merge option. Furthermore, you can adjust the marker that we use to determine if an event is partial via the partial_event_marker_field option.

State

This component is stateless, meaning its behavior is consistent across each input.