Datadog Logs Sink

The Vector datadog_logs sink streams log events to Datadog's logs via the TCP endpoint.

Configuration

vector.toml
[sinks.my_sink_id]
# General
type = "datadog_logs" # required
inputs = ["my-source-id"] # required
api_key = "${DATADOG_API_KEY_ENV_VAR}" # required
healthcheck = true # optional, default
# Encoding
encoding.codec = "json" # required
6 items
stringcommonrequired

api_key

Datadog API key

See Obtaining an API token for more info.

No default
View examples
tableoptional

buffer

Configures the sink specific buffer behavior.

int (events)commonoptional

max_events

The maximum number of events allowed in the buffer.

Only relevant when: type = "memory"
Default: 500 (events)
View examples
int (bytes)required*

max_size

The maximum size of the buffer on the disk.

Only required when: type = "disk"
No default
View examples
stringenumcommonoptional

type

The buffer's type and storage mechanism.

Default: "memory"
Enum, must be one of: "memory" "disk"
View examples
stringenumoptional

when_full

The behavior when the buffer becomes full.

Default: "block"
Enum, must be one of: "block" "drop_newest"
View examples
tablecommonrequired

encoding

Configures the encoding specific sink behavior.

stringenumcommonrequired

codec

The encoding codec used to serialize the events before outputting.

No default
Enum, must be one of: "json" "text"
View examples
[string]optional

except_fields

Prevent the sink from encoding the specified labels.

No default
View examples
[string]optional

only_fields

Limit the sink to only encoding the specified labels.

No default
View examples
stringenumoptional

timestamp_format

How to format event timestamps.

Default: "rfc3339"
Enum, must be one of: "rfc3339" "unix"
View examples
stringoptional

endpoint

The endpoint to stream logs to.

Default: "intake.logs.datadoghq.com:10516"
View examples
boolcommonoptional

healthcheck

Enables/disables the sink healthcheck upon start.

See Health Checks for more info.

Default: true
View examples
tableoptional

tls

Configures the TLS options for connections from this sink.

boolcommonoptional

enabled

Enable TLS during connections to the remote.

No default
View examples
stringoptional

ca_path

Absolute path to an additional CA certificate file, in DER or PEM format (X.509).

No default
View examples
stringcommonoptional

crt_path

Absolute path to a certificate file used to identify this connection, in DER or PEM format (X.509) or PKCS#12. If this is set and is not a PKCS#12 archive, key_path must also be set.

No default
View examples
stringoptional

key_pass

Pass phrase used to unlock the encrypted key file. This has no effect unless key_path is set.

No default
View examples
stringcommonoptional

key_path

Absolute path to a certificate key file used to identify this connection, in DER or PEM format (PKCS#8). If this is set, crt_path must also be set.

No default
View examples
booloptional

verify_certificate

If true (the default), Vector will validate the TLS certificate of the remote host. Do NOT set this to false unless you understand the risks of not verifying the remote certificate.

Default: true
View examples
booloptional

verify_hostname

If true (the default), Vector will validate the configured remote host name against the remote host's TLS certificate. Do NOT set this to false unless you understand the risks of not verifying the remote hostname.

Default: true
View examples

How It Works

Buffers

The datadog_logs sink buffers events as shown in the diagram above. This helps to smooth out data processing if the downstream service applies backpressure. Buffers are controlled via the buffer.* options.

Environment Variables

Environment variables are supported through all of Vector's configuration. Simply add ${MY_ENV_VAR} in your Vector configuration file and the variable will be replaced before being evaluated.

You can learn more in the Environment Variables section.

Health Checks

Health checks ensure that the downstream service is accessible and ready to accept data. This check is performed upon sink initialization. If the health check fails an error will be logged and Vector will proceed to start.

Require Health Checks

If you'd like to exit immediately upon a health check failure, you can pass the --require-healthy flag:

vector --config /etc/vector/vector.toml --require-healthy

Disable Health Checks

If you'd like to disable health checks for this sink you can set the healthcheck option to false.

Obtaining an API token

  1. Register for a free account at datadoghq.com

  2. Fetch your logs api key by going to the other options and selecting the fluentd option, it should then present you an api_key. This api key can now be used with Vector!

Streaming

The datadog_logs sink streams data on a real-time event-by-event basis. It does not batch data.