AWS Cloudwatch Metrics Sink

The Vector aws_cloudwatch_metrics sink streams metric events to Amazon Web Service's CloudWatch Metrics service via the PutMetricData API endpoint.

Configuration

vector.toml
[sinks.my_sink_id]
type = "aws_cloudwatch_metrics" # required
inputs = ["my-source-or-transform-id"] # required
compression = "none" # optional, default
healthcheck = true # optional, default
namespace = "service" # required
region = "us-east-1" # required, required when endpoint = ""
  • stringoptional

    assume_role

    The ARN of an IAM role to assume at startup. See AWS Authentication for more info.

    • No default
    • View examples
  • tableoptional

    batch

    Configures the sink batching behavior.

    • uint (events)commonoptional

      max_events

      The maximum size of a batch, in events, before it is flushed.

      • Default: 20 (events)
    • uint (seconds)commonoptional

      timeout_secs

      The maximum age of a batch before it is flushed.

      • Default: 1 (seconds)
  • stringenumcommonoptional

    compression

    The compression strategy used to compress the encoded event data before transmission.

    • Default: "none"
    • Enum, must be one of: "none" "gzip"
    • View examples
  • stringoptional

    endpoint

    Custom endpoint for use with AWS-compatible services. Providing a value for this option will make region moot.

    • Only relevant when: region = null
    • No default
    • View examples
  • boolcommonoptional

    healthcheck

    Enables/disables the sink healthcheck upon start. See Health Checks for more info.

    • Default: true
    • View examples
  • stringcommonrequired

    namespace

    A namespace that will isolate different metrics from each other.

    • No default
    • View examples
  • stringcommonrequired*

    region

    The AWS region of the target service. If endpoint is provided it will override this value since the endpoint includes the region.

    • Only required when: endpoint = null
    • No default
    • View examples

Env Vars

  • stringoptional

    AWS_ACCESS_KEY_ID

    Used for AWS authentication when communicating with AWS services. See relevant AWS components for more info. See AWS Authentication for more info.

    • No default
    • View examples
  • stringoptional

    AWS_SECRET_ACCESS_KEY

    Used for AWS authentication when communicating with AWS services. See relevant AWS components for more info. See AWS Authentication for more info.

    • No default
    • View examples

How It Works

AWS Authentication

Vector checks for AWS credentials in the following order:

  1. Environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.
  2. The credential_process command in the AWS config file. (usually located at ~/.aws/config)
  3. The AWS credentials file. (usually located at ~/.aws/credentials)
  4. The IAM instance profile. (will only work if running on an EC2 instance with an instance profile/role)

If credentials are not found the healtcheck will fail and an error will be logged.

Obtaining an access key

In general, we recommend using instance profiles/roles whenever possible. In cases where this is not possible you can generate an AWS access key for any user within your AWS account. AWS provides a detailed guide on how to do this.

Assuming Roles

Vector can assume an AWS IAM role via the assume_role option. This is an optional setting that is helpful for a variety of use cases, such as cross account access.

Environment Variables

Environment variables are supported through all of Vector's configuration. Simply add ${MY_ENV_VAR} in your Vector configuration file and the variable will be replaced before being evaluated.

You can learn more in the Environment Variables section.

Health Checks

Health checks ensure that the downstream service is accessible and ready to accept data. This check is performed upon sink initialization. If the health check fails an error will be logged and Vector will proceed to start.

Require Health Checks

If you'd like to exit immediately upon a health check failure, you can pass the --require-healthy flag:

vector --config /etc/vector/vector.toml --require-healthy

Disable Health Checks

If you'd like to disable health checks for this sink you can set the healthcheck option to false.

Metric Types

CloudWatch Metrics types are organized not by their semantics, but by storage properties:

  • Statistic Sets
  • Data Points

In Vector only the latter is used to allow lossless statistics calculations on CloudWatch side.

The following matrix outlines how Vector metric types are mapped into CloudWatch metrics types.

Vector MetricsCloudWatch Metrics
CounterData Point
GaugeData Point
Gauge Delta [*]Data Point
HistogramData Point
SetN/A

Streaming

The aws_cloudwatch_metrics sink streams data on a real-time event-by-event basis. It does not batch data.