AWS ECS metrics

Collect Docker container stats for tasks running in AWS ECS and AWS Fargate

status: stable role: sidecar delivery: at-least-once acknowledgements: no egress: batch state: stateless output: metrics
Page source Edit this page
Collects the docker container stats for tasks running in AWS ECS or AWS Fargate.

Configuration

Example configurations 

{
  "sources": {
    "my_source_id": {
      "type": "aws_ecs_metrics"
    }
  }
}
[sources.my_source_id]
type = "aws_ecs_metrics"

sources:
  my_source_id:
    type: aws_ecs_metrics

{
  "sources": {
    "my_source_id": {
      "type": "aws_ecs_metrics",
      "endpoint": "http://169.254.170.2/v2",
      "namespace": "awsecs",
      "scrape_interval_secs": 15,
      "version": "v2"
    }
  }
}
[sources.my_source_id]
type = "aws_ecs_metrics"
endpoint = "http://169.254.170.2/v2"
namespace = "awsecs"
scrape_interval_secs = 15
version = "v2"

sources:
  my_source_id:
    type: aws_ecs_metrics
    endpoint: http://169.254.170.2/v2
    namespace: awsecs
    scrape_interval_secs: 15
    version: v2

endpoint

optional string literal

Base URI of the task metadata endpoint.

If empty, the URI is automatically discovered based on the latest version detected.

By default:

  • The version 4 endpoint base URI is stored in the environment variable ECS_CONTAINER_METADATA_URI_V4.
  • The version 3 endpoint base URI is stored in the environment variable ECS_CONTAINER_METADATA_URI.
  • The version 2 endpoint base URI is 169.254.170.2/v2/.
default: http://169.254.170.2/v2

namespace

optional string literal

The namespace of the metric.

Disabled if empty.

default: awsecs

scrape_interval_secs

optional uint
The interval between scrapes, in seconds.
default: 15 (seconds)

version

optional string literal enum

The version of the task metadata endpoint to use.

If empty, the version is automatically discovered based on environment variables.

By default:

  • Version 4 is used if the environment variable ECS_CONTAINER_METADATA_URI_V4 is defined.
  • Version 3 is used if the environment variable ECS_CONTAINER_METADATA_URI_V4 is not defined, but the environment variable ECS_CONTAINER_METADATA_URI is defined.
  • Version 2 is used if neither of the environment variables ECS_CONTAINER_METADATA_URI_V4 or ECS_CONTAINER_METADATA_URI are defined.
Enum options string literal
OptionDescription
v2

Version 2.

More information about version 2 of the task metadata endpoint can be found here.

v3

Version 3.

More information about version 3 of the task metadata endpoint can be found here.

v4

Version 4.

More information about version 4 of the task metadata endpoint can be found here.

default: v2

Outputs

<component_id>

Default output stream of the component. Use this component’s ID as an input to downstream transforms and sinks.

Output Data

Metrics

blkio_recursive_io_merged_total

counter
Total number of bios/requests merged into requests.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.
device
Device identified by its major and minor numbers.
op
The operation type.

blkio_recursive_io_queued_total

counter
Total number of requests queued up at any given instant.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.
device
Device identified by its major and minor numbers.
op
The operation type.

blkio_recursive_io_service_bytes_total

counter
Number of bytes transferred to/from the disk.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.
device
Device identified by its major and minor numbers.
op
The operation type.

blkio_recursive_io_service_time_seconds_total

counter
Total amount of time in seconds between request dispatch and request completion for the IOs done.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.
device
Device identified by its major and minor numbers.
op
The operation type.

blkio_recursive_io_serviced_total

counter
Number of IOs completed to/from the disk.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.
device
Device identified by its major and minor numbers.
op
The operation type.

blkio_recursive_io_time_seconds_total

counter
Disk time allocated per device in seconds.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.
device
Device identified by its major and minor numbers.
op
The operation type.

blkio_recursive_io_wait_time_seconds_total

counter
Total amount of time in seconds the IOs spent waiting in the scheduler queues for service.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.
device
Device identified by its major and minor numbers.
op
The operation type.

blkio_recursive_sectors_total

counter
Number of sectors transferred to/from disk.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.
device
Device identified by its major and minor numbers.
op
The operation type.

cpu_online_cpus

gauge
Number of CPU cores.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

cpu_throttled_periods_total

counter
Number of periods throttled.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

cpu_throttled_time_seconds_total

counter
Throttling time in seconds.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

cpu_throttling_periods_total

counter
Number of periods.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

cpu_usage_kernelmode_jiffies_total

counter
Jiffies of CPU time spent in kernel mode by the container.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

cpu_usage_percpu_jiffies_total

counter
Jiffies of CPU time used by the container, per CPU core.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.
cpu
CPU core identifier.

cpu_usage_system_jiffies_total

counter
Jiffies of CPU time used by the system.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

cpu_usage_total_jiffies_total

counter
Jiffies of CPU time used by the container.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

cpu_usage_usermode_jiffies_total

counter
Jiffies of CPU time spent in user mode by the container.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_active_anonymous_bytes

gauge
Amount of memory that has been identified as active by the kernel. Anonymous memory is memory that is not linked to disk pages.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_active_file_bytes

gauge
Amount of active file cache memory. Cache memory = active_file + inactive_file + tmpfs.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_cache_bytes

gauge
The amount of memory used by the processes of this cgroup that can be associated with a block on a block device. Also accounts for memory used by tmpfs.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_dirty_bytes

gauge
The amount of memory waiting to get written to disk.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_hierarchical_memory_limit_bytes

gauge
The memory limit in place by the hierarchy cgroup.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_hierarchical_memsw_limit_bytes

gauge
The memory + swap limit in place by the hierarchy cgroup.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_inactive_anonymous_bytes

gauge
Amount of memory that has been identified as inactive by the kernel.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_inactive_file_bytes

gauge
Amount of inactive file cache memory.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_limit_bytes

gauge
Memory usage limit of the container, in bytes.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_major_faults_total

counter
Number of times that a process of the cgroup triggered a major page fault.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_mapped_file_bytes

gauge
Indicates the amount of memory mapped by the processes in the cgroup. It doesn’t give you information about how much memory is used; it rather tells you how it is used.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_max_used_bytes

gauge
Maximum measured memory usage of the container, in bytes.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_page_charged_total

counter
Number of charging events to the memory cgroup. Charging events happen each time a page is accounted as either mapped anon page(RSS) or cache page to the cgroup.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_page_faults_total

counter
Number of times that a process of the cgroup triggered a page fault.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_page_uncharged_total

counter
Number of uncharging events to the memory cgroup. Uncharging events happen each time a page is unaccounted from the cgroup.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_rss_bytes

gauge
The amount of memory that doesn’t correspond to anything on disk: stacks, heaps, and anonymous memory maps.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_rss_hugepages_bytes

gauge
Amount of memory due to anonymous transparent hugepages.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_total_active_anonymous_bytes

gauge
Total amount of memory that has been identified as active by the kernel.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_total_active_file_bytes

gauge
Total amount of active file cache memory.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_total_cache_bytes

gauge
Total amount of memory used by the processes of this cgroup that can be associated with a block on a block device.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_total_dirty_bytes

gauge
Total amount of memory waiting to get written to disk.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_total_inactive_anonymous_bytes

gauge
Total amount of memory that has been identified as inactive by the kernel.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_total_inactive_file_bytes

gauge
Total amount of inactive file cache memory.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_total_major_faults_total

counter
Total number of major page faults.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_total_mapped_file_bytes

gauge
Total amount of memory mapped by the processes in the cgroup.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_total_page_charged_total

counter
Total number of charging events.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_total_page_faults_total

counter
Total number of page faults.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_total_page_uncharged_total

counter
Total number of uncharging events.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_total_rss_bytes

gauge
Total amount of memory that doesn’t correspond to anything on disk: stacks, heaps, and anonymous memory maps.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_total_rss_hugepages_bytes

gauge
Total amount of memory due to anonymous transparent hugepages.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_total_unevictable_bytes

gauge
Total amount of memory that can not be reclaimed.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_total_writeback_bytes

gauge
Total amount of memory from file/anon cache that are queued for syncing to the disk.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_unevictable_bytes

gauge
The amount of memory that cannot be reclaimed.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_used_bytes

gauge
Memory used by the container, in bytes.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

memory_writeback_bytes

gauge
The amount of memory from file/anon cache that are queued for syncing to the disk.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.

network_receive_bytes_total

counter
Bytes received by the container via the network interface.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.
device
The network interface.

network_receive_errs_total

counter
Errors receiving packets.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.
device
The network interface.

network_receive_packets_drop_total

counter
Number of inbound packets dropped by the container.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.
device
The network interface.

network_receive_packets_total

counter
Number of packets received by the container via the network interface.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.
device
The network interface.

network_transmit_bytes_total

counter
Bytes sent by the container via the network interface.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.
device
The network interface.

network_transmit_errs_total

counter
Errors sending packets.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.
device
The network interface.

network_transmit_packets_drop_total

counter
Number of outbound packets dropped by the container.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.
device
The network interface.

network_transmit_packets_total

counter
Number of packets sent by the container via the network interface.
container_id
The identifier of the ECS container.
container_name
The name of the ECS container.
device
The network interface.

Telemetry

Metrics

link

component_discarded_events_total

counter
The number of events dropped by this component.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_type
The Vector component type.
host optional
The hostname of the system Vector is running on.
intentional
True if the events were discarded intentionally, like a filter transform, or false if due to an error.
pid optional
The process ID of the Vector instance.

component_errors_total

counter
The total number of errors encountered by this component.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_type
The Vector component type.
error_type
The type of the error
host optional
The hostname of the system Vector is running on.
pid optional
The process ID of the Vector instance.
stage
The stage within the component at which the error occurred.

component_received_bytes_total

counter
The number of raw bytes accepted by this component from source origins.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_type
The Vector component type.
container_name optional
The name of the container from which the data originated.
file optional
The file from which the data originated.
host optional
The hostname of the system Vector is running on.
mode optional
The connection mode used by the component.
peer_addr optional
The IP from which the data originated.
peer_path optional
The pathname from which the data originated.
pid optional
The process ID of the Vector instance.
pod_name optional
The name of the pod from which the data originated.
uri optional
The sanitized URI from which the data originated.

component_received_event_bytes_total

counter
The number of event bytes accepted by this component either from tagged origins like file and uri, or cumulatively from other origins.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_type
The Vector component type.
container_name optional
The name of the container from which the data originated.
file optional
The file from which the data originated.
host optional
The hostname of the system Vector is running on.
mode optional
The connection mode used by the component.
peer_addr optional
The IP from which the data originated.
peer_path optional
The pathname from which the data originated.
pid optional
The process ID of the Vector instance.
pod_name optional
The name of the pod from which the data originated.
uri optional
The sanitized URI from which the data originated.

component_received_events_count

histogram

A histogram of the number of events passed in each internal batch in Vector’s internal topology.

Note that this is separate than sink-level batching. It is mostly useful for low level debugging performance issues in Vector due to small internal batches.

component_id
The Vector component ID.
component_kind
The Vector component kind.
component_type
The Vector component type.
container_name optional
The name of the container from which the data originated.
file optional
The file from which the data originated.
host optional
The hostname of the system Vector is running on.
mode optional
The connection mode used by the component.
peer_addr optional
The IP from which the data originated.
peer_path optional
The pathname from which the data originated.
pid optional
The process ID of the Vector instance.
pod_name optional
The name of the pod from which the data originated.
uri optional
The sanitized URI from which the data originated.

component_received_events_total

counter
The number of events accepted by this component either from tagged origins like file and uri, or cumulatively from other origins.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_type
The Vector component type.
container_name optional
The name of the container from which the data originated.
file optional
The file from which the data originated.
host optional
The hostname of the system Vector is running on.
mode optional
The connection mode used by the component.
peer_addr optional
The IP from which the data originated.
peer_path optional
The pathname from which the data originated.
pid optional
The process ID of the Vector instance.
pod_name optional
The name of the pod from which the data originated.
uri optional
The sanitized URI from which the data originated.

component_sent_event_bytes_total

counter
The total number of event bytes emitted by this component.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_type
The Vector component type.
host optional
The hostname of the system Vector is running on.
output optional
The specific output of the component.
pid optional
The process ID of the Vector instance.

component_sent_events_total

counter
The total number of events emitted by this component.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_type
The Vector component type.
host optional
The hostname of the system Vector is running on.
output optional
The specific output of the component.
pid optional
The process ID of the Vector instance.

http_client_response_rtt_seconds

histogram
The round-trip time (RTT) of HTTP requests, tagged with the response code.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_type
The Vector component type.
host optional
The hostname of the system Vector is running on.
pid optional
The process ID of the Vector instance.
status optional
The HTTP status code of the request.

http_client_responses_total

counter
The total number of HTTP requests, tagged with the response code.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_type
The Vector component type.
host optional
The hostname of the system Vector is running on.
pid optional
The process ID of the Vector instance.
status optional
The HTTP status code of the request.

source_lag_time_seconds

histogram
The difference between the timestamp recorded in each event and the time when it was ingested, expressed as fractional seconds.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_type
The Vector component type.
host optional
The hostname of the system Vector is running on.
pid optional
The process ID of the Vector instance.

How it works

Context

By default, the aws_ecs_metrics source augments events with helpful context keys.

State

This component is stateless, meaning its behavior is consistent across each input.
Sidebar