AWS EC2 Metadata Transform

The Vector aws_ec2_metadata transform enriches log events with AWS EC2 environment metadata.

Requirements

Configuration

[transforms.my_transform_id]
type = "aws_ec2_metadata" # required
inputs = ["my-source-or-transform-id", "prefix-*"] # required
fields = ["instance-id", "local-hostname", "local-ipv4", "public-hostname", "public-ipv4", "ami-id", "availability-zone", "vpc-id", "subnet-id", "region"] # optional, default
namespace = "" # optional, default
refresh_interval_secs = 10 # optional, default
  • optionalstring

    endpoint

    Override the default EC2 Metadata endpoint.

    • Syntax: literal
    • Default: "http://169.254.169.254"
  • commonoptional[string]

    fields

    A list of fields to include in each event.

    • Default: ["instance-id","local-hostname","local-ipv4","public-hostname","public-ipv4","ami-id","availability-zone","vpc-id","subnet-id","region"]
  • commonoptionalstring

    namespace

    Prepend a namespace to each field's key.

    • Syntax: literal
    • Default: ""
  • commonoptionaluint

    refresh_interval_secs

    The interval in seconds at which the EC2 Metadata api will be called.

    • Default: 10

Output

This component outputs log events with the following fields:

{
"ami-id" : "ami-00068cd7555f543d5",
"availability-zone" : "54.234.246.107",
"instance-id" : "i-096fba6d03d36d262",
"local-hostname" : "ip-172-31-93-227.ec2.internal",
"local-ipv4" : "172.31.93.227",
"public-hostname" : "ec2-54-234-246-107.compute-1.amazonaws.com",
"public-ipv4" : "54.234.246.107",
"region" : "us-east-1",
"role-name" : "some_iam_role",
"subnet-id" : "subnet-9d6713b9",
"vpc-id" : "vpc-a51da4dc"
}
  • commonrequiredstring

    ami-id

    The ami-id that the current EC2 instance is using.

    • Syntax: literal
  • commonrequiredstring

    availability-zone

    The availability-zone that the current EC2 instance is running in.

    • Syntax: literal
  • commonrequiredstring

    instance-id

    The instance-id of the current EC2 instance.

    • Syntax: literal
  • commonrequiredstring

    local-hostname

    The local-hostname of the current EC2 instance.

    • Syntax: literal
  • commonrequiredstring

    local-ipv4

    The local-ipv4 of the current EC2 instance.

    • Syntax: literal
  • commonrequiredstring

    public-hostname

    The public-hostname of the current EC2 instance.

    • Syntax: literal
  • commonrequiredstring

    public-ipv4

    The public-ipv4 of the current EC2 instance.

    • Syntax: literal
  • commonrequiredstring

    region

    The region that the current EC2 instance is running in.

    • Syntax: literal
  • commonrequiredstring

    role-name

    The role-name that the current EC2 instance is using.

    • Syntax: literal
  • commonrequiredstring

    subnet-id

    The subnet-id of the current EC2 instance's default network interface.

    • Syntax: literal
  • commonrequiredstring

    vpc-id

    The vpc-id of the current EC2 instance's default network interface.

    • Syntax: literal

Telemetry

This component provides the following metrics that can be retrieved through the internal_metrics source. See the metrics section in the monitoring page for more info.

  • counter

    metadata_refresh_failed_total

    The total number of failed efforts to refresh AWS EC2 metadata. This metric includes the following tags:

    • component_kind - The Vector component kind.

    • component_name - The Vector component ID.

    • component_type - The Vector component type.

    • instance - The Vector instance identified by host and port.

    • job - The name of the job producing Vector metrics.

  • counter

    events_in_total

    The number of events accepted by this component either from tagged origin like file and uri, or cumulatively from other origins. This metric includes the following tags:

    • component_kind - The Vector component kind.

    • component_name - The Vector component ID.

    • component_type - The Vector component type.

    • container_name - The name of the container from which the event originates.

    • file - The file from which the event originates.

    • instance - The Vector instance identified by host and port.

    • job - The name of the job producing Vector metrics.

    • mode - The connection mode used by the component.

    • peer_addr - The IP from which the event originates.

    • peer_path - The pathname from which the event originates.

    • pod_name - The name of the pod from which the event originates.

    • uri - The sanitized uri from which the event originates.

  • counter

    processed_events_total

    The total number of events processed by this component. This metric includes the following tags:

    • component_kind - The Vector component kind.

    • component_name - The Vector component ID.

    • component_type - The Vector component type.

    • file - The file that produced the error

    • instance - The Vector instance identified by host and port.

    • job - The name of the job producing Vector metrics.

  • counter

    metadata_refresh_successful_total

    The total number of AWS EC2 metadata refreshes. This metric includes the following tags:

    • component_kind - The Vector component kind.

    • component_name - The Vector component ID.

    • component_type - The Vector component type.

    • instance - The Vector instance identified by host and port.

    • job - The name of the job producing Vector metrics.

  • counter

    events_out_total

    The total number of events emitted by this component. This metric includes the following tags:

    • component_kind - The Vector component kind.

    • component_name - The Vector component ID.

    • component_type - The Vector component type.

    • instance - The Vector instance identified by host and port.

    • job - The name of the job producing Vector metrics.

  • counter

    processed_bytes_total

    The number of bytes processed by the component. This metric includes the following tags:

    • component_kind - The Vector component kind.

    • component_name - The Vector component ID.

    • component_type - The Vector component type.

    • container_name - The name of the container from which the bytes originate.

    • file - The file from which the bytes originate.

    • instance - The Vector instance identified by host and port.

    • job - The name of the job producing Vector metrics.

    • mode - The connection mode used by the component.

    • peer_addr - The IP from which the bytes originate.

    • peer_path - The pathname from which the bytes originate.

    • pod_name - The name of the pod from which the bytes originate.

    • uri - The sanitized uri from which the bytes originate.

How It Works

State

This component is stateless, meaning its behavior is consistent across each input.