Metric Event

Description

A Vector metric event represents a numerical operation performed on a time series. Unlike other tools, metrics in Vector are first class citizens, they are not represented as structured logs. This makes them interoperable with various metrics services without the need for any transformation.

Vector's metric data model is heavily inspried by Prometheus, Statsd, and Datadog.

Schema

counter

{
"metric" : {
"counter" : {
"value" : 1
},
"kind" : "absolute",
"name" : "memory_available_bytes",
"namespace" : "host",
"tags" : {
"host" : "my.host.com",
"instance_id" : "abcd1234"
},
"timestamp" : "2020-10-10T17:07:36+00:00"
}
}
  • commonoptionaltable

    counter

    A single value that can only be incremented or reset to zero value, it cannot be decremented.

    • commonrequiredfloat
      value

      The value to increment the counter by. Can only be positive.

      • View examples
  • enumcommonrequiredstring

    kind

    The metric value kind.

    • Enum, must be one of: "absolute" "incremental"
    • View examples
  • commonrequiredstring

    name

    The metric name.

    • View examples
  • commonrequiredstring

    namespace

    The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.

    • View examples
  • commonrequiredtable

    tags

    The metric tags. Key/value pairs, nesting is not allowed.

    • commonoptional*
      *

      Key/value pairs, nesting is not allowed.

    • commonrequiredtimestamp

      timestamp

      The metric timestamp; when the metric was created.

      • View examples

    distribution

    {
    "metric" : {
    "distribution" : {
    "sample_rates" : [
    12,
    43,
    25
    ],
    "statistic" : "histogram",
    "values" : [
    12,
    43.3,
    25.2
    ]
    },
    "kind" : "absolute",
    "name" : "memory_available_bytes",
    "namespace" : "host",
    "tags" : {
    "host" : "my.host.com",
    "instance_id" : "abcd1234"
    },
    "timestamp" : "2020-10-10T17:07:36+00:00"
    }
    }
    • commonoptionaltable

      distribution

      A distribution represents a distribution of sampled values. It is used with services that support global histograms and summaries.

      • commonrequired[uint]
        sample_rates

        The rate at which each individual value was sampled.

        • View examples
      • enumcommonrequiredstring
        statistic

        The statistic to be calculated from the values.

        • Enum, must be one of: "histogram" "summary"
        • View examples
      • commonrequired[float]
        values

        The list of values contained within the distribution.

        • View examples
    • enumcommonrequiredstring

      kind

      The metric value kind.

      • Enum, must be one of: "absolute" "incremental"
      • View examples
    • commonrequiredstring

      name

      The metric name.

      • View examples
    • commonrequiredstring

      namespace

      The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.

      • View examples
    • commonrequiredtable

      tags

      The metric tags. Key/value pairs, nesting is not allowed.

      • commonoptional*
        *

        Key/value pairs, nesting is not allowed.

      • commonrequiredtimestamp

        timestamp

        The metric timestamp; when the metric was created.

        • View examples

      gauge

      {
      "metric" : {
      "gauge" : {
      "value" : 1
      },
      "kind" : "absolute",
      "name" : "memory_available_bytes",
      "namespace" : "host",
      "tags" : {
      "host" : "my.host.com",
      "instance_id" : "abcd1234"
      },
      "timestamp" : "2020-10-10T17:07:36+00:00"
      }
      }
      • commonoptionaltable

        gauge

        A gauge represents a point-in-time value that can increase and decrease. Vector's internal gauge type represents changes to that value. Gauges should be used to track fluctuations in values, like current memory or CPU usage.

        • commonrequiredfloat
          value

          A specific point-in-time value for the gauge.

          • View examples
      • enumcommonrequiredstring

        kind

        The metric value kind.

        • Enum, must be one of: "absolute" "incremental"
        • View examples
      • commonrequiredstring

        name

        The metric name.

        • View examples
      • commonrequiredstring

        namespace

        The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.

        • View examples
      • commonrequiredtable

        tags

        The metric tags. Key/value pairs, nesting is not allowed.

        • commonoptional*
          *

          Key/value pairs, nesting is not allowed.

        • commonrequiredtimestamp

          timestamp

          The metric timestamp; when the metric was created.

          • View examples

        histogram

        {
        "metric" : {
        "histogram" : {
        "buckets" : [
        1,
        2,
        5,
        10,
        25
        ],
        "count" : 1,
        "counts" : [
        1,
        10,
        25,
        100
        ],
        "sum" : 1
        },
        "kind" : "absolute",
        "name" : "memory_available_bytes",
        "namespace" : "host",
        "tags" : {
        "host" : "my.host.com",
        "instance_id" : "abcd1234"
        },
        "timestamp" : "2020-10-10T17:07:36+00:00"
        }
        }
        • commonoptionaltable

          histogram

          Also called a "timer". A histogram samples observations (usually things like request durations or response sizes) and counts them in configurable buckets. It also provides a sum of all observed values.

          • commonrequired[float]
            buckets

            The buckets contained within this histogram.

            • View examples
          • commonrequireduint
            count

            The total number of values contained within the histogram.

            • View examples
          • commonrequired[uint]
            counts

            The number of values contained within each bucket.

            • View examples
          • commonrequiredfloat
            sum

            The sum of all values contained within the histogram.

            • View examples
        • enumcommonrequiredstring

          kind

          The metric value kind.

          • Enum, must be one of: "absolute" "incremental"
          • View examples
        • commonrequiredstring

          name

          The metric name.

          • View examples
        • commonrequiredstring

          namespace

          The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.

          • View examples
        • commonrequiredtable

          tags

          The metric tags. Key/value pairs, nesting is not allowed.

          • commonoptional*
            *

            Key/value pairs, nesting is not allowed.

          • commonrequiredtimestamp

            timestamp

            The metric timestamp; when the metric was created.

            • View examples

          set

          {
          "metric" : {
          "kind" : "absolute",
          "name" : "memory_available_bytes",
          "namespace" : "host",
          "set" : {
          "values" : [
          "value1",
          "value2"
          ]
          },
          "tags" : {
          "host" : "my.host.com",
          "instance_id" : "abcd1234"
          },
          "timestamp" : "2020-10-10T17:07:36+00:00"
          }
          }
          • enumcommonrequiredstring

            kind

            The metric value kind.

            • Enum, must be one of: "absolute" "incremental"
            • View examples
          • commonrequiredstring

            name

            The metric name.

            • View examples
          • commonrequiredstring

            namespace

            The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.

            • View examples
          • commonoptionaltable

            set

            A set represents an array of unique values.

            • commonrequired[string]
              values

              The list of unique values.

              • View examples
          • commonrequiredtable

            tags

            The metric tags. Key/value pairs, nesting is not allowed.

            • commonoptional*
              *

              Key/value pairs, nesting is not allowed.

            • commonrequiredtimestamp

              timestamp

              The metric timestamp; when the metric was created.

              • View examples

            summary

            {
            "metric" : {
            "kind" : "absolute",
            "name" : "memory_available_bytes",
            "namespace" : "host",
            "summary" : {
            "count" : 54,
            "quantiles" : [
            0.1,
            0.5,
            0.75,
            1
            ],
            "sum" : 1,
            "values" : [
            2.1,
            4.68,
            23.02,
            120.1
            ]
            },
            "tags" : {
            "host" : "my.host.com",
            "instance_id" : "abcd1234"
            },
            "timestamp" : "2020-10-10T17:07:36+00:00"
            }
            }
            • enumcommonrequiredstring

              kind

              The metric value kind.

              • Enum, must be one of: "absolute" "incremental"
              • View examples
            • commonrequiredstring

              name

              The metric name.

              • View examples
            • commonrequiredstring

              namespace

              The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.

              • View examples
            • commonoptionaltable

              summary

              Similar to a histogram, a summary samples observations (usually things like request durations and response sizes). While it also provides a total count of observations and a sum of all observed values, it calculates configurable quantiles over a sliding time window.

              • commonrequireduint
                count

                The total number of values contained within the summary.

                • View examples
              • commonrequired[float]
                quantiles

                The quantiles contained within the summary, where 0 ≤ quantile ≤ 1.

                • View examples
              • commonrequiredfloat
                sum

                The sum of all values contained within the histogram.

                • View examples
              • commonrequired[float]
                values

                The values contained within the summary that align with the quantiles.

                • View examples
            • commonrequiredtable

              tags

              The metric tags. Key/value pairs, nesting is not allowed.

              • commonoptional*
                *

                Key/value pairs, nesting is not allowed.

              • commonrequiredtimestamp

                timestamp

                The metric timestamp; when the metric was created.

                • View examples

              Components

              View all metric-compatible components

              How it works

              Global histograms and distributions

              Some services, namely Datadog, support distributions. Distributions represent a distribution of sampled values and global statisctics are computed downstream by the service. This strategy enables global histograms avoiding the need to guess and determine buckets beforehand.

              Dogstatsd

              Vector supports the Dogstatsd variant of Statsd, enabling the use of distributions through Statsd.

              Interoperability

              Metrics are first-class citizens in Vector, they are not represented as structured logs. Therefore, metrics in Vector are fully interoperable betwee disparate metrics services. For example, Vector can ingest metrics from Statsd and output them to Prometheus without any special configuration.

              Namespaces

              Metric namespaces in Vector are treated separately from the metric name. This allows you to easily change namespaces at the source and sink level and it ensures Vector can map namespaces properly to different metrics services. Some metrics services require the namespace to be prepended to the metric name, while others requier namespaces to be separate.

              Tags, labels, & high-cardinality

              Metrics tags, also known as labels, allows you to augment metrics with context. Tags are key/value pairs that cannot be nested. A caveat with tags is using high-cardinality labels. You should avoid using high-cardinality labels whenever possible. Vector provides a tag_cardinality_limi transform to protect against the accidental use of high-cardinality values.