Metric Event
Description
A Vector metric event represents a numerical operation performed on a time series. Unlike other tools, metrics in Vector are first class citizens, they are not represented as structured logs. This makes them interoperable with various metrics services without the need for any transformation.
Vector's metric data model favors accuracy and correctness over ideological purity. Therefore, Vector's metric types are a conglomeration of various metric types found in the wild, such as Prometheus and Statsd. This ensures metric data is correctly interoperable between systems.
Schema
counter
{"metric" : {"counter" : {"value" : 1},"kind" : "absolute","name" : "memory_available_bytes","namespace" : "host","tags" : {"host" : "my.host.com","instance_id" : "abcd1234"},"timestamp" : "2020-10-10T17:07:36+00:00"}}
- optionaltable
counter
A single value that can only be incremented or reset to zero value, it cannot be decremented.
- requiredfloat
value
The value to increment the counter by. Can only be positive.
- View examples
- requiredstring
kind
The metric value kind.
- Enum, must be one of:
"absolute"
"incremental"
- View examples
- Enum, must be one of:
- requiredstring
name
The metric name.
- View examples
- requiredstring
namespace
The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.
- View examples
- requiredtable
tags
The metric tags. Key/value pairs, nesting is not allowed.
- optional*
*
Key/value pairs, nesting is not allowed.
- requiredtimestamp
timestamp
The metric timestamp; when the metric was created.
- View examples
distribution
{"metric" : {"distribution" : {"sample_rates" : [12,43,25],"statistic" : "histogram","values" : [12,43.3,25.2]},"kind" : "absolute","name" : "memory_available_bytes","namespace" : "host","tags" : {"host" : "my.host.com","instance_id" : "abcd1234"},"timestamp" : "2020-10-10T17:07:36+00:00"}}
- optionaltable
distribution
A distribution represents a distribution of sampled values. It is used with services that support global histograms and summaries.
- required[uint]
sample_rates
The rate at which each individual value was sampled.
- View examples
- requiredstring
statistic
The statistic to be calculated from the values.
- Enum, must be one of:
"histogram"
"summary"
- View examples
- Enum, must be one of:
- required[float]
values
The list of values contained within the distribution.
- View examples
- requiredstring
kind
The metric value kind.
- Enum, must be one of:
"absolute"
"incremental"
- View examples
- Enum, must be one of:
- requiredstring
name
The metric name.
- View examples
- requiredstring
namespace
The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.
- View examples
- requiredtable
tags
The metric tags. Key/value pairs, nesting is not allowed.
- optional*
*
Key/value pairs, nesting is not allowed.
- requiredtimestamp
timestamp
The metric timestamp; when the metric was created.
- View examples
gauge
{"metric" : {"gauge" : {"value" : 1},"kind" : "absolute","name" : "memory_available_bytes","namespace" : "host","tags" : {"host" : "my.host.com","instance_id" : "abcd1234"},"timestamp" : "2020-10-10T17:07:36+00:00"}}
- optionaltable
gauge
A gauge represents a point-in-time value that can increase and decrease. Vector's internal gauge type represents changes to that value. Gauges should be used to track fluctuations in values, like current memory or CPU usage.
- requiredfloat
value
A specific point-in-time value for the gauge.
- View examples
- requiredstring
kind
The metric value kind.
- Enum, must be one of:
"absolute"
"incremental"
- View examples
- Enum, must be one of:
- requiredstring
name
The metric name.
- View examples
- requiredstring
namespace
The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.
- View examples
- requiredtable
tags
The metric tags. Key/value pairs, nesting is not allowed.
- optional*
*
Key/value pairs, nesting is not allowed.
- requiredtimestamp
timestamp
The metric timestamp; when the metric was created.
- View examples
histogram
{"metric" : {"histogram" : {"buckets" : [1,2,5,10,25],"count" : 1,"counts" : [1,10,25,100],"sum" : 1},"kind" : "absolute","name" : "memory_available_bytes","namespace" : "host","tags" : {"host" : "my.host.com","instance_id" : "abcd1234"},"timestamp" : "2020-10-10T17:07:36+00:00"}}
- optionaltable
histogram
Also called a "timer". A histogram samples observations (usually things like request durations or response sizes) and counts them in configurable buckets. It also provides a sum of all observed values.
- required[float]
buckets
The buckets contained within this histogram.
- View examples
- requireduint
count
The total number of values contained within the histogram.
- View examples
- required[uint]
counts
The number of values contained within each bucket.
- View examples
- requiredfloat
sum
The sum of all values contained within the histogram.
- View examples
- requiredstring
kind
The metric value kind.
- Enum, must be one of:
"absolute"
"incremental"
- View examples
- Enum, must be one of:
- requiredstring
name
The metric name.
- View examples
- requiredstring
namespace
The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.
- View examples
- requiredtable
tags
The metric tags. Key/value pairs, nesting is not allowed.
- optional*
*
Key/value pairs, nesting is not allowed.
- requiredtimestamp
timestamp
The metric timestamp; when the metric was created.
- View examples
set
{"metric" : {"kind" : "absolute","name" : "memory_available_bytes","namespace" : "host","set" : {"values" : ["value1","value2"]},"tags" : {"host" : "my.host.com","instance_id" : "abcd1234"},"timestamp" : "2020-10-10T17:07:36+00:00"}}
- requiredstring
kind
The metric value kind.
- Enum, must be one of:
"absolute"
"incremental"
- View examples
- Enum, must be one of:
- requiredstring
name
The metric name.
- View examples
- requiredstring
namespace
The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.
- View examples
- optionaltable
set
A set represents an array of unique values.
- required[string]
values
The list of unique values.
- View examples
- requiredtable
tags
The metric tags. Key/value pairs, nesting is not allowed.
- optional*
*
Key/value pairs, nesting is not allowed.
- requiredtimestamp
timestamp
The metric timestamp; when the metric was created.
- View examples
summary
{"metric" : {"kind" : "absolute","name" : "memory_available_bytes","namespace" : "host","summary" : {"count" : 54,"quantiles" : [0.1,0.5,0.75,1],"sum" : 1,"values" : [2.1,4.68,23.02,120.1]},"tags" : {"host" : "my.host.com","instance_id" : "abcd1234"},"timestamp" : "2020-10-10T17:07:36+00:00"}}
- requiredstring
kind
The metric value kind.
- Enum, must be one of:
"absolute"
"incremental"
- View examples
- Enum, must be one of:
- requiredstring
name
The metric name.
- View examples
- requiredstring
namespace
The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.
- View examples
- optionaltable
summary
Similar to a histogram, a summary samples observations (usually things like request durations and response sizes). While it also provides a total count of observations and a sum of all observed values, it calculates configurable quantiles over a sliding time window.
- requireduint
count
The total number of values contained within the summary.
- View examples
- required[float]
quantiles
The quantiles contained within the summary, where 0 ≤ quantile ≤ 1.
- View examples
- requiredfloat
sum
The sum of all values contained within the histogram.
- View examples
- required[float]
values
The values contained within the summary that align with the
quantiles
.- View examples
- requiredtable
tags
The metric tags. Key/value pairs, nesting is not allowed.
- optional*
*
Key/value pairs, nesting is not allowed.
- requiredtimestamp
timestamp
The metric timestamp; when the metric was created.
- View examples
Components
How it works
Global histograms and distributions
Some services, namely Datadog, support distributions. Distributions represent a distribution of sampled values and global statisctics are computed downstream by the service. This strategy enables global histograms avoiding the need to guess and determine buckets beforehand.
Dogstatsd
Vector supports the Dogstatsd variant of Statsd, enabling the use of distributions through Statsd.
Interoperability
Metrics are first-class citizens in Vector, they are not represented as structured logs. Therefore, metrics in Vector are fully interoperable betwee disparate metrics services. For example, Vector can ingest metrics from Statsd and output them to Prometheus without any special configuration.
Namespaces
Metric namespaces in Vector are treated separately from the metric name. This allows you to easily change namespaces at the source and sink level and it ensures Vector can map namespaces properly to different metrics services. Some metrics services require the namespace to be prepended to the metric name, while others requier namespaces to be separate.
Tags, labels, & high-cardinality
Metrics tags, also known as labels, allows you to augment metrics with context.
Tags are key/value pairs that cannot be nested. A caveat with tags is using
high-cardinality labels. You should avoid using high-cardinality labels
whenever possible. Vector provides a
tag_cardinality_limi
transform to
protect against the accidental use of high-cardinality values.