Prometheus Exporter Sink

The Vector prometheus_exporter sink exposes metrics

Configuration

[sinks.my_sink_id]
type = "prometheus_exporter" # required
inputs = ["my-source-or-transform-id"] # required
address = "0.0.0.0:9598" # required
default_namespace = "service" # optional, no default
  • commonrequiredstring

    address

    The address to expose for scraping.

    • View examples
  • optional[float]

    buckets

    Default buckets to use for aggregating distribution metrics into histograms. See Histogram Buckets for more info.

    • Default: [0.005,0.01,0.025,0.05,0.1,0.25,0.5,1,2.5,5,10]
    • View examples
  • commonoptionalstring

    default_namespace

    Used as a namespace for metrics that don't have it. Typically namespaces are set during ingestion (sources), but it is optional and when missing, we'll use this value. It should follow Prometheus naming conventions.

    • View examples
  • optionaluint

    flush_period_secs

    Time interval between set values are reset.

    • Default: 60 (seconds)
  • optional[float]

    quantiles

    Quantiles to use for aggregating distribution metrics into a summary.

    • Default: [0.5,0.75,0.9,0.95,0.99]
    • View examples

Telemetry

This component provides the following metrics that can be retrieved through the internal_metrics source. See the metrics section in the monitoring page for more info.

  • counter

    processed_events_total

    The total number of events processed by this component. This metric includes the following tags:

    • component_kind - The Vector component kind.

    • component_name - The Vector component ID.

    • component_type - The Vector component type.

    • file - The file that produced the error

    • instance - The Vector instance identified by host and port.

    • job - The name of the job producing Vector metrics.

  • counter

    processed_bytes_total

    The total number of bytes processed by the component. This metric includes the following tags:

    • component_kind - The Vector component kind.

    • component_name - The Vector component ID.

    • component_type - The Vector component type.

    • instance - The Vector instance identified by host and port.

    • job - The name of the job producing Vector metrics.

Examples

Counter

Given the following Vector event:

{
"metric": {
"kind": "incremental",
"name": "logins",
"counter": {
"value": 1.5
},
"tags": {
"host": "my-host.local"
}
}
}

And the following configuration:

[sinks.prometheus_exporter]
type = "prometheus_exporter"
default_namespace = "service"

The following output will be produced:

HELP service_logins logins

TYPE service_logins counter

service_logins{host="my-host.local"} 1.5

How It Works

Histogram Buckets

Choosing the appropriate buckets for Prometheus histograms is a complicated point of discussion. The Histograms and Summaries Prometheus guide) provides a good overview of histograms, buckets, summaries, and how you should think about configuring them. The buckets you choose should align with your known range and distribution of values as well as how you plan to report on them. The aforementioned guide provides examples on how you should align them.

Default Buckets

The buckets option defines the global default buckets for histograms. These defaults are tailored to broadly measure the response time (in seconds) of a network service. Most likely, however, you will be required to define buckets customized to your use case.

Memory Usage

Like other Prometheus instances, the prometheus sink aggregates metrics in memory which keeps the memory footprint to a minimum if Prometheus fails to scrape the Vector instance over an extended period of time. The downside is that data will be lost if Vector is restarted. This is by design of Prometheus' pull model approach, but is worth noting if restart Vector frequently.