Sampler Transform

The Vector sampler transform samples events at a configurable rate.

Configuration

[transforms.my_transform_id]
# General
type = "sampler" # required
inputs = ["my-source-or-transform-id"] # required
rate = 10 # required
# Exclude
exclude.type = "check_fields" # example
exclude."message.eq" = "foo" # example
exclude."message.not_eq" = "foo" # example
exclude."message.exists" = true # example
exclude."message.not_exists" = true # example
exclude."message.contains" = "foo" # example
exclude."message.not_contains" = "foo" # example
exclude."message.ends_with" = "foo" # example
exclude."message.not_ends_with" = "foo" # example
exclude."message.ip_cidr_contains" = "10.0.0.0/8" # example
exclude."message.not_ip_cidr_contains" = "10.0.0.0/8" # example
exclude."message.regex" = " (any|of|these|five|words) " # example
exclude."message.not_regex" = " (any|of|these|five|words) " # example
exclude."message.starts_with" = "foo" # example
exclude."message.not_starts_with" = "foo" # example
  • commonoptionaltable

    exclude

    The set of logical conditions to exclude events from sampling.

    • commonoptionalstring

      *.contains

      Checks whether a string field contains a string argument, case sensitive. This may be a single string or a list of strings, in which case this evaluates to true if any of the list matches.

      • View examples
    • commonoptionalstring

      *.ends_with

      Checks whether a string field ends with a string argument, case sensitive. This may be a single string or a list of strings, in which case this evaluates to true if any of the list matches.

      • View examples
    • commonoptionalstring

      *.eq

      Check whether a field's contents exactly matches the value specified, case sensitive. This may be a single string or a list of strings, in which case this evaluates to true if any of the list matches.

      • View examples
    • optionalbool

      *.exists

      Check whether a field exists or does not exist, depending on the provided value being true or false respectively.

      • View examples
    • optionalstring

      *.ip_cidr_contains

      Checks whether an IP field is contained within a given IP CIDR (works with IPv4 and IPv6). This may be a single string or a list of strings, in which case this evaluates to true if the IP field is contained within any of the CIDRs in the list.

      • View examples
    • optionalstring

      .not_

      Allow you to negate any condition listed here.

      • commonoptionalstring

        *.regex

        Checks whether a string field matches a regular expression. Vector uses the documented Rust Regex syntax. Note that this condition is considerably more expensive than a regular string match (such as starts_with or contains) so the use of those conditions are preferred where possible.

        • View examples
      • commonoptionalstring

        *.starts_with

        Checks whether a string field starts with a string argument, case sensitive. This may be a single string or a list of strings, in which case this evaluates to true if any of the list matches.

        • View examples
      • enumcommonoptionalstring

        type

        The type of the condition to execute.

        • Default: "check_fields"
        • Enum, must be one of: "check_fields" "is_log" "is_metric"
        • View examples
    • optionalstring

      key_field

      The name of the log field whose value will be hashed to determine if the event should be passed. Consistently samples the same events. Actual rate of sampling may differ from the configured one if values in the field are not uniformly distributed. If left unspecified, or if the event doesn't have key_field, events will be count rated.

      • View examples
    • commonrequireduint

      rate

      The rate at which events will be forwarded, expressed as 1/N. For example, rate = 10 means 1 out of every 10 events will be forwarded and the rest will be dropped.

      • View examples

    Telemetry

    This component provides the following metrics that can be retrieved through the internal_metrics source. See the metrics section in the monitoring page for more info.

    • counter

      events_discarded_total

      The total number of events discarded by this component. This metric includes the following tags:

      • instance - The Vector instance identified by host and port.

      • job - The name of the job producing Vector metrics.

    • counter

      processed_events_total

      The total number of events processed by this component. This metric includes the following tags:

      • component_kind - The Vector component kind.

      • component_name - The Vector component ID.

      • component_type - The Vector component type.

      • file - The file that produced the error

      • instance - The Vector instance identified by host and port.

      • job - The name of the job producing Vector metrics.

    • counter

      processed_bytes_total

      The total number of bytes processed by the component. This metric includes the following tags:

      • component_kind - The Vector component kind.

      • component_name - The Vector component ID.

      • component_type - The Vector component type.

      • instance - The Vector instance identified by host and port.

      • job - The name of the job producing Vector metrics.