transmit

Apache Pulsar Sink

The Vector pulsar sink streams log events to Apache Pulsar via the Pulsar protocol.

Configuration

vector.toml
[sinks.my_sink_id]
  # General
  type = "pulsar" # required
  inputs = ["my-source-or-transform-id"] # required
  address = "pulsar://127.0.0.1:6650" # required
  healthcheck = true # optional, default
  topic = "topic-1234" # required


  # Encoding
  encoding.codec = "text" # optional, default
5 items
  • stringcommonrequired

    address

    A host and port pair that the pulsar client should connect to.

    • No default
    • View examples
  • tableoptional

    auth

    Options for the authentication strategy.

    • stringoptional

      name

      The basic authentication name.

      • No default
      • View examples
    • stringoptional

      token

      The basic authentication password.

      • No default
      • View examples
  • tablecommonoptional

    encoding

    Configures the encoding specific sink behavior.

    • stringenumcommonoptional

      codec

      The encoding codec used to serialize the events before outputting.

      • Default: "text"
      • Enum, must be one of: "json" "text"
      • View examples
    • [string]optional

      except_fields

      Prevent the sink from encoding the specified labels.

      • No default
      • View examples
    • [string]optional

      only_fields

      Limit the sink to only encoding the specified labels.

      • No default
      • View examples
    • stringenumoptional

      timestamp_format

      How to format event timestamps.

      • Default: "rfc3339"
      • Enum, must be one of: "rfc3339" "unix"
      • View examples
  • boolcommonoptional

    healthcheck

    Enables/disables the sink healthcheck upon start. See Health Checks for more info.

    • Default: true
    • View examples
  • stringcommonrequired

    topic

    The Pulsar topic name to write events to.

    • No default
    • View examples

How It Works

Environment Variables

Environment variables are supported through all of Vector's configuration. Simply add ${MY_ENV_VAR} in your Vector configuration file and the variable will be replaced before being evaluated.

You can learn more in the Environment Variables section.

Health Checks

Health checks ensure that the downstream service is accessible and ready to accept data. This check is performed upon sink initialization. If the health check fails an error will be logged and Vector will proceed to start.

Require Health Checks

If you'd like to exit immediately upon a health check failure, you can pass the --require-healthy flag:

vector --config /etc/vector/vector.toml --require-healthy

Disable Health Checks

If you'd like to disable health checks for this sink you can set the healthcheck option to false.

Streaming

The pulsar sink streams data on a real-time event-by-event basis. It does not batch data.

Support
Contents
Resources