OTLP Support

This was originally use_otlp_decoding, article was updated to reflect current config

Summary

We are excited to announce that the opentelemetry source now supports OpenTelemetry protocol decoding.

This is possible by setting decoding.codec to otlp. This setup allows shipping OTLP formatted logs to an OTEL collector without the use of a remap transform. The same can be done for metrics and traces. However, OTLP formatted metrics cannot be converted to Vector’s metrics format. As a workaround, the OTLP metrics are converted to Vector log events while preserving the OTLP format. This prohibits the use of metric transforms like aggregate but it enables easy shipping to OTEL collectors.

Example Configuration 1

Here is an example on how to setup an OTEL -> Vector -> OTEL pipeline:

sources:
  source0:
    type: opentelemetry
    grpc:
      address: 0.0.0.0:4317
    http:
      address: 0.0.0.0:4318
    decoding:
      codec: otlp
sinks:
  otel_sink:
    inputs:
      - source0.logs
    type: opentelemetry
    protocol:
      type: http
      uri: http://otel-collector-sink:5318/v1/logs
      method: post
      encoding:
        codec: otlp

The above configuration requires Vector 0.60.0 or later. For earlier versions, use use_otlp_decoding: true instead of decoding. See the migration guide for details.

Example Configuration 2

Here is another pipeline configuration that can achieve the same as the above:

otel_sink:
  inputs:
    - otel.logs
  type: opentelemetry
  protocol:
    type: http
    uri: http://localhost:5318/v1/logs
    method: post
    encoding:
      codec: protobuf
      protobuf:
        desc_file: path/to/opentelemetry-proto.desc
        message_type: opentelemetry.proto.collector.logs.v1.ExportLogsServiceRequest
        use_json_names: true
    framing:
      method: 'bytes'
    request:
      headers:
        content-type: 'application/x-protobuf'

The desc file was generated with the following command:

  protoc -I=/path/to/vector/lib/opentelemetry-proto/src/proto/opentelemetry-proto \\
    --include_imports \\
    --include_source_info \\
    --descriptor_set_out=opentelemetry-proto.desc \\
          $(find /path/to/vector/lib/opentelemetry-proto/src/proto/opentelemetry-proto -name '*.proto')