Vector 0.11 includes a new Adaptive Request Concurrency (ARC) feature that raises the performance and reliability of your entire observability infrastructure without any changes on your part. In short, it does away with static rate-limits and automatically optimizes HTTP concurrency limits based on downstream service responses. The underlying mechanism is a simple feedback loop inspired by TCP congestion control algorithms.
This feature, like all Vector features, will begin its life in public beta and be available on an opt-in basis. To get it, enable it for each sink:
[sinks.my-sink]type = "..." # any http-based sinkrequest.concurrency = "adaptive"# and remove the request.rate_limit_* settings