This MCP server provides an adaptive multi-tier rate limiter for LLM traffic, described as a “TPM/RPM” limiter. It combines token-bucket and sliding-window approaches to control throughput, and includes smooth leaky-bucket style queuing with burst wait estimation.
🛠️ Key Features
- Adaptive multi-tier TPM/RPM rate limiting
- Token-bucket and sliding-window control
- Leaky bucket-style queuing behavior
- Burst wait estimation
🚀 Use Cases
- Throttling LLM requests using token-per-minute and requests-per-minute constraints
- Managing bursty traffic by estimating wait time before rate limits apply
- Coordinating queued workloads across multiple tiers
⚡ Developer Benefits
- Built-in burst wait estimation for scheduling and backoff logic
- Predictable throughput management aligned with TPM/RPM metrics
⚠️ Limitations
- No additional capabilities, tool count, or topics are provided in the available source data.