Agent♥︎Age
Catalog

GenPark LLM Token Bucket & Sliding Window Rate Limiter

Official

by alphaparkinc · Python

Adaptive multi-tier TPM/RPM rate limiter with smooth leaky bucket queuing and burst wait estimation

This MCP server provides an adaptive multi-tier rate limiter for LLM traffic, described as a “TPM/RPM” limiter. It combines token-bucket and sliding-window approaches to control throughput, and includes smooth leaky-bucket style queuing with burst wait estimation.

🛠️ Key Features

  • Adaptive multi-tier TPM/RPM rate limiting
  • Token-bucket and sliding-window control
  • Leaky bucket-style queuing behavior
  • Burst wait estimation

🚀 Use Cases

  • Throttling LLM requests using token-per-minute and requests-per-minute constraints
  • Managing bursty traffic by estimating wait time before rate limits apply
  • Coordinating queued workloads across multiple tiers

⚡ Developer Benefits

  • Built-in burst wait estimation for scheduling and backoff logic
  • Predictable throughput management aligned with TPM/RPM metrics

⚠️ Limitations

  • No additional capabilities, tool count, or topics are provided in the available source data.

Topics

agent-skillsapi-gatewaylatency-optimizationllm-infrastructuremcppython-standard-libraryrate-limitersliding-windowtoken-buckettraffic-shapingtpm-limiter

Related servers

More in Security