This MCP server implements an Nginx-style smooth weighted round-robin load balancer with dynamic weight adjustment for multi-LLM routing. Its purpose is to distribute requests across multiple upstream targets while changing weights dynamically to reflect current conditions.
๐ ๏ธ Key Features
- Nginx-style smooth weighted round-robin load balancing
- Dynamic weight adjustment for multi-LLM request routing
๐ Use Cases
- Multi-LLM deployments requiring weighted request distribution
- Routing scenarios where target selection weights need to change over time
โก Developer Benefits
- Provides a load-balancing approach aligned with weighted round-robin behavior
- Supports dynamic re-weighting for multi-LLM traffic patterns
โ ๏ธ Limitations
- Details beyond weighted round-robin and dynamic weight adjustment are not provided in the available data