autobatcher

github.com ↗

Drop-in AsyncOpenAI replacement that transparently batches requests via the Batch API for cheaper LLM inference.