DeepSpeed-Mii

github.com ↗

MII makes low-latency and high-throughput inference, similar to vLLM powered by DeepSpeed.