guidellm

github.com ↗

- guidellm is a benchmarking and performance evaluation tool for large language model inference systems.