awesomelist.dev

Discover All lists Trending MCP server

Categories

AI & ML12 APIs & Data4 Databases3 Design5 DevOps & Cloud3 Game Dev5 Languages21 Learn & Career10 Meta & General11 Mobile4 Security & Ops10 Web15

103 lists indexed

GitHub snapshot · refreshed today

awesomelist.dev

REPOSITORY

Home › AI & ML › Awesome LLM › llama.cpp

L

llama.cpp

github.com ↗

LLM inference in C/C++.

Visit repository ↗← Back to Awesome LLM

READMEView on GitHub ↗

Category

AI & ML

Also in 2 other lists

Awesome NLP Awesome C

More from LLM Inference

DDeepSpeed-MiiMII makes low-latency and high-throughput inference, similar to vLLM powered by DeepSpeed.

Ddeploy-llms-with-ansibleEasily deploy any LLM on a VM with minimal configuration, using Ansible.

EexllamaA more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

FFastChatA distributed multi-model LLM serving system with web UI and OpenAI-compatible RESTful APIs.

FFasterTransformerNVIDIA Framework for LLM Inference(Transitioned to TensorRT-LLM)

HHaystackan open-source NLP framework that allows you to use LLMs and transformer-based models from Hugging Face, OpenAI and Cohere to interact with your own data.

IInfinityInference for text-embeddings in Python

LLiger-KernelEfficient Triton Kernels for LLM Training.

All lists Trending MCP server Sitemap

103 lists · 61.4k projects · refreshed today

esc

↵ to searchesc to close

awesomelist.dev

Discover All lists Trending MCP server