A high-throughput and memory-efficient inference and serving engine for LLMs
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Faster Whisper transcription with CTranslate2
SOTA Open Source TTS
LLM inference with 7x longer context. Pure C, zero dependencies. Lossless KV cache compression + single-header library.
Pre-trained Neural Network models in Axon (+ 🤗 Models integration)
AICI: Prompts as (Wasm) Programs
