A high-throughput and memory-efficient inference and serving engine for LLMs
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Faster Whisper transcription with CTranslate2
SOTA Open Source TTS
Pre-trained Neural Network models in Axon (+ 🤗 Models integration)
LLM inference with 7x longer context. Pure C, zero dependencies. Lossless KV cache compression + single-header library.
AICI: Prompts as (Wasm) Programs
