A high-throughput and memory-efficient inference and serving engine for LLMs
Faster Whisper transcription with CTranslate2
Fast inference engine for Transformer models
AICI: Prompts as (Wasm) Programs