sci-ml/amd-gaia::stuff
- Ebuilds: 2, Testing: 0.21.2 Description:
GAIA is AMD's open-source agent framework for local AI agents on
Ryzen AI hardware (NPU + iGPU). It orchestrates LLM-driven workflows
over any OpenAI-compatible inference endpoint, with built-in
integrations for Docker, Jira, code-search, RAG, MCP servers, and
Whisper / Kokoro voice pipelines. The reference local backend is
Lemonade Server (sci-ml/lemonade); GAIA itself is hardware-agnostic
so long as the upstream LLM API is OpenAI-compatible.
Homepage:https://github.com/amd/gaia License: MIT
sci-ml/bitnet::bennypowers
- Ebuilds: 1, Snapshot: 9999 Description:
BitNet is the official inference framework for 1-bit LLMs (BitNet b1.58).
It provides optimized kernels for fast and lossless inference of 1.58-bit
quantized models on CPU, with architecture-specific optimizations for
x86 (TL2) and ARM (TL1) platforms.
Homepage:https://github.com/microsoft/BitNet License: MIT
sci-ml/fastflowlm::stuff
- Ebuilds: 3, Testing: 0.9.45, Snapshot: 9999 Description:
FastFlowLM (FLM) is a lightweight LLM inference runtime purpose-built
for AMD Ryzen AI NPUs (XDNA2 architecture). It provides an Ollama-style
CLI and OpenAI-compatible server API for running language models entirely
on the NPU with no GPU or CPU compute required.
Supported hardware: Ryzen AI 300-series (Strix Point, Strix Halo),
400-series (Gorgon Point), and Z2 Extreme. XDNA1 (Ryzen AI 7000/8000)
is NOT supported.
The orchestration code and CLI are MIT-licensed. NPU compute kernels
(xclbins) are proprietary binaries, free for commercial use under
$10M annual company revenue.
Homepage:
https://fastflowlm.com/
https://github.com/FastFlowLM/FastFlowLM
License: MIT FastFlowLM-Binary
sci-ml/gaia::bennypowers
- Ebuilds: 5, Testing: 0.21.2 Description:
AMD Gaia is an AI agent framework that provides tools for building
and deploying AI agents. It includes support for various AI models
and frameworks, with integration for AMD hardware acceleration.
Homepage:https://github.com/amd/gaia License: MIT
sci-ml/kokoro-onnx::bennypowers
- Ebuilds: 1, Testing: 0.5.0 Description:
kokoro-onnx provides local text-to-speech using the Kokoro-82M
neural TTS model via ONNX Runtime. Supports multiple languages
and voices with CPU inference by default. GPU acceleration is
available via the ONNX_PROVIDER environment variable when
onnxruntime is built with CUDA or MIGraphX support.
Homepage:https://github.com/thewh1teagle/kokoro-onnx License: MIT
sci-ml/kokoros::stuff
- Ebuilds: 1, Snapshot: 9999 Description:
Kokoros is a Rust implementation of the Kokoro-82M text-to-speech
model. Provides the `koko` CLI and an OpenAI-compatible HTTP server
used as the kokoro:cpu backend by sci-ml/lemonade.
Tracks upstream lucasjinreal/Kokoros directly. The lemonade-sdk
fork only diverges in CI infrastructure plus a bundled espeak-ng-data
copy that ::gentoo already provides via app-accessibility/espeak-ng,
so source-build users get the same binary either way.
Runtime model files (kokoro-v1.0.onnx + voices-v1.0.bin) are not
bundled — see pkg_postinst for a quick fetch recipe.
Homepage:https://github.com/lucasjinreal/Kokoros License: Apache-2.0
llama-cpp (available in: sci-ml/llama-cpp::bentoo, sci-ml/llama-cpp::gentoo-zh, sci-ml/llama-cpp::gentooplusplus, sci-ml/llama-cpp::zGentoo)
- Ebuilds: 7, Testing: 9608, 0_pre9756, Snapshot: 9999 Description:
llama.cpp is an inference engine for large language models written in plain
C/C++, with no external runtime dependencies. It loads quantized models in
the GGUF format and runs them on CPU, or with optional acceleration through
CUDA, ROCm/HIP, Vulkan or OpenCL backends.
The package provides llama-cli (interactive and one-shot inference),
llama-server (an HTTP server exposing an OpenAI-compatible API, optionally
with an embedded WebUI), llama-quantize, llama-bench and the ggml libraries
used by many downstream projects.
Models are not shipped with this package and must be downloaded separately,
typically from Hugging Face. The CPU backend is selected through
CPU_FLAGS_X86, so enabling the AVX-512 and AMX flags supported by the host
processor has a large impact on inference throughput.
Note that ebuild is ported from the gentoo-zh overlay (sci-ml/llama-cpp).
Homepage:https://github.com/ggml-org/llama.cpp License: MIT
sci-ml/lmstudio-bin::deftera
- Ebuilds: 1, Testing: 0.4.18 Description:
LM Studio is a proprietary desktop application by Element Labs for
discovering, downloading and running large language models locally.
It provides a chat UI and an OpenAI-compatible local server, with
GPU acceleration via bundled Vulkan and ROCm/llama.cpp backends.
This package installs the prebuilt amd64 binary published by
upstream as a .deb release artifact under /opt/LM-Studio. The
Electron 38 / Chromium 140 runtime and its supporting libraries are
bundled inside that prefix; only a small set of system X11/GTK/glibc
libraries is pulled in via RDEPEND.
Models and settings are stored under ~/.lmstudio. A modern x86-64
CPU with AVX2 is required.
Homepage:https://lmstudio.ai/ License: LM-Studio
ollama-bin (available in: sci-ml/ollama-bin::bentoo, sci-ml/ollama-bin::pingwho-overlay)
- Ebuilds: 5, Testing: 0.31.2, Snapshot: 9999 Description:
Ollama is a tool for running large language models (LLMs) locally on your
machine. It provides a simple interface to download, run, and manage models
like Llama 3.2, Mistral, Gemma, and many others.
This is a binary distribution package that installs pre-built binaries from
the official Ollama releases. The binaries are provided under the MIT license
and include GPU acceleration support for both NVIDIA (CUDA) and AMD (ROCm)
graphics cards.
Key features:
- Easy model management with pull, push, and create commands
- Built-in API server for programmatic access
- GPU acceleration support (CUDA and ROCm)
- Efficient memory management with automatic model loading/unloading
- Support for multiple models and concurrent requests
- Compatible with OpenAI API format
Models are stored in /var/lib/ollama and can range from 2GB (3B parameters)
to 40GB+ (70B parameters) in size. GPU acceleration significantly improves
inference speed but requires compatible hardware.
Security Note: This package installs pre-compiled binaries. Security
hardening features (ASLR, PIE, stack protections) depend on upstream's
build configuration. The service runs as a dedicated 'ollama' user with
restricted permissions for defense in depth.
Homepage:https://ollama.com/ License: MIT
sci-ml/sherpa-onnx::stuff
- Ebuilds: 2, Testing: 1.13.4 Description:
sherpa-onnx is a speech-stack toolkit from the k2-fsa project:
speech-to-text, text-to-speech, speaker diarization, voice activity
detection, source separation, and keyword spotting, all running on
ONNX Runtime (no PyTorch dependency).
Source build against system sci-libs/onnxruntime. For the prebuilt
-bin alternative (faster install, ships upstream's manylinux wheels)
see sci-ml/sherpa-onnx-bin.
The CMake build vendors a dozen small deps (eigen, asio, cargs, json,
kaldi-{decoder,native-fbank,fst}, openfst, kissfft, simple-sentencepiece,
hclust-cpp, optionally espeak-ng + piper-phonemize + portaudio +
websocketpp + pybind11) via FetchContent. The ebuild pre-fetches them
all via SRC_URI and stages into ${S} for the cmake fallback paths;
no network access during build.
Runtime model files for each task (ASR, diarization, TTS, etc.) live
upstream — see https://k2-fsa.github.io/sherpa/onnx/pretrained_models/
Homepage:
https://k2-fsa.github.io/sherpa/onnx/
https://github.com/k2-fsa/sherpa-onnx
License: Apache-2.0
sci-ml/sherpa-onnx-bin::stuff
- Ebuilds: 3, Testing: 1.13.4 Description:
sherpa-onnx is a speech-stack toolkit from the k2-fsa project:
speech-to-text, text-to-speech, speaker diarization, voice activity
detection, source separation, and keyword spotting, all running on
ONNX Runtime (no PyTorch dependency). Suited to CPU-only deployment
and embedded targets.
This -bin ebuild ships upstream's manylinux wheels (sherpa-onnx-core
for the C++ shared libraries plus a per-CPython-ABI wheel for the
Python bindings). Runtime model files are not bundled — see the
post-install message for download pointers.
Homepage:
https://k2-fsa.github.io/sherpa/onnx/
https://github.com/k2-fsa/sherpa-onnx
https://pypi.org/project/sherpa-onnx/
License: Apache-2.0
sci-ml/tensorzero::pingwho-overlay
- Ebuilds: 4, Testing: 2026.6.0, Snapshot: 9999 Description:
TensorZero is an open-source stack for industrial-grade LLM applications.
Gateway: access every LLM provider through a unified API low latency
Observability: monitor your LLM systems, programmatically or with a UI
Optimization: optimize your prompts, models, and inference strategies
Evaluations: benchmark individual inferences or end-to-end workflows
Experimentation: deploy with built-in A/B testing, fallbacks, etc.
Homepage:https://www.tensorzero.com/ License: Apache-2.0