sci-ml/amd-gaia::bentoo
- Ebuilds: 1, Testing: 0.23.0 Description:
GAIA is AMD's agent framework for machines that run their models
locally. It provides the CLI (gaia, gaia-cli), an MCP bridge
(gaia-mcp), a chat and agent runtime, a RAG pipeline and a browser UI,
and it drives all of them through an OpenAI-compatible HTTP endpoint.
GAIA does not perform inference itself and contains no acceleration
backend: it is a client. The server it is designed against is Lemonade
(sci-ml/lemonade in this overlay), which is what reaches Ryzen AI NPU
and iGPU hardware; any other OpenAI-compatible endpoint works too.
Because the model is reached over HTTP and the code is pure Python,
nothing in this package is architecture specific.
Upstream ships a long list of optional extras. Only the ones whose
every dependency is already packaged are exposed as USE flags here;
the rest are documented, with the missing package named, in a comment
in the ebuild rather than silently dropped.
The browser UI bundle under gaia/apps/webui/dist is shipped
pre-compiled by upstream and is installed as-is — this package needs
no npm or Node.js at build time.
Homepage:https://github.com/amd/gaia https://pypi.org/project/amd-gaia/ License: MIT
sci-ml/ik_llama-cpp::bentoo
- Ebuilds: 1, Testing: 0_pre20260815 Description:
ik_llama.cpp is a fork of llama.cpp by Iwan Kawrakow, the author of most
of the k-quants, carrying additional state-of-the-art quantization types
(the IQK families) and CPU matrix-multiplication kernels that are
substantially faster than upstream for quantized inference.
It builds the same llama-cli, llama-server, llama-quantize and
llama-bench programs as sci-ml/llama-cpp. To let both be installed at
once, every binary is installed with an ik_ prefix (ik_llama-cli,
ik_llama-server, and so on) and the shared libraries live in a private
subdirectory of libdir, so nothing collides with sci-ml/llama-cpp. No
headers or pkg-config files are installed: this package is an
application, not a development target. Link against sci-ml/llama-cpp
instead.
Upstream publishes no release tags, so the package is pinned to a dated
commit snapshot fetched as a commit archive.
The CPU backend is tuned through CPU_FLAGS_X86. This matters more here
than for sci-ml/llama-cpp, because the fork's quantization kernels are
written for AVX2 and AVX-512; without those flags the generic fallback
gives up most of the fork's advantage.
Keywording is deliberately restricted to ~amd64. The SOTA quants are
hand-tuned for AVX2/AVX-512, the ARM paths are far less exercised
upstream, and no arm64 build of this snapshot has been verified.
Note that this ebuild is adapted from the guru overlay
(sci-misc/ik_llama-cpp).
Homepage:https://github.com/ikawrakow/ik_llama.cpp License: MIT
sci-ml/koboldcpp::bentoo
- Ebuilds: 1, Testing: 1.119 Description:
KoboldCpp is a local AI server built on ggml/llama.cpp that bundles LLM
text generation (the KoboldAI Lite web UI plus an OpenAI-compatible
API), Stable Diffusion image generation, Whisper speech-to-text and
TTS behind a single pure-Python launcher.
This ebuild builds the ggml backends from source and installs the
launcher (koboldcpp.py) together with the embedded web UI and
resources. It ships neither the upstream PyInstaller binary nor any
model weights, and it deletes the prebuilt helper binaries the source
tarball carries for Windows so that no executable blob is installed.
The CPU backend is built portable rather than tuned to the build host,
and its x86 instruction set is selected through CPU_FLAGS_X86: upstream
offers three tiers (AVX2+FMA+F16C, AVX only, SSE3 only). GPU
acceleration is optional in every direction - Vulkan (upstream's
officially supported path for both AMD and NVIDIA), CUDA/cuBLAS, and
ROCm/hipBLAS - and a CPU-only build works with all of them disabled.
Homepage:https://github.com/LostRuins/koboldcpp License: AGPL-3+ MIT
sci-ml/lemonade::bentoo
- Ebuilds: 1, Testing: 11.7.0 Description:
Lemonade runs large language models locally and exposes them over an
OpenAI-compatible HTTP API, so any client that speaks that API can talk
to a model on the machine instead of a hosted service. It ships two
programs: lemond, the server daemon, and lemonade, the command line
client used to pull, run, benchmark and manage models.
The inference backends themselves — llama.cpp, whisper.cpp, vLLM, ONNX
Runtime GenAI, Ryzen AI (the AMD NPU stack), stable-diffusion.cpp,
Kokoro and the others — are not compiled into this package. lemond
knows about each of them through a registry compiled into the binary
and downloads the matching prebuilt backend at runtime, picking one
suited to the hardware it finds: NVIDIA and AMD GPUs, Intel GPUs, AMD
NPUs or plain CPU. That is why app-arch/unzip is a runtime dependency
and why no acceleration backend appears among the build dependencies.
The daemon runs as the dedicated lemonade user and keeps its models and
its config.json under /var/lib/lemonade. Both a systemd unit (system
and user scope, installed by upstream) and an OpenRC service are
provided.
Not built here: the Tauri desktop shell and the AppIndicator system
tray, which upstream builds from separate npm and cargo dependency
sets. With USE=webapp the browser UI that lemond serves is bundled
from a pre-populated node_modules distfile, since upstream's build
otherwise runs npm ci and the sandbox has no network.
Homepage:https://lemonade-server.ai/ https://github.com/lemonade-sdk/lemonade License: Apache-2.0 webapp? ( BSD MIT )
sci-ml/lemonade-bin::bentoo
- Ebuilds: 1, Testing: 11.7.0 Description:
Lemonade is a local LLM server that exposes an OpenAI-compatible HTTP API
plus a browser UI, and configures state-of-the-art inference engines for
the hardware it finds: AMD GPUs (ROCm/Vulkan), NVIDIA GPUs, Intel, Ryzen
AI NPUs and plain CPU. It ships a daemon (lemond) and a CLI client
(lemonade) able to download, load and route models from providers such as
Hugging Face.
This package installs upstream's prebuilt "embeddable" binaries instead of
building from source. The binaries are taken from the embeddable tarball,
which links its Debian-specific dependencies statically; the official .deb
is used only as a source of architecture-independent data (web UI, JSON
resources, schemas, examples, man pages and systemd units), because its
executables require Debian-13 library sonames that do not exist on Gentoo.
Inference backends are not packaged: lemond downloads them at runtime into
its cache directory.
Homepage:https://lemonade-server.ai/
https://github.com/lemonade-sdk/lemonade License: Apache-2.0 BSD MIT
sci-ml/llama-cpp::bentoo
- Ebuilds: 1, Testing: 0.2.0 Description:
llama.cpp is an inference engine for large language models written in plain
C/C++, with no external runtime dependencies. It loads quantized models in
the GGUF format and runs them on CPU, or with optional acceleration through
CUDA, ROCm/HIP, Vulkan or OpenCL backends.
The package provides llama-cli (interactive and one-shot inference),
llama-server (an HTTP server exposing an OpenAI-compatible API, optionally
with an embedded WebUI), llama-quantize, llama-bench and the ggml libraries
used by many downstream projects.
Models are not shipped with this package and must be downloaded separately,
typically from Hugging Face. The CPU backend is tuned through the
architecture's CPU_FLAGS: on x86 through CPU_FLAGS_X86, where the AVX-512
and AMX flags supported by the host processor have a large impact on
inference throughput, and on ARM through CPU_FLAGS_ARM, which ggml gates
at compile time via an accumulated -march string rather than per-feature
build options.
Note that ebuild is ported from the gentoo-zh overlay (sci-ml/llama-cpp).
Homepage:https://github.com/ggml-org/llama.cpp License: MIT
sci-ml/llama-swap::bentoo
- Ebuilds: 1, Testing: 251 Description:
llama-swap sits in front of one or more OpenAI-compatible inference
servers and starts, stops and swaps them on demand, so a single
endpoint can serve many models without keeping them all resident.
It works with llama.cpp, vLLM and anything else exposing the same API.
The daemon has no authentication of its own and both shipped service
files bind 127.0.0.1 by default. Put an authenticating proxy in front
of it before exposing it to a network you do not control.
Written in pure Go with no cgo, which is why it carries ~arm64 while
the C++ inference packages in this overlay do not.
Homepage:https://github.com/mostlygeek/llama-swap License: MIT Apache-2.0 BSD BSD-2 ISC
sci-ml/ollama-bin::bentoo
- Ebuilds: 1, Testing: 0.32.15 Description:
Ollama is a tool for running large language models (LLMs) locally on your
machine. It provides a simple interface to download, run, and manage models
like Llama 3.2, Mistral, Gemma, and many others.
This is a binary distribution package that installs pre-built binaries from
the official Ollama releases. The binaries are provided under the MIT license
and include GPU acceleration support for both NVIDIA (CUDA) and AMD (ROCm)
graphics cards.
Key features:
- Easy model management with pull, push, and create commands
- Built-in API server for programmatic access
- GPU acceleration support (CUDA and ROCm)
- Efficient memory management with automatic model loading/unloading
- Support for multiple models and concurrent requests
- Compatible with OpenAI API format
Models are stored in /var/lib/ollama and can range from 2GB (3B parameters)
to 40GB+ (70B parameters) in size. GPU acceleration significantly improves
inference speed but requires compatible hardware.
Security Note: This package installs pre-compiled binaries. Security
hardening features (ASLR, PIE, stack protections) depend on upstream's
build configuration. The service runs as a dedicated 'ollama' user with
restricted permissions for defense in depth.
Homepage:https://ollama.com/ License: MIT
sci-ml/sherpa-onnx::bentoo
- Ebuilds: 1, Testing: 1.13.6 Description:
sherpa-onnx is the speech toolkit of the k2-fsa project:
speech-to-text, text-to-speech, speaker diarization, voice activity
detection, source separation and keyword spotting, all running on ONNX
Runtime with no PyTorch dependency.
This is a source build linked against the system sci-libs/onnxruntime
rather than a bundled copy, which is what
SHERPA_ONNX_USE_PRE_INSTALLED_ONNXRUNTIME_IF_AVAILABLE selects.
Upstream's CMake build resolves roughly a dozen small dependencies
through FetchContent at configure time — eigen, asio, cargs, json,
kaldi-decoder, kaldi-native-fbank, kaldifst, openfst, kissfft,
simple-sentencepiece, hclust-cpp, and optionally espeak-ng,
piper-phonemize, portaudio, websocketpp and pybind11. This ebuild
declares every one of them in SRC_URI, pinned by tag or full commit
SHA, and stages them into the source directory so CMake resolves them
locally. The build therefore needs no network access and carries no
RESTRICT="network-sandbox".
Model files are not bundled; each task loads its own from upstream.
See https://k2-fsa.github.io/sherpa/onnx/pretrained_models/
Homepage:
https://k2-fsa.github.io/sherpa/onnx/
https://github.com/k2-fsa/sherpa-onnx
License: Apache-2.0
sci-ml/whisper-cpp::bentoo
- Ebuilds: 1, Testing: 1.9.3 Description:
whisper.cpp is a plain C/C++ implementation of OpenAI's Whisper automatic
speech recognition model, built on the ggml tensor library and free of
external runtime dependencies. It transcribes and translates audio using
quantized models in the GGML format, on CPU or with optional acceleration
through CUDA, ROCm/HIP, Vulkan or OpenCL backends.
The package provides whisper-cli (one-shot transcription), whisper-server
(an HTTP server with an OpenAI-compatible endpoint), whisper-bench and
whisper-quantize, plus the parakeet-cli and parakeet-quantize tools for
NVIDIA's Parakeet TDT models, which upstream added alongside Whisper in the
1.9 series. With USE=sdl2 the real-time microphone examples whisper-stream,
whisper-command, whisper-talk-llama and whisper-lsp are built as well.
Models are not shipped with this package and must be downloaded separately,
typically from Hugging Face. The CPU backend is tuned through the
architecture's CPU_FLAGS: on x86 through CPU_FLAGS_X86, and on ARM through
CPU_FLAGS_ARM, which ggml gates at compile time via an accumulated -march
string rather than per-feature build options.
The ggml libraries this package builds are installed into a private
subdirectory rather than the system libdir, because sci-ml/llama-cpp
vendors the same ggml tree and owns those paths.
Note that the ebuild is adapted from the istitov/stuff overlay
(app-accessibility/whisper-cpp).
Homepage:https://github.com/ggml-org/whisper.cpp License: MIT