sci-ml/ik_llama-cpp::bentoo
- Ebuilds: 1, Testing: 0_pre20260802 Description:
ik_llama.cpp is a fork of llama.cpp by Iwan Kawrakow, the author of most
of the k-quants, carrying additional state-of-the-art quantization types
(the IQK families) and CPU matrix-multiplication kernels that are
substantially faster than upstream for quantized inference.
It builds the same llama-cli, llama-server, llama-quantize and
llama-bench programs as sci-ml/llama-cpp. To let both be installed at
once, every binary is installed with an ik_ prefix (ik_llama-cli,
ik_llama-server, and so on) and the shared libraries live in a private
subdirectory of libdir, so nothing collides with sci-ml/llama-cpp. No
headers or pkg-config files are installed: this package is an
application, not a development target. Link against sci-ml/llama-cpp
instead.
Upstream publishes no release tags, so the package is pinned to a dated
commit snapshot fetched as a commit archive.
The CPU backend is tuned through CPU_FLAGS_X86. This matters more here
than for sci-ml/llama-cpp, because the fork's quantization kernels are
written for AVX2 and AVX-512; without those flags the generic fallback
gives up most of the fork's advantage.
Keywording is deliberately restricted to ~amd64. The SOTA quants are
hand-tuned for AVX2/AVX-512, the ARM paths are far less exercised
upstream, and no arm64 build of this snapshot has been verified.
Note that this ebuild is adapted from the guru overlay
(sci-misc/ik_llama-cpp).
Homepage:https://github.com/ikawrakow/ik_llama.cpp License: MIT
sci-ml/lemonade::bentoo
- Ebuilds: 1, Testing: 11.5.1 Description:
Lemonade runs large language models locally and exposes them over an
OpenAI-compatible HTTP API, so any client that speaks that API can talk
to a model on the machine instead of a hosted service. It ships two
programs: lemond, the server daemon, and lemonade, the command line
client used to pull, run, benchmark and manage models.
The inference backends themselves — llama.cpp, whisper.cpp, vLLM, ONNX
Runtime GenAI, Ryzen AI (the AMD NPU stack), stable-diffusion.cpp,
Kokoro and the others — are not compiled into this package. lemond
knows about each of them through a registry compiled into the binary
and downloads the matching prebuilt backend at runtime, picking one
suited to the hardware it finds: NVIDIA and AMD GPUs, Intel GPUs, AMD
NPUs or plain CPU. That is why app-arch/unzip is a runtime dependency
and why no acceleration backend appears among the build dependencies.
The daemon runs as the dedicated lemonade user and keeps its models and
its config.json under /var/lib/lemonade. Both a systemd unit (system
and user scope, installed by upstream) and an OpenRC service are
provided.
Not built here: the Tauri desktop shell and the AppIndicator system
tray, which upstream builds from separate npm and cargo dependency
sets. With USE=webapp the browser UI that lemond serves is bundled
from a pre-populated node_modules distfile, since upstream's build
otherwise runs npm ci and the sandbox has no network.
Homepage:https://lemonade-server.ai/ https://github.com/lemonade-sdk/lemonade License: Apache-2.0 webapp? ( BSD MIT )
sci-ml/lemonade-bin::bentoo
- Ebuilds: 1, Testing: 11.5.1 Description:
Lemonade is a local LLM server that exposes an OpenAI-compatible HTTP API
plus a browser UI, and configures state-of-the-art inference engines for
the hardware it finds: AMD GPUs (ROCm/Vulkan), NVIDIA GPUs, Intel, Ryzen
AI NPUs and plain CPU. It ships a daemon (lemond) and a CLI client
(lemonade) able to download, load and route models from providers such as
Hugging Face.
This package installs upstream's prebuilt "embeddable" binaries instead of
building from source. The binaries are taken from the embeddable tarball,
which links its Debian-specific dependencies statically; the official .deb
is used only as a source of architecture-independent data (web UI, JSON
resources, schemas, examples, man pages and systemd units), because its
executables require Debian-13 library sonames that do not exist on Gentoo.
Inference backends are not packaged: lemond downloads them at runtime into
its cache directory.
Homepage:https://lemonade-server.ai/
https://github.com/lemonade-sdk/lemonade License: Apache-2.0 BSD MIT
sci-ml/llama-cpp::bentoo
- Ebuilds: 1, Testing: 0_pre10229 Description:
llama.cpp is an inference engine for large language models written in plain
C/C++, with no external runtime dependencies. It loads quantized models in
the GGUF format and runs them on CPU, or with optional acceleration through
CUDA, ROCm/HIP, Vulkan or OpenCL backends.
The package provides llama-cli (interactive and one-shot inference),
llama-server (an HTTP server exposing an OpenAI-compatible API, optionally
with an embedded WebUI), llama-quantize, llama-bench and the ggml libraries
used by many downstream projects.
Models are not shipped with this package and must be downloaded separately,
typically from Hugging Face. The CPU backend is tuned through the
architecture's CPU_FLAGS: on x86 through CPU_FLAGS_X86, where the AVX-512
and AMX flags supported by the host processor have a large impact on
inference throughput, and on ARM through CPU_FLAGS_ARM, which ggml gates
at compile time via an accumulated -march string rather than per-feature
build options.
Note that ebuild is ported from the gentoo-zh overlay (sci-ml/llama-cpp).
Homepage:https://github.com/ggml-org/llama.cpp License: MIT
sci-ml/ollama-bin::bentoo
- Ebuilds: 1, Testing: 0.32.5 Description:
Ollama is a tool for running large language models (LLMs) locally on your
machine. It provides a simple interface to download, run, and manage models
like Llama 3.2, Mistral, Gemma, and many others.
This is a binary distribution package that installs pre-built binaries from
the official Ollama releases. The binaries are provided under the MIT license
and include GPU acceleration support for both NVIDIA (CUDA) and AMD (ROCm)
graphics cards.
Key features:
- Easy model management with pull, push, and create commands
- Built-in API server for programmatic access
- GPU acceleration support (CUDA and ROCm)
- Efficient memory management with automatic model loading/unloading
- Support for multiple models and concurrent requests
- Compatible with OpenAI API format
Models are stored in /var/lib/ollama and can range from 2GB (3B parameters)
to 40GB+ (70B parameters) in size. GPU acceleration significantly improves
inference speed but requires compatible hardware.
Security Note: This package installs pre-compiled binaries. Security
hardening features (ASLR, PIE, stack protections) depend on upstream's
build configuration. The service runs as a dedicated 'ollama' user with
restricted permissions for defense in depth.
Homepage:https://ollama.com/ License: MIT
sci-ml/whisper-cpp::bentoo
- Ebuilds: 1, Testing: 1.9.1 Description:
whisper.cpp is a plain C/C++ implementation of OpenAI's Whisper automatic
speech recognition model, built on the ggml tensor library and free of
external runtime dependencies. It transcribes and translates audio using
quantized models in the GGML format, on CPU or with optional acceleration
through CUDA, ROCm/HIP, Vulkan or OpenCL backends.
The package provides whisper-cli (one-shot transcription), whisper-server
(an HTTP server with an OpenAI-compatible endpoint), whisper-bench and
whisper-quantize, plus the parakeet-cli and parakeet-quantize tools for
NVIDIA's Parakeet TDT models, which upstream added alongside Whisper in the
1.9 series. With USE=sdl2 the real-time microphone examples whisper-stream,
whisper-command, whisper-talk-llama and whisper-lsp are built as well.
Models are not shipped with this package and must be downloaded separately,
typically from Hugging Face. The CPU backend is tuned through the
architecture's CPU_FLAGS: on x86 through CPU_FLAGS_X86, and on ARM through
CPU_FLAGS_ARM, which ggml gates at compile time via an accumulated -march
string rather than per-feature build options.
The ggml libraries this package builds are installed into a private
subdirectory rather than the system libdir, because sci-ml/llama-cpp
vendors the same ggml tree and owns those paths.
Note that the ebuild is adapted from the istitov/stuff overlay
(app-accessibility/whisper-cpp).
Homepage:https://github.com/ggml-org/whisper.cpp License: MIT