Gentoo Packages

sci-misc/llama-cpp (stuff)

Search

Install

Install this package:

emerge -a sci-misc/llama-cpp

Package Information

Description:
llama.cpp runs LLaMA-family models in C/C++ on CPU and on several GPU backends. This ebuild was forked from the ::guru overlay at build b9090 and has diverged since. Versions follow ::guru's naming. A release keeps upstream's own X.Y.Z and is keyworded. A snapshot of upstream build bN is named X.Y.Z_pN after the release it follows, so it sorts above that release and below the next one; it tracks the tip between releases and stays unkeyworded, so it has to be requested per version through package.accept_keywords.
Homepage:
https://github.com/ggml-org/llama.cpp
License:
MIT

Versions

Version EAPI Keywords Slot
9999 8 0
0.8.0_p12032 8 0
0.8.0_p12020 8 0
0.8.0_p12001 8 0
0.8.0 8 ~amd64 ~arm64 0
0.7.0 8 ~amd64 ~arm64 0
0.6.0 8 ~amd64 ~arm64 0

Metadata

Description

Maintainers

Upstream

Raw Metadata XML 
<pkgmetadata>
	<maintainer type="person">
		<email>iohann.s.titov@gmail.com</email>
		<name>Ivan S. Titov</name>
	</maintainer>
	<longdescription>
    llama.cpp runs LLaMA-family models in C/C++ on CPU and on several GPU
    backends. This ebuild was forked from the ::guru overlay at build
    b9090 and has diverged since.

    Versions follow ::guru's naming. A release keeps upstream's own
    X.Y.Z and is keyworded. A snapshot of upstream build bN is named
    X.Y.Z_pN after the release it follows, so it sorts above that
    release and below the next one; it tracks the tip between releases
    and stays unkeyworded, so it has to be requested per version through
    package.accept_keywords.
  </longdescription>
	<use>
		<flag name="blis">Build a BLIS backend</flag>
		<flag name="flexiblas">Build a FlexiBLAS backend</flag>
		<flag name="rocm">Build a HIP (ROCm) backend</flag>
		<flag name="openblas">Build an OpenBLAS backend</flag>
		<flag name="opencl">Build an OpenCL backend, so far only works on Adreno and Intel GPUs</flag>
		<flag name="openssl">Use openssl to support HTTPS</flag>
		<flag name="webui">Embed the llama-server web UI, using the prebuilt
    assets upstream publishes with each release (the live ebuild builds
    or downloads them at configure time); disable for a server binary
    with only the HTTP API.</flag>
	</use>
	<upstream>
		<remote-id type="github">ggml-org/llama.cpp</remote-id>
	</upstream>
</pkgmetadata>

Lint Warnings

USE Flags

Manage flags for this package: euse -i <flag> -p sci-misc/llama-cpp | euse -E <flag> -p sci-misc/llama-cpp | euse -D <flag> -p sci-misc/llama-cpp

Flag Description 9999 0.8.0_p12032 0.8.0_p12020 0.8.0_p12001 0.8.0 0.7.0 0.6.0
( ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
) ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
amx_bf16 ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
amx_int8 ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
amx_tile ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
avx ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
avx2 ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
avx512_bf16 ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
avx512_vnni ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
avx512f ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
avx512vbmi ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
avx_vnni ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
blis Build a BLIS backend ✓ ✓ ✓ ✓ ✓ ✓ ✓
bmi2 ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
cuda Build the CUDA (cuda_v13) llama-server GPU backend via <pkg>dev-util/nvidia-cuda-toolkit</pkg> ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
debug Enable debug messages ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
examples Build and install the example programs ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
f16c ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
flexiblas Build a FlexiBLAS backend ✓ ✓ ✓ ✓ ✓ ✓ ✓
fma3 ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
openblas Build an OpenBLAS backend ✓ ✓ ✓ ✓ ✓ ✓ ✓
opencl Build an OpenCL backend, so far only works on Adreno and Intel GPUs ✓ ✓ ✓ ✓ ✓ ✓ ✓
openmp Build the host backend with OpenMP threading ⚠️ ⊕ ⊕ ⊕ ⊕ ⊕ ⊕ ⊕
openssl Use openssl to support HTTPS ⊕ ⊕ ⊕ ⊕ ⊕ ⊕ ⊕
rocm Build a HIP (ROCm) backend ✓ ✓ ✓ ✓ ✓ ✓ ✓
sse4_2 ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
vulkan Build the Vulkan llama-server GPU backend ⚠️ ✓ ✓ ✓ ✓ ✓ ✓ ✓
webui Embed the llama-server web UI, using the prebuilt assets upstream publishes with each release (the live ebuild builds or downloads them at configure time); disable for a server binary with only the HTTP API. ⊕ ⊕ ⊕ ⊕ ⊕ ⊕ ⊕

Files

Manifest

Type File Size Versions
DIST ggml-org_models_tinyllamas_stories15M-q4_0-99dd1a73db5a37100bd4ae633f4cfce6560e1567.gguf 19077344 bytes 9999, 0.8.0_p12032, 0.8.0_p12020, 0.8.0_p12001, 0.8.0, 0.7.0, 0.6.0
DIST llama-b11429-ui.tar.gz 3099802 bytes 0.6.0
DIST llama-b11669-ui.tar.gz 3140519 bytes 0.7.0
DIST llama-b11960-ui.tar.gz 3244974 bytes 0.8.0
DIST llama-b12001-ui.tar.gz 3241897 bytes 0.8.0_p12001
DIST llama-b12020-ui.tar.gz 3243410 bytes 0.8.0_p12020
DIST llama-b12032-ui.tar.gz 3243407 bytes 0.8.0_p12032
DIST llama-cpp-0.6.0.tar.gz 37874349 bytes 0.6.0
DIST llama-cpp-0.7.0.tar.gz 38183379 bytes 0.7.0
DIST llama-cpp-0.8.0.tar.gz 38680037 bytes 0.8.0
DIST llama-cpp-0.8.0_p12001.tar.gz 38811955 bytes 0.8.0_p12001
DIST llama-cpp-0.8.0_p12020.tar.gz 38845552 bytes 0.8.0_p12020
DIST llama-cpp-0.8.0_p12032.tar.gz 38858446 bytes 0.8.0_p12032
Unmatched Entries
Type File Size