Gentoo Packages

sci-ml/koboldcpp (bentoo)

Search

Install

Install this package:

emerge -a sci-ml/koboldcpp

Package Information

Description:
KoboldCpp is a local AI server built on ggml/llama.cpp that bundles LLM text generation (the KoboldAI Lite web UI plus an OpenAI-compatible API), Stable Diffusion image generation, Whisper speech-to-text and TTS behind a single pure-Python launcher. This ebuild builds the ggml backends from source and installs the launcher (koboldcpp.py) together with the embedded web UI and resources. It ships neither the upstream PyInstaller binary nor any model weights, and it deletes the prebuilt helper binaries the source tarball carries for Windows so that no executable blob is installed. The CPU backend is built portable rather than tuned to the build host, and its x86 instruction set is selected through CPU_FLAGS_X86: upstream offers three tiers (AVX2+FMA+F16C, AVX only, SSE3 only). GPU acceleration is optional in every direction - Vulkan (upstream's officially supported path for both AMD and NVIDIA), CUDA/cuBLAS, and ROCm/hipBLAS - and a CPU-only build works with all of them disabled.
Homepage:
https://github.com/LostRuins/koboldcpp
License:
AGPL-3+ MIT

Versions

Version EAPI Keywords Slot
1.119 8 ~amd64 0

Metadata

Description

Maintainers

Upstream

Raw Metadata XML 
<pkgmetadata>
	<maintainer type="person">
		<email>lucascs@proton.me</email>
		<name>Lucas C.S.</name>
	</maintainer>
	<longdescription lang="en">
		KoboldCpp is a local AI server built on ggml/llama.cpp that bundles LLM
		text generation (the KoboldAI Lite web UI plus an OpenAI-compatible
		API), Stable Diffusion image generation, Whisper speech-to-text and
		TTS behind a single pure-Python launcher.

		This ebuild builds the ggml backends from source and installs the
		launcher (koboldcpp.py) together with the embedded web UI and
		resources. It ships neither the upstream PyInstaller binary nor any
		model weights, and it deletes the prebuilt helper binaries the source
		tarball carries for Windows so that no executable blob is installed.

		The CPU backend is built portable rather than tuned to the build host,
		and its x86 instruction set is selected through CPU_FLAGS_X86: upstream
		offers three tiers (AVX2+FMA+F16C, AVX only, SSE3 only). GPU
		acceleration is optional in every direction - Vulkan (upstream's
		officially supported path for both AMD and NVIDIA), CUDA/cuBLAS, and
		ROCm/hipBLAS - and a CPU-only build works with all of them disabled.
	</longdescription>
	<use>
		<flag name="cuda">Build the CUDA/cuBLAS backend (koboldcpp_cublas) for
			NVIDIA GPUs, for every architecture the installed toolkit
			supports</flag>
		<flag name="rocm">Build the ROCm/hipBLAS backend (koboldcpp_hipblas)
			for AMD GPUs, for the architectures selected in
			AMDGPU_TARGETS</flag>
		<flag name="vulkan">Build the Vulkan backend (koboldcpp_vulkan),
			upstream's officially supported GPU path for both AMD and
			NVIDIA</flag>
	</use>
	<upstream>
		<changelog>https://github.com/LostRuins/koboldcpp/releases</changelog>
		<bugs-to>https://github.com/LostRuins/koboldcpp/issues</bugs-to>
		<remote-id type="github">LostRuins/koboldcpp</remote-id>
	</upstream>
</pkgmetadata>

Lint Warnings

USE Flags

Manage flags for this package: euse -i <flag> -p sci-ml/koboldcpp | euse -E <flag> -p sci-ml/koboldcpp | euse -D <flag> -p sci-ml/koboldcpp

Flag Description 1.119
cpu_flags_x86_avx ⚠️ ✓
cpu_flags_x86_avx2 Select the AVX2-optimized binary on amd64 (default on modern CPUs). ⚠️ ✓
cpu_flags_x86_f16c ⚠️ ✓
cpu_flags_x86_fma3 ⚠️ ✓
cuda Build the CUDA/cuBLAS backend (koboldcpp_cublas) for NVIDIA GPUs, for every architecture the installed toolkit supports ✓
rocm Build the ROCm/hipBLAS backend (koboldcpp_hipblas) for AMD GPUs, for the architectures selected in AMDGPU_TARGETS ✓
vulkan Build the Vulkan backend (koboldcpp_vulkan), upstream's officially supported GPU path for both AMD and NVIDIA ⊕

Manifest

Type File Size Versions
DIST koboldcpp-1.119.gh.tar.gz 88763488 bytes 1.119
Unmatched Entries
Type File Size