Gentoo Packages

Search

Install

Install this package:

emerge -a sci-misc/llama-swap

Package Information

Description:

llama-swap is an OpenAI/Anthropic-compatible HTTP proxy that starts and stops local LLM inference servers (llama.cpp, vllm, mlx-server, etc.) on demand based on the requested model. Lets a single API endpoint serve many models without keeping them all resident in GPU/NPU memory.

Homepage:

https://github.com/mostlygeek/llama-swap

License:

MIT

Versions

Metadata

Description

• llama-swap is an OpenAI/Anthropic-compatible HTTP proxy that starts and stops local LLM inference servers (llama.cpp, vllm, mlx-server, etc.) on demand based on the requested model. Lets a single API endpoint serve many models without keeping them all resident in GPU/NPU memory.

Maintainers

• Ivan S. Titov (iohann.s.titov@gmail.com) - Type: person

Upstream

• Remote IDs:
  • github: mostlygeek/llama-swap
  • github: istitov/extra-stuff
  • gitlab: istitov/extra-stuff
  • codeberg: istitov/extra-stuff

<pkgmetadata>
	<maintainer type="person">
		<email>iohann.s.titov@gmail.com</email>
		<name>Ivan S. Titov</name>
	</maintainer>
	<longdescription>
    llama-swap is an OpenAI/Anthropic-compatible HTTP proxy that
    starts and stops local LLM inference servers (llama.cpp, vllm,
    mlx-server, etc.) on demand based on the requested model. Lets
    a single API endpoint serve many models without keeping them
    all resident in GPU/NPU memory.
  </longdescription>
	<use>
		<flag name="openrc">Install OpenRC init.d/conf.d files for a
    supervise-daemon-managed service</flag>
		<flag name="systemd">Install a systemd template service unit
    (llama-swap@.service)</flag>
		<flag name="ui">Build and embed the Svelte web UI (pulls
    <pkg>net-libs/nodejs</pkg>; runs npm at build time, requires
    network access during compile).</flag>
	</use>
	<upstream>
		<remote-id type="github">mostlygeek/llama-swap</remote-id>
		<remote-id type="github">istitov/extra-stuff</remote-id>
		<remote-id type="gitlab">istitov/extra-stuff</remote-id>
		<remote-id type="codeberg">istitov/extra-stuff</remote-id>
	</upstream>
</pkgmetadata>

Lint Warnings

• [Error] Missing md5-cache for ebuild llama-swap-217

USE Flags

Manage flags for this package: euse -i <flag> -p sci-misc/llama-swap | euse -E <flag> -p sci-misc/llama-swap | euse -D <flag> -p sci-misc/llama-swap

Files

• llama-swap.confd
• llama-swap.initd
• llama-swap.service

Manifest