Gentoo Packages

Search

Install

Install this package:

emerge -a sci-ml/llama-swap

Package Information

Description:

llama-swap sits in front of one or more OpenAI-compatible inference servers and starts, stops and swaps them on demand, so a single endpoint can serve many models without keeping them all resident. It works with llama.cpp, vLLM and anything else exposing the same API. The daemon has no authentication of its own and both shipped service files bind 127.0.0.1 by default. Put an authenticating proxy in front of it before exposing it to a network you do not control. Written in pure Go with no cgo, which is why it carries ~arm64 while the C++ inference packages in this overlay do not.

Homepage:

https://github.com/mostlygeek/llama-swap

License:

MIT Apache-2.0 BSD BSD-2 ISC

Versions

Metadata

Description

• llama-swap sits in front of one or more OpenAI-compatible inference servers and starts, stops and swaps them on demand, so a single endpoint can serve many models without keeping them all resident. It works with llama.cpp, vLLM and anything else exposing the same API. The daemon has no authentication of its own and both shipped service files bind 127.0.0.1 by default. Put an authenticating proxy in front of it before exposing it to a network you do not control. Written in pure Go with no cgo, which is why it carries ~arm64 while the C++ inference packages in this overlay do not.

Maintainers

• Lucas C.S. (lucascs@proton.me) - Type: person

Upstream

• Bugs: https://github.com/mostlygeek/llama-swap/issues
• Remote IDs:
  • github: mostlygeek/llama-swap

<pkgmetadata>
	<maintainer type="person">
		<email>lucascs@proton.me</email>
		<name>Lucas C.S.</name>
	</maintainer>
	<longdescription lang="en">
		llama-swap sits in front of one or more OpenAI-compatible inference
		servers and starts, stops and swaps them on demand, so a single
		endpoint can serve many models without keeping them all resident.
		It works with llama.cpp, vLLM and anything else exposing the same API.

		The daemon has no authentication of its own and both shipped service
		files bind 127.0.0.1 by default. Put an authenticating proxy in front
		of it before exposing it to a network you do not control.

		Written in pure Go with no cgo, which is why it carries ~arm64 while
		the C++ inference packages in this overlay do not.
	</longdescription>
	<use>
		<flag name="ui">Build and embed the Svelte web interface. Requires npm at build time; without this flag the binary serves the API only and /ui returns nothing</flag>
	</use>
	<upstream>
		<bugs-to>https://github.com/mostlygeek/llama-swap/issues</bugs-to>
		<remote-id type="github">mostlygeek/llama-swap</remote-id>
	</upstream>
</pkgmetadata>

Lint Warnings

• [Warning] USE flag 'systemd' in ebuild 250-r1 is not documented in metadata.xml
• Bypassing the network sandbox using `RESTRICT="network-sandbox"` violates Gentoo's core packaging policies. Ebuilds must support offline and reproducible builds, meaning all assets must be fetched via `SRC_URI` during the fetch phase and verified using Manifest checksums. To resolve this, you should package the required dependencies (e.g., node modules, cargo crates) into a tarball, host it, reference it in `SRC_URI` and perform an offline installation.
• [Error] Manifest entry for unused DIST file 'llama-swap-250.tar.gz'
• [Error] Repository is missing a repo-name in metadata/layout.conf or profiles/repo_name

USE Flags

Manage flags for this package: euse -i <flag> -p sci-ml/llama-swap | euse -E <flag> -p sci-ml/llama-swap | euse -D <flag> -p sci-ml/llama-swap

Files

• llama-swap.confd
• llama-swap.env.example
• llama-swap.initd
• llama-swap.service

Manifest