Key changes:
- add cuda13 container for Ampere to Blackwell GPUs
- support DGX Spark (GB10) with arm64 cuda 13 manifests
- enable ik_llama on vulkan container
- map `LLAMA_SWAP_*` env vars to llama-server CLI flags with backwards support
- refactor GHA/docker build to accept variants
closes: #905, #1066
supercedes: PR #841, PR #1084
Expose llama-swap through Tailcat virtual TCP listeners and route peer
requests through Tailcat transports.
- validate server identities, caller allowlists, and published model IDs
- restrict the default remote HTTP surface and retain Tailcat request sources
- add UI status, activity attribution, configuration docs, and schema support
- make Tailcat transport diagnostics opt-in with server.tailcat.debug
fixes: #1073
- Add option to specify CUDA version and architectures during image
building. If not provided, it should behave just like now.
- Include Stable Diffusion UI in SD binaries.
- Fix ik_llama.cpp build for arm64 architecture:
https://github.com/ikawrakow/ik_llama.cpp/issues/1684.
Big refactor to split the build stages for the unified container to be built in parallel.
- parallel building of binaries to speed up full container build (~6+hr to 1.5hr)
- split cuda and vulkan pipelines to be independent (vulkan is much faster)
- establish pattern for building building binaries for final image (easier to add new resources)
- use llama-swap-build for build containers to avoid untagged clean up script
Fixes: #1069
Add an Docs agent to the playground that can help the user with more
advanced configuration.
- add mcp 2026-07-28 (stateless mcp) framework
- add MCP tools for doc search and config search
- add initial set of guides for key topics
- add Docs to the Playground
- removed out of data documentation and plans
Replace TARGETARCH build-arg with runtime arch detection via uname -m.
BuildKit's TARGETARCH injection was unreliable for the multi-arch cpu
build, causing the arm64 image variant to download and embed the x86_64
llama-swap binary — resulting in "exec format error" on arm64 hosts.
With QEMU user-space emulation, uname -m correctly returns aarch64
inside an arm64 container build, so the download always fetches the
right binary for the actual target architecture. Also adds --fail to
curl so HTTP 404s produce a build error instead of silently embedding an
HTML error page.
fixes#818
Co-authored-by: Claude <noreply@anthropic.com>
Encountered a similar problem as in
https://github.com/mostlygeek/llama-swap/issues/709 but in my case I
only needed the :cpu version.
So decided to add the github action to build arm64 combined with the
amd64 version on the same :cpu tag. Already tested it from this fork:
ghcr.io/rhtenhove/llama-swap:cpu and it works perfectly fine.
Adding GPU support is a whole other beast, needing quite a bit more work
and isn't something I can test.
Install uv after the cpp tool binaries are copied and before the
llama-swap binary, enabling `uv run` usage for Python-based inference
backends like vLLM.
- add python3-pip to runtime apt installs
- add `pip install uv --break-system-packages` after cpp installs
fixes#628
Co-authored-by: Claude <noreply@anthropic.com>
Build the root image once, then derive the rootless variant from it
using a small inline Dockerfile that adds the non-root user and chowns
the writable directories. This halves the number of CI jobs (4 → 2) and
eliminates the redundant full CUDA compilation for the rootless variant.
- remove RUN_UID build arg from build-image.sh
- derive rootless image inline after root build completes
- collapse variant matrix out of unified-docker.yml
- push both root and rootless tags in a single CI job
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Expose CMAKE_CUDA_ARCHITECTURES as a Docker build ARG so users can
customize CUDA architectures via --build-arg without editing the
Dockerfile.
- convert hardcoded ENV to ARG with default, feeding into ENV
- replace silent fallback defaults (:-) in scripts with :? guards
to fail fast if the env var is missing
- add usage example to Dockerfile header
Follow up to: #624https://claude.ai/code/session_01EWiUe7jNABX7Uz95dUGJqK
Co-authored-by: Claude <noreply@anthropic.com>
multiple fixes to vulkan build:
- use ubuntu 26.04 to be compatible with AMD 395+ (Strix halo) hardware
- add home directory in container
- fix stable-diffusion install to actually enable vulkan
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- set up a GHA scheduled job to build the container nightly
- enabling pushing a llama-swap:unified and a llama-swap:unified-Y-M-D
image to ghcr.io
- tidy up Dockerfile to use a non-root user and llama-swap as an entry
point
Add `cuda13` as a supported build architecture, targeting the
`ghcr.io/ggml-org/llama.cpp:server-cuda13` upstream base image.
The `server-cuda13` image ships with CUDA 13 libraries, providing
improved performance on recent NVIDIA hardware compared to the existing
`server-cuda` (CUDA 12) image. Users with newer GPUs (e.g., RTX
50-series) benefit from reduced model load latency and higher token
throughput.
- Add `cuda13` to the allowed architectures list in
`docker/build-container.sh`
- Add `cuda13` to the CI matrix in `.github/workflows/containers.yml` so
the container is built and pushed automatically
## Summary
- Add `--provenance=false` to docker build commands in
`build-container.sh`
- BuildKit attestation manifests are stored as untagged images in GHCR,
and the `delete-untagged-containers` cleanup job deletes them, breaking
the manifest list and causing `manifest unknown` errors on pull
- ref: https://github.com/actions/delete-package-versions/issues/162
* docker: add .env usage in build-container.sh
* .github,docker: add rocm, improve logging
* .github,CLAUDE.md: fix workflow and update guidelines
Update containers workflow to only push images when triggered
manually or on schedule, not on workflow file changes.
- add push trigger for workflow file changes in containers.yml
- update push condition to skip on regular push events
- update CLAUDE.md commit message guidelines
* docker: remove comma in build-container.sh
* .github,docker: improve container build workflow
Add pagination support for fetching llama.cpp tags and improve debugging.
- add build-container.sh to workflow trigger paths
- implement fetch_llama_tag() with pagination support
- replace .env with local testing instructions
- add DEBUG_ABORT_BUILD flag for testing
* feat: Add support for custom llama.cpp base image and forked llama-swap repositories
- Introduce BASE_LLAMACPP_IMAGE env var to customize llama.cpp base image
- Introduce LS_REPO env var to customize llama-swap source
- Use GITHUB_REPOSITORY env var to automatically detect forked repos
- Update container tagging to use dynamic repo paths
- Pass build args for BASE_IMAGE and LS_REPO to Containerfile
- Enable flexible release downloads from forked repositories
* chore: quote entire curl options, appease coderabbitai
Change the user back to root for containers. Additionally, built a "non-root" labeled container for users who wish to have the additional security of running llama-swap as a lower privileged user.
Refactor the container build script to resolve llama.cpp base image for CPU, also tag these builds accordingly.
- For CPU containers, now fetch the latest 'server' tagged llama.cpp image instead of using a generic 'server' tag
- Cleans up the docker build command to use dynamic BASE_TAG variable
- Maintains existing push functionality for built images
- Replace `addgroup` with `groupadd` for system group creation
- Replace `adduser` with `useradd` for system user creation
- Maintain same functionality while using more standard POSIX commands
Set default container user/group to lower privilege app user
* refactor: update Containerfile to support non-root user execution and improve security
- Updated LS_VER argument from 89 to 170 to use the latest version
- Added UID/GID arguments with default values of 0 (root) for backward compatibility
- Added USER_HOME environment variable set to /root
- Implemented conditional user/group creation logic that only runs when UID/GID are not 0
- Created necessary directory structure with proper ownership using mkdir and chown commands
- Switched to non-root user execution for improved security posture
- Updated COPY instruction to use --chown flag for proper file ownership
* chore: update containerfile to use non-root user with proper UID/GID
- Changed default UID and GID from 0 (root) to 10001 for security best practices
- Updated USER_HOME from /root to /app to avoid running as root user
