A `setParams` / `setParamsByID` key ending in `?` is set-if-undefined:
the value is applied only when the request doesn't already carry that
parameter. Plain keys keep forcing their values, and a config that
doesn't use the suffix behaves exactly as before.
Fixes: #1052
Hide tailnet server token in the UI and add an easy to copy/paste
Peer config that is easy to send to connect to the server.
- Add missing configuration overview documentation
- Fix out of date docs
- Fix minor bug for listing tailcat server models
Fixes#1092
Refresh documentation suggestions and streamline the Help experience.
- align sidebar footer links and add topic refresh action
- remove the speculative-decoding article and stale search hint
- increase agent loop limit while hiding iteration display
The Docs Agent's get_config tool would fail on very large
configs with lots of models. Replace it with a jq expression
so the agent can request very specific parts of the configuration
without adding too many extra tokens to the context.
- add jq expression support to Doc Agent's Tools
- move Help out of the Playground into the main sidebar
- add more tips to the Help front page
Updates #1085
Expose llama-swap through Tailcat virtual TCP listeners and route peer
requests through Tailcat transports.
- validate server identities, caller allowlists, and published model IDs
- restrict the default remote HTTP surface and retain Tailcat request sources
- add UI status, activity attribution, configuration docs, and schema support
- make Tailcat transport diagnostics opt-in with server.tailcat.debug
fixes: #1073
Add an Docs agent to the playground that can help the user with more
advanced configuration.
- add mcp 2026-07-28 (stateless mcp) framework
- add MCP tools for doc search and config search
- add initial set of guides for key topics
- add Docs to the Playground
- removed out of data documentation and plans
Add a dedicated /comfyui/ passthrough backed by the reserved
comfyui_auto local model.
- preserve escaped upstream paths such as encoded workflow separators
- allow only the root path to start an unloaded ComfyUI model
- enforce a minimum concurrency limit of 50 for the reserved model
- add example upstream.ignorePaths to prevent unintentional swaps by
ComfyUI
- add model.workarounds.ignoreWebsockets so ComfyUI does not block
swapping
fixes: #1001fixes: #1000
Resolve configuration macros against an untyped YAML representation
before decoding the typed Config. This materializes YAML anchors and
preserves scalar types without maintaining field-specific replacement
paths.
- apply environment, global, and model macros in their defined scopes
- centralize MODEL_ID, PORT, PID, key, and unknown-macro handling
- remove the capability-specific raw decoder and replacement loops
- document macro ordering, scope, and runtime behavior
- add coverage for anchors, scope isolation, validation, and runtime
macros
fixes#919
Address peer models by fully qualified names across routing, selectors,
model listings, and the UI.
- support fully qualified peer/model routing names
- support peer models in selector spillover targets
- show peer models in Playground model pickers
fixes#944
Add configurable model ID profiles that can be switched without
restarting.
- rewrite request and upstream model IDs through active profile pins
- expose profile selection through API, SSE, and UI
- validate and document profile configuration
Updates #933
Supercedes: #774
Track request and streamed response metadata for active model requests.
- publish keyed updates through a non-blocking recoverable outbox
- add compact configurable activity columns and clock-safe elapsed time
- tag playground requests with a page-scoped session ID
Fixes#912
Store activity metrics in SQLite and use the store as the source for
activity APIs.
- add SQLite-backed activity store with migrations and activity stats
queries
- add new store.path configuration option
- add activity pagination and stats API endpoints
- update the activity UI to fetch paginated rows and server-side stats
- add optional store.path configuration for durable activity history
Fixes#310, #729
This is a huge backend change that essentially started with rewriting
the concurrency handling for processes and blew up to a refactor of the
entire application. In short these are the improvements:
**Better state and life cycle management:**
Life cycle management of processes has always been the trickiest part of
the code. Juggling mutex locks between multiple locations to reduce race
conditions was complex. Too complex for my feeble brain to build a
simple mental model around as llama-swap gained more features. All of
that has been refactored. Most of the locks are gone, replaced with a
single run() that owns all state changes. There is one place to start
from now to understand and extend routing logic.
The improved life cycle management makes it easier to implement more
complex swap optimization strategies in the future like #727.
**Collation of requests:**
llama-swap previously handled requests and swapping in the order they
came in. For example requests for models in this order ABCABC would
result in 5 swaps. Now those requests are handled in this order AABBCC.
The result is less time waiting for swap under a high churn request
queue. This fixes#588#612.
A possible future enhancement is to support a starvation parameter so
swap can be forced when models have been waiting too long.
**Shared base implementation for groups and swap matrix:**
During the refactor it became clear that much of the swapping logic was
shared between these two implementations. That is not surprising
considering the swap matrix was added many moons after groups. Now they
share a common base and their specific swap strategies are implemented
into the swapPlanner interface.
Requests for bespoke or specific swapping scenarios is a common theme in
the issues. Now users can implement whatever bespoke and weird swapping
strategy they want in their own fork. Just ask your agent of choice to
implement swapPlanner. I'll still remaining more conservative on what
actually lands in core llama-swap and will continue to evaluate PRs if
the changes is good for everyone or just one specific use case.
**AI / Agentic Disclosure:**
I paid very close attention to the low level swap concurrency design and
implementation. It's important to keep that essential part reliable,
boring and no surprises. Backwards compatibility was also maintained,
even the one way non-exclusive group model loading behaviour that people
have rightly pointed out be a weird design decision.
With the underlying swap core done the web server, api and UI sitting on
top were largely ported over with Claude Code and Opus 4.7 in multiple
phases. If you're curious I kept the changes in docs/newrouter-todo.md.
I did several passes to make sure things weren't left behind.
However, even frontier LLMs at the time of this PR still make small
decisions that don't make a lot of sense. They get shit wrong all the
time, just in small subtle way.
That said, there's likely to be some new bugs introduced with this
massive refactor. I'm fairly confident that there's no major
architectural flaws that would cause goal seeking agents to make dumb,
ugly code decisions.
For a little while the legacy llama-swap will be available under
cmd/legacy/llama-swap. The plan is to eventually delete that entry point
as well as the proxy package.
On a bit of a personal note, this PR is exciting and a bit sad for me. I
hand wrote much of the original code and this PR ultimately replaces
much of it. While the old code served as a good reference for the agent
to implement the new stuff it still a bit sad to eventually delete it
all.
- update README.md with new docker instructions
- update docs/configuration.md
- update .github/workflows to have pinned action versions
- gofmt events package
- fix small bugs in CI scripts
- reduce config options for internal/perf/monitor and config. A ring buffer is used to keep 1hr of entries at max 5s granularity. For long term stats use prometheus monitoring on /metrics
Fixes#744
Add a comprehensive performance monitoring system that collects CPU, memory, swap, load average, network IO, and GPU stats. Provides both a REST API for the UI and a Prometheus /metrics endpoint.
Backend changes:
- New internal/perf package with configurable interval-based stats collection
- GPU monitoring via LACT (Unix socket) and nvidia-smi fallback on Linux
- Ring buffer (internal/ring) for time-series stat storage
- Prometheus /metrics endpoint with all system and GPU metrics
- Moved LogMonitor to internal/logmon package
- New PerformanceConfig for hot-reloadable monitoring settings
- REST /api/performance endpoint replacing SSE streaming
UI changes:
- New Performance page with real-time charts for CPU, memory, GPU, and network
- Reusable PerformanceChart component
- LLAMA_SWAP_URL environment variable support
- Improved capture dialog display
Other:
- Example Grafana dashboard for Prometheus metrics
- monitor-test standalone binary
- Config schema and example updates
fixes#596
Add configurable HTTP timeout settings to both models and peers to support installations that requires longer timeouts than the current hardcoded defaults.
Closes#618
This PR allows a single llama-swap to be the central proxy for models served by other inference servers. The peer servers can be another llama-swap or any API that supports the /v1/* inference endpoint.
Updates: #433, #299Closes: #296
Add configuration support for api keys that are enforced by llama-swap. Keys are stripped before sending them to upstream servers.
Updates: #433, #50 and #251
The new logToStdout option controls what is logged to stdout. The
default has been changed to just the proxy logs, which contain swap and
http request logs.
There are four supported settings: none, proxy, upstream, both. The
"both" setting is the legacy setting where everything was spewed to
stdout.
