mirror of
https://github.com/mostlygeek/llama-swap.git
synced 2026-09-07 16:37:22 +02:00
cphlipotandGitHub
51d1969c19
add a -api-key option that the user can optionally provide to give wol-proxy a private api key it will use for authenticating its health checks against the llama-swap server. This is api key is not used as part of any client request, and clients connecting to the wol-proxy will still need to provide their own api keys matching one of the ones in the destination llama-swap's config.yaml Fixes: #1051
wol-proxy
wol-proxy automatically wakes up a suspended llama-swap server using Wake-on-LAN when requests are received.
When a request arrives and llama-swap is unavailable, wol-proxy sends a WOL packet and holds the request until the server becomes available. If the server doesn't respond within the timeout period (default: 60 seconds), the request is dropped.
This utility helps conserve energy by allowing GPU-heavy servers to remain suspended when idle, as they can consume hundreds of watts even when not actively processing requests.
Usage
# minimal
$ ./wol-proxy -mac BA:DC:0F:FE:E0:00 -upstream http://192.168.1.13:8080
# everything
$ ./wol-proxy -mac BA:DC:0F:FE:E0:00 -upstream http://192.168.1.13:8080 \
# use debug log level
-log debug \
# altenerative listening port
-listen localhost:9999 \
# seconds to hold requests waiting for upstream to be ready
-timeout 30 \
# API key sent as Bearer token to the upstream SSE endpoint
# (can also be set via the LLAMA_SWAP_API_KEY env var; the flag wins if both are set)
-api-key <key>
API
GET /status - that's it. Everything else is proxied to the upstream server.