Hot-swap local model backends for agent workflows with llama-swap
Use llama-swap to give agents one OpenAI/Anthropic-compatible endpoint that loads, unloads, and switches local model servers on demand.
npx skills add agentskillexchange/skills --skill hot-swap-local-model-backends-for-agent-workflows-with-llama-swap
Use llama-swap when an agent or operator needs a stable local inference endpoint while different model backends are started, stopped, routed, or replaced behind the scenes. The workflow is to define model IDs in a llama-swap config, point OpenAI-compatible or Anthropic-compatible clients at llama-swap, let requests load the right upstream server, and use profiles, TTLs, unload APIs, logs, metrics, and optional swap matrices to control local GPU and CPU resources. Invoke this instead of manually launching individual llama.cpp, vLLM, stable-diffusion.cpp, audio.cpp, ComfyUI, or similar servers when agent runs need repeatable model selection, predictable endpoint URLs, and observable local backend switching. The scope boundary is local model routing and lifecycle control for agent/operator workflows; it is not a generic local-LLM product card, model-serving framework listing, MCP-server listing, or broad SDK entry.
What this skill actually does
Inputs and prerequisites: llama-swap binary or container, a config.yaml with model definitions, local model servers such as llama.cpp or vLLM, and an OpenAI-compatible or Anthropic-compatible agent/client endpoint.
Setup notes: Install with Homebrew using brew tap mostlygeek/llama-swap && brew install llama-swap, pull a GHCR container such as ghcr.io/mostlygeek/llama-swap:unified-cuda13, use release binaries, or build from source. Create a config.yaml with models and cmd entries, then run llama-swap –config path/to/config.yaml –listen localhost:8080 and point compatible agent clients at that endpoint.
Source and verification boundary: use https://github.com/mostlygeek/llama-swap as the canonical reference before running the workflow; keep commands, API calls, CLI usage, and generated outputs reviewable against that upstream source.
Framework fit: publish this as a Multi-Framework workflow only when the operator can invoke the documented toolchain directly, rather than treating the upstream project as a generic product listing.