Self hosted · local · api key required

One set of weights on one card, in front of every service.

An OpenAI shaped endpoint over llama.cpp. Before this each site carried its own model server, which on an 8GB card meant whichever one you used last evicted the other. This holds the model, hands out keys, and writes down every prompt and every completion that passes through.

• A proxy, not a library. Anything that already speaks chat completions needs a base url and a header, not a rewrite.
• Keys per caller, so usage is attributable and revoking one stops it on the next request.
• Every call logged in full, prompt and completion, with tokens and throughput, aged out on a retention window.
• Loaded on demand and unloaded after three idle minutes, so a desktop that is also a workstation gets its card back.