One endpoint, many providers

Introduction

At about 11:00 PM an editor agent declined a directory. Not the repository: a subtree, a few hundred thousand tokens of code it had already read once that day, in a session that had been going for an hour. It said the model’s context was 128,000 tokens, so it would summarise first and work from the summary.

The model behind that endpoint serves a million tokens and answers with up to 384,000. It had done so a dozen times that week for other callers. Nothing was down, every health check was green, and the only wrong number in the estate was in a JSON document 9router had served from a registry entry nobody had read since the image was built. The agent had read the catalogue, believed it, and sized its own request to fit inside a limit that did not exist. It was being careful. The endpoint had told it a lie.

That is the subject of this post, and it is not really about routing. 9router is a self-hosted, OpenAI-compatible gateway in front of every model the estate uses, and a gateway like that is a small piece of infrastructure with a large number of assumptions baked into it. Each route encodes a belief about the system behind it: that it listens on this address, that it wants eight sampling steps and not twenty, that its context window is a million tokens, that the event it emits at the end of a response carries an output array. When the belief is right, the machinery is invisible and pleasant. When it is wrong, nothing crashes. A client clamps its own context, an image takes five times as long as it should, a strict client rejects a response that is 98 per cent correct.

What follows is 9router as it runs today, the six patch layers that make it behave, and the parts of it that are still fragile.

What 9router is, and what it is not

It is one self-hosted, OpenAI-compatible HTTPS endpoint in front of two hosted inference plans, Command Code and Opencode Go. Both are flat: ten dollars a month each, with no per-token billing anywhere in the arrangement. Callers speak the shapes they already know: /v1/chat/completions for most of the traffic, /v1/responses for the clients that moved to it, /v1/messages for the ones that speak that dialect, and a legacy prefixed path for a script I have not rewritten. The same endpoint also serves image generation through SD.Next, which owns the checkpoint and the GPU on the other machine, plus text to speech, transcription and web search, because a client holding one base URL is a client I do not have to configure twice.

Every device on the estate holds one base URL and one key. That is the whole proposition. The editor agents on two machines, LibreChat on the same host, and a handful of scripts all authenticate against the same place, and the provider keys live behind it in one file that one process reads.

LibreChat is worth naming because it is part of the deployment rather than a client of it: a chat application on the same host, holding 9router’s key, and the reason the speech and SearXNG routes exist. SearXNG is a local metasearch engine, and it is the one private service the egress guard has to be taught to allow.

LibreChat also runs code on request through its execute_code capability, and that path is out of scope here: it is a code interpreter API on its own bridge network, with an egress gateway as the only route out and a manifest for each execution.

Before this existed the same estate held several provider keys across several configuration blocks. Each key had its own rate limit, its own invoice and its own idea of what a model was called, and no single query could answer what a month had cost. The migration was not about saving money; it was about having one answer to a question I ask every week.

What it is not matters more, because the shape invites wrong assumptions. It is not multi-tenant: one user, my keys. It is not highly available: one host, one compose file, one SQLite file, and a reboot is an outage measured in seconds. It is not a security boundary, and the routing is not cost-aware because there is no cost to be aware of: both plans are flat, and the router spreads load rather than pricing it. It is a maintenance object with a Dockerfile, and the Dockerfile is where most of the interesting decisions live.

nginx publishes it, as it publishes everything else on that host. Two directives are worth naming because they are the ones that were wrong first. proxy_read_timeout 600 because a long-context completion can legitimately take minutes and the default minute is not a timeout, it is a coin toss. client_max_body_size 1000M because an image comes back base64-encoded inside a JSON body, and the largest single thing 9router has ever had to carry is one of those responses. The limit is not a design statement about how big requests should be; it is the size at which the proxy stops lying about what it can carry.

The shape of it

One compose.yaml, two services. 9router is built from an upstream release base image plus the patches described later, and publishes one port. Headroom, a proxy that compresses long prompts before they leave the host, sits on its own port with a /health route that returns status: healthy. Everything else is state.

That state is a single bind mount, and the contents are the whole memory of the deployment: a SQLite database, a machine identifier, a JWT secret, a CLI secret, a cached model catalogue, the write-ahead log, and timestamped backups. The catalogue is 620 KB raw and 316 KB normalised, covering 26 providers, synced from upstream rather than hand-written. It is what the /v1/models route answers from, which makes it the most consequential file in the mount.

One directory buys three things. Backup is a copy. Restore is a copy back. Moving the whole thing is a copy plus a compose file, the only disaster-recovery plan I have ever actually tested at 2:00 AM. What it costs is in the failure modes: one file is one point of truth, backed up, not replicated.

flowchart LR
    E["Editor agents"] --> N["nginx, TLS"]
    F["LibreChat"] --> N
    S["Scripts"] --> N
    N --> R["9router"]
    R --> P1["Command Code"]
    R --> P2["Opencode Go"]
    R --> D["SD.Next, ARM64 desktop"]
    R --> T["Speech route"]
    R --> W["SearXNG"]
    R --> Z["Headroom"]
    R --> DB["SQLite on a bind mount"]

Read the diagram as one gate and four kinds of thing behind it. Inference is two flat monthly plans, ten dollars each, and between them they carry almost exactly half the traffic apiece. Images are SD.Next on a machine on my desk. Speech is a route inside 9router, and search goes out to SearXNG. Headroom sits in the path of text requests rather than beside it, which is the difference between a route and an interceptor.

Addresses and domains are placeholders in what follows. The code is not: the paths, the environment variables and the literals are the ones inside the built image.

One endpoint, many models

The unit of configuration is a combo: a named list of <provider-alias>/<model-id> entries, resolved by a round-robin fallback strategy. There are two, and clients ask for a combo by name. One is the default for the editor agents, the other is the one LibreChat uses for chores, which is mostly generating conversation titles from the first exchange. Titles are cheap, they are generated constantly, and giving them a connection of their own rather than letting them compete with real work is the sort of decision that pays for itself in a week of noise.

Round-robin does not mean random. A request goes to the next connection in the list that is not in trouble, and “in trouble” is a per-connection state machine rather than a guess. The state lives in the database, one row per connection, and the fields are a small vocabulary of failure: priority and isActive say whether a connection is in play at all; testStatus and lastUsedAt say when it was last proven; consecutiveUseCount counts how hard it has been leaned on; backoffLevel, errorCode, lastErrorAt and rateLimitedUntil say why it is being avoided and for how long. Per-model locks sit beside that, so two requests cannot both decide to repair the same model at once.

There is also a thinking mode set per connection, max in the current configuration, because thinking is a property of the deployment rather than of the request in this estate.

Two more records live per provider alias rather than per connection: disabled models and custom models, the latter carrying a caps block for vision and reasoning. That is how a model the upstream registry has never heard of can be addressed by name, and how a model that misbehaves can be removed without editing a combo.

The count that made me trust the arrangement was 66: sixty-six distinct model ids appear in the usage rollups across the two providers. That is not sixty-six deliberate choices, it is upstream catalogues moving and clients pinning names that were current in September. A combo absorbs that churn for the clients that ask for a combo; the ones that pin a model id get what they asked for, which is occasionally nothing.

sequenceDiagram
    autonumber
    actor U as Me
    participant F as LibreChat
    participant R as 9router
    participant X as Headroom
    participant A as Command Code
    participant B as Opencode Go
    participant D as SQLite
    U->>F: send a turn
    F->>R: POST /v1/chat/completions with a combo name
    R->>D: read the combo and its connections
    D-->>R: ordered connections with health and backoff
    R->>X: pass the request body through
    X-->>R: compressed or unchanged body
    R->>A: first connection, round robin
    alt the provider answers
        A-->>R: completion with token counts
    else the provider fails or is rate limited
        R->>D: record errorCode, backoffLevel, rateLimitedUntil
        R->>B: next connection
        B-->>R: completion with token counts
    end
    R->>D: usageHistory row and usageDaily rollup
    R-->>F: the answer, streamed
    F-->>U: rendered reply

The fallback is the part people assume is clever and is actually just disciplined. Whether a failed request is tried on the next connection is decided by its error class rather than by the fact that it failed, so the class a connection returns determines whether the request is handed on. That distinction is why errorCode is stored rather than logged and forgotten.

Capability metadata is the contract

GET /v1/models returns one entry per model, and the entry is richer than the OpenAI shape most clients expect. Alongside the identifiers there is a capabilities block: vision, pdf, audioInput, videoInput, imageOutput, audioOutput, search, tools, reasoning, thinkingFormat, thinkingCanDisable and thinkingRange. Alongside that, context and output limits, and here the catalogue speaks both dialects at once, reporting contextWindow and context_length, maxOutput and max_completion_tokens, because different clients read different keys and 9router serves both.

Clients act on those numbers. The editor agent uses the context window to decide when to compact a session. LibreChat uses the capability block to decide whether to offer an attachment picker, and whether to warn before sending a PDF. Scripts decide from the same document whether a task is worth attempting. The catalogue is not documentation a human reads once; it is the input to decisions made on every request, by software with no way to check it.

Which is why a wrong entry is worse than a missing one. The mis-specified model claimed contextWindow: 128000 and maxOutput: 64000. The model behind it genuinely serves 1,000,000 and 384,000. The claim was not an outage, a 500 or a log line: it was a 128k ceiling installed in every client that read the catalogue, by a client doing exactly what it was told. The editor agent works under it and summarises; nothing anywhere records that a limit was hit for no reason.

The fix is patch layer 5, which rewrites two numbers in the registry and two minified literals in the compiled bundle. /v1/models now reports contextWindow: 1000000 and maxOutput: 384000 for the combo, and the agent stops compacting a session that fits.

That is the lesson, and it generalises past 9router. A gateway that reports what it wishes were true, rather than what the system behind it can do, has not described a capability; it has installed a policy nobody voted for. When the number is generous the failure is a rejected request and a clear error. When it is mean, the failure is silence, and silence costs months.

Headroom

Headroom sits in front of text requests, and 9router has a toggle for it in the settings table, so it can be taken out of the path without a rebuild.

It is a separate service for two reasons. It is not code I patch: keeping it in its own container makes an upstream release a tag change, and the six layers below apply to the 9router image alone. And it is optional: compression in the request path is worth it on long prompts and pointless on short ones, and a toggle means the trade is judged by the usage table rather than argued about.

The costs are the obvious ones. It is another hop, so latency has a new floor. It is another thing that can be down, and when it is down the symptom is every text request failing at once while 9router’s own health check is perfectly happy, because 9router is fine; its dependency is not.

Speech out, speech in

Text to speech was the easiest thing in the estate, which is worth recording because it makes the images section look like what it is.

POST /v1/audio/speech takes {model|voice, input} and returns audio/mpeg. One sentence measured 0.63 seconds and 16,128 bytes: MPEG-1 layer III, 32 kbps, 16 kHz mono. A sentence of speech costs about 16 KB and two thirds of a second, cheap enough that reading a paragraph aloud is a reasonable thing to ask for on a whim, and cheap enough that caching the output is mostly pointless.

Voice identifiers are namespaced edge-tts/<locale>-<Name>Neural in LibreChat’s configuration: 62 voice ids across nineteen locales, of which fourteen are English, four French and one Lithuanian. The US locale alone carries 17 of the 62. 9router does not care about the naming; it forwards a voice string and returns audio. The namespace lives in LibreChat because that is where the list is displayed, and duplicating it in 9router would be a second list to keep in step.

LibreChat’s speech.tts block points at 9router’s speech route with 9router’s key, and carries an allowedAddresses entry for the host 9router runs on. That second clause is the interesting one. LibreChat has an egress allowlist of its own, so a new host in the path is a configuration change in two places, and forgetting the second produces a refusal that reads like a network fault. Two allowlists are two places to get wrong; they are also two places to notice.

Transcription is the road not taken. 9router carries an sttCore route, wired and working, and nothing in the estate uses it yet. I keep it because a route in the base image costs nothing to leave alone, and the day a meeting needs transcribing the alternative is building the same thing badly somewhere else.

The lesson from speech is the contrast with images: when the upstream product already routes a capability through the compatible interface, the work is a key and an allowlist. When it does not, the work is a Dockerfile.

Images: the workstation on the desk

SD.Next runs on a separate machine: an ARM64 desktop with 64 GB of unified memory, sitting on the desk rather than in a rack. It is launched with --api --listen --port 7860 --ckpt <diffusers path> --debug against a diffusers-format checkpoint directory. Two checkpoints are present: Z-Image-Turbo, a distilled 6B-class model, which is the default, and an ordinary SDXL-class base checkpoint I keep for the cases where I want the slower answer.

There are two paths from the estate to that machine, and they are not the same path.

The first is the route through 9router. POST /v1/images/generations is handled by the image provider, which posts to {base}/sdapi/v1/txt2img with {prompt, width, height, cfg_scale: 1.0, steps: 8, batch_size: n}, receives {images: [b64]}, and normalises it to the OpenAI shape {created, data: [{b64_json}]}. Measured: 512 × 512, one image, 27.3 seconds through 9router.

The second is a tool inside LibreChat, patched to call SD.Next’s API directly with environment-driven defaults: SD_WEBUI_URL, SD_WEBUI_STEPS=8, SD_WEBUI_CFG_SCALE=1.0, and an optional override_settings.sd_model_checkpoint pointing at a diffusers-format checkpoint directory. That call bypasses 9router’s SSRF guard deliberately, because the guard would correctly refuse to let a chat tool reach a private address.

The measurement that taught me the most was not the 27.3 seconds. It was the second request, which named a model id I had already deleted from the registry: 25.9 seconds, one image, no error. The model id on this route is a label, not a selector. The handler never sends it, so what gets generated is whichever checkpoint SD.Next happens to have loaded. A client asking for one model and receiving another is a bug that will never produce an error message, and the only reason I know is that I timed two requests and compared them.

sequenceDiagram
    autonumber
    participant C as LibreChat tool
    participant R as 9router
    participant H as Image handler
    participant S as SD.Next
    C->>R: POST /v1/images/generations
    R->>H: route to the image provider
    Note over H: rewrite one, at build time, points the registry at the workstation
    H->>S: POST /sdapi/v1/txt2img with cfg_scale 1.0 and steps 8
    S-->>H: images as base64 inside JSON
    Note over H: rewrite two, per response, becomes created plus data with b64_json
    H-->>R: OpenAI-shaped JSON
    R-->>C: one 512 by 512 image in 27.3 s

Why eight steps and a guidance scale of one is not a preference. Z-Image-Turbo is distilled, and eight steps at 1.0 is what it was trained to be used with. The upstream handler hardcoded twenty steps, which on this machine is roughly five times the wait for a worse image: the sampler keeps refining past the point the distillation was tuned for, and the result is overcooked. The first time I ran it with the stock defaults I assumed the machine was slow. It was doing exactly what it was told, for longer, to worse effect.

That is the phrase I would put above the whole setup: every route is a set of instructions to something else, and the defaults are written by somebody who has never seen your hardware.

The injections: six patch layers

9router is an upstream product and I am not its maintainer. The Dockerfile is where I argue with it: six patch layers, applied in build order, each in its own RUN, each ending with an assertion that the patch landed. The file grew from 13 lines to 128 across five edits between 21 September and 13 October, and every revision is backed up beside it, because a patch that no longer applies is a fact I want to be able to look up rather than reconstruct.

The paths below are the ones inside the built image, so a snippet copied from here edits the same files the container runs.

Layer 1: the endpoint rewrite

/app/open-sse/providers/registry/sdwebui.js hardcodes localhost:7860. On this estate SD.Next is a different machine, so the address is wrong the moment the image is built. The layer rewrites it to the workstation’s address, in the source and in the compiled route bundle, and fails the build if any localhost:7860 survives.

Layer 2: the sampling defaults and the model label

/app/open-sse/handlers/imageProviders/sdwebui.js sends twenty steps and no guidance scale; the layer sends eight steps and a guidance scale of 1.0. It also replaces the registry’s two stock model entries, a v1.5-era model and an older base checkpoint, with a single entry naming Z-Image-Turbo. This is the layer that must be applied twice: the same literals exist in readable source and again, minified, in the compiled route bundle. Both are patched and both asserted, and that duplication is the single most important thing I know about this image.

Layer 3: the outbound request allowlist

The application’s SSRF guard blocks any private hostname, which is correct and which also blocks the internal SearXNG instance that 9router uses for web search. The guard took new URL(request.url).hostname, compared it against a blocklist, and threw Blocked URL: internal host. The layer patches the two guard functions in the compiled bundle so they consult NINEROUTER_SSRF_ALLOWED_HOSTS, a comma-separated list read before the block decision, and the compose file sets that variable to ${SEARCH_HOST}.

I want to be blunt about this one, because it is the layer I am least comfortable with. It is a security control taught to skip itself for a list of names. The list has exactly one entry, it is reviewed whenever it changes, and the day it grows to three I should be asked why.

Layer 4: the token clamp

/app/open-sse/translator/request/openai-to-commandcode.js defaulted max_tokens from the provider’s own default, which silently truncated long-context requests. The layer replaces that with a clamp against the caller’s own value: Math.min(body.max_tokens ?? body.max_output_tokens ?? DEFAULT_MAX_TOKENS, 200000). The ceiling is deliberately generous, since the largest completion this estate has ever produced is 47,572 tokens, so the clamp is a guardrail rather than a limit anyone reaches in ordinary work.

Layer 5: the capability correction

/app/open-sse/providers/capabilities.js claims a 128,000-token context and a 64,000-token output for a model that serves 1,000,000 and 384,000. The layer patches both numbers, in source and in the bundle, where they appear as 128e3 and 64e3. This is the layer that ends the story in the introduction, and it is four numbers in two files.

Layer 6: the Responses transform

/app/open-sse/transformer/responsesTransformer.js emits the response.completed event for the Responses endpoint without an output array. Loose clients do not notice; strict ones reject the event and the answer never arrives. The layer adds output: []. It is written in node -e with an explicit old and new pattern, exits non-zero if the old pattern is absent, and verifies after writing.

Two snippets carry the shape of the whole file. The first is a layer that must land in two places, which is where the verification idiom earns its keep:

sed -i "s#localhost:7860#${WORKSTATION}:7860#g" /app/open-sse/providers/registry/sdwebui.js
sed -i "s#localhost:7860#${WORKSTATION}:7860#g" /app/.next/server/app/api/v1/images/generations/route.js
! grep -q "localhost:7860" /app/open-sse/providers/registry/sdwebui.js
! grep -q "localhost:7860" /app/.next/server/app/api/v1/images/generations/route.js
grep -q "${WORKSTATION}:7860" /app/open-sse/providers/registry/sdwebui.js
grep -q "${WORKSTATION}:7860" /app/.next/server/app/api/v1/images/generations/route.js

And the sampling layer, which is one line of substitution that decides whether an image takes 27 seconds or two and a half minutes:

sed -i "s#steps: 20, batch_size: n#cfg_scale: 1.0, steps: 8, batch_size: n#g" /app/open-sse/handlers/imageProviders/sdwebui.js
sed -i "s#steps:20,batch_size:n#cfg_scale:1,steps:8,batch_size:n#g" /app/.next/server/app/api/v1/images/generations/route.js
! grep -q "steps: 20" /app/open-sse/handlers/imageProviders/sdwebui.js
grep -q "steps: 8" /app/open-sse/handlers/imageProviders/sdwebui.js

The six layers divide into two groups, and the division is the honest summary of the work. Layers 1 and 2 are what make the image path work: without them the route points at nothing and samples at the wrong settings. Layers 3, 4, 5 and 6 are what stopped everything else misbehaving: search was refused, long prompts were cut, the catalogue lied, strict clients rejected a valid completion. Layer 6 is in that group because it is the seam where a new route met an old assumption.

flowchart TD
    A["Upstream release base image"] --> L1["Layer 1: endpoint rewrite"]
    L1 --> V1{"old literal gone?"}
    V1 -- no --> F["build fails"]
    V1 -- yes --> L2["Layer 2: sampling defaults and model label"]
    L2 --> V2{"source and bundle patched?"}
    V2 -- no --> F
    V2 -- yes --> L3["Layer 3: egress allowlist"]
    L3 --> V3{"guard consults the list?"}
    V3 -- no --> F
    V3 -- yes --> L4["Layer 4: token clamp"]
    L4 --> V4{"clamp present?"}
    V4 -- no --> F
    V4 -- yes --> L5["Layer 5: capability correction"]
    L5 --> V5{"1M and 384k present?"}
    V5 -- no --> F
    V5 -- yes --> L6["Layer 6: responses output array"]
    L6 --> V6{"output array added?"}
    V6 -- no --> F
    V6 -- yes --> G["image tagged and deployed"]

Verification discipline: fail the build, not the night

The idiom in every layer is the same four steps: search for the old literal, patch it, assert the old literal is gone, assert the new one is present. grep -q before and after, ! grep -q in between. It reads like belt and braces and it is not, because the thing being patched is a minified bundle whose content changes with every upstream release.

A sed that matches nothing exits zero. A patch layer that no longer applies therefore produces a green build, a container that starts, a route that answers, and behaviour that is subtly the old behaviour. On the image route that is a two-and-a-half-minute wait instead of twenty-seven seconds; on the catalogue it is the 128k lie, back again, silently. The assertion is the only thing standing between a release and a quiet regression.

The cost is that the build breaks when upstream renames something, which happens, and that is the deal I want: a broken build at 9:00 AM is a task, a broken default at 11:00 PM is an evening. I would rather maintain a file that fails loudly than a container that runs quietly wrong.

What the asserts do not catch is semantic drift: a literal that survives but no longer means what it did. If a release keeps steps: 20 in a different code path and moves the sampled value elsewhere, the patch applies, the assert passes, and the image is generated with the wrong settings. The only defence I have found is to measure the behaviour, write the number down, and re-measure after an upgrade.

Storage and accounting

Seven tables carry the memory of the deployment. providerConnections holds the per-connection state described earlier. combos holds the two named lists. kv holds the disabled-model and custom-model records per provider alias. settings holds the fallback strategy, the thinking modes, the Headroom toggle and a password hash. apiKeys holds the keys the estate authenticates with. And two of them carry the history: usageDaily, a per-day rollup of requests, prompt tokens, completion tokens, cached tokens and cost, broken down by provider and model; and usageHistory, one row per request with the timestamp, provider, model, connection, key, endpoint, token counts, cost, status, a tokens JSON with the cached and reasoning breakdown, and a meta field.

An eighth table, requestDetails, is empty on purpose. Full request logging is off. That is a privacy decision and it has a price: when a response looks wrong, I can see the token counts and the endpoint and cannot see what was asked. I have paid that price twice and would pay it again, but it belongs in the list of things 9router cannot answer. Two more exist and are empty: a provider-node table and a proxy-pool table, surfaces the schema carries and this deployment does not use.

The database is about 14 MB after seven weeks, with the write-ahead log enabled. Backups are written beside it on demand and named data.sqlite.bak-<ISO timestamp>, which sorts, which is the only property a backup name needs.

The last column of the table below is 9router’s own arithmetic rather than a bill; the next section explains why.

The endpoint mix validates keeping the old paths: /v1/chat/completions 26,984 requests, /v1/responses 225, /v1/messages 186, and the legacy /api/v1/chat/completions 36. The long tail is a fraction of a per cent, and every entry in it is a client I would otherwise have rewritten.

Two flat plans and a shadow price

47 days of use, from 8 September to 27 October 2026: 27,431 requests, all with status: ok, 2.2 billion prompt tokens and 25.71 million completion tokens. The money that actually left the account is about $31: two plans at ten dollars a month across a month and a half. No part of the arrangement bills per token.

Every other dollar figure in this post is 9router’s own arithmetic. The cost column in usageDaily and usageHistory multiplies tokens by a per-model price table the router ships, and against flat plans that produces a shadow price at list price: $177.87 for the same seven weeks, roughly six times what I paid. It is not a bug and it is not an invoice; it is the number that lets one model be compared with another, which is all a price table can do when nobody pays by the token.

The busiest day was 26 October with 3,143 requests and 177.8 million prompt tokens, and $16.39 of shadow cost for that day alone. The largest single request carried 550,038 prompt tokens and the largest completion was 47,572 tokens. Neither number has a bill attached to it.

The two plans carried 13,234 and 14,095 requests: almost exactly half the traffic each, for the same ten dollars a month. The only asymmetry in the accounting is that Opencode Go reports 1.007 billion cached prompt tokens, 86.6 per cent of its prompt total, and Command Code reports no caching at all.

Two requests on the same day show it. A 150,817-token prompt with 150,016 tokens cached came out at $0.0012 on Opencode Go; a 62,232-token prompt on Command Code came out at $0.0089. Neither was billed: the seven-times gap is two price lists and two reporting habits, not two invoices.

So what did 9router save? Never money: two ten-dollar plans would have cost about $31 with or without it. It saved one key per device, one catalogue, one place to add a plan, one table that answers what the month did by endpoint and model, and two cheap flat plans behind a single name so no client knows which one answered.

What I would change, and the failure modes that are still there

The model id on the image route is decorative. The handler does not send it, so the label names a checkpoint the client is not getting. Fixing it means passing the id through as a checkpoint override, which means 9router has to know SD.Next’s checkpoint names well enough to be honest about them.

Patches to minified bundles are the most brittle thing in the setup. They break on every upstream release by design, and the build asserts are the only reason that is a nuisance rather than a regression. The real fix is an upstream seam, and that is not mine to land.

The SSRF allowlist is a security control taught to skip itself. It must stay a list of exactly one host. Every addition is a decision about which private service 9router may reach on behalf of a request that may have been shaped by content it read, and that decision deserves a reviewer.

Headroom in the path is another thing that can be down. Its failure looks like a total outage of text requests while 9router reports healthy, because 9router is healthy. The toggle is the mitigation and it is a human one.

One SQLite file is a single point of truth. It is copied beside itself before I touch something, not on a schedule, and not replicated at all; the two backups that exist as I write are ten minutes apart. A corrupted database costs the accounting history, not the service, because the routing state can be rebuilt from the provider catalogue. I would not want to test that sentence.

Usage numbers are only as honest as the provider’s accounting. Command Code reports no cache hits at all; Opencode Go reports 1.007 billion of them. The 86.6 per cent figure is a fact about what one provider chose to report, not a fact about the world. And the cost column is a price table rather than an invoice, which with two flat plans it never was, so it can drift a long way from reality without anything looking wrong.

The direct image path is invisible to accounting. LibreChat’s own tool calls SD.Next without going through 9router, so those images never reach usageHistory. They cost electricity rather than tokens, easy to forgive and easy to forget.

Routing should optimise headroom, not price. Round-robin is fair and blind. With two flat plans there is no price to weigh, so a smarter strategy would weigh headroom and cache behaviour: send long-context work to the plan that has room and reports its cache hits, leave the other for what it does best. The two same-day requests above are the illustration, though the seven-times gap belongs to the price table rather than to the plans.

Closing checklist for anyone building the same setup

1. Publish it behind a real proxy, with a read timeout and a body limit you chose rather than inherited.
2. Put every credential in one file read by one process, and give each device a base URL and a key.
3. Treat /v1/models as the contract: if a number is in it, some client will act on it.
4. Assert every patch. A sed that matches nothing exits zero, and silence is the failure mode you cannot see.
5. Patch the readable source and the compiled bundle, and assert both, because both ship.
6. Keep any egress allowlist at exactly the hosts you can defend naming aloud.
7. Measure the path end to end and write the number down: 0.63 s for a sentence, 27.3 s for an image.
8. Decide what you are not logging before you need the log, and accept the price of that decision.

None of this is exotic. It is one compose file, one bind mount, a proxy with two directives that matter, and six patch layers that exist because a product made assumptions about hardware it had never met. What 9router changes is where the decisions live: the capability of a model is a field in a catalogue rather than a guess inside an agent, the invoice is one number a month with a shadow table beside it, and the assumption behind a route is a line in a Dockerfile with an assertion under it.

That is the same argument as the rest of the AI-augmented engineering work: a written contract, a small surface, an audit line. The routing is not the interesting part. Being able to say, six months later, why a request took twenty-seven seconds and which assumption put it there, is.