The front door

Introduction

A chat application is the smallest part of a chat deployment.

I have written that sentence in notes three times since September, and each time it arrived from a different side. This time it was the model list. The picker offers two models, and both are named combos the router serves. The list is typed by hand into a configuration file because discovery is switched off for that endpoint. Switch it on and the app asks the router what it has; the router answers with 56 model ids — the two combos, and 54 provider models sitting behind them — and the picker stops being a short list of things this deployment is for and becomes a catalogue of everything the estate could reach. So discovery stays off, the list is curated, and it will go on naming two combos after one of them is renamed, with no warning when the day comes.

That is not a defect in the chat application. It is the shape of the thing. A chat front-end is a browser UI, a login system, a conversation store and an adapter layer, and the deployment is everything the adapter talks to: a router, a search engine, a vector store, an ingestion pipeline, a speech route, an image server, a sandbox. The app draws the front door. Behind the door is a building, and the building has its own keys, its own addresses and its own opinions about who may talk to whom.

This one took twelve hours to wire, on 20 and 21 September, and the evidence is the backups: thirteen copies of .env, five of librechat.yaml, three of the compose override, all written inside that window. It has run since with a restart count of zero on all seven containers. It has also been mostly idle: three users, twenty conversations, seventy-three messages, last login 18 October, last web search 13 October.

What follows is what that deployment actually is, where its decisions live, and which parts of it I would not defend. The application appears throughout mainly as the place where other people’s decisions become visible. The router has its own post and is not described again here. What is new is the front of the stack: one file, two mounts, two allowlists, one token scheme, and the gap between what is configured and what has been used.

What actually runs

Seven containers on one compose bridge: the app, an admin panel, MongoDB, Meilisearch, pgvector, a RAG API and SearXNG. Six of the seven exist because they are separate programs with separate release cycles, not because microservices are a good idea for a personal chat client. The reasons are worth writing down, because the same argument decides what you patch and what you upgrade.

Three tags are pinned and four float, which is the inverse of what I would recommend to anyone else. The three pinned ones are pinned because they hold state whose format is version-specific: a Mongo data directory, a Meilisearch index, a PostgreSQL data directory with an extension. Losing that bet costs a restore, so the tag does not move by itself. The four that float are the four that hold no state of their own, and the app is one of them: librechat-dev:latest is a floating tag over a release candidate, and the running image reports v0.8.8-rc3 at commit ba44443f. That number will change without a compose edit, and it is the reason the patch described later has an upgrade story rather than a maintenance story.

One bridge means every service is reachable by name and nothing is reachable by port. Only three containers publish anything: the app, the admin panel and the search UI. Mongo, Meilisearch, pgvector and the RAG API sit on the bridge with no host binding at all, which is the correct default and also the reason the next paragraph matters.

Mongo runs as mongod --noauth --port 27018. No authentication, on a network shared with six other containers, one of which is a search engine that scrapes arbitrary pages for a living. Nothing outside the host can reach it. Everything inside can read every conversation with one connection string, and the only thing standing between a compromised sidecar and the message history is that nobody has compromised a sidecar. For a personal deployment that is a considered trade. It is not a security boundary, and I would not describe it as one.

What “up” tells you is worth being precise about. docker ps shows all seven running, and RestartCount is zero on every one. Only the admin panel declares a healthcheck: a bun -e snippet that fetches /health on its own port, every 30 seconds, with a 5-second timeout, a 15-second start period and three retries. The other six have no healthcheck at all, so “Up 13 days” for the app means the process has not exited in thirteen days. It does not mean the database connection is alive, the router is answering, or the tool layer can load. A container that is up and a service that works are different claims, and only one of them is being tested here.

The storage tells the same story in miniature. Mongo occupies 8.3 MB on its bind mount around 2.3 MB of actual data. The pgvector volume is 47 MB and the vector table inside it is 240 kB, of which 32 kB is the heap. Meilisearch holds 140 kB. Uploads are 71 files and 509 kB. Generated images are 19 files and 2.5 MB. Logs are 92 kB. The whole deployment, minus images and the database engine’s own overhead, is smaller than one photograph from a phone.

And the data is a personal deployment’s data: 3 users, 20 conversations, 73 messages, 89 files, 570 transactions and a long tail of empty collections, including balances, sessions, prompts, memories and every agent table except one. Pouring a seven-container stack around 2.3 MB of records is not efficient. It is, however, the only way to find out which of the six sidecars you actually use, and the answer is in the numbers rather than in the feature list.

Addresses and domains are placeholders in what follows. The configuration keys, the numbers and the file paths inside the image are not.

The configuration file is the product

librechat.yaml is 100 lines, and 62 of them are text-to-speech voice identifiers. The other 38 lines are the deployment. They declare a schema version, a cache setting, one custom endpoint, two models, a title model, a search provider with one allowlisted address, a speech block with the same router and a different allowlist entry, and eight agent capabilities. That is the whole product surface: a hundred lines, mounted read-only into the container, and everything the app can do is downstream of them.

The first line is the one that will break first. version: 1.3.16 is the configuration schema the app validates against, not the app version. The app validates the file at boot and refuses to start on a mismatch it does not recognise, which is a good property and a coupling: an image on a floating tag can move the accepted schema while the file stays where it was, and the failure is a refused boot rather than a warning. Pinning the image would make the coupling explicit. Floating it makes the coupling a surprise with a schedule.

The mount is read-only and there is no reload path. Every change to these 38 lines is a host edit and a container restart, which makes the file the one part of the deployment where a mistake is an outage measured in seconds rather than a slow drift. That is also why the wiring session left five backups of this one file: five edits, five restarts, and no way to roll back from inside the app.

cache: true is set with no Redis anywhere in the stack. The cache therefore runs on the in-process path, which is fine for one app container and wrong the moment there are two, because each would keep its own copy and invalidation would stop meaning anything. It is a one-line setting that quietly encodes a one-replica topology.

The endpoint block is the interesting part:

- name: "9Router"
  apiKey: "${NINEROUTER_API_KEY}"
  baseURL: "http://host.docker.internal:20128/v1"
  models:
    default:
      - name: "DeepSeekCombo"
      - name: "DSv4.1"
    fetch: false
  titleConvo: true
  titleModel: "DeepSeekCombo"
  modelDisplayLabel: "DeepSeek Combo"

baseURL reaches the router over the docker bridge address, and apiKey is a ${VAR} placeholder substituted at boot from the bind-mounted .env. The credential is in one file that one process reads, which is the right arrangement and the reason the same key can be reused for speech without a second copy existing anywhere.

fetch: false is the line that started this post, and it is curation rather than laziness. With fetch on, the app asks the endpoint for its model list and populates the picker from the answer. The route is authenticated, and answers a bare request with a 401, but the app sends the bearer and the bearer is enough: the router replies with 56 model ids, two combos and the 54 provider models they fall back to. A picker of 56 entries, most of which are not meant to be addressed directly, is worse than a picker of two. So discovery is off and both names are typed by a human, and the price is that nothing reconciles them with the router’s combos. The list will keep naming a combo after it has been renamed, and nothing will tell me until a conversation fails on a model name the router has never heard of.

titleConvo: true with titleModel: 'DeepSeekCombo' is the other cost, and it is on the hot path. Every new conversation makes a second model call to generate a title from the first exchange. The call is small and the model is the cheap combo, so the money is nothing, but the latency is not: the first message of every conversation waits for two round trips rather than one, and the second one is to the same rate-limited router. Turning it off means a sidebar full of untitled conversations, which is worse for a tool used by three people who need to find things later.

modelDisplayLabel renames the endpoint in the interface. It is cosmetic and it earns its line: the endpoint is called 9Router in configuration and something human in the picker, so the config can be named for the thing it points at and the UI for the thing it does.

The precedence between these layers is the part that has already cost me an hour. There are three places a value can come from, and they do not rank the way the file layout suggests.

flowchart TD
C["compose environment:"] --> W["value seen by the process"]
E[".env on a bind mount"] --> W
Y["librechat.yaml with VAR substitution"] --> W
D["MONGO_URI and HOST in .env"] -. "shadowed, dead" .-> E

The live proof is two values. The .env sets MONGO_URI to a loopback address and HOST to localhost. Compose sets MONGO_URI to a service name and HOST to a wildcard bind, so the process never sees either .env value. They are dead lines. A file with 1,222 non-empty lines that has been edited by hand for two days will contain a few of those, and the person debugging at midnight will read the file, believe it, and spend an hour on a connection string that was never used. Precedence is not documentation. It has to be demonstrated, or it is folklore.

One line, eight capabilities

The single most consequential line in the file is an array:

endpoints:
  agents:
    capabilities:
      [
        "execute_code",
        "file_search",
        "web_search",
        "artifacts",
        "context",
        "tools",
        "chain",
        "ocr",
      ]

Eight entries, one line, and the app stops being a chat client. execute_code gives an agent a sandbox to run programs in. file_search gives it retrieval over uploaded documents, which reaches through the RAG API into pgvector. web_search gives it SearXNG and a scraper. artifacts gives it rendered documents and diagrams. context gives it a way to keep material across a conversation. tools gives it function calling. chain gives it multi-step runs. ocr gives it text out of images.

Each of those is a subsystem, and the line turns all eight on at once. There is no per-capability cost in the app; the cost is in everything the capability reaches. execute_code is the clearest example, because it is the reason a sandbox tier exists at all: a second compose project, its own bridge network, an egress gateway, a runner, a file server and a tool-call server, none of which would be running if that one word were absent.

Read the list again as infrastructure rather than features. chain changes the shape of a turn: one message becomes a plan of steps, and the interface has to show them or the user sits through a long silence. artifacts needs somewhere to put what it renders, which is the same images directory the generated pictures land in. ocr is a model call over an image, so it inherits every limit the endpoint has. context and tools assume the model behind the endpoint can hold material and call functions, which is a claim in the router’s catalogue rather than one this file can make good on.

The array is scoped to the agents endpoint and nothing else: an ordinary conversation still gets a text box and a send button. That boundary matters when something behaves strangely, because an agent in the interface is the only mode in which six of the eight exist, and the other two are the ones the file already needed.

Turning it on took seconds. Making it safe took the rest of the session, and the safety lives outside the app: network isolation, an egress gateway as the only route out, and a token scheme described further down. That asymmetry is the thing to notice about capability flags in general. The flag is a UI decision. The blast radius is an infrastructure decision. A product that lets you enable eight capabilities in one line has made the first decision cheap and left the second one to you.

Two allowlists and three private targets

The app fetches URLs on behalf of a model. That is a server-side request forgery problem wearing a feature’s clothes, and upstream handles it the way a careful product should: a dispatcher refuses private addresses unless the operator names them. The names live in allowedAddresses, and there are two entries in this deployment.

The first is webSearch.allowedAddresses: ['searxng:8080'], the search container by service name rather than a published port. The distinction matters and I got it wrong first: the host publishes a port for the search UI, but from inside the app’s container that address is unreachable, because a published port on the host is not the same network as the compose bridge. The correct target is the service name and the container’s own port, and searxngInstanceUrl carries the rest of the path, so the URL the app builds is the service, the port and /search.

The second is speech.tts.allowedAddresses: ['host.docker.internal:20128'], the same router port the chat endpoint uses, entered a second time for a second capability. One router, two allowlist entries, because the guard does not know that two features point at the same host. The shared bearer token is the same too, which is the one place where the deployment is tidier than the guard.

The third private target has no entry at all. The image tool posts to a diffusion server on the LAN at an address that comes from operator environment, not from a user field or a model output. Upstream only wraps user-supplied URLs in the SSRF-safe dispatcher, so an operator-configured URL goes out through the ordinary fetch path, unguarded and unlisted. Three private destinations, two guard behaviours, one of them implicit. If you audit this deployment by reading allowedAddresses, you will find two of the three and conclude you are done.

Search configuration is otherwise thin: searchProvider: searxng, scraperProvider: keenable, rerankerType: none, and a search container running with default engines enabled and the rate limiter off. That last pair is what makes the search logs read the way they do. The metasearch layer returns links without much trouble; the scrape layer then fetches them and the sites refuse.

sequenceDiagram
    autonumber
    actor U as Me
    participant A as LibreChat
    participant G as Outbound guard
    participant S as SearXNG
    participant W as The open web
    U->>A: a question that needs the web
    A->>G: GET searxng:8080/search, format json
    alt the host is on the allowlist
        G->>S: forward
    else private host, not listed
        G-->>A: refused, private address
    end
    S->>W: query the default engine set
    W-->>S: links
    S-->>A: eight results
    A->>W: scrape the top results directly
    W-->>A: three pages, five refusals
    A-->>U: an answer built from what came back

The numbers in the log are the honest description of that second hop. One turn recorded ok=3 chars=18.5k failed=5 reasons={http_403:5}: eight links found, five sites refusing the scraper outright, three pages delivered, eighteen and a half thousand characters for the model to read. Other turns skew the other way, with http_429 refusals dominating and one turn returning a single page. That is not a bug in the deployment. It is the open web declining to be a database, and a search feature that reports success while half the corpus says no. The failed=5 counter is the most useful thing in those lines, because it is the only place the refusal is recorded at all.

The limiter being off is the decision I would revisit first. It exists so a self-hosted instance on a private network does not throttle its only user. It also means the container will happily hammer an engine until that engine starts returning 429 to everything, which is what the http_429 lines look like. For one user on a LAN the trade is defensible. For anything with a second user it is a way to get the estate’s IP range suspended by a search engine, and the fix is one line in the search container’s settings.

The patch

One file in the app is not the file upstream shipped. patches/StableDiffusion.js, 8,158 bytes, is bind-mounted read-only over /app/api/app/clients/tools/structured/StableDiffusion.js. Beside it sits StableDiffusion.js.orig, 7,871 bytes, byte-identical to the upstream file at commit ba44443f, kept for one purpose: it is the only diff anchor that survives an image pull. Without it, an upgrade leaves you comparing a patched file with a file you no longer have.

The delta is four numbers and one clause. cfg_scale: 4.5 becomes Number(process.env.SD_WEBUI_CFG_SCALE ?? 1.0), steps: 22 becomes Number(process.env.SD_WEBUI_STEPS ?? 8), width and height go through the environment with 1024 as the default, and one conditional adds payload.override_settings = { sd_model_checkpoint: process.env.SD_WEBUI_MODEL } when that variable is set.

Upstream’s defaults are for an SD1.5-class checkpoint: 22 steps and a guidance scale of 4.5. This deployment runs 8 steps at 1.0 because the checkpoint is Z-Image-Turbo, a distilled model that was trained to be sampled that way. Running it at 22 steps does not produce a better image, it produces an overcooked one more slowly. The first time I used the unpatched tool I assumed the machine was slow. It was doing exactly what it had been told, for longer, to worse effect.

The clause is the part that does more than tuning. Passing SD_WEBUI_MODEL as sd_model_checkpoint makes the chat app choose which checkpoint the other machine loads, and the value is a path on that machine’s filesystem. So a chat tool in a container is expressing a model choice as a directory layout on a different host, and the two hosts now share an invisible contract: if the path moves, image generation silently reverts to whatever checkpoint was loaded before. That is model routing, implemented as an environment variable pointing at a folder.

The reason this works without a rebuild is worth stating because it is not obvious and it is not guaranteed. The image runs the source directly — npm run backend, which is node api/server/index.js, with no compiled dist/ anywhere — so require() reads the bind-mounted file when the tool module loads. A built image would have bundled the file at build time and the mount would have shadowed a file nothing reads. The patch works because of a packaging decision made upstream and could stop working for the same reason.

flowchart TD
    U["image at ba44443f: StableDiffusion.js"] --> M["bind mount replaces the file at start"]
    P["patches/StableDiffusion.js on the host"] --> M
    M --> R["tool module loads through require()"]
    R --> O["8 steps, guidance 1.0, checkpoint override"]
    N["upgrade: new image, same mount"] --> Q{"path and module shape unchanged?"}
    Q -- yes --> S["new code, old patch, no assertion"]
    Q -- no --> T["mount lands on nothing, tool fails at load"]
    S --> X["silently the old sampling settings"]

Nothing asserts the patch is in place. There is no build step, no healthcheck, no test, and no startup check that the mounted file is the patched one rather than the upstream one. Drop the mount from the override and the app still starts, the tool still answers, and the only difference is that images take two minutes and look worse. That is the failure mode I keep coming back to in this estate: not a crash, a regression that no one measures.

The one check available is cheap and I run it by hand after any image change: hash the file inside the running container and hash the file on the host. If they match, the mount is doing what the compose file says. It catches the mount disappearing. It does not catch upstream renaming a function inside the file, which is the failure that matters more, because the file still mounts and still loads and then calls something that no longer exists.

The upgrade story is therefore three sentences long. Pull the image, check whether that path still exists and still exports the same shape, and re-cut the patch against the new upstream file if it does. If the path is gone, Docker will create a directory where the mount points, the app will try to require a directory, and the tool will fail at load with an error that names a module rather than a mount. That is a better failure than the silent one, and it is not the one I expect.

The seams

Every edge out of the app is a URL and a credential, and the useful discipline is that each tier gets one and only one of each. Not because the code requires it, but because an edge you cannot name is an edge you cannot revoke.

The router is the first. A base URL and a bearer token in the YAML, and the same bearer reused for text to speech because both routes live in the same process behind the same key. One credential to rotate, and one place where rotating it breaks two features at once — which is the trade for having only one.

The sandbox is the second, and it is the only edge whose credential is not an API key. LIBRECHAT_CODE_BASEURL names the sandbox API, and the auth is a short-lived EdDSA JSON Web Token minted per call: issuer librechat, audience codeapi, a single allowed algorithm, a 300-second time to live with the same figure as a hard ceiling, one tenant, a token id, and an auth-context hash in the claims. The app signs with an Ed25519 private JWK that never leaves it. The sandbox holds only the public half and verifies. There is no shared secret to leak from the sandbox side, and there is nothing to rotate when a token is spent, because tokens expire in five minutes.

The search edge is one variable: SEARXNG_INSTANCE_URL, plus the allowlist entry described above. The speech edge is the YAML url and the shared bearer. The diffusion edge is SD_WEBUI_URL plus the three tuning values. The embeddings edge is four variables — provider ollama, model bge-m3, the Ollama base URL, and the RAG API’s own address — and it points at the same workstation as the diffusion server.

That last detail is the one with a blast radius. One machine on the LAN serves both image generation and embedding. When it is off, new images fail and new documents cannot be ingested, and the two failures look unrelated: one is a tool error in a conversation, the other is a spinner during an upload. The shared dependency is invisible in the app, because the app only knows two URLs. It is visible in the list of environment variables, which is where I found it.

flowchart LR
    B["Browser"] --> A["LibreChat app"]
    A -->|"bearer, shared"| R["9router"]
    A -->|"EdDSA JWT, 300 s"| C["Sandbox API"]
    A -->|"service name"| S["SearXNG"]
    A -->|"operator URL, no allowlist"| D["SD.Next on the workstation"]
    A -->|"ollama base URL"| E["Embeddings on the workstation"]
    A --> M["MongoDB, no auth"]
    A --> V["RAG API to pgvector"]
    C --> G["Egress gateway"]

Read the diagram for the asymmetry it carries. Two edges go to the same router through different features. Two edges go to the same workstation through different capabilities. One edge is unguarded because it is operator-configuration rather than user input. One edge is unauthenticated because Mongo was set up for a private bridge. Four different credential patterns for six edges, and only one of them was designed as a security decision rather than inherited as a default.

The sandbox as one tier

execute_code is the capability with the longest shadow, so it is worth describing what it actually reaches. The sandbox is not a subprocess and not a container the app can spawn. It is a separate compose project with its own API, a service worker, a sandbox runner, a file server, a tool-call server and an egress gateway, and the only path from the app is one HTTP endpoint named by LIBRECHAT_CODE_BASEURL.

Each call carries the EdDSA token described above. Five minutes is a short life for a credential and the right length for this one: it covers a cold start and an execution, and it expires long before a leaked token is worth harvesting. The token names a tenant and an auth-context hash, so the sandbox can refuse a token minted for a different user context even while the signature verifies. The signing key lives in the app as a private JWK, and the sandbox has the public half. That is the correct direction for trust: the verifier cannot mint.

Inside, the runner has no route out except the egress gateway, and the gateway is what makes the tier meaningful. Code execution without egress control is a shell with a network. With it, the interesting question stops being “can it reach the internet” and becomes “what did it ask to reach”, which is a question a log can answer.

What the whole tier has actually done is sixty-one executions. Sixty of them happened on 20 and 21 September while I was wiring it, most within a few minutes of each other, which is what a person testing a tool looks like. The sixty-first ran on 13 October at 20:50 and took 99 milliseconds: one request received, one execution completed, nothing else. Since then the sandbox has answered health checks and 404s and has not run any code.

Sixty-one is the number that decides how much of this tier is worth defending. Two compose projects, an egress gateway and a token scheme are a fair price for a capability that runs code once every five weeks only if the alternative is worse — and it is. The alternative is enabling the capability with no isolation at all, which is what the capability flag does if you have not built the tier. The tier is the difference between a feature and an incident.

Dormant features and the nine vectors

The gap between what is configured and what is used is the most honest measurement in the deployment, and it is wide. Everything below is wired, healthy and unvisited.

Speech to text has no speech.stt block and no key at all. Stateful code sessions are configured in a comment: the stateful base URL is commented out, so every execution starts cold, and so is the prewarm and the 35-minute cold threshold that would keep a session warm between turns. Meilisearch runs with SEARCH=false and an empty master key, holds 140 kB of data, and its sync log for the only day it ran is zero bytes. Balances are off with zero documents. OAuth, SAML and LDAP are blank, and the admin panel logs OpenID check failed: 404 at startup for an endpoint nobody configured — a health probe for a feature that was never turned on, failing in a way that looks like a fault and is not.

Then there is retrieval, which is the best example because it works. The RAG API runs the -lite variant, embeds through Ollama on the workstation with bge-m3, and writes 1024-dimension vectors into pgvector. The pipeline is healthy: on 23 September it took a 44 kB PDF, split it into nine chunks, embedded them at a batch size of 500 in one consumer, and inserted nine rows in 2.05 seconds with a peak resident set of 182 MB. The collection is named testcollection, belongs to user public, and holds nine rows in 240 kB. There has been one ingestion and one evening of queries, on 23 September, and nothing since. Thirty-four days idle.

The DDL logs are the part I would fix first. Three migration flags sit at their defaults and the API says so on every start: Skipping legacy vector indexes, Skipping cmetadata JSONB migration, Skipping cmetadata GIN index. So the vector table has no approximate-nearest-neighbour index and no index on its metadata column. At nine rows that is irrelevant; PostgreSQL scans nine rows faster than it reads an index. At nine hundred thousand rows a query that should take milliseconds takes seconds, and the fix is a migration flag that will have been off since the first day. The default is right for a test and wrong for a corpus, and the log line announcing it is the only warning you get.

What that pattern says about feature surface is the lesson. Every one of these features was enabled because enabling was cheap: a flag, a key, a block of YAML. None of them was proven by use. The deployment therefore carries the configuration cost of retrieval, transcription, full-text search, session reuse and single sign-on while carrying the operational cost of none of them, and the only reason I know which is which is that I counted rows. A feature surface that is configured but unused is not a capability. It is a liability with a schema.

Closing

A checklist for wiring a chat front-end to a stack, written from the seven containers above and the parts of them that have already gone wrong.

1. Pin what you patch. A floating tag over a release candidate is a fine way to try a product and a poor way to carry a bind mount across an upgrade.
2. Assert your mounts. Hash the file inside the container against the file on the host after every image change; a mount that silently disappears looks exactly like a patch that never worked.
3. Turn discovery off only when you accept its cost. fetch: false buys a two-item picker out of a fifty-six-item catalogue, and the price is a hand-maintained list that drifts without a warning.
4. Keep one key per tier, and name the tier. A shared bearer for chat and speech is one rotation and two breakages; an EdDSA token per sandbox call is nothing to rotate at all.
5. Allowlist explicitly. Two entries are listed and a third private target is implicit because its URL comes from operator configuration, which is precisely the one you will forget when you audit.
6. Count what you use. Nine vectors and sixty-one executions are the real feature list, and they are cheaper to read than a changelog.
7. Put the healthcheck on the thing that does the work, not on the console. Six containers with no healthcheck will report healthy minutes after the database has gone away.
8. Copy the state you cannot rebuild. The Mongo bind mount holds every conversation in this deployment and has no dump, no snapshot and no copy anywhere.

None of this is exotic and none of it is specific to this product. It is the same argument as the rest of the AI-augmented engineering work, and the same one as the router post: a written contract, a small surface, an audit line. The front door is the part people see. The building is where the decisions are, and a deployment you can describe from memory — seven containers, two allowlists, one patch, nine vectors — is one you can fix at midnight without guessing.