Glossary — every term, one card each

Every term on this site is a card: click any dashed-underlined word in a chapter and its definition, origin, a sketch and a short scenario open in place, without leaving the page. This page lists all of those cards in one alphabetical run so you can skim the vocabulary end to end, or land here from a search. Each heading below is the card's title and each paragraph is its one-line definition; the full card, with example and scenario, opens from the term wherever it appears in the chapters.

Agentic Loop — one turn, nine stages

The server-side life of a turn: guardrail in → session → query rewrite → profile retrieval → general retrieval → context pack → agent + tools → generate (+ output guard) → write-back. In the middle, the model itself chooses the path.

Autoscaling

Changing capacity automatically with load — out under pressure, in when quiet, sometimes all the way to zero. It is what makes serving ten users and ten million users the same architecture with a different number.

Canary — a small slice first

Sending a small percentage of live traffic to a new version, watching SLOs and eval scores, then ramping if they hold and rolling back if they do not. Bad changes reach few people and reverse automatically.

Context Pack

The bundle a worker assembles fresh for one turn: exact profile facts (keyed lookup) ⊕ a tone template chosen by segment ⊕ the top-K episodic memories retrieved inside the user's own namespace. It is the whole of what makes the answer personal.

Context Window

The maximum number of tokens a model can take in at once — instructions, the context pack, history and the message combined. It is a hard budget every turn must fit inside, which is why the pack is budgeted.

Control Plane

The part of the system that decides how much capacity should exist and drives the fleet toward it, as opposed to the data plane that actually answers turns. It observes, compares desired with actual, and acts — continuously.

Error Budget

The amount of failure an SLO permits over its window — 99.9% availability leaves 0.1% of turns allowed to fail. Spend it on risky rollouts; when it runs low, slow down.

Eval — measuring quality before users do

A repeatable test: run a fixed set of prompts through the system, score the answers (for correctness and for tone), and compare against the last known-good run. A regression fails the deploy.

Fallback — degrade gracefully

A pre-planned lesser path when the preferred one fails: a second provider, a smaller model, or an answer composed from the context pack alone. The person gets something useful instead of an error.

Golden Set

A curated collection of prompts with expected answers or rubrics, kept per persona or segment, that the eval runs on every change. It is the system's memory of what 'good' looked like.

Grounding — answers tied to evidence

Making the model answer from material actually in the context pack — retrieved memories, documents, tool results — and cite it, rather than from its training-time impressions. The defence against confident-wrong.

Guardrail — checks the model cannot talk past

Deterministic code that inspects what goes into the model (length, injection patterns, sensitive data) or what comes out of it, and blocks or degrades regardless of how persuasive the text is. Prompts steer; guardrails enforce.

HPA — Horizontal Pod Autoscaler

The Kubernetes controller that adds or removes replicas of a workload to hold a metric (CPU, requests per second, a custom signal) at its target. It is the reconcile loop, productized.

Keyed Lookup — exact facts by key

Reading a structured fact (name, plan, language, timezone) by its exact key rather than by similarity. It is the retrieval path with a guarantee: the same key always returns the same fact.

MCP — Model Context Protocol

An open standard by which a tool server describes what it offers and an AI application calls it — discover, then invoke — so any assistant can use any tool without bespoke glue. N applications × M tools becomes N + M integrations.

Memory Namespace — one person's data boundary

A partition key stamped on every memory row and enforced on every read, so one person's recall can never surface in another person's turn. One shared store, logically partitioned per user.

Model Routing

Choosing, per turn, which model (and which provider) should answer — by intent, required capability, cost and latency — behind one abstraction, with a fallback when the first choice is unavailable.

Observability

Being able to answer 'what is the system doing, and why' from what it emits — metrics, logs and traces correlated by turn id — including questions nobody thought to ask in advance.

Output Guardrail

The mirror of the input check: before a reply leaves, code scans it for leaked instructions or secrets, verifies that every citation points at something actually in the context pack, and redacts or degrades if not.

Persona — a fictional user for illustration

An invented person with an invented profile, segment, tone preference and memory namespace, used to show personalization and to seed golden sets. Never a real user's data.

Prefix Caching

Reusing the model's computation for a prompt prefix that repeats across turns — the system instructions, the tone template, a stable profile block — so only the new suffix is processed at full price and full latency.

Query Rewrite

A cheap, bounded model call that turns a context-dependent message ('what about there this weekend?') into a standalone query using recent conversation, so retrieval searches for what the person meant.

Queue Depth

How much asynchronous work is waiting — memory write-backs, eval runs, summaries — measured as items in the queue. A rising depth is an early warning that the background fleet is under-sized.

Reconcile Loop — desired state vs actual state

The continuous cycle at the heart of a control plane: observe the world, compute the desired state, compare with the actual state, act to close the gap, repeat. It scales out under load and scales in when idle.

Session — short-term conversation state

The last few turns and hot preferences for one person, kept in a fast shared store keyed by user and loaded at the start of every turn. It is what gives a stateless worker a sense of 'what we were just talking about'.

SLO — Service Level Objective

A measurable target for how the service should behave — for example p95 latency under 800 ms and 99.9% of turns succeeding — chosen deliberately and tracked continuously. Not a promise to customers (that is an SLA); an engineering contract with yourself.

Stateless Worker

A process that holds nothing about any user between turns: it pulls session, profile and memory from shared stores, answers, writes back, and forgets. Any worker can serve any person, which is what lets a fleet be scaled by a control loop.

Temporal Decay — fresh memories outrank stale ones

Re-ranking retrieved memories by similarity multiplied by a freshness factor, so a preference learned last week outranks a contradictory one from two years ago. Old memories fade in rank; they are not deleted.

Tiering — small models for easy turns

Serving the simple majority of turns on a smaller, cheaper model and reserving the frontier model for the hard minority. At scale it is the single largest lever on the bill, because the bill is tokens.

Token — the unit of cost

The piece of text a model reads or writes — roughly three-quarters of a word in English. Models are priced and rate-limited per token, so at consumer scale the bill is tokens, not servers.

Tone Template — personality by segment

A small instruction layer, chosen by user segment, that sets how the assistant sounds: warmth, brevity, formality, how often it asks a follow-up. Personalization is facts and feeling.

Tool Use — letting the model act

Inside a turn the model may choose a tool from a described catalogue, the system runs it, the result is fed back, and the model decides again. The model decides; the surrounding code executes, under caps.

Users Per Instance — the scaling ratio

A capacity policy: how many concurrently active people one worker is expected to serve. Desired replicas = ceil(active users ÷ this ratio). Dial it down and the fleet spreads out; dial it up and it consolidates.

Vector Search

Finding the stored items whose embeddings are nearest to a query embedding — approximate nearest-neighbour search over millions or billions of vectors, using an index (graph- or cluster-based) instead of brute force.

Write-Back — memory that compounds

After a turn is answered, durable facts and preferences are extracted from it, embedded, de-duplicated and written into the user's memory namespace off the request path, so the next turn can recall them.