Three people ask the same assistant the same question — "give me a quick dinner idea" — and get three genuinely different answers: one short and visual with no peanuts in it, one with a cost-per-serving line, one gentle and one-handed for a parent holding a toddler. None has their own model or server. What each has is a context pack: a small, budgeted bundle assembled on every turn by a worker that forgets them the moment the reply is sent. Three fictional people — Maya, Raj and Lena — show how.
Why isn't personalization just a longer prompt?
The intuitive design puts everything known about a person into the prompt. It fails three ways at once. The context window is finite, and a year of conversation does not fit. Cost is tokens, so every turn would carry the price of the whole history. And quality degrades: a model handed a transcript attends to whatever is loudest, not whatever is relevant now.
The alternative treats personalization as a data-architecture problem. A person is represented by three things, each in a store with the right access pattern, and a stateless worker composes a small pack from them on every turn:
TEXT
context_pack = profile(keyed) ⊕ tone_template(segment) ⊕ topK(memory, own namespace)
Exact facts by keyed lookup. A voice chosen by segment. A few episodic memories retrieved by vector search inside the person's memory namespace, re-ranked by temporal decay. Keeping them apart is the whole trick.
flowchart LR
U[(User id)] --> K[Keyed lookup profile card]
U --> S[Segment] --> T[Tone template]
U --> N[(Memory namespace)] --> V[Vector search top-K]
Q[Rewritten query] --> V
V --> D[Temporal decay re-rank]
Q --> GV[Vector search]
G[(Shared corpus)] --> GV
K --> B{Allocation budget}
T --> B
D --> B
GV --> B
B --> P[Context pack]
Why are identity facts never left to similarity?
Maya has a severe peanut allergy. When she asks for a lunch spot, "lunch spot near me" embeds nowhere near "peanut allergy"; a system relying on similarity would miss the allergy exactly when it matters. Similarity ranks by topic; it has no notion of importance.
So the pack has two retrieval paths. Facts that must be right every time live on a profile card fetched by keyed lookup: one indexed read by user id, injected every turn under a heading the model treats as ground truth. Everything episodic is retrieved by vector search, scoped to the person's namespace first and ranked second, then re-scored with temporal decay so last week's preference outranks a contradictory one from three years ago.
Keyed lookup
Vector search
Question it answers
"what is true about this person?"
"what has this person said that relates to this?"
Guarantee
exact, deterministic
approximate, ranked
Keyed by
user id
meaning (an embedding), inside a user filter
Cost per turn
one indexed read, ~1 ms
one embedding + one scoped nearest-neighbour scan, tens of ms
Present every turn?
yes — always relevant
only when similar to the query
Ages?
no — edited, never decayed
yes — temporal decay
Role in the pack
authoritative block
cited context, budgeted
Wrong when
used for fuzzy recall
used for facts that must be exact
flowchart TD
F[A fact about the person] --> A{Must it be exactly right every single turn?}
A -- "yes: allergy, city, name" --> K[Profile card keyed lookup]
A -- "no: episodic, evolving" --> M[Episodic memory vector search + temporal decay]
K --> P[Authoritative block in the pack]
M --> C[Cited context under budget]
W[Write-back after a turn] --> M
E[Edited by the person] --> K
The card changes when the person edits it; episodic memory grows through write-back after every turn, which is why it needs decay and de-duplication — see Memory.
⚠️ Pitfall — one index for everything. Putting the profile card into the vector index beside episodic memory feels tidy and is the most common personalization bug. Identity gets buried under fuzzier, newer, more numerous memories; the assistant knows the person's favourite typeface and forgets their allergy. Keep exact facts keyed.
How does a segment choose the voice?
Facts are half of personalization; the other half is feeling, and the right answer in the wrong register reads as tone-deaf. Rather than a bespoke prompt per person, the pack carries a tone template chosen by the person's segment — one of a few named registers, each with a system-prompt layer and an explicit empathy target.
A relational template opens with warmth, acknowledges feelings before facts, keeps replies short, and reassures under stress. A productivity template leads with the answer, backs it with numbers, states assumptions, and respects expertise without being curt. Maya and Lena are relational; Raj is productivity. Each persona adds a one-line empathy target — Maya "crisp and warm", Lena "reassuring and brief", Raj "precise, impatient with fluff".
Templates number in the dozens while people number in the millions. A template is a shared, stable prefix in front of every prompt in its segment — the shape prefix caching rewards — and it is testable: a golden set can score whether Lena's answers stayed brief and gentle, not only whether they were correct (Quality).
How is the pack budgeted?
Retrieval returns more than the pack should carry, and similarity scores from the person's memory and from the shared corpus are not on the same scale. Pouring both lists together and sorting lets a high-scoring general document silently evict a personal memory that mattered more. The pack instead uses an allocation budget: a fixed number of slots per source, filled in priority order — three personal and two general is a sound default — with the keyed card always present and a similarity floor below which a slot stays empty.
Each survivor gets a citation marker — [1], [2] for personal memory, [g1] for general — so the answer can point at its sources and the output guardrail can confirm every marker used was actually retrieved. It is a product dial too: more personal slots feels intimate, more general feels encyclopaedic.
What does one pack look like?
Maya asks about trails. The worker assembles roughly this — a structural sketch, not a schema:
score is similarity × e^(−λ·age_days): the twelve-day-old memory is gently discounted, the one learned last turn is not. The profile block carries no score; it is not competing. And the whole pack is about six hundred tokens — small, fixed, predictable, which is what makes being personal affordable a billion times a day.
How do three people get three assistants from one worker?
flowchart LR
Q["Give me a quick dinner idea"] --> W[Stateless worker]
W --> PM["Maya's pack SF · peanut allergy · visual · relational"] --> AM["Short, visual, peanut-free"]
W --> PR["Raj's pack NY · data-first · numbers · productivity"] --> AR["Direct, cost per serving, a source"]
W --> PL["Lena's pack Austin · toddler Theo · one-handed · relational"] --> AL["Gentle, brief, one-handed"]
Maya (product designer, San Francisco): severe peanut allergy, prefers examples over walls of text, a relational template. A five-line recipe with no peanut ingredients.
Raj (finance analyst, New York): data-first, wants sources and numbers, a productivity template, memories about stated assumptions. The dish first, then cost per serving and prep time, and where the estimate comes from.
Lena (new parent on leave, Austin, with an eighteen-month-old named Theo): time-poor, wants short reassuring answers, a relational template, memories about one-handed meals. Two sentences, something made with a toddler on one hip, no lecture.
Same worker, same model, same turn; the difference is entirely in the pack — three stores with the right access patterns and one assembly step with a budget.
🪤 Misconception — "to personalize, fine-tune a model per user." Fine-tuning changes weights: slow, expensive, impossible for a hundred million people, and it bakes yesterday's facts into a model that cannot be told the person moved house. The pack updates the moment the store does and leaves the model shared.
🎯 At consumer scale, the pack makes personalization a constant cost per turn. Every ingredient is scoped: one keyed read, one nearest-neighbour scan over one person's few thousand memories, one of a few dozen templates. A lab serving 100M weekly users scales personalization by sharding stores by user and adding stateless workers, never by making a query bigger. The template layer is also where prefix caching pays: the expensive prefix is identical for everyone in a segment.
📖 Story: how the pack got its three doors
Personal assistants went through three ideas. The first was one prompt with everything the person had ever said — slow, expensive, and less personal as conversations grew. The second was a model tuned per person, which cannot work for ten million people. The third was less glamorous: notice that "what is true about you", "how you like to be spoken to" and "what you have told me" are three different kinds of data, and store each the way it wants to be stored — a keyed row, a handful of shared templates, a scoped vector index. The pack is not a clever prompt; it is the admission that personalization is a data-architecture problem.
🔍 See it happen — Raj's pack, assembled
Raj (fictional finance analyst, New York) asks: "Should I shift more of my portfolio into bonds right now?"
Keyed lookup — New York; data-first, wants sources and numbers; a broad-market index fund, one large-cap stock, short-term treasury bills. The authoritative block, no score.
Segment → tone template — productivity: lead with the answer, state assumptions, back it with numbers.
Vector search, Raj's namespace only — "evaluating whether to rebalance toward bonds as rates shift" (0.84, 20 days → 0.69); "dislikes hand-wavy estimates" (0.58, 60 days → 0.32); "prefers explicit numbers, assumptions stated" (0.63, 90 days → 0.26). Three personal slots used.
Shared corpus — one note on bond duration and rate sensitivity (0.71); the second candidate fell below the floor.
Budget — personal 3/3, general 1/2, about 680 tokens.
The answer — a framing rather than a verdict, its assumption named, a numeric illustration across his holdings, citations [1] and [g1], and a not-financial-advice line from the segment's policy layer.
Swap in Lena's relational template and the same facts arrive wrapped in reassurance. That is the lever.
🧭 Enterprise mapping. The pack keeps its shape and tightens every field. A person's keyed profile becomes a tenant's — and an employee's — entitlement record, sourced from identity systems rather than self-reported. The segment template becomes a policy: a regulated-advice register, a support register, each versioned and signed off. The memory namespace becomes the tenant data boundary, region-pinned, with retention and deletion rules. The allocation budget and the citation registry become audit evidence: for any answer, the pack shows exactly which facts and documents were in front of the model. See Consumer → Enterprise.
Where next?
How the episodic half is written, decayed, de-duplicated and sharded is Memory. The turn that spends the pack is One Turn. Why the template layer is a serving-cost lever is Model Serving; how tone becomes a test is Quality; why the namespace filter is a privacy guarantee is Trust. Every term here has a card in the Glossary.