The industry calls it a memory harness. I think memory is the wrong word. Recall was never the hard part. Part 2 of a series on agent-ready projects: the salience problem.
This spring, one of my systems confidently re-investigated a problem it had already solved.
The short version: a process-management setup had once looked like it wouldn't survive a reboot. Alarming, plausible, worth checking. And it had been checked, months earlier, and disproven with a single command. But the disproof lived in a chat thread, and the chat thread was gone the moment the session ended. So the alarm regrew from the same soil, got re-raised with the same confidence, and cost the same investigation a second time. Nothing had been forgotten, exactly. The original design docs, the configs, the code were all right there, findable by any search. What was missing was the verdict.
That incident taught me the distinction this whole post is about. The system didn't have a recall problem. Every relevant document was retrievable. It had no way to know which of the retrievable things was in charge.
Part 1 of this series ended with a single instruction file groaning under its own success: 400 lines where 60 used to govern, the one warning that mattered buried at paragraph nineteen. The obvious next move is the one everybody makes, and it's the right move. The file splits. But how you think about what it splits into determines whether you've solved the problem or just distributed it across more files.
First, the practical part, because it's genuinely simple. When one file stopped working, here's what worked for me. Not a framework, just the shape the pressure produced:
| File | What it holds | Write pattern |
|---|---|---|
| State | Where the work stands right now: what's in flight, what's blocked, what's next | Rewritten constantly; the freshest thing in the repo |
| Decisions | What was chosen, when, and the one-line why. Where "we already tried that" becomes permanent | Append-only |
| Lessons | Things that bit us, with cause and fix. Including negative findings: the alarming hypotheses that turned out to be false, written down so they can't regrow | Append-only |
| Architecture | The distilled shape of the system. Not generated API docs; the load-bearing walls, in prose | Rewritten rarely, deliberately |
| Work items | One file per piece of in-flight work, self-sufficient enough that a fresh session can execute from it alone | Created, worked, archived |
To be clear about what this table is: it's the recurring shapes across my repos, not a checklist. Very few of my projects have all five. Each file earns its place when the work demands it, and not before. If nothing has bitten you yet, you don't need a lessons file; the third time a session re-litigates a settled question, the decisions file will invent itself.
The lessons file's negative findings are what the reboot incident taught me: the disproof that lived only in a chat thread gets written down somewhere it can't die with the session.
The instruction file from Part 1 stays, but it shrinks back to being an entry point: identity, rules of engagement, and pointers to the rest. Sixty lines again. Sixty lines that govern.
You'll notice I'm describing files and not naming the directory they live in. That's deliberate. I've watched this space long enough to see the names calcify into products. The moment there's an official folder name, there's an official structure, and the moment there's an official structure, people cargo-cult the structure and skip the thinking. Mine has a name that only makes sense to me. Call yours whatever you want. The files aren't the point.
The point is the question this structure exists to answer, and here's where I have a quarrel with the word everyone, including me, reaches for.
The industry has mostly settled on calling this kind of thing a memory harness: external memory for a stateless agent, structured and versioned. I use the term too. It's searchable, people know what you mean.
But the name bothers me, and the reason it bothers me is the actual thesis of this post.
Memory, in the sense of storage and recall, is the part we've already solved. Embarrassingly well. Vector databases retrieve. Search indexes recall. Generated wikis remember every function signature you've ever written. If the problem were that agents can't find text, we'd be done; an agent today can find more text about your project in four seconds than a human colleague could surface in a week.
And yet the failures keep coming. When you autopsy them, almost none are recall failures. The agent that re-implements the rejected design found the design doc. That's the problem: it found the old one, and nothing told it the doc had lost its authority. My reboot incident wasn't a failure to retrieve; it was a failure to know that a verdict existed which outranked the alarm.
We don't have a memory problem. We have a salience problem.
Salience is just relevance, taken seriously. Not "relevant" the way a search engine means it, where relevance is similarity to the query. Relevant to the action you're about to take: what actually bears on it, what should govern it, what would change it if you knew. Retrieval answers "what looks related to this question?" Salience answers "what is binding on this next move?" A vector database will hand you the superseded decision and the current one side by side, with nearly identical scores, and it is doing its job perfectly when it does. Similarity has no concept of which one wins. That gap, between text that looks related and text that should govern, is where most agent failures I've seen actually live.
The legal system is the cleanest picture of the difference I know. Law has world-class retrieval: every statute, every ruling ever made, indexed and searchable. But nobody confuses search results with law. What makes the system function is the doctrine sitting on top of retrieval, which ruling is binding, which precedent is current, which jurisdiction applies. A repo full of docs and chat logs is all case law and no doctrine. What the file-splitting exercise actually builds, underneath the storage, is doctrine.
When I look at how agent sessions actually go wrong in my projects, the failures sort into four questions the repo failed to answer:
Which of these is the truth? The README says one thing, the instruction file another, a generated doc a third. All retrievable, all confident, no marked winner. The agent picks by vibes.
Is this still true? The state file describes March. It doesn't announce that it describes March. Stale text doesn't look stale, and that's what makes it dangerous.
Does this apply here? The deploy runbook is law for production and noise for the prototype next to it. Text carries no jurisdiction unless you give it one.
What wins on conflict? When two files disagree, is there a rule, or an improvisation? My rule is boring and explicit: the decisions file beats the architecture file beats the README, and anything written down beats anything that only ever lived in a chat thread, because chat is where verdicts go to die. Yours can differ. Having one is what matters.
But there's a fifth thing, and I think it's the one that genuinely can't be solved by better search, ever:
An agent only searches for what it already suspects matters. Retrieval is pull. It requires the agent to form the question. The catastrophic failures are unknown-unknowns: the fragile deploy step it had no reason to ask about, the stakeholder it didn't know existed. No retrieval system, however good, surfaces the answer to a question that never gets asked. Governing context has to be pushed, placed in the small set of things an arriving agent reads unprompted, before it forms any intentions at all. That's why the entry-point file survives the split, and why what goes in it is the highest-stakes editorial decision in the repo. The hot path is the only channel you have to a mind that doesn't yet know what it doesn't know.
Which leads to the quiet punchline of all this: salience is a write-time property. You mostly can't compute it at read time. Not with embeddings, not with rerankers, not with a smarter agent. You curate it: by deciding where a fact lives, by promoting the verdict and demoting the superseded draft, by writing the negative finding into the lessons file so the false alarm can't regrow. Retrieval systems add recall. Only curation adds judgment.
I want to revisit Dory, because this is where my thinking has moved since that post.
The Dory frame, a brilliant coworker with no long-term memory, casts the human as a kind of caretaker: build the external memory as an accommodation, because she can't do it herself. I still think that's true. But it's incomplete in a way that matters more the more capable agents get.
Dory forgets. Agents confabulate. Forgetting is an innocent deficit. A blank, and blanks ask questions. An agent fills the blank with a confident, plausible invention and then acts on it at machine speed, with whatever permissions you've granted. And bad context doesn't just sit there being wrong. An agent amplifies it, building correct, well-tested structures on top of a false premise faster than you can notice the premise.
Which changes what the harness is for. An accommodation is something you build out of kindness. This is something you build because you need control over what counts as memory. Deciding what's canonical, what's expired, what governs: that's not a favor to the agent. That's the human staying in charge of the premises the agent acts from. The harness is a safety system wearing a filing system's clothes.
That's also the accountability answer, and it's why I'm suspicious of fully self-organizing memory. Whatever survives between sessions is going to govern future action, so someone has to own it. Every fact in my files is there because a person (fine, me) decided it deserved to be. When it's wrong, that's on me, findably and fixably. "The retrieval system surfaced it" is not an owner.
Part 1 had the new-chat test. Here's this post's version, and it's even simpler:
Pick something your project believes in two places. Ask a fresh agent which one is true, and how it knows.
If the repo contains a rule that answers it, a marked canonical source, a precedence order, a dated decision, then you have a harness. If the agent has to guess, you don't have memory or salience. You have a pile of text with good search. The agent will guess well most days. The day it guesses wrong will be the day you find out which tier your repo was really in.
So: structured files, explicit precedence, curated hot path, owned verdicts. I've described all of it in the present tense, like a machine that works.
Here's the problem, and it's the one this series has been walking toward the whole time.
Everything I just described is write-time curation, and time doesn't stop after write time. The state file that was the freshest thing in the repo describes last month, eventually. The distilled architecture summary drifts from the architecture. And here's the vicious part: distillation concentrates error exactly as efficiently as it concentrates signal. A summary the agent trusts is a liability the moment it goes wrong. Worse than the raw pile, because the raw pile at least invites verification, while the confident summary forecloses it. I built the harness so the agent would trust curated text over found text. Which means when the curated text rots, I've built a machine for injecting stale premises directly into the hot path, with authority.
Curation isn't an act. It's a metabolism. The memory doesn't just need to be written — it needs to be consolidated, compressed, challenged, and sometimes deliberately forgotten, on a rhythm, forever.
Which is why the third post is called what it's called. Agents need sleep.
Part 2 of a three-part series. Part 1: Making a Codebase Agent-Ready. Part 3: why agents need sleep.