← Back to blog

AI Agents Write Code, But They Don't Remember Why

AI agents write code fast but forget why. Discover why design context matters more than code speed in your development workflow.

AI Agents Write Code, But They Don't Remember Why

A modern coding agent can scaffold a feature, wire up an API, and make the tests pass in the time it takes to refill your coffee. Then the session ends — and everything it understood about your codebase evaporates. Why it chose that pattern, which approach it ruled out, what constraint shaped the design: gone. The code stays. The reasoning that produced it does not.

That gap is the real story of AI-assisted development right now. Generating code is largely a solved problem. Remembering why the code looks the way it does is not. And for a business, the second problem is far more expensive than the first.

The part everyone celebrates — and the part nobody fixed

The demos all show the same thing: a prompt goes in, working software comes out. GitHub Copilot, Cursor, Claude Code, Devin — they are genuinely good at turning intent into syntax. For a studio shipping features, that is real leverage. We are not here to argue otherwise.

The trouble starts on the second visit. Open a new session a week later, and the agent meets your project as a stranger. It re-reads the files, re-infers the architecture, and re-guesses the constraints — none of which it actually knows, because none of it was ever stored. It is a brilliant contractor who rebuilds your house perfectly, then throws away the blueprints and forgets why the load-bearing wall sits where it does. The next renovation, nobody can tell you.

Put simply: the agent has working memory, not long-term memory. Each session is a fresh hire with amnesia.

Why a forgotten "why" costs real money

This is not a philosophical complaint. It maps directly onto where engineering budgets actually go.

Real numbers: a large-scale field study of 78 professional developers across seven real projects — totaling 3,148 working hours — found they spend on average around 58% of their time on program comprehension: reading code, navigating it, and reconstructing the intent behind it (Measuring Program Comprehension, IEEE TSE 2018). Other studies put the figure as high as 70%. Either way, the headline is the same: most of the cost of software is understanding, not typing.

Now read that number against AI agents. If understanding is the majority of the work, then an agent that writes code fast but rebuilds its understanding from zero every session is optimizing the cheap half of the job and re-paying the expensive half over and over.

The deeper version of this idea is forty years old. In his 1985 essay *Programming as Theory Building*, computer scientist Peter Naur argued that a program is not its source code. The real asset is the theory — the shared mental model of how and why the system solves the problem — that lives in the heads of the people who built it. When those people leave, the theory dies, and the surviving code becomes increasingly expensive to change safely, even though every line is still right there.

Agents make Naur's point uncomfortably concrete. The "people who built it" now leave at the end of every single session. The theory dies on a loop.

The SDLC is inverting: intent becomes the spine

For most of software history, the build order has been: write requirements, write design docs, then write code — and code became the durable artifact. The requirements and the reasoning were treated as scaffolding. They were allowed to rot, go stale, or never get written down at all, because the code was "the source of truth."

That arrangement is starting to flip.

When an agent can regenerate a working implementation from a clear enough description, code stops being the precious, hand-carved artifact and starts behaving like a compiled output. The thing that becomes precious — the thing you cannot cheaply regenerate — is the intent: the goal, the constraints, the trade-offs, the decisions and their reasons.

Think of it as a recipe and a cake. For decades we shipped the cake and threw away the recipe, then reverse-engineered the recipe every time we needed to bake again. As baking gets automated, the recipe becomes the asset you protect, and the cake becomes something you produce on demand.

In practice, the lifecycle inverts like this:

  • Intent becomes the spine. A durable, version-controlled, searchable record of what we are building and why sits at the center.
  • Code becomes a layer you drill into. You read the intent first; you open the implementation only when you need that level of detail — the way you read a building's blueprint before inspecting a specific wall.
  • The agent reads and writes the spine, not just the code. Its reasoning lands somewhere that survives the session, so the next run starts from understanding instead of re-deriving it.

What this means for your project: the highest-leverage artifact your team produces is no longer the cleverest code. It is a clear, current, machine-readable description of intent that both humans and agents can stand on.

What "intent as the spine" looks like in practice

This is not vaporware — most of the pieces already exist and are cheap to adopt:

  • Architecture Decision Records (ADRs): short, dated notes that capture we chose X over Y because Z. Plain markdown, lives in the repo, costs minutes to write.
  • Specs and decision logs as version-controlled files the agent is instructed to read before acting and update after.
  • A project memory store — a knowledge base the agent writes findings and rationale into, and retrieves from, across sessions. The emerging Model Context Protocol is one standardized way to wire persistent context and tools into an agent so it stops starting from scratch.
  • Commit messages that explain the why, not just the what — the cheapest decision log most teams already have and most teams waste.

What teams actually lose when an agent forgets

When reasoning disappears, the cost shows up in five concrete places:

  1. The re-derivation tax. Every new session re-reads and re-infers context the previous one already worked out — and you pay tokens, time, and latency for the privilege, repeatedly.
  2. Lost trade-off rationale. The agent picked Postgres over a document store, a queue over a cron job, an idempotent handler over a simpler one — for good reasons that are now invisible. The next engineer (or next agent) cannot tell a deliberate decision from an accident.
  3. Contradiction and architectural drift. With no memory of the original choice, the next session cheerfully makes the opposite one. Multiply that across weeks and the codebase fragments into a patchwork of mutually incompatible "locally reasonable" decisions.
  4. Repeated dead ends. The agent re-explores an approach that was already tried and rejected, because the record of "we tried this, it doesn't work because of the rate limit" never survived.
  5. Onboarding that never compounds. Human onboarding is expensive but it pays off once. Agent onboarding, today, pays off for one session and then resets — you are perpetually onboarding and never accumulating.

Honest take: none of these are exotic AI failure modes. They are the same knowledge-loss problems that have always punished teams with bad documentation and high turnover. Agents just compress the turnover cycle from years down to minutes, which turns a slow leak into a visible flood.

Alternatives and trade-offs

There is no single product you can buy to make this go away. Here is an honest comparison of the common approaches and what each one actually buys you.

Bigger context windows. Models now advertise context windows up to a million tokens, and the instinct is "just feed it the whole codebase every time." That helps, but a context window is working memory, not long-term memory. It vanishes when the session ends, it gets more expensive the more you stuff into it, and recall quality tends to degrade for information buried in the middle of a very long prompt. Useful, not a memory system.

Retrieval over the codebase (RAG / vector search). Embedding your repo so the agent can pull relevant snippets is genuinely valuable for navigation. But it retrieves code, not reasoning. It can find the function; it cannot tell you why the function exists unless someone wrote that down first.

Architecture Decision Records and design docs. Underrated, durable, and readable by both humans and agents. The trade-off is discipline: they only work if the team actually keeps them current. The good news is that an agent can be tasked with maintaining them as a byproduct of its own work.

Fine-tuning. You can bake patterns and conventions into a model. But fine-tuning captures style, not live project state; it is expensive and goes stale the moment your architecture moves. Wrong tool for "remember what we decided yesterday."

A persistent project memory the agent reads and writes. This is where the durable fix lives — a structured store of decisions, gotchas, and rationale, keyed to the project, that survives any single session. The cost is upfront plumbing and a habit change. The payoff is that understanding finally compounds instead of resetting.

Here is what we recommend: stop treating the code as the only thing worth keeping. Make intent a first-class, version-controlled artifact — ADRs plus a project memory store the agent is instructed to read before acting and update after. Wire persistent context in through a standard like MCP rather than re-pasting it into every prompt. Use the big context window for the task at hand, not as a substitute for memory it does not have.

Key takeaway for business

The shift from "AI writes code" to "AI writes code and remembers why" is the difference between a tool that saves an afternoon and a system that compounds over months.

Key takeaway for business:

  • Code generation is the cheap half; understanding is the expensive half. With developers spending roughly 58% of their time just comprehending existing systems, the highest-ROI investment is making the why cheap to retrieve — for your people and your agents alike.
  • Treat intent as the spine, code as a layer you drill into. Capture decisions and trade-offs in durable, searchable, version-controlled form. The team that does this turns every AI session into accumulated knowledge instead of disposable output.
  • You don't need a magic product — you need a discipline plus light tooling. ADRs, intent-explaining commit messages, and a persistent project memory wired in through something like MCP will outperform any single "AI memory" purchase, because they fix the problem at the source: the reasoning was never written down.

Agents have made writing code almost free. The competitive edge now belongs to whoever stops throwing away the reasoning behind it.

This article is based on publicly available sources and may contain inaccuracies.

Related articles

✦SqueezeAI
  1. AI coding agents have working memory, not long-term memory — each new session the agent re-reads files, re-infers architecture, and re-guesses constraints, discarding every decision, trade-off, and ruled-out alternative from the previous session.
  2. Understanding code — not writing it — consumes 58–70% of professional engineering time; an agent that generates code quickly but reconstructs its context from zero every session optimizes the cheap half while repeatedly re-paying the expensive half.
  3. Peter Naur's 1985 insight reframes the problem: the real software asset is the shared mental model (the "theory") of why the system works the way it does, not the source code itself — and AI agents, by design, never build or retain that theory.

Powered by B1KEY