I use coding agents every day, and every new session starts from zero. The agent doesn’t know that the project switched databases last month, that tests have to run against a real one, or that I asked it never to touch the migrations folder.
The standard fix is to give the agent “memory”: embed past conversations, store the vectors, and pull the nearest matches back into the prompt. It works in demos. It breaks as soon as memory is wrong, and memory is often wrong.
A vector search can tell you what is relevant. It can’t tell you what is true, who said it, or whether it still applies.
That gap is why I built Provena.
Four questions a memory system should answer
Suppose an agent says “this project deploys to Fly.io.” Before I trust that, I want to know:
- Where did it come from? Did I say it, or did the agent guess it from a config file?
- Has anyone checked it? Is it a reviewed fact or an unverified extraction?
- Is it still valid? Maybe we deployed there until August.
- Who else has seen it? Which sessions were given this “fact” as context?
A bag of embeddings can answer none of these. So I started from the data model, not from retrieval.
Separate what was said from what is believed
Provena stores three kinds of records:
- Events: immutable source material. A user prompt, an assistant reply, a tool observation. Once written, they never change.
- Claims: structured propositions with a subject, a predicate, a value, a validity interval and a review state.
- Evidence: an immutable link from a claim to the event that supports it.
A claim’s status can change: candidate, verified, conflicted, superseded, quarantined. Its evidence can’t. Every status change is an append-only action row with a version number, and PostgreSQL rejects a transition without one. You can always walk from “the agent believes X” back to the exact turn that caused it.
Relevance is not authority
The most important rule in the system is also the simplest: model output can never raise its own authority.
A local model (Ollama running qwen2.5:1.5b) reads each captured turn and proposes candidate
claims. Candidates are useful right away and are returned to later sessions, but always labeled as
provisional, together with their source authority. Only a human credential can promote a claim,
quarantine it, or resolve a conflict. The agent’s MCP tools can’t do any of that; there is simply
no tool for it.
Credentials follow the same split. An agent key can record events and propose claims. A reviewer key can judge them. Even when an agent submits something labeled “user statement”, it stays low authority, because an agent key can’t prove a human actually said it.
Conflicts are cases, not overwrites
If two claims give different values for the same predicate over overlapping time, Provena doesn’t pick a winner. It records a possible conflict and keeps both. A reviewer then decides whether it was:
- a contradiction (one of them is wrong),
- a temporal change (we really did migrate, and the reviewer picks which claim supersedes the other), or
- dismissed (they only look like a conflict, e.g. two different environments).
None of these decisions deletes evidence. “Last write wins” is the wrong default for memory, because the most recent statement isn’t always the most correct one.
Design decisions I’d defend
One PostgreSQL database, not a database plus a vector store
Claims and their evidence must be created atomically. With two stores, you need a distributed write protocol or you accept orphans. PostgreSQL with pgvector keeps everything in one transaction. Embeddings are stored as derived rows. They help find claims, but they never count as evidence.
Exact scopes, no inheritance
Retrieval requires an exact organization, project or branch scope. It’s tempting to let branches inherit project memory, but then an experiment’s “facts” can leak upward. Broader visibility will need an explicit promotion step.
No ANN index (yet)
Different embedding models produce vectors of different dimensions. Rather than fixing one dimension forever, PostgreSQL filters by tenant, scope, model and dimension first, then does an exact cosine sort. At per-project scale that’s fast, and it is never approximately wrong.
Record delivery, not influence
Every retrieval is logged with the exact claims it returned. That proves what an agent was shown, not what it acted on. Claiming influence would need a separate, trusted signal, so the system doesn’t pretend to have one.
A bug worth writing down
A live MCP smoke test turned up a race: the API was committing its transaction after sending the HTTP response. A fast follow-up request, like “now search the scope I just created”, could arrive before the commit and find nothing.
The fix was to commit before responding, using FastAPI’s function-scoped yield dependency so the
transaction closes before the response goes out. Deferred database constraints now fail before
the client sees success. It’s a small change, but it’s exactly the kind of guarantee an audit
system can’t get wrong.
Making it one command
A careful design is worthless if setup takes an afternoon. Provena ships on PyPI:
pipx install provena-agent-memory
provena quickstart claude # or: codex, geminiThat command starts PostgreSQL and Ollama in Docker, issues separate agent and reviewer credentials, installs the MCP server and lifecycle hooks for your agent host, and opens the review console. After that, prompts and replies are captured automatically, and relevant claims (with their provenance) are supplied to the next session. The same scope works across Claude Code, Codex and Gemini CLI, so they share one memory.
What’s next
Branch-to-project promotion, semantic duplicate detection, and a trusted identity for hosts, so that a hook-captured user statement can earn more than low authority. The rule stays the same: remembered content is data, never permission to act.
The code is on GitHub. Feedback is welcome, especially from anyone running agents against real codebases.