Giving AI a Long-Term Memory: How I Built a Semantic Knowledge System Across Multiple Repos

AI coding assistants are powerful, but they have a fundamental limitation: they forget everything between sessions. Every new conversation starts from zero. Your agent doesn’t know what you built yesterday, what decisions you made last week, or why you chose Dapper over Entity Framework three sessions ago.

I decided to fix that. Enter the Semantic Knowledge System.

Over the course of a single day, I built a system that gives Claude Code persistent, searchable memory across multiple repositories — using PostgreSQL, pgvector, and the Model Context Protocol (MCP). By the end of the day, I could ask Claude Code a question in my NGINX proxy repo and it would pull context from a code review conducted in my Angular frontend repo to give me a precise, cross-repo answer.

Here’s how it works and how I built it.

Semantic Knowledge Systems – The Problem

Claude Code is excellent at understanding the code in front of it. But it has no memory of anything outside the current session.

That means:

• It doesn’t know you already fixed the issue it’s warning about.
• It can’t see prior code reviews unless you paste them in again.
• It has no idea why a design decision was made.
• It can’t connect work in one repository to work in another.

Every session, you end up re-explaining context that should already be known.

I wanted Claude Code to feel less like a brilliant intern with amnesia—and more like a long-term teammate who remembers everything we’ve worked on together.

The Architecture

The system has three components:

1. An ingestion tool that scans repositories and stores their content as vector embeddings in PostgreSQL with pgvector. It processes source files, configuration, documentation, prompt history, and git commit messages. Each piece of content is chunked, embedded via OpenAI’s text-embedding-3-small model, and stored alongside metadata like file path, repository name, and content hash.
2. An MCP server that exposes semantic search over the database as tools that Claude Code can call natively. No REST APIs, no curl commands — Claude Code discovers the tools on startup and calls them automatically when it thinks the context would help.
3. A session workflow that ensures the database stays current. At the end of every Claude Code session, the agent commits changes with a detailed message, documents the session in a prompt catalog, and runs the ingest tool to sync everything to the database.

The whole thing cost about one cent in embedding API calls to ingest three full repositories.

The Ingestion Pipeline

The ingest tool is a C# console application that walks a repository and processes every meaningful file. It’s selective about what it ingests — source code, configuration, documentation, SQL schemas, and prompt history all go in. Binary files, build artifacts, node_modules, and anything that looks like a secret gets filtered out.

Each file is chunked into segments that fit within the embedding model’s context window, then sent to OpenAI for embedding. The resulting vectors are stored in PostgreSQL using pgvector’s vector type with an HNSW index for fast cosine similarity search.

The key design decision: hash-based change detection. Every file’s SHA-256 hash is stored alongside its embeddings. On subsequent runs, the tool checks the hash first. If the file hasn’t changed, it skips the embedding API call entirely. This means re-running the ingest after a session where you changed three files only costs three embedding calls, not three hundred.

Git Commit Messages as Context

File content tells Claude Code what the code does. But it doesn’t explain why. That context lives in commit messages.

The ingest tool has a --commits flag that runs git log, extracts every commit message with its metadata (author, date, hash), and ingests them as searchable documents. Each commit becomes a vector in the database, so a query like “why did we switch from EF Core to Dapper” matches the commit message that explains the decision — something file content alone would never surface.

Multi-Repo Support

Every document in the database is tagged with its repository name, auto-detected from git remote get-url origin. This means you can ingest as many repos as you want into the same database. The MCP tools accept an optional repo filter so Claude Code can scope searches to the current project or search across everything.

The MCP Server

The Model Context Protocol is what makes this feel seamless rather than bolted on. MCP is a standard that lets AI assistants discover and call external tools. Claude Code has native MCP support — you drop a .mcp.json file in your project root, and it connects on startup.

The MCP server is another C# console app that exposes four tools:

search_context takes a natural language query, generates an embedding, and runs a cosine similarity search against pgvector. It returns the most relevant chunks from across the database with their source files and similarity scores. This is the workhorse — it’s what Claude Code calls when it needs to understand something about the project’s history or architecture.

get_interaction_history retrieves past Claude Code sessions filtered by file path or topic. Useful for questions like “what did we do to this file last time?”

get_file_summary returns all stored context for a specific file — its content chunks, last update time, and content hash.

list_recent_interactions shows the most recent activity across the project, giving Claude Code a sense of what’s been happening lately.

Claude Code calls these tools on its own when it determines the context would help. You don’t have to tell it to search — it just does.

The Session Workflow

A system like this is only useful if the database stays current. I solved this with a simple convention: a session closing checklist embedded in each project’s CLAUDE.md file.

At the end of every session, Claude Code:

1. Commits all changes with a detailed commit message documenting what changed, why, key decisions, and any issues encountered
2. Writes a session document to a prompts/ directory — a dated markdown file that catalogs every prompt and result from the session
3. Runs the ingest tool with the --commits flag to sync all changes to pgvector

This takes about 30 seconds and ensures that the next session starts with full context from this one. The commit messages, the session documentation, and the code changes all become searchable vectors in the database.

Where It Gets Interesting: Cross-Repo Context

The real payoff isn’t within a single repo — it’s across repos.

I maintain an Angular SSR application and the NGINX reverse proxy configuration it runs behind. They live in separate repositories, maintained in separate Claude Code sessions. But they’re deeply coupled: the Angular app generates CSP nonces that NGINX needs to handle correctly. Cache keys need to align. Proxy timeouts need to match SSR timeouts.

With both repos ingested into the same database, I can open Claude Code in the NGINX repo and ask:

“What changes were deferred from the codematters code review that need to be done in this repo?”

Claude Code searches the database, finds the code review from the Angular repo, identifies the specific items that were deferred because they required NGINX changes, and presents them with the exact config lines and code changes needed. It pulls context from a completely different repository to inform work in the current one.

This also transformed how I run code reviews. My production readiness review prompt now tells Claude Code to check pgvector for all prior reviews before starting. It pulls the findings from reviews 041 through 045, checks which items have been fixed (with commit references), which are still open, and which have regressed. Then it only reports new issues or status changes. Each review builds on the last one instead of starting from scratch.

What I Learned

Commit messages are underrated context. They’re small, cheap to embed, and packed with intent. They answer the “why” questions that file content can’t.

Hash-based change detection makes re-ingestion free. I was initially worried about API costs from running the ingest tool after every session. In practice, a session that changes five files costs five embedding calls — a fraction of a cent.

The MCP protocol is the right abstraction. I initially considered building a REST API, but MCP tools feel native to Claude Code. There’s no friction — the agent discovers the tools on startup and uses them when relevant without being told to.

Convention beats automation. The session closing checklist is just text in a markdown file. There’s no CI pipeline enforcing it. But because Claude Code reads CLAUDE.md at the start of every session, it knows to do it. Simple conventions that the agent follows are more maintainable than complex automation.

One cent. Three repositories, 570 embedding requests, 562K tokens. Total cost: approximately one cent. The economics of this are essentially free.

What’s Next

The system works well for my workflow, but there are clear extensions:

• Automated ingestion triggered by git hooks rather than the session closing checklist
• A web API layer so other AI tools (not just Claude Code) can query the context
• Smarter chunking that respects code structure (functions, classes) rather than splitting on character count
• Embedding model comparison to see if a code-specific model like Voyage Code outperforms the general-purpose OpenAI model for code search
• A shared team database where multiple developers’ sessions and decisions accumulate into collective project memory

The Bottom Line

AI coding assistants are dramatically more useful when they can remember. Not just the current file, not just the current repo, but the full history of decisions, reviews, and cross-project dependencies that inform how software should evolve.

The tools to build this exist today: pgvector for storage, MCP for integration, and a simple ingest pipeline to keep it all current. The whole system took a day to build, costs essentially nothing to run, and fundamentally changed how productive my Claude Code sessions are.

The best part: Claude Code built most of it.

More coding articles