Solving the Memory Problem

Every AI tool wants to be your only AI tool. Each one builds its own memory silo: your preferences, your history, your context, all locked inside one provider's ecosystem. Use Claude for a month, switch to a different model, and you're starting from scratch. The memory doesn't travel.

I decided to fix this by building a persistent memory layer that sits outside any single LLM, with its own API, its own storage, and its own retrieval logic that any tool can query.

It took a few months to get right, and it has completely changed how I work.

What the system actually does

The memory layer stores structured facts, freeform notes, and a knowledge graph with entities, relationships, and semantic search. Every tool I use — Claude Code, Claude Desktop, Google's Codex, Claude Co-Work — pulls from the same memory on every request. When I tell one tool something, every other tool knows it. When I make a decision in one context, that decision carries forward everywhere.

This sounds simple. In practice, most people's AI setup is a collection of disconnected conversations where the same context gets re-explained in every session, every tool, every day. The memory layer eliminates that entirely.

The system also handles workspaces, API authorisation for external services, LLM querying across providers, and brief generation that pulls from stored facts automatically. Last week I asked one tool to prepare a project brief — it pulled my preferences, prior decisions, and relevant notes from across three months of work without me pointing it at anything. That's what a shared memory layer buys you.

The lock-in problem nobody talks about

Every major AI provider has an incentive to make their memory layer proprietary. OpenAI's memory, Claude's project knowledge, Google's context caching — all designed to keep your information inside their ecosystem.

This is the real lock-in. Not the model quality or the pricing or the feature set. Once you've spent six months building up memory inside one system, switching providers means abandoning all of it. The memory is the moat.

Building the memory layer externally breaks that dependency. I can swap models without losing context. I can use the best tool for each task without fragmenting my knowledge across five different systems. The memory belongs to me, not to any provider, and that turns out to be worth more than any single model improvement.

What happened when I connected it to Google Drive

The most instructive example of what autonomous tooling can do came when I connected Claude Co-Work to my Google Drive through the memory layer.

It scanned my entire Drive, indexed the contents, created a schema and taxonomy for the information it found, and ported the whole structured index back into the Drive itself. Not a summary but a working knowledge architecture that I can now query, reference, and build on.

I didn't sit there tagging files or building folder hierarchies. The system looked at what I had, figured out how it related, and organised it. That would have taken me days to do manually, and I would never have maintained it.

Why this matters beyond my workflow

The broader point isn't that everyone should build their own memory system. It's that persistent memory across tools is the layer that makes AI actually useful for sustained work rather than one-off queries.

Right now, most people use AI the way they used early search engines: type a question, get an answer, move on. The context evaporates. The next session starts cold. Every tool is an island. That's fine for simple lookups, but it falls apart the moment you're working on anything that spans more than a single conversation.

The organisations and individuals who figure out how to give their AI tools shared, persistent context are likely to have a structural advantage over those who don't, because their models know more about the work. The quality of the output is bounded by the quality of the context, and right now most context is thrown away after every session.

What building this taught me

AI as a chat interface is useful for the duration of the conversation. AI as infrastructure compounds over time, because every fact stored and every relationship mapped makes every subsequent interaction a little better.

In my experience, most of the conversation about AI productivity focuses on prompt technique or model capability. Those things matter, but they're not where the real leverage lives. The real leverage is in the memory layer, the tool connections, and the autonomous operations that let the system do useful work without being asked.

The tools will keep getting better. The memory is what makes them yours.