Agent memory as a file format

August 2026 Memoryfields - a vastly simpler way to do agent memory [Floppy disk insertion noise] WOW - I know the corporate VLAN configuration Many model benchmarks start from a blank context window. The tabula rasa of AI. To some extent, this makes sense, to keep the benchmarks fair. But real agents should never start from a blank context window. They should start with as much relevant information available to the agent as possible. Your AI agents should start with memories. Why existing agent memory systems don't seem to work The trouble is, a lot of agent memory systems are actually pretty rubbish. I think there are roughly three popular kinds of memory system at the moment, each of them not working in their own way. The first are ones that deliberately tie you into a specific harness - usually written by the lab that rents you that harness. Said lab desperately wants to transition out of the (highly competitive) "API business" and into the (much more lucrative) "platform business". This form of system usually works by mining information out of your conversation history, with the result that most of their memories are all about you, even though information about the world is generally much more useful. Another kind is ludicrously complicated. I know of one prominent system that needs pgvector, a Neo4j graph database and an LLM of its own just to decide what's worth remembering. This complexity is not only difficult to administer, but, for reasons I will explain: these Big Systems bamboozle the models as well. They also fail to scale with the model frontier as it moves forward. The final kind is the "High Modernist" variety, which imagine an idealised, rationalist form of memory. Inevitably, this involves a graph, and sometimes logical propositions as well. This kind systematically strips information from its context and leaves it isolated and senseless to the agent (and you). How useful, after all, is a simple list of "distilled facts"? What they have in common is that they treat memory as a process. But memory - especially to a model - is much better represented as data. Memory should be a data format, not a multi-stage pipeline Brooks said: Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious. So, here is the "memoryfield" portable memory file format: my-memories.memoryfield.zip ├── carbon-fibre-woks.md ├── finnish-bureaucracy-tips.md ├── [... many more md files...] ├── wec-2026-season-notes.md └── nomic-embed-text-v1.5.sqlite3 A memoryfield is: Markdown "pages", with (optional) YAML frontmatter and (optional) SQLite vector index for semantic search Agents work best with files. Allow me to explain. Design decision 1: use prose, not chunks or "facts" The main reason why RAG pipelines can be very complicated is that they are trying to make a mass of existing, human-authored documents legible to an AI agent. Often these documents are very hard for the agent to read directly, eg: because they are big PDFs. But agent memories are not complicated legacy documents. A memory, at the time it is being formed, is occurring directly to an AI agent which is fully able to write prose. That prose does not need to be chunked, enriched, double-summarised or otherwise mechanically processed: just have the agent write the memory directly in its favourite format (which is Markdown). A memoryfield page looks like this: --- title: Carbon Fibre Woks created: '2026-03-01T09:00:00Z' updated: '2026-08-22T14:30:00Z' uuid: 6aa615f0-486f-48a7-a210-ba4f5ff18c8b summary: Thermal properties of carbon fibre cookware --- Carbon fibre woks conduct heat evenly, but... The one limitation, admittedly, is that the page has to be short enough to fit into a vector embedding: so there is a soft limit of about 8kb (~2000 tokens). But this is a highly beneficial restriction in practice: 8,000 characters is about 1,300 words, or the length of a medium-length magazine article. That is, in fact, a restriction it would make sense to impose anyway. To add more detail, add more pages - agents do not struggle to do this. Design decision 2: semantic jump, not graph walking A key piece of prior art was Karpathy wikis. Karpathy wikis are oriented around hyperlinked Markdown files: modelled on those used by Roam or Obsidian. The idea was that the agent would walk the "knowledge graph" to find relevant pages. But in practice, having an AI agent traverse a knowledge graph is slow and unreliable - as well as being confusing for the agent. A beautiful knowledge graph - it's a real shame that your AI agent absolutely hates it Traversal is slow because the model needs to frequently stop to make serial tool calls to read successive pages. The rough algorithm for an agent to walk a knowledge graph: Read wiki front page [tool call] find relevant links Read linked page(s) [tool call] find relevant links Decide if enough relevant information has been found If not, go to #2 If the relevant information is N steps deep in the knowledge graph, N+1 tool calls are required to retrieve it. This is slow, as your billion (trillion?) dollar LLM model has to pause for each tool call, each of which takes maybe 2-3 seconds. It also heavily penalises deeply nested knowledge graphs, which frankly cuts across the whole point of them. Knowledge graphs are also unreliable. Because the AI can only tell if the material is relevant by looking at the link text, or maybe page title, if that is externalised somehow. That puts great pressure on the agent to do 1990s-SEO-style page metadata hacking to ensure that the link text/title/caption of each page is snappy and accurate. Doing so punishes digression, the ambient noting of side details and the kind of implicit lore that is both common and highly useful in larger text corpuses. In practice, relevant information is often missed in Karpathy wikis because it is not titled or captioned in a way which looks appealing enough to the searching agent. And knowledge graphs are also confusing to the agent because they often have to pore over a lot of irrelevant information as they walk around the graph. Inadvertently reading irrelevant information (the frontpage is often the main offender) puts a bunch of noise into the model's context window, which lowers the quality of their output and makes them look fixated on weird stuff. This is all solved by using semantic search to just jump directly to all the relevant pages (based on their actual content, not their page metadata) and having the agent read all relevant pages, at once, in parallel - which the vast majority of them will do now. So in a memoryfield, at most 2 tool calls are required (#1 to search, #2 to read in parallel). Relevant stuff actually gets found and irrelevant input tokens are minimised. Design decision 3: More model, less mechanism One of the issues posed by "high mechanism" memory systems - the kind that include a lot of specially crafted APIs or databases - is that to use them, agents must navigate an interface maze to achieve their goal. If the interface is large, then you're loading a lot of openapi.json into the context. If the interface is small, then it is limiting. Even if the balance is right, often the API is still wrong: recall the times when you had to use an API written by someone else who hadn't foreseen your needs. Did you enjoy that experience? Memoryfields then, being a "low mechanism" system (just a file format), gives agents much greater latitude to invent their own access patterns. While some (hopefully) helpful tooling is provided, agents are fully free to use whatever access patterns they like. For example using perl to do find-and-replaces across the whole corpus, or putting inline CSV files inside memories that they then query with SQLite (both real examples I have personally seen). Being "low mechanism" also means that memoryfields scale with the model frontier. As models get better, agents think of more stuff to do. One of the recentish breakthroughs is that the models are accidentally very good at bash. They are good at Markdown too. And SQLite. One of the reasons that I think memoryfields work well inside real agents is that agents fundamentally can "get" what is going on from their training data (which is all you have until you can read your memories) in a way that as a disembodied LLM call within a "memory pipeline" they cannot. As models get better, they automatically start to write memories a bit more cleverly. The memory systems of the "bag-on-the-side" rarely do this. There are only so many ways to more imaginatively use a fixed set of API endpoints. Memoryfields will scale with the model frontier. Design decision 4: Open format, interchangeable, transport invariant As your collection of memories builds, they start to become precious. Your built up treasure of learned lessons and hard-won established facts. You don't want to be locked in to a specific harness, model or agent. I've written an RFC-style spec for the file format - mainly to remove ambiguities and avoid tying it to a specific embedding function. If you want, you can surely vibe code whatever tooling you need from the spec alone. But I also provide a skill and an agent-optimised command line tool to go with it. The canonical "archival" format of a memoryfield is as a zipfile. That's to make data exchange as easy as possible. But I've deliberately left the spec open to being served from local files, Amazon S3, on GitHub or over HTTP. In fact, anything that has files works. I personally use a mixture of these transports: Syncthing for personal memoryfields, S3 for those I share with others. Getting started You could have your agent pull down SPEC.md and vibe an implementation, but probably the simplest way to get started is to use my tooling: # Requires: ollama, uv and npx (comes with npm) # # 1. Pull the embedding model: