Every few weeks, someone on my team or a client's team asks the same question: "Should we use GraphRAG or Self-RAG? What about HyDE? We heard Corrective RAG handles hallucinations better."
I've sat in this meeting a dozen times. Smart engineers with whiteboards full of retrieval pipeline diagrams, benchmarking embedding models, A/B testing chunk sizes. It all looks productive. Nobody is asking the right question.
And if the meetings weren't enough, your LinkedIn feed is full of armchair retrieval architects who've never instantiated a vector store, screaming that GraphRAG is "the future" and Self-RAG is "dead." They speedrun every acronym in the RAG cinematic universe to farm impressions. The only thing they're retrieving is engagement.
The RAG conversation has turned into a framework selection exercise. Pick your variant, wire it up, ship it, tune until the eval metrics stop embarrassing you. I've spent years architecting production AI infrastructure, including systems that serve real-time financial intelligence across multi-agent pipelines, and I think the whole framing is off.
RAG isn't a search problem. It's a memory management problem. Until you see it that way, you're optimizing the wrong layer.
The Taxonomy Trap
The RAG ecosystem has turned into a zoo of acronyms. GraphRAG, Self-RAG, HyDE, Corrective RAG, Adaptive RAG, Agentic RAG, RAG-Fusion, Speculative RAG, LongRAG, Cache-Augmented Generation. Each paper claims to improve on vanilla RAG. Each one gets a Medium post and a LangChain integration.
Strip away the branding and look at what each variant actually does. They're all trying to answer the same question: what goes into the LLM's limited working memory, and what stays out?
That's a memory management question, not a search question.
Vanilla RAG is demand paging with a nearest-neighbor lookup. Query in, find the closest vectors, load those chunks into the context window. Fine until "nearest vectors" stops meaning "actually relevant."
GraphRAG replaces the flat vector index with a graph: entities, relationships, community summaries. You traverse for relevant subgraphs and load those into context. Different page replacement policy. "Relevance" becomes structural proximity in a knowledge graph, not cosine similarity in embedding space.
HyDE generates a hypothetical answer first, then retrieves using that embedding. Speculative prefetching. The system guesses what the working set will look like and pre-loads. Clever, except the guess is made without ground truth. For earnings data, policy changes, market signals, the model's priors are often stale before you even query.
Self-RAG adds a reflection step after retrieval and generation. The model checks whether the chunks actually supported the answer before you treat them as part of the working set. Cleaner cache, extra LLM call per cycle.
Cache-Augmented Generation pre-computes the whole knowledge base into the model's KV cache at startup. You pin the working set in RAM. Fast, no retrieval latency, but only works when the corpus fits in the context window and doesn't change much.
Corrective RAG scores retrieval quality before generation and falls back to web search when the documents look weak. A cache miss handler with a slower storage tier behind it.
Same idea, six different names. All of them are memory policies dressed up as retrieval architectures.
The Reframe: Your Context Window Is RAM
Karpathy has been making this argument since 2023: the LLM is the CPU, the context window is RAM, tools are system calls. He said it on X, then again in his "Intro to Large Language Models" talk, then at Sequoia's AI event with the Software 3.0 framing. Context window as RAM, model weights as CPU, prompting as programming.
The Berkeley team behind MemGPT (now Letta) went further and built the virtual memory layer. Main context as RAM, recall storage as disk, archival storage as cold storage, function calls as memory management operations. Their agents do better on document analysis and multi-session conversations because they manage memory on purpose instead of hoping the context window sorts itself out.
What I haven't seen anyone connect cleanly is RAG. Karpathy gives you the architecture diagram. MemGPT gives you paging mechanics. But if retrieval is memory management, what are GraphRAG and Self-RAG actually doing in OS terms? That's where the taxonomy above stops being a framework shopping list and starts reading like a field guide to page replacement policies.
Your context window is RAM: fixed, expensive, fast. Everything the model can reason about in one inference pass lives there, with a hard cap at 8K, 128K, or 1M tokens. Every token you add pushes something else out.
Your corpus is disk. Big, slower to reach, indexed in ways that may or may not match how your workload actually accesses it.
Your retrieval pipeline is the page fault handler. When the model needs something that isn't in context, a fault fires and the retrieval system decides what to pull from disk into RAM.
Belady's optimal algorithm, from 1966, says the best page replacement policy evicts the page that won't be needed for the longest time. You can't run that in production because it requires knowing the future. Operating systems approximate with LRU, LFU, clock algorithms, and the rest.
RAG hits the same wall. When the context window is full and new information has to come in, something gets dropped. Most teams truncate old turns, cut the lowest-scoring chunks, or cram everything in and hope the model copes. That's FIFO. It works until it doesn't.
I think you can do better than FIFO for context windows, and knowledge graphs are part of the answer, in a way most GraphRAG setups don't use yet. I'll get into that in the next post.
What Changes When You See It This Way
You stop treating RAG variants as isolated fixes and start asking what the full memory hierarchy should look like.
Four tiers.
The system prompt is L1. It loads on every call and never gets evicted, so it had better earn its token cost. A lot of teams park boilerplate instructions there that could live in retrieval instead. That's expensive real estate wasted on low-value text.
Recent tool results and conversation turns are L2: session-scoped, high churn, your active working set. Summarize or drop stale turns instead of keeping them verbatim.
Retrieved chunks sit in main memory, where your RAG pipeline does its work. Bad retrieval here is a bad cache hit rate: the system runs, but you're paying for I/O that doesn't help.
The full corpus is disk. Flat vector store, knowledge graph, relational DB, markdown wiki. The shape of that tier limits what retrieval can even attempt. Vector similarity only gets you so far if everything is chunked the same way.
Most teams split ownership across these tiers. One person owns the system prompt, another owns retrieval, conversation management is an afterthought, and nobody owns eviction policy at the boundaries.
That's the gap. Not picking a RAG variant. Managing the hierarchy.
The Missing Piece
Nobody has built the operating system for this yet, and it bothers me.
Vector databases work. Knowledge graph tooling is improving, slowly. Agent frameworks can chain retrieval steps. But the layer that watches the active thread, predicts what's needed next, loads proactively, and evicts stale context is still held together with prompt engineering and hardcoded retrieval calls.
In an OS, that's the virtual memory manager. It took decades to get right.
For LLM systems we're still swapping segments by hand and calling it done.
Knowledge graphs might get us closer, but not the way GraphRAG uses them today. Graph distance as a proxy for semantic proximity ties into a result from the paging literature that I want to unpack in the next post.
The Takeaway
If your team is in a meeting debating GraphRAG vs Self-RAG, you're in the wrong meeting.
Ask instead who owns your memory hierarchy, what the eviction policy is at each tier, and what happens when the context window fills up.
Get that right and the retrieval variant is an implementation detail. Get it wrong and swapping frameworks won't fix your hallucination rate.
Less retrieval tuning. More memory design.