The Karpathy Series · Part 1 of 3
Karpathy’s wiki saves tokens. It won’t fix your company’s AI bill.
The architecture of enterprise AI cost, and why the savings your AI team is excited about won’t fix the bill on your desk.
Watch the video
The architecture of enterprise AI cost, and why the savings your AI team is excited about won’t fix the bill on your desk.
"Five or seven percent of every knowledge worker’s salary and twenty percent of every engineering salary as additional token costs."Rory O’Driscoll, Scale Venture Partners: 20VC Podcast, 2026
Andrej Karpathy’s wiki architecture delivers real savings, at the right layer. Here’s why enterprise finance, ops, and RevOps teams are paying a different bill entirely.
In April, Andrej Karpathy described how he uses an LLM to compile his research into a Markdown wiki, and the replies calculated 70 to 90% token savings on repeated questions. Within a week, finance teams were forwarding the thread to their AI leads with one question: can we do this?
LLM Knowledge Bases
— Andrej Karpathy (@karpathy) April 2, 2026
Something I'm finding very useful recently: using LLMs to build personal knowledge bases for various topics of research interest. In this way, a large fraction of my recent token throughput is going less into manipulating code, and more into manipulating…
You can, and it won’t move your bill. Karpathy isn’t wrong. His savings come from not re-reading prose, and most enterprise tokens go somewhere else.
Steelman first. He’s solving a real problem with a clean architecture:
raw/ directory holds source material, papers, articles, repos, web clippings.wiki/ of Markdown files: per-source summaries, concept articles, backlinks, auto-maintained indexes.The architecture is elegant. The savings are real. At his scale (about 100 articles, ~400K words) we’ve costed the maintenance at roughly $200 a month. For a researcher who lives in that corpus 40 hours a week, that’s an excellent trade.
The objection going around, "do you know how much that wiki will cost to maintain?", is wrong at his scale.
Three sources, in order of magnitude:
All three are content-layer optimizations. They reduce how much prose the model has to re-read on each query. If your workload is prose-heavy (research, legal contracts, policy docs) these savings show up as advertised.
Now compare that with where enterprise tokens go.
Knowledge workers in your finance, ops, and revenue teams aren’t asking "summarize this 80-page paper." They’re asking:
These don’t waste tokens by re-reading prose. They waste tokens by re-disambiguating meaning on every single call. What’s "pipeline coverage", booked, qualified, weighted, in the last 30 days? Does "offen" mean open or pending or unbilled? Which of the five tables called "revenue" is the right one for Q1 Germany?
When an AI agent runs into ambiguous data (and it does, on most enterprise queries) it doesn’t fail gracefully. It fires clarifying sub-queries, retrieves more context, generates hedged outputs, and retries. We’ve measured 4,200 tokens spent on a query that should have cost 1,100.
Multiply by 10,000 queries a day. That’s the CFO bill.
A Karpathy-style wiki doesn’t help here. A wiki article called "Revenue" will be written, and so will four others, because the model dutifully compiled what it found across SAP, Salesforce, MongoDB, and the data lake. Five articles, five definitions, and the same problem as before.
"People go nuts with Claude: one person spent a thousand dollars in token costs over one weekend, on a report you can build with your standard CRM."
Head of RevOps & Enablement, mid-market SaaS companyKarpathy precomputes content. The enterprise needs to precompute meaning.
That’s the architectural difference. Everything else flows from it.
Precomputing content gets you out of re-reading prose. Precomputing meaning gets you out of re-resolving ambiguity, and then lets the query run deterministically. Your raw definitions never touch the model; they're abstracted with probabilistic methods first, and the model is used once, at setup, to resolve "revenue means X under context Y" from those abstractions into governed code. After that, queries run directly from that code, executed against the source systems, and the LLM is not in the hot path. The bill on the millionth identical query approaches zero.
That’s not a marginal improvement on the wiki. It’s a different cost curve.
If you want to know whether your bill looks more like Karpathy’s problem or the enterprise problem, sort one day’s worth of AI queries by intent.
If they’re mostly "find me the right paragraph in the right document" (research, contracts, policy lookups) you have a content-layer problem. Build a wiki. The savings will be real.
If they’re mostly "what’s the right number for this metric, in this context, this quarter" (finance, ops, RevOps, executive reporting) a Markdown wiki will not save you. The waste isn’t in re-reading prose; it’s in re-resolving definitions that haven’t been governed.
In our experience with mid-market and enterprise customers, 60 to 80% of queries fall into that second bucket: repeatable questions about metrics that, once their meaning is governed, never need to hit the model at all. Yours may differ, so look before you copy.
Karpathy did the field a real favor. He took an architectural pattern that almost nobody was using at personal scale and made it obvious. He raised the floor on how people think about LLM cost.
The next move isn’t to argue with the architecture. It’s to ask which surface your bill is being burnt against. If your surface is prose, his answer is right. If your surface is meaning (which is most of what the enterprise is paying for) the right shape of compression looks different.
That’s the layer we built LazyFox at. We compile your definition graph the way Karpathy compiles his wiki: once, with the model doing real work. After that, every query you ask runs deterministically, against your actual systems, with no LLM call in the path. The cost curve flattens. The CFO bill stops growing with query volume.
Andrej Karpathy's proposed knowledge-base architecture for AI memory is elegant, but this post argues it won't solve enterprise data consistency: the problem isn't how a model stores what it learns, it's that different enterprise systems disagree about what a metric means in the first place.
The fix isn't in the model layer. It's a governed semantic layer beneath it, which is what reconciles those definitions before any model, knowledge base or otherwise, ever sees them.
Share your email and we’ll run a free token cost audit, showing where your knowledge worker queries are burning budget and what a 90-day reduction path looks like.