Comparative Audits and Architecture Trials
Empirical head-to-head comparisons, multi-implementation reviews, and architectural audits evaluating wiki performance, schema rigor, and retrieval accuracy.

An empirical comparative audit evaluating three Obsidian knowledge base plugins for Claude Code: AgriciDaniel's claude-obsidian, Ar9av's obsidian-wiki, and Ars Contexta. Benchmarks sandbox vault setup, ingestion performance, plan hash pinning, provisional vs. inferred claim marking, and hook safety behaviors like automated Git commits.

An architectural audit and comparative review analyzing 10 company brain implementations across four families: self-hosted patterns (Claude Code, GBrain, Sylph), memory SDKs (mem0, Zep, Letta), vertical setups (Pletor, Gorgias Cortex), and dedicated platforms (Carrara, Slite Agent). Evaluates signals, storage shapes, maintenance loops, and query interfaces against empirical survey findings from 149 teams.
Hybrid RAG vs LLM-Wiki: 13-Question A/B Evaluation (Kevin Wu)osisdie.github.ioAn empirical comparative audit benchmarking Karpathy's LLM Wiki against Hybrid RAG (BM25 + dense + RRF) across a 60-document knowledge base. Highlights scorecard metrics (13/13 vs. 12/13 accuracy, 9.3× token overhead, 2.5× p50 latency) and diagnoses three critical LLM Wiki failure modes: cross-tenant contamination, baked-in page synthesis noise, and paraphrase over-simplification.
I Built Karpathy's LLM Wiki Twice — Once as Code, Once as a .md (Leandro Bernardo on Towards AI)pub.towardsai.netA comparative case study outlining the tradeoffs between a programmatic pipeline (predictability, typed contracts, token budgets) and an agentic instruction-based Markdown setup.

A comparative empirical case study examining the impact of knowledge organization on Claude Code reasoning. Compares a rigidly structured Obsidian vault (strict SCHEMA.md, YAML metadata) against a minimally structured one, demonstrating how over-structuring can lead to confidently incorrect answers when data is missing and outlining a progressive structuring threshold at ~300 notes.

A comparative review of five open-source LLM Wiki implementations, focusing on automation and workflow integration.
I Tested Karpathy's Second Brain Against the AI Setup I Already Use (James Presbitero on Unpromptable)unpromptable.substack.comAn empirical comparative trial benchmarking Karpathy's LLM Wiki against a flat file folder using Claude Code across 166 business documents and 20 evaluation questions. Analyzes build duration (154 minutes), API token costs ($54.39 build, 4.6% query token savings), multi-LLM council grading across five question types, and contradiction detection during vault linting.

A practitioner build of an LLM wiki, with four maintenance rules, two zero-token drift checks, and an audit of the author's own vault.

An empirical comparative review and audit examining 12 open-source implementations of Karpathy's LLM Wiki pattern across desktop applications, Obsidian plugins, agent skills, and compilers. Evaluates interfaces, model providers, and offline capabilities alongside an empirical study of 46 public LLM wikis on GitHub.

An architectural comparative review evaluating five codebase wiki implementations alongside LangChain's OpenWiki: DeepWiki, Google Code Wiki, OpenDeepWiki, deepwiki-open, and repowise. Analyzes tradeoffs across in-repo agent-maintained Markdown vaults, hosted public readers, self-hosted enterprise portals, and AST code-graph extraction without model token spend.
OpenWiki: The LLM Wiki Pattern as Running Code (Rohit Ghumare)aiengineeringfromscratch.comAn architectural audit and comparative review contrasting two production implementations of the LLM Wiki pattern: LangChain's file-based CLI product (openwiki) and III HQ's decomposed runtime worker. Compares their storage layers (committable OKF Markdown in Git vs. runtime KV state with commit-pinned line citations), execution interfaces (CLI/CI vs. HTTP/SSE/MCP), and incremental refresh loops to evaluate when to ship wikis as files versus services.
Teaching an AI to Remember: What I Learned Building a Wiki LLM (Ido Efrati on Cornell AI Innovation Hub)innovationhub.ai.cornell.eduA case study evaluating Karpathy's three-layer LLM Wiki architecture (sources, wiki, schema) against traditional RAG. Stress-tests chapter-by-chapter ingestion and contradiction reconciliation on The Hound of the Baskervilles, utilizing a CLI tool (
wiki_tool.py) for catalog indexing, metadata validation, and orphan-page linting.
When is the Karpathy Wiki Better than RAG (Learning Aloud Blog)learningaloud.comA follow-up case study comparing pre-compiled local wiki query outputs against direct cloud RAG query results.
Why Your AI Agent Needs a Wiki (And Why RAG Isn't Enough) (Eyal)eyal454160.substack.comAn empirical case study benchmarking Karpathy's LLM Wiki against raw file RAG across six tournament rounds on a 134-file production codebase (~50K LOC). Highlights how pre-synthesized wikis dominate cross-file architecture tracing (94 vs. 84) and retain deleted historical context, while a hybrid pipeline (wiki context plus code verification) swept every tournament across both architectural and exact code retrieval questions.
