I Turned Karpathy’s Second Brain Into an AI Operating System
Channel: AI Impact Video: Watch on YouTube
Karpathy Joins Claude — What It Means
Andrej Karpathy — the “AI Godfather,” creator of the term vibe coding, pioneer of knowledge graphs, and creator of the auto research system — has joined Claude (Anthropic). This move signals a massive industry shift: AI companies are realizing that the real power isn’t hidden in the models — it’s in the layer on top.
“The power of AI is actually not hidden in the models. It’s actually hidden in the layer on top.” — AI Impact
Karpathy and the broader industry have found that you need an operating system customized to you on top of the AI. The model alone is not enough to bring true value to people’s lives.
The AI OS Concept
The AI Operating System (AI OS) is a pattern that is starting to solidify. In a fast-moving field like AI, you want to invest in what is not going to change. The models will keep changing — but the need for structured, queryable, personal/company knowledge is permanent.
Two Categories of Data for LLMs
The data you feed to LLMs breaks into two categories:
- Quantitative Data — SQL, business intelligence, numbers. Straightforward, well-understood.
- Qualitative Data — written content, strategies, brand guidelines, visuals, images. This is becoming increasingly important and is what you should spend most of your time on.
Other categories include run-time states and canonical knowledge, but quantitative and qualitative are the primary focus.
Why Knowledge Graphs?
Traditional file systems use a strict hierarchy (one parent → many children, one-to-many relationships). Knowledge graphs break this constraint entirely.
With a knowledge graph, you can say: “Here is my Klaviyo draft — it ties to the Memorial Day project, the email tool, the marketing department workspace, the seasonal offer, and our company voice.” That single file can have six references — it’s a flattened node brain rather than a rigid hierarchy.
AIs love this kind of data. They can fly through it, see connections humans can’t, and perform self-exploration across files. On traditional file paths, they get very limited and spend way more time just finding things.
It doesn’t matter if you use OpenAI, Claude, or a super AI three years from now — AI will always need knowledge, and specifically your company’s knowledge and your personal knowledge.
The Infinite Brain System
AI Impact had previously not liked a lot of “second brain” content because it was focused on human readers — but you’re never going to open your knowledge graph yourself. Instead, AI opens it, explains things, and writes details for you.
Key Architecture
The old approach used hierarchical folders containing agents, skills, and workflows. The new approach replaces hierarchy with entity types:
- Agents (persona + capability)
- Skills
- Workflows (if-this-then-that automations)
- Rules
- Tools
- Knowledge (knowledge-within-knowledge, topic-specific knowledge spaces)
- Projects
- Outputs (everything AI creates gets saved)
Edge Types
A key innovation often missed by others: edge types in the knowledge graph. These define the relationships between entities, making the graph far more expressive and useful for AI traversal.
Indexing (Most Important Part)
If you have 5,000 notes on a topic, you need an index that says: “Here are all marketing images tied to this product, here are all products tied to this campaign.” Each piece gets a short summary so the AI can quickly read indexes and know where to find details. The AI can read ~500 notes before having to come back with an answer — it reads across, finds relevant things, and injects them into its context.
Three Pillar Tools (Free & Open Source)
The entire system is built on three free tools:
- Obsidian — for viewing and navigating knowledge graphs visually. Free, great for exploring the infinite brain.
- n8n — for workflows (if-this-then-that automation). Self-hosted = free. Gold standard for workflow automation.
- Paperclip (Shadow Departments) — for storing agents in a compatible OS format. Open source, self-hostable.
Everything is stored in markdown (plain English text). No special coding languages needed. You just need to know English and how the components work together.
Model Agnostic
The AI OS is completely model-agnostic:
- If Claude gets way better → move your OS to Claude.
- If Grok enters the race → move your OS to Grok.
- If Google pulls ahead → move your OS to Google.
You are not locked into any third-party vendor. The OS sits on top and defines your AI architecture.
How It Works in Practice
- Every team member gets their own personal AI operating system folder.
- They can add agents, workflows, skills, and knowledge.
- Claude Code / Codex can build everything out super fast — just describe the problem and it converts it to an Infinite Brain shaped solution.
- Individual innovations can be promoted upward: a Meta Ads analyst creates an amazing report → it moves to the Meta Ads department knowledge graph.
- Individual repos → apartment-level repos → department operating systems → company operating systems.
This architecture allows companies to scale horizontally (across the entire organization) and vertically (across all problems) without being tied to a specific AI vendor.
What CTOs and Tech Leaders Say
CTOs and traditional tech leaders who have seen this system agree it makes sense. The only concern raised is whether business power users will be overwhelmed by GitHub and these tools — but in practice, when people are empowered with this setup, they are quick to take it and run with it.
Open Source Release
The Infinite Brain system will be open sourced for free on June 10th. A template will be available for anyone. The creator’s school community will also release a structured course alongside it.
“We believe in open source and having free options. We want AI to be free and people to push it forward.” — AI Impact
Key Takeaways
- Models change, knowledge is permanent — invest in building your knowledge graph and AI OS now.
- Knowledge graphs are the substrate — they are what AIs use to understand your context, your company, your brand.
- AI-first, not human-first — design your second brain for AI to read and traverse, not for human browsing.
- Markdown is the lingua franca — everything lives in plain English markdown, accessible to any AI.
- Free and open — the three core tools (Obsidian, n8n, Paperclip) are all free and self-hostable.
- Model agnostic — your AI OS works with any model, now and in the future.
Related
- Karpathy LLM Wiki
- Understand Anything — code-to-knowledge-graph tool
- AI Engineering From Scratch — structured AI learning
Full Transcript Summary
The video covers (in order):
- Karpathy joining Claude and what it signals about the industry’s focus shift from models to the layer on top.
- The need for a personalized AI operating system.
- Two data categories: quantitative and qualitative (with qualitative being the focus).
- What knowledge graphs are and why AIs love them (flattened node brain vs. hierarchical file systems).
- Why second brain content needs to shift from human-reader focus to AI-reader focus.
- The Infinite Brain architecture: entity types, edge types, indexing.
- The three free tools: Obsidian (view), n8n (workflow), Paperclip (agent OS).
- How this architecture enables company-wide AI scaling, bottom-up knowledge promotion, and model agnosticism.
- Open source release plans and community resources.