Overview
You design one system: the next version of the platform behind Northdesk, a company that sells an AI helpdesk to 240 other companies. Its employees ask questions answered from their own company's permissioned documents (Ask), hand routine requests to a bounded agent that calls their company's systems (Cases), and upload documents that are processed in the background (Intake). The product team's target is 100,000 daily users asking 10 questions each, with a 3-second response. Module 3 shows, with five minutes of arithmetic, that the target cannot be met as written, and the rest of the course designs what can. Every decision is proved with a number or a small component that runs in Colab, and written down as a design document, an architecture decision record, a numbers sheet and a failure-mode table. Northdesk is fictional and its data is simulated; every lesson also opens with a real, cited case.
What you will learn
- 107 written parts, yours for good
- 103h 14m of reading, measured not estimated
- Written for the advanced level
- Every future revision included
Prerequisites
- A Google account, to run code in Google Colab. Nothing to install; no GPU needed.
- For about half the lessons, one free API key from Groq or Google Gemini, or no key at all if you run a model on your own computer with Ollama.
- For the parts Colab cannot run (Docker Compose, PostgreSQL with pgvector, Qdrant and Redis as servers, Langfuse, Temporal), your own computer with Docker. These parts are optional; every lesson's point is proved in Colab first.
Curriculum
17 sections · 107 parts · 103h 14m3h 28m free to read · 99h 46m with the course
01Course guide6 parts · 80 min
- AI System Design: The Complete CoursePreview6 min
- How This Course WorksPreview7 min
- Setup: Colab, Model Providers, Libraries and PricesPreview13 min
- The Project: NorthdeskPreview19 min
- Shared Northdesk DataPreview30 min
- Run Locally: the Repository, Ollama and the Parts Colab Cannot RunPreview5 min
02Module 1: How AI Systems Differ6 parts · 128 min
- Module 1: How AI Systems DifferPreview1 min
- Lesson 1: Eight ways an AI system breaks the old rulesPreview30 min
- Lesson 2: What still carries over from classical designPreview25 min
- Lesson 3: Five habits that keep an AI design boringPreview22 min
- Lab: The Northdesk repository and its first recordsPreview39 min
- Incident: A provider change broke a quarter of requestsPreview11 min
03Module 2: Requirements6 parts · 169 min
- Module 2: Requirements1 min
- Lesson 1: Frame the problem before you design it29 min
- Lesson 2: Setting the quality bar before the design28 min
- Lesson 3: The seven constraints that shape Northdesk's design26 min
- Lab: Northdesk's requirements as data the gates read54 min
- Incident: A chatbot invented a crime about a real man31 min
04Module 3: Capacity and Cost Estimation9 parts · 262 min
- Module 3: Capacity and Cost Estimation1 min
- Lesson 1: From daily users to requests in flight21 min
- Lesson 2: Counting the tokens one request really sends25 min
- Lesson 3: How big the index, the GPUs and the stores must be31 min
- Lesson 4: What one answer costs, and who pays29 min
- Lesson 5: Splitting three seconds across every hop27 min
- Lesson 6: Cost decisions you build in, not bolt on24 min
- Lab: The estimator and Northdesk's numbers sheet76 min
- Incident: A test job ran up 72,000 dollars overnight28 min
05Module 4: Architecture Selection6 parts · 227 min
- Module 4: Architecture Selection1 min
- Lesson 1: The architecture ladder, one rung at a time32 min
- Lesson 2: Choosing an architecture by the numbers26 min
- Lesson 3: Where to draw the lines between components32 min
- Lab: The model port, the ladder and ADR-001102 min
- Incident: A distributed design hit its limit at 5% load34 min
06Module 5: Data Architecture6 parts · 239 min
- Module 5: Data Architecture1 min
- Lesson 1: Getting documents in, and keeping them fresh27 min
- Lesson 2: Where each piece of Northdesk's data lives28 min
- Lesson 3: Checks every document passes before Ask reads it27 min
- Lab: Intake, from source listing to searchable chunks126 min
- Incident: A migration served incomplete data for six days30 min
07Module 6: Retrieval Architecture6 parts · 277 min
- Module 6: Retrieval Architecture1 min
- Lesson 1: Retrieval's core pipeline, decided by the numbers31 min
- Lesson 2: Retrieval beyond one vector search30 min
- Lesson 3: Retrieval that holds up in production33 min
- Lab: Permissioned retrieval, from Intake's store to cited answers149 min
- Incident: A confidential label honoured on one path only33 min
08Module 7: Model Layer7 parts · 309 min
- Module 7: Model Layer1 min
- Lesson 1: Choosing a model for each Northdesk task23 min
- Lesson 2: Routing, cascades and fallback behind one gateway28 min
- Lesson 3: What happens inside the model server28 min
- Lesson 4: When the model changes under you32 min
- Lab: The model gateway, wired to a registry180 min
- Incident: A regional model endpoint failed for six hours17 min
09Module 8: Orchestration and State7 parts · 346 min
- Module 8: Orchestration and State1 min
- Lesson 1: Control flow: who decides the next step31 min
- Lesson 2: Sessions, context budgets and a cache that survives31 min
- Lesson 3: Persistent memory that helps and cannot be turned30 min
- Lesson 4: Long-running work that survives crashes and retries30 min
- Lab: The Ask workflow, its sessions and its jobs210 min
- Incident: A retried weekly email went out twice13 min
10Module 9: Agentic Architecture9 parts · 485 min
- Module 9: Agentic Architecture1 min
- Lesson 1: Design decisions for a bounded agent34 min
- Lesson 2: Designing the tool surface the agent sees34 min
- Lesson 3: Running a tool layer for many tenants27 min
- Lesson 4: One agent first, then a team, then A2A35 min
- Lesson 5: Where the agent's code runs, and what it may touch29 min
- Lesson 6: Running agents in production, and knowing what they did39 min
- Lab: The bounded Cases agent over MCP266 min
- Incident: An agent's cleanup destroyed production20 min
11Module 10: Deployment and Serving9 parts · 496 min
- Module 10: Deployment and Serving1 min
- Lesson 1: Serving Ask fast, cheaply and safely34 min
- Lesson 2: Choosing the shape each part of Northdesk runs in28 min
- Lesson 3: Containers, GPUs and scaling a slow resource24 min
- Lesson 4: Cloud platforms: what you rent and what it locks25 min
- Lesson 5: Networks, secrets and keeping tenants apart22 min
- Lesson 6: Shipping model, prompt and config changes safely35 min
- Lab: The Northdesk service, packaged and gated317 min
- Incident: One bad config value crashed every pod10 min
12Module 11: Reliability5 parts · 441 min
- Module 11: Reliability1 min
- Lesson 1: Deciding in advance how each part fails37 min
- Lesson 2: Timeouts, retries, breakers and limits under load34 min
- Lab: Gateway version 3 and the 22 September replay353 min
- Incident: Retries kept a recovered service down16 min
13Module 12: Trust Boundaries5 parts · 474 min
- Module 12: Trust Boundaries1 min
- Lesson 1: Map every way in and every way out31 min
- Lesson 2: Controls that still hold when the model is fooled35 min
- Lab: The trust map, one policy and control tests381 min
- Incident: An agent leaked data from the server side26 min
14Module 13: Quality Architecture5 parts · 517 min
- Module 13: Quality Architecture1 min
- Lesson 1: Evaluation built into the platform47 min
- Lesson 2: Seeing what production is doing34 min
- Lab: Quality as part of the platform421 min
- Incident: The router broke and answers got worse14 min
15Module 14: Scale and Evolution5 parts · 533 min
- Module 14: Scale and Evolution1 min
- Lesson 1: What breaks first when traffic grows tenfold30 min
- Lesson 2: Changing the architecture without stopping it32 min
- Lab: Ten times the traffic, and a migration457 min
- Incident: A provider cut capacity on five days' notice13 min
16Module 15: Communicating a Design8 parts · 663 min
- Module 15: Communicating a Design1 min
- Lesson 1: A design, in order, out loud35 min
- Lesson 2: Show judgement with Northdesk's own numbers22 min
- Lesson 3: Six practice designs, each from a real one27 min
- Lesson 4: Four more practice designs, each from a real one21 min
- Lesson 5: Debugging a design that got slower and dearer33 min
- Lab: The design review, from the repository514 min
- Incident: One shared dependency, and a blind monitor10 min
17Module 16: Capstone2 parts · 548 min
- Module 16: Capstone1 min
- Lab: The Northdesk design review, proved in one run547 min
Architecting a whole AI system rather than building its parts: what to put where, what to leave out, and which tradeoffs you are choosing deliberately. This is the layer above every other course in the series, and it doubles as preparation for AI system design interviews, which reward the same thinking that produces good architecture.
You design one system: the next version of the platform behind Northdesk, a company that sells an AI helpdesk to 240 other companies. Its employees ask questions answered from their own company's permissioned documents (Ask), hand routine requests to a bounded agent that calls their company's systems (Cases), and upload documents that are processed in the background (Intake). The product team's target is 100,000 daily users asking 10 questions each, with a 3-second response. Module 3 shows, with five minutes of arithmetic, that the target cannot be met as written, and the rest of the course designs what can. Every decision is proved with a number or a small component that runs in Colab, and written down as a design document, an architecture decision record, a numbers sheet and a failure-mode table. Northdesk is fictional and its data is simulated; every lesson also opens with a real, cited case.
Why this order
Classical system design assumes a deterministic core, a fixed cost per request and failures that announce themselves. An AI system has none of those, and every later decision follows from that (Module 1). The course then follows the order the roadmap recommends for any design, on a whiteboard or in a document: clarify the problem, set the quality bar, run the numbers, pick the simplest architecture the numbers permit, draw the data path, draw the request path, mark the trust boundaries, say how you would know it works, name the failure modes, state the tradeoffs. The arithmetic comes before the architecture on purpose: the numbers decide the design far more often than the design decides the numbers.
Reviews
to review this course once you have finished it.