CourseAI System Design: The Complete Course · Module 10: Deployment and Serving · part 71 of 107
Part 71 · Module 10: Deployment and Serving

Lesson 2: Choosing the shape each part of Northdesk runs in

28 min read·
Module 10 · Lesson 2 · About 50 minutes · Needs: Module 3 lessons 1 and 3 (peak traffic, GPU sizing), Module 7 lesson 3 (managed inference against your own), Module 8 lesson 4 (queues and workers). No key and no model calls

The situation

Week 10. Funmi must say where each part of Northdesk runs. Ana has heard serverless is cheaper; Gareth asks why it is not all one process.

The running example is one Tallis Retail Group question, "How many days of annual leave do I get?", at Joel's Q2 2027 peak of 46.3 requests a second. Three jobs must not ride inside it: a Cases run waiting up to 39 minutes for approval, Marlow's drive sync and the Monday Digest for 240 tenants.

One question, three jobs Tallis: How many days of annual leave do I get? Active flow Background or async 1 Processing step number Front Door front door gateway, balancer Request Tier request tier CPU LLM Generation Layer inference tier model Client Interface answer 46.3 RPS peak 1 2 3 Worker Tiers RQ workers off the request path 4 Cases approval waits up to 39 min Marlow's sync background Monday Digest 240 tenants
The running example: one Tallis question takes the request path (solid arrows, in step order, the model in blue), while three slower jobs (dashed, background) leave it for the worker tiers (yellow).

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.

Buy this course · ₹399See what is inside

Or open everything with a plan