CourseAI System Design: The Complete Course · Module 5: Data Architecture · part 37 of 107
Part 37 · Module 5: Data Architecture

Lesson 3: Checks every document passes before Ask reads it

27 min read·
Module 5 · Lesson 3 · About 50 minutes · Needs: Module 5 lesson 1 (the dead-letter list) and enough Python to run a cell. No model calls and no key.

The situation

Week 5. Intake can fetch every tenant's documents. Before anything is indexed, Leila asks: which documents are refused, are personal phone numbers removed, can a facilities page be read by the whole company, and can we say which file and which code produced a quoted sentence? Gareth adds: how do we know the parser kept the text?

Follow one document: Tallis Retail Group's team page t-tallis-d014, shared with facilities, ending "Out of hours, call Aisha on 07700 900140."

Five controls before the index One tenant document on the data path Active flow 1 Processing step number Data Source Tallis drive t-tallis-d014 Ingestion pipeline parse html-v1 validate the gate Leave the path dead letter rejected 1 2 Governance controls ACL facilities redact phone replaced provenance one row Storage index Module 6 3 4 5 6
Every document from a tenant (pale yellow, untrusted) passes five Northdesk controls (the ingestion and governance panels) before the index; what the gate rejects goes to the dead-letter list (pale red). Numbers give the order.

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.

Buy this course · ₹399See what is inside

Or open everything with a plan