ProjectBuild an AI-Powered Digital Bank with RAG: the explanation · Module 8: Document ingestion, from file to chunks · part 10 of 20
Part 10 · Module 8: Document ingestion, from file to chunks
Module 8: Document ingestion, from file to chunks
24 min read·
About 40 minutes · Needs: Module 6 (auth and roles), Module 5 (the data model for documents and chunks) helps · Code: backend/app/rag/ingest.py, backend/app/rag/parsing.py, backend/app/rag/chunking.py, backend/app/api/routes/documents.py
Where this fits
Ingestion (blue) runs from the admin's upload through the API and Redis queue into Postgres, where retrieval later finds it.
The assistant can only quote rules that are in its index. This module is how a rule gets there: an admin uploads a file, and a pipeline turns it into small, searchable pieces in Postgres. Module 9 then retrieves the right pieces. If ingestion gets a heading or a table wrong, every later answer built on it is wrong too.
The rest of this project is yours to keep
This project is bought on its own, once, and stays readable afterwards, including the parts added to it later.