Lesson 1: How a vector index finds neighbours fast

Module 5 · Lesson 1 · About 45 minutes · Needs: Module 4 Lesson 1 (bge-small vectors, cosine similarity, unit length)

The situation

Kwame wants to index Hollin's tickets and forum next. They change hourly and daily, so the 21 public documents from Module 4 will one day be tens of thousands of chunks. Today Ellie Shaw's question, "the smoothie maker stopped and felt hot", is compared with every document. Kwame asks: when does that stop being fast enough, and what replaces it?

This lesson traces Ellie's question through Hollin's real vectors, then measures each index type on 50,000 simulated vectors standing in for Hollin's future chunks. The Hollin data is simulated.