Lesson 5: Duplicates, versions and the record every chunk inherits

Module 2 · Lesson 5 · About 50 minutes · Needs: Lessons 1 to 4 of this module (content hashes, tombstones, loaders, the rejection list, normalise), basic Python sets and classes

The situation

On 24 September an agent asked the prototype "repair turnaround time". Two of the three results were the same answer: faq-warranty-claim and faq-warranty-claim-copy, copied by hand in a help-centre migration. The third was pol-repairs. The agent got one answer twice, and nothing about the repair centre move, with its 2 extra days, made the list.

Kwame asks for three things before anything is chunked: find copies, keep only the current version of each document, and give every document one typed shape that says where it came from. Hannah adds a fourth: anything left out must be listed, with a reason.