ProjectIntent Over Keywords: From Stitched Embeddings to a Production Fashion Agent · Module 3: Frames and CLIP · part 5 of 26
Part 5 · Module 3: Frames and CLIP
Module 3: Frames and CLIP
23 min read·
About 50 minutes · Needs: Module 1 (the problem with multimodal search), Module 2 (the stitched architecture) · Code: part1-video-rag-stitched/src/extractors/media.py, src/embedders/local_models.py, Dockerfile, originals/v1/video-rag-old/src/extractors/frame_extractor.py, .../embedders/clip_embedder.py
Where this fits
This module (highlighted): frames from ffmpeg, the CLIP model (amber, because it is a model) and the p1_visual collection its vectors go into.
This module is Part 1's picture path: how a video becomes a set of still frames, how CLIP turns each frame into 512 numbers, and how those vectors land in the p1_visual collection. It also covers where the CLIP model lives (baked into the image, run offline). The speech path is Module 4; how the two collections are searched and merged is Module 5.
The rest of this project is yours to keep
This project is bought on its own, once, and stays readable afterwards, including the parts added to it later.