Lab: Hana's answer, costed and split, tested

Module 9 · Lab · About 60 minutes · Needs: lessons 1 to 4 of this module, and Module 8's lab

What you are building

Sol 426 at Aurora Station. Hana Aoki asked one question across four lessons: what would a model 1,000 times bigger than base cost to train on hired GPUs on Earth, and would it be worth it? Her cap for one hired run is $1,500. The lessons answered in pieces, and they did not quite agree on what "1,000 times" meant.

Today you settle it in tinyformer-lab. The 1,000-times model is Lesson 4's big: width 2,048, 16 layers, context 2,048. You save the grid and fit as scaling.py, the memory and dollar arithmetic as costs.py, and the four splits as tinyformer/parallel/, with one launcher that every distributed script shares. Eight tests pin them, one script writes Hana's answer as a manifest, and the tag is v0.9.0.