Lesson 2: Spending a compute budget, then splitting the batch
27 min read·13 Oct 2026
Module 9 · Lesson 2 · About 50 minutes · Needs: Lesson 1 (what a scaling law is); Module 6's train; Module 8, Lesson 4 (6ND and tokens per parameter)The situation
Sol 426 at Aurora Station. Hana Aoki, the Compute Officer, asked what a model 1,000 times bigger than base would cost to train on GPUs hired on Earth. Her cap for any single hired run is $1,500. Lesson 1 fitted a scaling law and said what it can claim.
She now wants two answers. For a fixed budget, how big should the model be, and on how many tokens? And how do several GPUs share one run without changing the result? The running example is her model, base times 1,000 (Lesson 4's big), and one batch of 8 station windows.