CourseAI / ML Engineer: The Complete Course · Module 9: Serving and Inference · part 57 of 93
Part 57 · Module 9: Serving and Inference

Lesson 2: A scoring service that bends instead of breaking

28 min read·
Module 9 · Lesson 2 · About 60 minutes · Needs: lesson 1 (the scoring API), Module 5 lesson 4 (training-serving skew), Module 3 lesson 3 (the logistic baseline). The models are trained in one cell here.

The situation

Week 9, Thursday. Lesson 1's service works on a quiet laptop. Rhys asks what happens at 18:00, when every depot scans its collections at once, when a model file is corrupt, or when a GPU model needs more machines than are running. Tomasz asks whether the service builds its inputs exactly as training did.

The running example is KP0104204 (simulated), collected from Dartford at 18:00 on 2 September 2026, in the evening peak. This lesson's model gives it 0.588.

WHERE THIS FITS The Kestrel system this lesson SHARED SERVICES Depot scans parcels, routes Ingestion nightly batch Validation Pandera checks Data lake Parquet, DuckDB Experiments MLflow runs Training LightGBM, Optuna Online store Feast, Redis Features point in time Gates vs the champion Registry staged versions Serving API FastAPI, ONNX Consumers vans, customers Retraining trigger, budget Alerting on call Monitoring drift, Evidently Late labels a day later Orchestrator Prefect flows Versions DVC, Git CI/CD GitHub Actions Containers Docker images
This lesson hardens Kestrel's serving part and the containers it runs in (blue); the rest of the system is grey for now.

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.

Buy this course · ₹399See what is inside

Or open everything with a plan