Lesson 7: Deployment

Container image and environment promotion

Module 4 covered the minimal Dockerfile. Production adds a multi-stage build and a non-root user.

FROM python:3.12-slim AS builder

COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
WORKDIR /app

COPY pyproject.toml uv.lock ./
RUN uv sync --locked --no-dev --no-install-project

COPY src/ ./src/
RUN uv sync --locked --no-dev


FROM python:3.12-slim AS runtime

RUN useradd --create-home --uid 1000 appuser
WORKDIR /app

COPY --from=builder --chown=appuser:appuser /app/.venv /app/.venv
COPY --from=builder --chown=appuser:appuser /app/src /app/src

ENV PATH="/app/.venv/bin:$PATH" \
    PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1

USER appuser
EXPOSE 8000

CMD ["uvicorn", "recipe_extractor.main:app", "--host", "0.0.0.0", "--port", "8000"]

[VOLATILE: base image tags and the uv container installation pattern change. Verify before publishing.]

What each production addition buys. The multi-stage build leaves the build toolchain behind, producing a smaller image with a smaller attack surface. The non-root user limits what a compromise achieves. PYTHONUNBUFFERED makes logs appear immediately rather than sitting in a buffer, which matters when a container is killed. The dependency layer still precedes the source layer, for the caching reason from Module 4.

On worker processes. Running Uvicorn directly with one worker per container replica is a common and defensible pattern when an orchestrator handles restarts and scaling. A process manager running multiple workers per container is the alternative on plainer infrastructure. The decision is about who manages processes, and running both means two things managing the same lifecycle.

Environment promotion means one artifact moves through tiers.

build once  →  staging  →  production

The image is identical at every tier and only configuration differs. Rebuilding per environment means the thing you tested is not the thing you shipped, which defeats the purpose of testing it.

Tag images immutably by commit hash rather than by a moving tag, so a deployment names an exact artifact and a rollback names a different exact artifact.

Rollback strategy

Every deployment needs a way back, decided before it is needed.

1. Deploy the new version alongside the old
2. Wait for readiness to pass on the new instances
3. Shift traffic gradually
4. Watch error rate, p95 latency, and spend rate
5. Roll back on any breach, or complete the shift

Rollback must be faster than diagnosis. The correct response to a spike in errors after a deploy is to roll back and then investigate, not to investigate while users are affected.

What makes rollback hard, and the answers.

Database migrations that are not backwards compatible, which Module 7 answered with expand and contract: the old code must run against the new schema during the overlap.

Cache entries written by the new version, which Module 11 answered with versioned cache keys, so a rollback does not read entries the old code would misinterpret.

Queued jobs enqueued by the new version and processed by the old, which needs the payload to be readable by both.

Practise it. A rollback path that has never been executed is a plan rather than a capability.

Feature flags for model and prompt changes

Model and prompt changes are the highest-risk changes in this system, and they are the ones a code review cannot evaluate. Flags let you change them without a deployment and revert without one.

@dataclass(frozen=True)
class ModelFlags:
    generation_model: str
    prompt_version: PromptVersion
    rerank_enabled: bool
    retrieval_top_k: int
    rollout_percentage: int = 100


async def flags_for(user_id: str, deps: Dependencies) -> ModelFlags:
    """Resolve flags at request time, with stable per-user assignment."""
    base = await deps.flags.current()
    if base.rollout_percentage >= 100:
        return base
    digest = hashlib.sha256(f"{base.prompt_version}:{user_id}".encode()).hexdigest()
    in_rollout = int(digest[:8], 16) % 100 < base.rollout_percentage
    return base if in_rollout else await deps.flags.previous()

Three properties. Resolved at request time from a store, so a change takes effect in seconds rather than a deployment. Stable per user, using Module 11's assignment rule, so nobody sees two versions. And percentage-based, so a change reaches one percent of traffic before all of it.

Record the flag values on every request, alongside the cost and quality data from Modules 10 and 11. Without that, you cannot attribute a change in quality or spend to the flag that caused it, which is the entire point of rolling out gradually.

Flags accumulate and must be removed. A codebase with forty flags has two to the fortieth possible configurations, only one of which is tested. Remove a flag once its rollout completes, and treat that removal as part of the work rather than as tidying.

[IMAGE PROMPT M12-4
Purpose: Show one artifact promoted through environments with a gradual rollout, monitored gates, and two distinct revert paths.
Visual type: Promotion pipeline diagram with a rollout ramp and revert arrows.
Prompt: A clean educational diagram reading left to right. On the far left, a box labelled "build once" with a sub-label "tagged by commit hash". An arrow leads to a box labelled "staging" with a sub-label "same image, different config". Another arrow leads to a box labelled "production" drawn wider and containing an internal horizontal ramp divided into four labelled segments reading "1%", "10%", "50%", "100%". Above the ramp, three small monitor icons are labelled "error rate", "p95 latency", "spend rate", with a bracket labelled "watched at every step". Two return arrows curve back beneath: a short one from the ramp labelled "flag revert: seconds, no deployment", and a longer one from production back to a box labelled "previous image" labelled "image rollback: one deployment". A note at the bottom reads "roll back first, diagnose after".
Required elements: A single build artifact promoted through two environments, a four-stage percentage ramp, three monitored signals with a bracket, two distinct revert paths of differing lengths and speeds, the closing note.
Style: Clean educational illustration, professional, uncluttered, high contrast, flat vector.
Layout: Horizontal left to right pipeline, ramp inside the production box, revert arrows curving beneath.
Text labels: "build once", "tagged by commit hash", "staging", "same image, different config", "production", "1%", "10%", "50%", "100%", "error rate", "p95 latency", "spend rate", "watched at every step", "flag revert: seconds, no deployment", "previous image", "image rollback: one deployment", "roll back first, diagnose after".
Aspect ratio: 16:9
Accessibility: Label both revert paths with their speed in text, and name every monitored signal rather than using icons alone.
Avoid: Cloud provider logos, decorative icons, tiny text, watermarks.
Alt text: Deployment pipeline showing one image built once and promoted through staging to production with a one, ten, fifty, one hundred percent rollout ramp watched for error rate, latency, and spend, plus a fast flag revert and a slower image rollback.
END IMAGE PROMPT]