Setup: Colab, Model Providers, Libraries and Prices

This takes about ten minutes. You do it once.

Many lessons need no key at all: the arithmetic, the estimator, the ingestion pipeline, the queue, the web service and the simulations on the shared data all run inside the notebook. The lessons that call a hosted model are marked in each module README and in this table:

Modules 5, 14 and 15 make no hosted calls.

1. Colab

All code runs in Google Colab on the free CPU. Open a new notebook, paste each code block into a cell in order, and run it. No lesson needs a GPU.

The course targets the Python version Colab runs, which was 3.12 at the last check recorded for this series (13 October 2026); the course's own test environment is Python 3.12.15. Print it once in your first notebook:

import sys
print(sys.version)

Code walkthrough

• sys.version is the version of Python running your notebook. If it does not start with 3.12 (or a later 3.x), some pinned libraries may refuse to install; the run-locally page says which.

2. Choose a model provider

You only need one provider. You switch by changing one word in the setup cell.

In Colab, choose Groq or Gemini. They need a free key and nothing else. Ollama runs the model on your machine, so no key is needed and no text leaves your computer, which is one of the options Leila weighs in Module 7 when a tenant's contract restricts where its data may go.

Free plans limit requests per minute and per day. If you hit a limit you get an error with the number 429. Wait a minute and try again, or switch provider. Module 14 reads the limit headers and treats them as a capacity ceiling.

3. Get a key

Groq

1. Go to console.groq.com and sign in.
2. Open API Keys and create a key.
3. Copy it. You will not be shown it again.

Model list: console.groq.com/docs/models

Google Gemini

1. Go to aistudio.google.com and sign in.
2. Choose Get API key and create a key.
3. Copy it.

Model list: ai.google.dev/gemini-api/docs/models

4. Store the key safely in Colab

Never paste a key into a code cell. Notebooks get shared, and a key in a shared notebook is a key someone else can spend. (Module 10 lesson 5 is about the same rule at company scale: secrets never live in code.)

1. In the left sidebar of your notebook, click the key icon (Secrets).
2. Add a secret. The name must be exactly GROQ_API_KEY (for Groq) or GOOGLE_API_KEY (for Gemini). The value is your key.
3. Turn on Notebook access for it.

5. The setup cell

Every lesson that calls a hosted model begins with these cells. Paste them, set PROVIDER, and run.

The course uses LangChain, a widely used Python library that gives every model provider the same interface, so the rest of your code does not care which one you chose. That is also the first architectural idea of the course: the model is a dependency behind one boundary, not something written all through the code (Module 4 lesson 3).

%pip install -q -U langchain langgraph langchain-groq langchain-google-genai langchain-ollama

import os
from langchain.chat_models import init_chat_model

PROVIDER = "groq"   # "groq" or "gemini" in Colab; "ollama" on your own computer
MODELS = {"groq": "groq:openai/gpt-oss-20b", "gemini": "google_genai:gemini-3.5-flash-lite", "ollama": "ollama:llama3.2:3b"}
KEY = {"groq": "GROQ_API_KEY", "gemini": "GOOGLE_API_KEY", "ollama": None}[PROVIDER]
if KEY and KEY not in os.environ:
    from google.colab import userdata   # keys live in Colab's Secrets panel
    os.environ[KEY] = userdata.get(KEY)
llm = init_chat_model(MODELS[PROVIDER], temperature=0, max_retries=6)

print(llm.invoke("Reply with the single word: ready").text)

If it prints ready (or something close), you are set up. In the course's test run on 18 October 2026 it printed ready on both Groq and Gemini.

Code walkthrough

• %pip install ... installs the libraries once per notebook: langchain (the framework), langgraph (its companion for multi-step workflows, used from Module 8) and one connector per provider. This line is the same in every course in the series.
• PROVIDER = "groq" is the one word you change to switch provider.
• MODELS names the model for each provider as provider:model-name. This is the only place model names appear, so when a name changes you edit one line.
• KEY is the name of the secret the provider needs. Ollama needs none.
• The if block runs only in Colab when the key is not already set: it copies your key from the Secrets panel into an environment variable, which is where LangChain looks for it.
• init_chat_model(...) creates the model object, llm. temperature=0 asks for the most consistent output. max_retries=6 makes the client wait and try again when the provider says "too many requests" (a 429).
• llm.invoke(...) sends one message and returns a reply; .text is the reply as a string.

Two tiers, for the routing lessons. From Module 7 the gateway routes between a small, cheap model and a large one. Lessons that need the large tier add this line after the setup cell, and nothing else changes:

LARGE = {"groq": "groq:openai/gpt-oss-120b", "gemini": "google_genai:gemini-3.8-flash", "ollama": "ollama:llama3.2:3b"}
big = init_chat_model(LARGE[PROVIDER], temperature=0, max_retries=6)

Code walkthrough

• LARGE names the stronger model for each provider; Ollama keeps the same local model, because a laptop rarely runs a second, larger one.
• big is the large-tier model object, created the same way as llm. The gateway decides which of the two each request goes to.

What the providers report back

Every reply carries usage_metadata, the provider's own count of tokens. The estimator in Module 3 is checked against it. In the course's test run on 18 October 2026 the shapes were:

When the provider rejects a request or a reply

Sometimes a provider returns an error instead of a reply: a malformed answer, a model name that no longer exists, a brief outage. max_retries does not help when the request itself is the problem. The course handles it the same way everywhere: try once more, and if that also fails, say so plainly instead of crashing.

def ask(model, messages):
    """Returns (reply, None), or (None, reason) after two refusals."""
    for attempt in range(2):                     # one retry: the next reply is usually fine
        try:
            return model.invoke(messages), None
        except Exception as err:
            last = err
    return None, f"provider_error: {type(last).__name__}"

Code walkthrough

• ask sends the messages and returns the reply with no reason, or no reply with a reason.
• On an error it tries once more; after a second error it gives up and names the error. In the test run, a made-up model name gave provider_error: NotFoundError on Groq and provider_error: GoogleModelNotFoundError on Gemini.

Lessons use this shape: a request the model could not answer gets the result "provider_error" and is counted as a failure in every score, so one bad reply never stops a notebook and never quietly improves a result. This small function is also the seed of the gateway: Module 7 turns "try once more" into a fallback chain across providers, and Module 11 adds timeouts, a retry budget and a circuit breaker.

Lessons that run the golden sets (54 Ask questions, 18 Cases tasks) cache every reply to disk, so a rerun does not pay twice and stays inside the free plans.

6. Prices and limits (checked 18 October 2026)

This is the course's one table of model prices. Lessons that count cost say "prices from course-guide/02" and use these numbers (the same numbers are PRICE_SNAPSHOT in 06-shared-data.md). When a provider changes them, this table changes and the lessons follow. US dollars per 1,000,000 tokens, standard paid tier.

The roadmap's illustrative prices. The worked estimate in Module 3 uses "$1 per million input, $5 per million output" and "a tier at a fifth the price". Those are round numbers for arithmetic, not any provider's price, and the course labels them illustrative wherever they appear.

Limits. Groq's rate-limits page lists 30 requests a minute, 1,000 a day, 8,000 tokens a minute and 200,000 tokens a day for the two gpt-oss models, labelled as base limits for its Developer plan; your own limits are on your console's limits page. Google's rate-limits page says Gemini limits depend on your usage tier and are shown in Google AI Studio.

Deprecations to know about. Groq's deprecations page lists llama-3.1-8b-instant and llama-3.3-70b-versatile as shut down on 16 August 2026, although its models page still listed them on 18 October 2026. Do not use them. Google lists gemini-2.0-flash and gemini-3-pro-preview as shut down. Module 7 lesson 4 uses these as the example of deprecation as a planned event.

Sources (all read 18 October 2026): Groq, Supported models, Prompt caching, Rate limits, Deprecations; Google, Gemini models, Gemini Developer API pricing, Rate limits; Ollama, llama3.2. These are the numbers in this course most likely to go out of date.

7. Closed and managed models, shown but not run

Lessons show how the same call looks on OpenAI or Anthropic models and on the three big cloud platforms, using the official SDK or LangChain snippet from their current documentation, labelled not run here. The course does not call them, so no key is needed and no number in the course comes from them.

Versions are the latest on PyPI on 18 October 2026, recorded so a snippet names a real version. With LiteLLM installed (Module 7), pip chooses openai 2.54.0, because LiteLLM 1.106.0 requires a version below 3.0.

8. The libraries

Colab already ships numpy, pandas, torch and transformers. Each lesson installs only what it imports, in one cell near the top, with the version pinned so your run matches the course's. numpy, pandas and torch stay at Colab's own versions, because forcing them can make Colab ask you to restart the runtime.

Versions are the latest releases on PyPI, read on 18 October 2026; "Released" is the upload date of that release.

Libraries used only on your own computer (uvicorn, psycopg, pgvector, langfuse, a Temporal server for temporalio, docker) are on 07-run-locally.md.

The huggingface-hub pin. The newest huggingface-hub was 2.2.0 (8 October 2026), but sentence-transformers 6.1.0, langchain-huggingface 1.2.3 and litellm 1.106.0 all require a version below 2.0. Colab may already have a 2.x version. Every cell that installs one of them pins it:

%pip install -q sentence-transformers==6.1.0 huggingface-hub==1.33.0

Code walkthrough

• sentence-transformers==6.1.0 installs the embedding library at the course's version; huggingface-hub==1.33.0 keeps the download library below 2.0, which sentence-transformers needs. If Colab then asks you to restart the runtime, do so and run the cells again from the top.

Two libraries the course does not use, and why. arq (a Redis job queue) requires redis below 6, which conflicts with the pinned redis 8.1.0, so the course uses RQ. langchain-mcp-adapters 0.3.2 requires mcp below 2.0, so Module 9 writes its own short adapter between MCP tools and LangGraph instead.

Checked on paper, not yet in Colab. On 18 October 2026 a pip install --dry-run of the course's libraries (with huggingface-hub==1.33.0, leaving out arize-phoenix and the serving engines), for Python 3.12 on Linux, wheels only, resolved with no conflict: 193 packages. That proves the pins fit together; the first lesson pilot installs and imports them in Colab and records the result here.

9. Open models from Hugging Face (no key)

Each lesson names any model it downloads, with its licence read from the Hugging Face Hub on the day it was written; licences can change, so check the model card before you ship one.

10. When something goes wrong