Module 1: The problem a clinic phone line has

About 35 minutes · Needs: nothing, this is where the project starts · Code: research/llm.py, backend/app/services/seed.py, backend/data/synthetic/patients.md, backend/app/voice/session.py

Where this fits

This module looks at the whole system before any of it is built: who calls, what they ask, and what must never go wrong. The highlighted parts are the ones a caller and a staff member meet directly: the call page, the staff app, the ears, the brain and the voice. Every later module opens one box on this map and explains why it exists by pointing back to a requirement you meet here.

What it does

Medilane Clinic is a fictional clinic in Pune with five doctors and one front desk. The phone rings all day. Most calls are routine: book a visit, move it, cancel it, ask about Sunday hours or fasting before a blood test. A few are not routine at all, such as a caller with chest pain. The person at the desk is also checking in patients who are standing in front of them, so calls wait or go unanswered.

The project builds an AI voice receptionist: a program that answers the call, listens, understands English, Hindi and Hinglish, looks things up in the clinic's own data, books or changes appointments only after checking who is calling, and speaks the answer. Staff watch every call live and take over when the receptionist hands off. This module turns that idea into a list of things the system must do and how well it must do them.

At a glance

This module owns four production practices. Each is explained in the sections below; the table is the short version.

Use cases and examples

AI voice agent for routine calls

• Rahul Verma calls at 9 PM, after the desk has closed, and books Dr. Mehta for Monday morning. Without the agent he would call again the next day and maybe go elsewhere.
• A hospital's outpatient line gets hundreds of "what are the visiting hours?" calls; an agent answers them from the hospital's own page so nurses stop picking up the phone.
• An online shop's delivery line tells callers where their parcel is by reading the order system. Without it, every "where is my order?" waits in a queue behind real problems.
• When not to use one: a bereavement call to an insurer. The caller needs a person, and the agent should hand off at once.

Latency as a feature

• At Medilane a simple answer reaches the caller's ear in 1.1 to 1.5 seconds. At four or five seconds callers start saying "hello?" into the silence.
• A bank's card-blocking line must say "I am blocking your card" before the caller panics and hangs up, even if the blocking itself takes longer.
• A school's absence line can be slow and nobody minds, because it is a one-way message. Latency matters most when people take turns.

Functional and non-functional requirements

• Functional, at Medilane: "never book without a spoken yes". Non-functional: "first audio within about two seconds on simple turns" and "no date of birth in stored transcripts". All three must hold before a release.
• A government office's appointment line must book the right counter (functional) and work for callers who speak only Marathi (non-functional). Missing the second shuts people out.
• A SaaS company's support bot must reset passwords (functional) and cost less per call than the staff time it saves (non-functional).

Synthetic test data

• Medilane's demo caller, Rahul Verma, born 12 March 1988, exists only in seed.py. Anyone can call as him without exposing a real patient.
• A hospital testing a new booking screen with copied real records would leak diagnoses the day that test server is breached.
• An insurer builds a claims bot with invented claims for theft, flood and fire, so developers never carry real addresses on their laptops.

How it works

What callers want

A week of calls at a clinic like Medilane falls into four kinds. Appointment requests ask to book, move or cancel. Information questions ask about timings, fees, insurance or preparation. Emergencies describe symptoms that need help now. Everything else, such as "is my sugar of 180 normal?", needs a person, because the receptionist must never give medical advice. Each kind needs a different response, so the system has to tell them apart within one sentence.

Three languages on one line

Callers in Pune switch languages freely. Rahul's request mixes them in one sentence: "Mujhe kal Dr. Mehta ka appointment chahiye, morning mein." The grammar is Hindi, "appointment" and "morning" are English, and the whole sentence is written in Latin letters. This mix is called Hinglish. Another caller asks in pure Hindi: "क्या रविवार को क्लिनिक खुला है?" (Is the clinic open on Sunday?). The receptionist must understand all three and answer in the language the caller used (Module 10 explains how the language is detected).

What an AI voice agent is

An AI voice agent is a program that holds a spoken conversation and acts on it. It has four parts. Ears turn the caller's speech into text (speech to text, Whisper in this project). A brain, a language model, reads the text and decides what to say. Hands are tools, small functions the brain may call to check a patient or book a slot. A mouth turns the reply back into speech (text to speech, Kokoro here). The brain never touches the database directly; it asks the tools.

Why a voice call is harder than a chat

A text chatbot can show buttons, a calendar and a summary to tick. A phone call has none of that. The caller cannot scroll back, so every important detail must be read back aloud. The ears can mishear, so the system must cope with wrong words. And people interrupt: a caller who has heard enough starts talking, and the receptionist has to stop mid-sentence (Module 14 covers barge-in).

Latency is part of the answer

Latency is the time between the caller finishing and the caller hearing the first word of the reply. Treating it as a feature means measuring it on every turn and designing for it. Here is one live English call from 24 October 2026, "Is the clinic open on Sunday?", which took 2,812 ms. The system first waits 500 ms of silence to be sure the caller has finished. Whisper then took 281 ms, the model 1,275 ms to its first words and Kokoro 706 ms to the first audio. The remaining 50 ms is everything else.

Different turns cost different amounts. The Hindi emergency call was answered in 1,096 ms because no model ran at all. Simple turns took 1.1 to 1.5 seconds. Booking turns that called several tools in a row took 1.9 to 3.7 seconds in the completed live booking; the slowest was the turn that verified Rahul and looked up slots. They are still the slowest turns in the system. Knowing which turns are slow tells you where to spend effort.

What a plain model does

Before writing any product code, the team ran a small experiment, a spike, on 23 October 2026: three open models on Groq got the receptionist's instructions and three caller sentences. The results set many later rules. Qwen answered an English question in Hinglish. gpt-oss-20b wrote times as Devanagari digits (१०:३०), which a speech engine then reads badly. Only gpt-oss-120b answered in the right language every time.

The most serious result: asked whether the clinic opens on Sunday, all three models gave clinic hours, and none had been given any. They invented them. A caller who hears a confident wrong time walks to a closed clinic. The live system searches the clinic's documents first and answers "Sunday 10 AM to 1 PM, General Physician only, walk-in", which is what Medilane's timings document says (Module 12 covers the search).

Emergencies come first

Some callers describe chest pain, breathlessness or heavy bleeding. The only right answer is "call 112 now", spoken immediately in the caller's language, while the front desk gets an alert. This check must not depend on a model, because a model can be slow, rate-limited or wrong. In the live Hindi test the emergency reply started 1,096 ms after the caller stopped, and the model stage took 7 ms: it was skipped (Module 15 covers the emergency rules).

Anyone can call

A phone line has no login. Anyone who knows Rahul's name could ask about his appointments. So the receptionist checks identity in the call: full name plus date of birth, matched against the clinic's records. Rahul Verma, 12 March 1988, matches. "Sunita Rao, 5 June 1975" matches nobody, so nothing is shared or changed. After three failed attempts the call goes to a person.

Nothing changes without a yes

Speech recognition and language models both make mistakes, so a booking must never happen on a guess. The receptionist prepares the change, reads it back with the doctor, day, time and fee, and books only when the caller agrees. In the live Hinglish test on 24 October it read back "Monday 26 October 10:30, Rs 600" to Rahul. (Rahul had asked for "kal", Sunday, which is walk-in only, so Monday was the first bookable morning.) His "हाँ, कर दो" came back from Whisper as "हांकर दोए", still counted as a yes, and the booking APT-FQHH was made. One honest flaw: before Rahul chose a time, the model read back the first free slot, 09:00, on its own; he then said "साढ़े दस" (half past ten) and the read-back changed. Nothing was booked until his yes. The yes check is done by code, not by the model (Module 11 covers confirmation).

Six ways it can go wrong

From these cases come six failures the design must prevent. Invented facts: hours, fees or rules the clinic never set. Wrong language: an answer the caller cannot follow. Missed emergency: chest pain treated as a booking. Leaked details: appointments told to a stranger. Unwanted change: a booking nobody agreed to. Silence: an answer so slow the caller hangs up.

Requirements

Each failure turns into one functional requirement, something the system must do: answer clinic facts only from the clinic's data, reply in the caller's language and script, catch emergencies before any model, verify identity before any patient detail, confirm before any change, and offer a person whenever it cannot help.

The non-functional requirements decide whether it can go on a phone line at all. Speed: first audio around one to two seconds on simple turns. Languages: English, Hindi and Hinglish. Privacy: no dates of birth or phone numbers in stored transcripts or logs. Cost: open models on Groq's free tier, which allowed 200,000 tokens per day per gpt-oss model during testing. Visibility: staff see every live call.

Synthetic data

The project uses only synthetic data: invented people and records that look real enough to test with. Every patient name, date of birth, phone number and Aadhaar suffix is generated. The seed creates 5 doctors with their weekly hours, 40 patients, 2 staff users and 5 finished demo calls. It also creates two weeks of past and two weeks of future appointments around the day it runs: 607 in the dev database on 24 October 2026.

Code tour

Four pieces of code turn this module's ideas into something you can run: the spike that showed what plain models do, the synthetic doctors and patients, the generator that fills in their private details, and the stopwatch that measures latency on every turn.

The spike script lives in research/, next to source-code/, because it was run before the build. It needs a Groq key and is shown here, not run again (the daily quota was used up during testing).

# research/llm.py (project root, next to source-code/)
SYS = ("You are the voice receptionist of Medilane Clinic, Pune. Reply in 1-2 short spoken sentences. "
       "Reply in the caller's language. If the caller speaks Hindi or Hinglish, write your whole reply in Devanagari script "
       "(write English words like appointment or morning in Devanagari too: अपॉइंटमेंट, मॉर्निंग). Write numbers and times as words. "
       "Available slots tomorrow with Dr. Ananya Mehta: 10:00, 10:30, 11:45.")
TURNS = ["Hi, mujhe kal Dr. Mehta ka appointment chahiye, morning mein.", "क्या रविवार को क्लिनिक खुला है?", "Can I see the skin doctor on Saturday?"]
def run(model, extra):
    for u in TURNS:
        t0=time.perf_counter(); first=None; out=""
        for ch in c.chat.completions.create(model=model, stream=True, temperature=0.3, max_tokens=400,
                messages=[{"role":"system","content":SYS},{"role":"user","content":u}], **extra):
            d = ch.choices[0].delta.content or ""
            if d and first is None: first=time.perf_counter()-t0
            out+=d
        lat = sum(1 for x in out if "a"<=x.lower()<="z"); dev = sum(1 for x in out if "ऀ"<=x<="ॿ")
        print(f"{model:22s} first {first or -1:4.2f}s total {time.perf_counter()-t0:4.2f}s latin {lat:3d} deva {dev:3d} | {out.strip()[:120]}")

Code walkthrough

• SYS is the system prompt, the standing instructions sent before every caller sentence. It asks for short spoken replies, the caller's language, Devanagari for Hindi and Hinglish, and numbers as words.
• The last line of SYS gives the model three free slots for Dr. Mehta. It gives no clinic hours at all. That is deliberate: the Sunday question tests whether a model admits it does not know.
• TURNS holds three caller sentences: Rahul's Hinglish request, the Hindi Sunday question and an English question about the skin doctor.
• def run(model, extra): sends each sentence to one model. extra carries settings that differ per model, such as reasoning_effort.
• t0=time.perf_counter() starts a stopwatch; first=None means "no text yet".
• c.chat.completions.create(..., stream=True) asks Groq to send the reply in small pieces as it is written, instead of all at the end. The loop receives each piece as ch.
• d = ch.choices[0].delta.content or "" takes the new text in this piece; or "" turns an empty piece into an empty string.
• if d and first is None: records the time of the first real text. This is the model's share of latency.
• lat counts Latin letters and dev counts Devanagari letters (the Unicode block from ऀ to ॿ). A Hindi reply full of Latin letters failed the script rule.
• The print line shows the model, first-text time, total time, both letter counts and the first 120 characters of the reply.

New concept: lists and tuples A list, written [a, b, c], is an ordered collection you can change. A tuple, written (a, b, c), is an ordered collection you cannot change after it is made. Code often keeps fixed records as tuples inside a list: each tuple is one row, and its position tells you what each value means, like columns in a table.

# backend/app/services/seed.py
# name, speciality, fee (INR), languages, lane colour, [(weekdays, start, end, slot minutes)]
# Names, specialities, fees, days and colours are the approved prototype's (prototype/index.html).
DOCTORS = [
    ("Dr. Ananya Mehta", "General Physician", 600, ["en", "hi", "mr"], "#5b4bdb",
     [((MON, TUE, WED, THU, FRI, SAT), "09:00", "13:00", 15), ((MON, TUE, WED, THU, FRI, SAT), "16:00", "20:00", 15)]),
    ("Dr. Rohan Kulkarni", "Paediatrics", 700, ["en", "hi", "mr"], "#22a39a",
     [((MON, WED, FRI), "10:00", "13:00", 30), ((MON, WED, FRI), "15:00", "18:00", 30)]),
# ...
# full name, date of birth, sex (only to pick a plausible doctor), preferred language
PATIENTS = [
    ("Rahul Verma", "1988-03-12", "m", "hinglish"),
    ("Priya Sharma", "1992-07-24", "f", "en"),
# ...
DEMO_PATIENT = ("Rahul Verma", date(1988, 3, 12))

Code walkthrough

• The first comment is the column list for each doctor tuple: name, speciality, fee in rupees, languages spoken, a colour for the staff calendar and the working sessions.
• DOCTORS = [ starts a list with one tuple per doctor. Dr. Ananya Mehta is a General Physician, charges 600 rupees and speaks English, Hindi and Marathi ("mr").
• Her sessions are a list of two tuples: Monday to Saturday 09:00 to 13:00 and 16:00 to 20:00, in 15-minute slots. MON to SAT are the numbers 0 to 5, set near the top of the file.
• Dr. Rohan Kulkarni, Paediatrics, 700 rupees, works Monday, Wednesday and Friday in 30-minute slots. # ... hides the other three doctors.
• PATIENTS holds 40 tuples. Rahul Verma is first: born 12 March 1988, male, prefers Hinglish. Sex is only used to choose a plausible doctor when filling the calendar.
• DEMO_PATIENT names the patient the demo calls use. date(1988, 3, 12) is a real date object, not text, so it can be compared with the database.
• Nothing here is real: the file and data/synthetic/patients.md both say so.

New concept: a seeded random generator random.Random(SEED) creates a generator of random-looking numbers that starts from a fixed number, the seed. The same seed always produces the same sequence. That makes synthetic data deterministic: every install gets the same phone numbers, and a test that fails today fails the same way tomorrow.

# backend/app/services/seed.py
def seed_patients(db: Session) -> int:
    existing = set(db.execute(select(Patient.name_key, Patient.date_of_birth)).all())
    rng = random.Random(SEED)
    added = 0
    used_phones: set[str] = set()
    for full_name, dob, _sex, lang in PATIENTS:
        while True:
            phone = "+91" + rng.choice("6789") + "".join(rng.choice("0123456789") for _ in range(9))
            if phone not in used_phones:
                used_phones.add(phone)
                break
        aadhaar_last4 = f"{rng.randrange(10000):04d}"  # drawn even when skipped, so the sequence never shifts
        if (name_key(full_name), date.fromisoformat(dob)) in existing:
            continue
        added += 1
        db.add(Patient(full_name=full_name, name_key=name_key(full_name), date_of_birth=date.fromisoformat(dob),
                       phone_e164=phone, aadhaar_last4=aadhaar_last4, preferred_language=lang))
    db.flush()
    return added

Code walkthrough

• existing reads the name and birth date of every patient already in the database into a set, a collection built for fast "is this in it?" questions.
• rng = random.Random(SEED) creates the seeded generator. SEED is the number 20261023.
• for full_name, dob, _sex, lang in PATIENTS: unpacks each tuple into four names. The underscore in _sex says this loop does not use it.
• The while True: loop builds an Indian mobile number: +91, a first digit from 6 to 9 (Indian mobile numbers start with one of these), then nine random digits. If the number was already used, it draws again.
• aadhaar_last4 invents the last four digits of an Aadhaar number, padded to four digits by :04d. The comment explains why it is drawn even for patients who already exist: skipping the draw would shift every later number.
• if ... in existing: continue skips patients already stored, so running the seed twice does not create duplicates.
• name_key(full_name) stores a normalised form of the name, lower case without extra spaces, for matching what a caller says (Module 7 covers patient lookup for booking).
• db.add(Patient(...)) queues a new row; db.flush() sends the queued rows to the database inside the current transaction.

New concept: time.perf_counter() time.perf_counter() returns a number of seconds from a precise clock that only ever moves forward. On its own the number means nothing; the difference between two readings is an exact duration. Code that measures latency takes one reading when something starts and another when it ends.

# backend/app/voice/session.py
def _ms(t0: float | None, t1: float | None = None) -> int | None:
    if t0 is None:
        return None
    return max(0, round(((t1 or time.perf_counter()) - t0) * 1000))
# ...
    def _audio_start(self, ts: TurnState, sr: int) -> None:
        now = time.perf_counter()
        ts.audio_t0 = now
        ts.guard_frame = (self.detector.frames_seen if self.detector else 0) + self.guard_frames
        if ts.t_end is not None:
            ts.first_audio_ms = _ms(ts.t_end, now)
            FIRST_AUDIO_SECONDS.labels(lang=ts.lang).observe(ts.first_audio_ms / 1000)
        self.status("speaking")
        self.send("audio_start", ts.n, turn=ts.n, sample_rate=sr)

Code walkthrough

• _ms(t0, t1) turns two stopwatch readings into whole milliseconds. With no t1 it uses "now". max(0, ...) guards against a negative result.
• _audio_start runs once per reply, at the moment the first piece of audio is ready to send.
• ts.audio_t0 = now remembers when the audio started; barge-in uses it later to work out which words the caller heard (Module 14).
• ts.guard_frame marks a short window after the audio starts in which the receptionist's own voice, echoing into the microphone, must not count as an interruption.
• ts.t_end is when the caller's speech ended. first_audio_ms = _ms(ts.t_end, now) is the latency this module is about.
• FIRST_AUDIO_SECONDS...observe(...) records the same number, in seconds and labelled by language, for the monitoring dashboards (Module 18 covers metrics).
• self.send("audio_start", ...) tells the browser that audio for this turn follows, at sample rate sr (24,000 per second).

What happens when the seed creates Rahul Verma

1. docker compose up runs the migrate service, which runs python -m app.cli bootstrap. Bootstrap creates the staff users, then, because SEED_DEMO_DATA=true, calls seed.seed_clinic, which runs seed_doctors, seed_patients, seed_appointments and seed_demo_calls.
2. existing is an empty set on a fresh database. rng starts from 20261023.
3. The first tuple is ("Rahul Verma", "1988-03-12", "m", "hinglish"). The first draws give the number +918438539506; it is not in used_phones, so it is kept.
4. The next draw gives aadhaar_last4 = "1260".
5. name_key("Rahul Verma") is "rahul verma". That pair is not in existing, so added becomes 1 and a Patient row is queued with preferred_language="hinglish".
6. The loop repeats for the other 39 patients. db.flush() writes them and the function returns 40.
7. On any other install, the same seed gives Rahul the same number. Staff see it masked in transcripts, and the masking code replaces it with [PHONE] (Module 15 covers masking).

Design decisions

Version one keeps to what can be checked, verified and confirmed. Medical advice, reading lab reports, payments and a real phone number all stay out, and every one of them ends in an offer to connect the caller to the front desk.

The appointments are generated, not typed. For each doctor and each day from 14 days back to 14 days ahead, seed_appointments fills each slot with a probability: 45% on past days, 60% today, then 3 points less per day ahead, never below 18%. Past slots become completed, no-show or cancelled; future ones booked or cancelled. Three of Dr. Mehta's morning slots (10:00, 10:30 and 11:45) are always kept free for the demo on the next day she works, so a seed run on a Saturday keeps Monday's slots free. Because dates are relative to the day the seed runs, your counts will differ slightly from the 607 here.

When to choose differently

These first decisions fit a demo clinic with synthetic data; a real clinic meets other situations early.

What can go wrong

Each failure from How it works has a defence built in a later module and proven by a test in backend/tests/.

Invented facts. The code searches the clinic's documents before the model speaks on information questions; test_information_question_is_searched_before_the_model_speaks proves it, and test_invented_fee_is_never_spoken proves a fee not found in the tool results is never said aloud.

Missed emergency. test_emergency_short_circuits_without_the_model sends four emergencies (chest pain in English, a fainted father in Hindi, breathlessness and heavy bleeding in Hinglish) and checks that the 112 reply comes without any model call.

Leaked details. test_patient_tools_refuse_before_verification checks that appointment tools refuse until the caller is verified, and test_three_failed_verifications_hand_off checks the hand-off after three failures.

Unwanted change. test_booking_reads_back_then_books_only_after_yes checks that nothing is booked until the caller says yes.

Synthetic voices are not real callers. All test audio comes from Kokoro voices and macOS voices. Real callers on poor connections, with background noise and accents, will be harder. This is a known limit, stated in the facts sheet.

Fixed: turning the seed off still created the doctors. A code review found that bootstrap always ran seed_doctors, so a real clinic with SEED_DEMO_DATA=false would still get five fictional doctors. Now seed_clinic returns at once when the setting is off: no synthetic doctors, patients, appointments or calls, and admins add the real doctors in the staff app. test_seed_clinic_without_demo_data_creates_no_doctors in tests/test_cli.py proves it.

The free quota runs out. During testing both gpt-oss models hit 200,000 tokens per day. The receptionist then said it was not sure and offered the front desk instead of guessing. That is safe, but a real clinic needs a paid plan.

Try it yourself

cd source-code/backend
uv run python -m app.cli describe

What you should see

The five doctors with their speciality, fee and sessions, such as Dr. Ananya Mehta General Physician Rs 600 Mon/Tue/Wed/Thu/Fri/Sat 09:00-13:00 (15 min); ..., and the line Demo caller: Rahul Verma, born 12 March 1988. No database or key is needed.

cd source-code
docker compose exec postgres psql -U medilane -c "select status, count(*) from appointments group by 1"

What you should see

Four rows. On 24 October 2026 they were completed 279, booked 259, cancelled 49 and no_show 20, which add up to 607. Your numbers depend on the day your stack was seeded.

Then open http://localhost:8090, start a call and ask "Is the clinic open on Sunday?". Watch the metrics under the reply: first_audio_ms is the latency this module is about. Then say "Sunita Rao, 5 June 1975" when asked who you are, and check that nothing about any appointment is shared.

Check yourself

1. A caller asks "Kya Dr. Sheikh Saturday ko aati hain?". Which kind of call is it, which language, and what must the answer come from?

An information question in Hinglish. The answer must come from the clinic's data (Dr. Farah Sheikh works Tuesday, Thursday and Saturday), not from the model's memory.

1. The Sunday call took 2,812 ms to first audio, but Whisper, the model and Kokoro add up to only 2,262 ms. Where did most of the rest go?

The 500 ms of silence the system waits for before deciding the caller has finished. About 50 ms is everything else.

1. Why does the emergency check run before the model instead of asking the model "is this an emergency?"

A model call can be slow, rate-limited or wrong. A fixed check in code runs in milliseconds and always runs, so the caller hears "call 112" at once.

1. Why does seed_patients draw the Aadhaar digits even for a patient that already exists?

Skipping the draw would shift the random sequence, and every later patient would get different numbers. Drawing every time keeps the data the same on every run.

Key terms