ProjectBuild an AI Voice Receptionist: the explanation · Module 10: English, Hindi and Hinglish · part 12 of 23
Part 12 · Module 10: English, Hindi and Hinglish

Module 10: English, Hindi and Hinglish

24 min read·
About 45 minutes · Needs: Module 9 (speech to text with Whisper), Module 2 (the voice loop) · Code: backend/app/agent/language.py, backend/app/voice/normalize.py, backend/app/agent/speech_guard.py, backend/app/agent/prompts.py

Where this fits

WHERE THIS FITS The Medilane voice receptionist this module CLIENT React API LAYER FastAPI SAFETY SPEECH IN BRAIN LangGraph SPEECH OUT KNOWLEDGE AND DATA pgvector OPERATIONS Langfuse, Prometheus Caller app browser mic Staff app desk and admin REST API staff, sessions Voice socket audio both ways Auth + RBAC staff roles, voice tokens Guardrails emergency, PII VAD Silero, turns Whisper speech to text Language en, hi, Hinglish Agent tools, confirm Open models Groq: gpt-oss, Qwen Kokoro TTS en + hi voices Retrieval multilingual Ingestion clinic documents PostgreSQL clinic, calls, vectors Redis events, limits Front desk live events, hand-offs Observability traces, metrics Evaluation WER, safety, latency
The language step (blue: this module) sits between Whisper and the agent, and its normaliser sits in front of Kokoro.

Whisper hands over a transcript, and before the agent sees it, this module decides which language the caller is speaking. On the way out, it checks that each reply sentence is written in the script the voice can read, and turns digits and symbols into the words a person would say. A mistake here does not crash anything. The receptionist simply answers a Hindi speaker in English, or reads "₹600" as nonsense, and the caller hangs up.

The rest of this project is yours to keep

This project is bought on its own, once, and stays readable afterwards, including the parts added to it later.

Buy this project · ₹799See what is inside

Or open everything with a plan