You call a clinic. A robotic menu says "press 1 for appointments, press 2 for billing." You press 1. It says "please hold." You hold. For four minutes. Someone picks up, asks your name, asks why you're calling, asks to spell your email. Two transfers later, you have an appointment for next Thursday.
That entire experience can be replaced with a 45-second phone call with an AI.
This tutorial shows you how to build a Voice AI Agent that answers calls, understands natural speech, and books real appointments — using OpenAI for language and Orita for scheduling.
Why voice-first for scheduling?
Accessibility. Not everyone is comfortable with apps or chat widgets. Phone calls are universal — elderly patients, people with low digital literacy, and people calling while driving all reach for the phone.
Speed. Typing is slower than talking. A 3-minute chat conversation takes 45 seconds by voice.
Naturalness. "Book me something Thursday morning" is how people actually talk. Voice interfaces capture this naturalness without requiring users to learn a UI.
24/7 coverage. Most clinics, barbershops, and consulting firms get calls after hours that go to voicemail and never convert. A voice agent converts them.
Architecture
Caller
│
│ (phone call)
▼
Twilio
│
│ POST webhook (caller speech as text)
▼
Your Server ──── Agent Logic ──── Orita API
│ (check availability,
│ TwiML response book appointment)
│ (text → speech)
▼
Twilio TTS
│
▼
Caller hears response
The flow:
- Caller dials your Twilio number
- Twilio uses speech recognition to transcribe what the caller says
- The transcript goes to your agent via webhook
- The agent calls Orita to check availability and book
- Your agent responds with text
- Twilio converts text to speech and plays it to the caller
This tutorial simulates steps 3–5 — the agent logic is identical in production. Steps 1–2 and 6 are Twilio plumbing covered in the Production Setup section below.
Prerequisites
- Python 3.9+
- An Orita API key → Get one free
- An OpenAI API key → platform.openai.com
pip install openai orita-sdk
export ORITA_API_KEY=orita_your_key_here
export OPENAI_API_KEY=sk-your_key_here
Step 1: Voice-optimized tool design
Voice agents need simpler, faster tools than chat agents. On a phone call:
- The caller can't scroll through a list of 8 time slots
- You can't show a table of professionals
- Every second of silence feels like an eternity
Design principle: fewer tools, smarter defaults.
TOOLS = [
{
"type": "function",
"function": {
"name": "get_appointment_types",
"description": "Get available appointment types. Call first.",
"parameters": {"type": "object", "properties": {}, "required": []},
},
},
{
"type": "function",
"function": {
"name": "check_next_available",
"description": (
"Find the next available slot for an event type. "
"Automatically searches today through 7 days ahead. "
"Returns the first available slot."
),
"parameters": {
"type": "object",
"properties": {
"event_type_id": {"type": "string"},
"preference": {
"type": "string",
"enum": ["morning", "afternoon", "any"],
},
"preferred_date": {"type": "string", "description": "YYYY-MM-DD"},
},
"required": ["event_type_id"],
},
},
},
{
"type": "function",
"function": {
"name": "confirm_booking",
"description": "Book after caller confirms. Returns booking ID.",
"parameters": {
"type": "object",
"properties": {
"event_type_id": {"type": "string"},
"date": {"type": "string"},
"time": {"type": "string"},
"caller_name": {"type": "string"},
"caller_lastname": {"type": "string"},
"caller_email": {"type": "string"},
"reason": {"type": "string"},
},
"required": ["event_type_id", "date", "time",
"caller_name", "caller_lastname", "caller_email"],
},
},
},
]
Three tools. check_next_available is the key innovation — instead of calling /slots for a specific date and presenting options, it searches forward automatically and returns the single best slot. Less back-and-forth, faster call.
Step 2: The check_next_available implementation
def check_next_available(event_type_id: str, preference: str = "any",
preferred_date: str = None) -> dict:
start = date.fromisoformat(preferred_date) if preferred_date else date.today()
for days_ahead in range(8): # Search up to 7 days
check_date = start + timedelta(days=days_ahead)
check_date_str = check_date.isoformat()
try:
slots = orita.slots(event_type_id=event_type_id, date=check_date_str)
except OritaError:
continue
if not slots:
continue
# Apply time preference filter
filtered = slots
if preference == "morning":
filtered = [s for s in slots if int(s["value"].split(":")[0]) < 12]
elif preference == "afternoon":
filtered = [s for s in slots if int(s["value"].split(":")[0]) >= 12]
if filtered:
chosen = filtered[0]
return {
"found": True,
"date": check_date_str,
"time": chosen["value"],
"display_time": chosen["label"],
"days_from_today": days_ahead,
}
return {
"found": False,
"message": "No availability in the next 7 days.",
}
This is the voice agent equivalent of a human receptionist saying: "Let me check... we have Thursday at 9 AM, does that work?" — not reading out every available slot.
Step 3: Voice-optimized system prompt
The system prompt for a voice agent is fundamentally different from a chat agent:
SYSTEM_PROMPT = f"""You are an AI receptionist answering phone calls.
Today is {TODAY}.
VOICE RULES — CRITICAL:
- Keep responses SHORT. Max 2-3 sentences per turn.
- No bullet points, no markdown. This is speech.
- Speak naturally: "nine AM" not "9:00". "July twenty-ninth" not "2026-07-29".
- Be warm but efficient — callers want to book, not chat.
- Confirm back what you heard before booking.
FLOW:
1. Answer warmly, ask what kind of appointment.
2. Call get_appointment_types.
3. Ask morning or afternoon preference, and preferred day.
4. Call check_next_available — tell them the first slot.
5. If they agree, ask for full name and email.
6. Call confirm_booking.
7. Read back the booking reference number.
Never read out raw IDs or JSON. Keep it conversational.
"""
The VOICE RULES section is not optional — without it, GPT-4o defaults to chat-style responses with bullet points that sound terrible when read aloud.
Step 4: The agent loop
def run_voice_agent():
messages = [{"role": "system", "content": SYSTEM_PROMPT}]
booking_confirmed = False
# Agent picks up first (no user input needed)
messages.append({"role": "user", "content": "[caller connected]"})
opening = client.chat.completions.create(
model="gpt-4o", messages=messages, tools=TOOLS, tool_choice="auto"
)
msg = opening.choices[0].message
if msg.content:
print(f"Agent: {msg.content}\n") # → Twilio TTS in production
messages.append({"role": "assistant", "content": msg.content or ""})
while not booking_confirmed:
# In production: this comes from Twilio speech recognition webhook
caller_speech = input("Caller: ").strip()
messages.append({"role": "user", "content": caller_speech})
response = client.chat.completions.create(
model="gpt-4o", messages=messages, tools=TOOLS, tool_choice="auto"
)
msg = response.choices[0].message
if msg.tool_calls:
messages.append({
"role": "assistant",
"content": msg.content or "",
"tool_calls": [tc.dict() for tc in msg.tool_calls],
})
for tc in msg.tool_calls:
result = execute_tool(tc.function.name,
json.loads(tc.function.arguments))
result_data = json.loads(result)
if tc.function.name == "confirm_booking" and result_data.get("success"):
booking_confirmed = True
messages.append({
"role": "tool",
"tool_call_id": tc.id,
"content": result,
})
# Get spoken response after tool results
follow_up = client.chat.completions.create(
model="gpt-4o", messages=messages, tools=TOOLS, tool_choice="auto"
)
final_msg = follow_up.choices[0].message
if final_msg.content:
print(f"Agent: {final_msg.content}\n") # → TTS in production
messages.append({"role": "assistant", "content": final_msg.content or ""})
else:
if msg.content:
print(f"Agent: {msg.content}\n") # → TTS in production
messages.append({"role": "assistant", "content": msg.content or ""})
The comment # → Twilio TTS in production marks every output line. In production, instead of print(), you return a TwiML <Response><Say> to Twilio.
Demo: A complete simulated call
[PHONE RINGS]
Agent: Good afternoon! You've reached the clinic.
How can I help you today?
Caller: Hi, I'd like to book an appointment with a psychologist.
→ [Orita] get_appointment_types()
→ [Orita] check_next_available(event_type_id=evt_xxx, preference=any)
Agent: Of course. Our next available psychology session is
tomorrow morning at nine AM. Does that work for you?
Caller: Tomorrow morning sounds great.
Agent: Perfect! Could I get your full name and email address?
Caller: Sure, it's Carlos Fernández,
carlos.fernandez@example.com
Agent: Let me confirm: Carlos Fernández, tomorrow at nine AM.
Shall I go ahead and book that?
Caller: Yes, please.
→ [Orita] confirm_booking(date=2026-07-30, time=09:00, ...)
Agent: You're all booked! Your reference number is B-K-1-8-4-2-3.
We'll send a confirmation to carlos.fernandez@example.com.
Is there anything else I can help with?
Caller: No, that's great. Thanks!
Agent: Wonderful! Have a great rest of your day. Goodbye!
[CALL ENDED — Duration: 47 seconds]
47 seconds from ring to confirmed booking. No hold music. No transfers.
Production setup with Twilio
This simulation runs in the console. To make it answer real phone calls, you need three things:
1. A Twilio phone number
Sign up at twilio.com and buy a phone number. Twilio handles the telephony layer — calls in, audio out.
2. A webhook endpoint
Deploy your agent as an HTTP endpoint. When Twilio receives a call, it POSTs the transcribed speech to your URL. You return TwiML:
from flask import Flask, request
from twilio.twiml.voice_response import VoiceResponse, Gather
app = Flask(__name__)
@app.route("/voice", methods=["POST"])
def voice():
# Get transcribed speech from Twilio
caller_speech = request.form.get("SpeechResult", "")
# Run your agent logic (same as console version)
agent_response = get_agent_response(caller_speech)
# Return TwiML: say the response, then gather next input
response = VoiceResponse()
gather = Gather(input="speech", action="/voice", timeout=5)
gather.say(agent_response, voice="Polly.Joanna")
response.append(gather)
return str(response)
3. Configure the webhook in Twilio Console
Point your Twilio number's "A Call Comes In" webhook to https://your-server.com/voice.
That's it. The same agent logic runs on real calls.
Helpful resources:
For production, consider Twilio ConversationRelay for lower-latency, streaming voice interactions.
Performance notes
Latency matters in voice. A 3-second silence feels like the call dropped. Optimize for speed:
- Use
gpt-4o-minifor tool calls that don't need full reasoning power (slot checking, booking) — it's 10x faster and 10x cheaper - Stream the response with
stream=Trueand start sending audio before the full response is generated - Cache event types — the list of appointment types doesn't change between calls, so fetch it once on startup and reuse
Orita API calls are fast — typically under 200ms. The bottleneck is usually the LLM, not the scheduling backend.
Get the full code
git clone https://github.com/Alkilo-do/orita-python
cd orita-python
pip install -e .
export ORITA_API_KEY=orita_your_key
export OPENAI_API_KEY=sk-your_key
python examples/voice_agent_orita.py --demo
Start answering calls
Get an Orita API key and give your clinic, barbershop, or consulting firm a 24/7 AI receptionist.
Questions? hola@orita.online or GitHub.