FIELD NOTES / 01 · VOICE ARCHITECTURE
Inside an enterprise voice bot
From a phone call to business systems: components, conversation and failure handling, through my experience.
Over more than three years working on enterprise voice assistants, I focused on backend development, integrations and database administration. I learned that conversational quality also depends on what the caller cannot see: connections, waiting time, operation state and the ability to reconstruct a failure.
This is my reference architecture, shaped by that experience. Voice Gateway, Google STT and TTS, Watson Assistant and a Java Spring module reaching systems over a VPN are the starting point. Load balancing, session storage, secrets management, an event pipeline and a human handoff path complete the proposal; this is not a reproduction of a particular client’s system.
For this model I choose an application call from Watson Assistant to Spring APIs through an authenticated ingress. This is a design choice for the illustration. The bridge to Google STT/TTS is shown as an adapter: protocols, codecs and compatibility with the Voice Gateway version must be checked for the actual integration.
01 / ARCHITECTURE
The structure behind the voice
Select a component to explore its responsibility. Connections show logical dependencies; the sequence below makes message direction and order explicit.
Green: experience-based core · Blue: proposed components · Dashed: cross-cutting services
Components and responsibilities
Caller Phone caller
A spoken request; identity and authorisation are checked separately.
SBC / SIP Telephony ingress
Telephony boundary: SIP routing and voice traffic policies. The media path depends on telephony topology.
Voice Gateway Session and audio
Connects the call to the conversation and coordinates audio/text exchange with speech services through adapters.
Google STT / TTS Speech adapter
STT transcribes audio; TTS synthesises text. The adapter handles formats and protocols; native compatibility with every gateway version is not assumed.
Watson Assistant Dialogue and responses
Manages the turn and uses structured application outcomes to formulate replies or offer a transfer.
API ingress / LB Authentication and routing
Authenticates application requests and routes load to healthy Spring instances. Endpoints and connectivity to Watson need explicit design.
Java Spring × N Adapters and resilience
Runs application logic, checks authorisation and isolates integrations with timeouts, bulkheads and circuit breakers. Redis does not replace authoritative systems.
VPN → Systems Business systems
The tunnel protects connectivity to external APIs. Systems retain ownership of data and operations; authentication and authorisation are still required.
Redis / Session Expiring context
Shares temporary context between Spring instances with TTLs and limits. Idempotency keys require their own durability and rules, not just a cache.
Contact center Agent and context
Receives a transferred call and a reference to minimal required context. Queue availability and transfer failure are explicitly handled.
Event queue Async events
Decouples conversation from analytics. Backpressure, bounded retries, deduplication and a dead-letter queue are part of the design.
Analytics Consumer → database
A consumer transforms events for reports and dashboards in a separate store. No dashboard query blocks the spoken response.
Secrets manager Credentials and rotation
Supplies secrets to authorised services with least privilege and rotation. It does not carry conversation content.
Observability Metrics · logs · traces
Links events through a correlation ID. Measures stage latency, errors, pool saturation, queues and transfers without recording sensitive content by default.
02 / THE FLOW
Follow a call
Illustrative request: “What is the status of my service request?” The user is already authenticated for this operation; caller ID alone is not proof of identity. Playback timing is for teaching, not a latency measurement.
Sequence diagram
Read from top to bottom. Solid arrow: request or command. Dashed: response. Orange: degraded outcome. Purple: asynchronous publication. API ingress, adapters and VPN are grouped into the Spring → Systems lane; STT and TTS share a lane.
Horizontally scrollable diagram. The full text sequence is available directly below.
Normal call
Text sequence
- Caller → Gateway Call through SBC / SIP
- Gateway → STT / TTS Audio → Google STT
- STT / TTS → Gateway Turn transcript
- Gateway → Watson Text and turn context
- Watson → Spring Authenticated request via API ingress
- Spring → Systems Status read through adapter and VPN
- Systems → Spring Application result: status available
- Spring → Watson Validated structured result
- Watson → Gateway Response text
- Gateway → STT / TTS Text → Google TTS
- STT / TTS → Gateway Synthesised audio
- Gateway → Caller Play response to caller
- Spring → Events Minimised event (asynchronous)
Slow backend
Text sequence
- Caller → Gateway Call through SBC / SIP
- Gateway → STT / TTS Audio → Google STT
- STT / TTS → Gateway Turn transcript
- Gateway → Watson Text and turn context
- Watson → Spring Authenticated request via API ingress
- Spring → Systems Status read through adapter and VPN
- Systems → Spring Timeout: result unavailable
- Spring → Watson Degraded outcome; no invented success
- Watson → Gateway Explain the issue and offer alternatives
- Gateway → STT / TTS Synthesise fallback message
- STT / TTS → Gateway Synthesised audio
- Gateway → Caller Useful message within the budget
- Spring → Events Timeout event (asynchronous)
Human handoff
Text sequence
- Caller → Gateway Call through SBC / SIP
- Gateway → STT / TTS Audio → Google STT
- STT / TTS → Gateway Turn transcript
- Gateway → Watson Text and turn context
- Watson → Spring Prepare minimal agent context
- Spring → Operator Deliver context to contact center
- Operator → Spring Reference to received context
- Spring → Watson Reference ready for transfer
- Watson → Gateway Request agreed transfer
- Gateway → Operator Telephony transfer through SBC
- Operator → Caller Conversation with available agent
- Spring → Events Handoff event (asynchronous)
Events appear last for readability: they do not wait for audio playback. Handoff shows the successful path; unavailable agents require an alternative. An already-open circuit can reject a request before it reaches business systems.
03 / DECISIONS AND TRADE-OFFS
The decisions that make the model useful
The waiting budget belongs to the conversation
I would define a total budget for each turn and distribute it across recognition, dialogue, application calls and synthesis. Each service timeout must leave room for a useful response. Thresholds should come from measurements of the actual path, not a universal number. On timeout, Watson receives an explicit outcome rather than empty data that looks like success.
A failure must not occupy every resource
Within Spring, adapters isolate clients for each external system. Timeouts bound waiting; bulkheads and bounded pools contain concurrency; circuit breakers suspend calls once configured thresholds indicate a persistent failure. One timeout does not necessarily open the circuit. Retrying a side-effecting operation requires idempotency and outcome reconciliation: a timeout does not mean the operation never happened.
Scaling Spring does not automatically save a call
The load balancer can distribute new requests to healthy instances; Redis can share application context with an expiry. Neither automatically recovers an interrupted audio session. Telephony, gateways, VPN and stores need their own availability strategies and failover exercises. Redis and the queue can also become failure points, so their unavailability needs bounded, explicit handling.
Analytics stays off the response path
The module publishes minimised events to a queue; a consumer feeds the analytics database. The caller does not wait for the dashboard. Consumers must handle duplicates, delays and unprocessable messages, with bounded retries and a dead-letter queue. If an event must be guaranteed together with a persistent change, I would consider a transactional outbox; non-critical telemetry can use an explicitly best-effort policy.
A VPN does not replace application controls
The API ingress authenticates Watson and applies limits; Spring checks user authorisation before querying systems. A secrets manager supplies rotatable credentials. Logs should use correlation identifiers without recording transcripts, tokens or personal data by default. Data retention should follow operational need.
Handoff is a service capability
When the bot cannot complete a request, Watson can offer a human handoff. The backend prepares minimal context and the telephony layer handles the transfer. Transfer failure and unavailable agents also need a defined response: a clear explanation and another support channel, without promising a connection that is not available.