Architecting and delivering an AI documentation agent embedded in a mobile clinical application, converting surgeons' voice dictation into structured operative notes ready for billing.
Market-Leading Clinical Software Provider
12/1/2025
Surgeons dictated operative notes after every procedure, but the audio was sent out for human transcription, adding up to 72 hours before documentation was final and billable and leaving physicians to catch transcription errors after the fact.
Designed a decoupled, stateless AI engine embedded in the client's existing mobile web application, providing real-time transcription, template structuring, and voice-driven editing on HIPAA-compliant infrastructure.
Our Role
Team Led
Delivery
Compliance
A clinical software provider serving tens of thousands of physicians had a workflow problem sitting inside its revenue cycle. Surgeons preferred to dictate operative notes, but that audio went out for human transcription, and documentation could take up to 72 hours to return finalized and ready to bill.
We designed and delivered a documentation agent that removes the outsourced transcription step, embedded in the mobile application surgeons already used.
Client: A market-leading vendor of clinical documentation software, serving surgical practices nationwide.
Engagement: Technical lead of a team of five across discovery, architecture, MVP build, pilot, and handoff, over sixteen weeks.
The asynchronous workflow carried three costs.
Revenue cycle delay. Billing and reimbursement waited on transcription turnaround.
Physician friction. Surgeons liked dictating. They did not like waiting days, or finding transcription errors after the note was filed.
A hard technical constraint. The core application was a mobile HTML5 environment. Any voice solution had to run in the browser, in real time, at clinical accuracy, without forcing a rewrite to a native app.
That last constraint shaped the design more than anything else, and it pointed in a useful direction: build around the habit that already exists. Surgeons dictate. The job was to make dictation produce a finished note, not to replace it with a form.
We built the AI engine as a standalone, stateless microservice, decoupled from the client’s existing application backend. The separation let us iterate on models and prompts without touching legacy core systems, and it kept the security boundary clean. The service processes; it does not store.
Audio capture and streaming. Audio captured in the existing HTML5 app and streamed to the engine over a persistent connection, with HTTP fallbacks for degraded networks.
Speech to text. A high-fidelity, low-latency transcription model tuned for medical terminology, returned to the surgeon as they speak, with continuous disambiguation applied to the live transcript.
Template parsing. A smaller, faster model maps the raw transcript into structured operative note templates, handling both variable filling and freeform insertion. Tiered model routing was a deliberate cost and latency decision: the expensive model where fidelity is critical, the efficient one everywhere else.
Conversational editing. Surgeons correct the note by speaking to it, and the edit applies to the structured note immediately. No one returns to a keyboard to fix a number.
Security. HIPAA-compliant infrastructure throughout, with BAA-aligned protocols, voice files held in secure blob storage under defined retention limits, and per-physician feature enablement through the enterprise backend.
Word Error Rate is the standard transcription metric and it is insufficient for medicine. A system can score well on WER while mangling the terms that carry clinical weight.
We built an evaluation framework covering all three flows:
Alongside it, an observability framework covering latency to final transcript, connection stability, edits per note, template versus freeform usage, note abandonment, and token and cost tracing.
These thresholds became deployment gates in CI/CD. A build that falls below the accuracy baseline does not ship, which keeps quality a standing property of the system rather than a launch-day measurement.
Dictation now produces a structured, reviewable note inside the application in real time, removing the two-to-three-day transcription wait the previous workflow imposed on billing. Turnaround against the MVP target is tracked through the pilot instrumentation above.
The workflow lives in the application surgeons already open and mirrors the habit they already have. The architecture was designed from the start to support tiered licensing of the capability as a commercial feature.
Client identity, product details, and implementation specifics are withheld under confidentiality.
