This build wasn't a side project. It was at a home-care provider I worked for, and I started on it in late 2022, within weeks of ChatGPT's public release. I did nearly all of it myself, alongside leading teams who were building another product. I'm including it here, even though it predates this lab, because it's the clearest example I have of how much of an AI product is not the AI.
The problem
When someone starts receiving home care, a supervisor visits them for an initial assessment, which typically takes around an hour and a half: their health, what they can and can't do, their routines, their preferences, the risks in their home. Everything gathered in that conversation then has to be typed into a highly structured digital care plan, spread across dozens of sections, conditional questions and risk assessments. The information arrives as a conversation and has to leave as a form.
Writing the plan took at least five hours, often split over several days, which created a risk that care could begin before a plan was in place. Supervisors saw it as the most tedious part of the job and the biggest drain on their time. My question was whether a supervisor could simply hold the conversation, record it, and have AI draft the care plan in real time for them to review, more than halving the time it took to write one. If it worked, it would remove one of the biggest frustrations of the role and reduce that risk.
Getting access to a model good enough
At the time the choice of models was narrow, and their quality varied enormously. I compared what was available, including Google's models and open models such as Llama, and none of them could hold their own against GPT-4, which wasn't yet public; I negotiated early access to it for the proof of concept, and we set up a private, gated environment to run it in, so that recordings of real people's assessments never went anywhere they shouldn't. For speech, OpenAI's Whisper worked best in my trials.
What I built, and how
I started by turning the care plan itself into something a model could work with: a JSON schema of every section and field, with its type, permitted options, whether it was required, and the conditions under which it applied. Each field carried its own micro-prompt describing what good looked like.
I then wrote a mock initial assessment for a fictional client, and used it to trial different speech-to-text tools and language models. Once that was promising, I recorded real initial assessments on both an iPhone and the model of tablet branch staff actually used, uploading the audio to the gated environment and deleting it from the device straight away. Consent was captured when the assessment was booked and confirmed again at the start of the visit. Whisper with GPT-4 was the only combination that worked reliably. I ran every stage by hand myself first, to see that the approach worked end to end, before bringing in a contract engineer with some AI experience, one day a week for a few weeks in June and July 2023, to automate it.
The pipeline ended up with three stages, the first two added because of problems I hit:
- Who said what. A care assessment isn't one person talking. The supervisor, the client, a family member and sometimes a carer are all in the room, and a care plan depends on knowing whose words are whose. The transcript was split into speakers and then passed to the model with a description of the possible roles, so it could work out who was the supervisor, who was the client and who was a relative.
- Correcting the transcript. Speech to text made errors, especially with medication names, so the transcript went back through the model to correct anything that looked wrong, using the context of the rest of the conversation.
- Extraction. Only then did the model pull out the relevant information and apply each field's micro-prompt.
Knowing who said what turned out to be harder than it sounds. In one assessment I sat in on, the client struggled to answer and needed a lot of prompting. Asked what she liked to eat for her tea, she didn't really answer, and the supervisor offered an example of what she herself liked to eat, to help her along. A model reading the transcript could very easily have taken that as the client's own answer. With extra family members in the room, and sometimes a separate conversation going on at the same time, the question isn't only who spoke, but whose words should count as the client's.
What I learned from listening to real assessments
The most useful discoveries weren't about models at all.
- People don't say what they can see. Supervisors refer to documents, medicines on the table, the layout of a room, things everyone present can see and nobody needs to say aloud; none of that reaches a transcript. The product would only work if supervisors learned to verbalise things they wouldn't naturally say.
- Meaning is often implied. A question about whether someone used pads wasn't reliably recognised as meaning they had continence needs. Supervisors would need to be trained to refer to things in particular ways, and the model couldn't be trusted to infer what experienced staff would.
- Not every answer comes from the person it's about. Supervisors prompt with their own examples, other family members are in the room, and side conversations carry on in the background. A care plan has to record the client's needs and preferences, not whatever was said nearest to the question.
- "No", "not mentioned" and "assumed" are different answers. Early outputs filled in values the conversation never established. In a care plan, "doesn't need help with medication", "medication wasn't discussed" and "the model guessed" are three very different statements, and a yes/no field collapses them into one.
- The schema fought back. As the schema grew, outputs came back truncated or malformed. At the time OpenAI offered no way to guarantee a schema: it added JSON mode in November 2023, which still didn't guarantee one, and Structured Outputs, which did, in August 2024.
Designing for the right amount of trust
I wanted supervisors to trust the draft, but not too much: the risk with a fluent draft is that people stop reading it properly. That's why the design required supervisors to review the draft care plan inside the platform field by field, and purposefully accept, reject or edit each suggestion. Every suggestion would show where it came from: the transcript, guidance from the care quality team, or documents already held about the client. Because the review happened inside the care-planning system, its rules still applied: conditional questions still opened up, and a risk assessment that should have been triggered couldn't simply be missed.
Risk assessments carried the most risk, so I planned to trial that part on its own first. The first phase would test the review mechanism using risk suggestions hard-coded by the quality team rather than generated by AI: for example, if someone uses an emollient cream, there should be a fire-risk assessment with a stock set of mitigations, all of which the supervisor could edit to fit the client and their home. Proving that people would review suggestions properly came before letting a model make them.
The parts no model could solve
Some clients lived in homes with no phone signal or internet, so the design had to work offline, and the recording had to be good enough in a busy living room, which meant thinking about microphone arrays and choosing the right hardware rather than assuming any tablet would do. I developed a phased plan for the whole thing, facilitated a design sprint with a head of UX, a tech lead and a care expert, and ran two pre-mortems before the build: one with engineers and data scientists, and one with clinical, quality, branch and health-and-safety colleagues, so that the people who'd have to live with the risks were the ones naming them.
Why it was parked
Then an urgent piece of work needed my time and my team's. I put together a business case for three more people, which would have let us do both, but the company was in a hiring freeze and wouldn't make an exception, so in autumn 2023 the scribe was parked. I think that's worth saying plainly, because it's how most promising work actually stops: not because the idea failed, but because something more urgent needed the same people.
What the market built, and what it didn't
In April 2025, NHS England published guidance on using AI-enabled ambient scribing products in health and care, and looking at those products against what I'd designed was instructive. Recording, transcription and drafting a document, the part I'd found easiest in 2023, was exactly what the market built first; a polished recording experience was something we hadn't got round to building ourselves. The parts I'd designed around the model weren't available out of the box in the products I compared:
- populating the care-planning system's own structured fields, rather than producing a separate document;
- enforcing its conditional rules, so that every risk assessment an answer should trigger actually gets triggered;
- a review designed so that each suggestion is deliberately accepted, rejected or edited, inside the system of record;
- showing where every suggestion came from;
- starting risk assessments from the quality team's own rules before letting a model propose anything.
That gap is the most useful thing the prototype taught me: the easiest layer of an AI product is the one that gets commoditised first, and the value, and most of the risk, sits in the workflow around it.
In a later role, I rolled out an AI report-drafting tool through a governance framework I'd set up. More on how I approach that on emilysas.com
Why this matters to me now
Most of what made this hard in 2022 and 2023 has become easier, but the parts beyond the model (speaker roles, implied meaning, unstated context, provenance, reviewing field by field, trust that's earned rather than assumed, consent and hardware in someone's living room) are, I think, still where these products succeed or fail. That's the clearest evidence I have that leading an AI product is mostly about the workflow and the people around the model.
If you're building or buying an ambient scribe for health or care, I'd be glad to compare notes on the parts that aren't the model. Get in touch.