This lesson is the one to read properly, and not only because it is about safety. The way this product treats AI is also the reason it can be in your clinic at all.
The rule
The assistant drafts. You decide. Nothing clinical happens because the software concluded something.
That is not a preference setting — it is built into what each surface can do:
- The differential is labelled not a diagnosis and cannot become one on its own
- Orders are drafts and stay drafts until you press Confirm
- Codes are suggested only until you tick an attestation box at sign-off
- The note is unsigned, and nothing writes into it on its own
- No image and no signal is ever interpreted
Why it is built this way
Software that interprets an image or tells you what a patient has is regulated as a medical device. Software that surfaces information, shows its working, and leaves the clinician to decide is not — provided the clinician can independently review the basis for what is suggested.
That last phrase is why the Why? button exists on every card. It is not a nicety. It is the thing that keeps the product on the right side of the line, and it only works if you actually use it.
Reading the differential
Above the list, always: For clinician review only — not a diagnosis. Ranked most → least likely.
Each card gives a rank, the condition, a likelihood badge (high, moderate, low, unsure), an ICD hint, the rationale, and a one-line evidence summary: ▲ n supporting and ▼ n against.
Early in a visit you will often see a single card reading Unsure — gathering information: Not enough has been said yet to rank possibilities. The differential updates live as the visit continues. That is the system declining to guess, which is the behaviour you want.
Checking the working
Every suggestion has a Why? button with a count. It opens Why suggested:
Inputs that triggered this suggestion. A clinician can independently review the basis (transparency requirement). No image or signal is interpreted.
Inside are the actual inputs — the lines from the conversation, the chart values — that led to the suggestion. Read them when the suggestion surprises you, and read them when it doesn't.
The badges that mark thin ice
Suggestions carry an availability badge. Two of them are warnings:
| Badge | What it means |
|---|---|
| available | Derived from data present in this prototype |
| stale | Source exists but may be out of date |
| source-needed | The real source (payer policy / guideline) is not ingested in this prototype. Do not treat as authoritative. |
| license-needed | Requires a licensed code set (e.g., CPT/AMA) not bundled with this prototype. |
A source-needed badge on a payer-criteria suggestion means exactly what it says: the payer's actual policy is not in the system, and the criteria shown are generic placeholders. The prior-authorization card repeats it in bold — do not assert coverage.
Accept, Modify, Reject
Every card offers Accept, Modify and Reject (questions get only Accept and Reject). Rejecting is a normal, expected action, not a failure of the system. A visit where you reject half the suggestions and accept the other half is the product working.
What is recorded
Every AI draft is written to the audit trail as a draft-only agent action. The console says so: This AI draft is persisted as a draft-only, audited agent action, with View audit trail for this AI draft beside it.
So there is a permanent, tamper-evident record of what the AI proposed and what you did with it. That protects you as much as it documents you.
What usually goes wrong
Accepting a good-looking differential without opening Why?. The suggestion is only as good as what it heard, and the transcript flags its own uncertainty.
Reading a likelihood badge as a probability. It is a ranking of what to consider, produced from a conversation. It is not a calculated risk.
Assuming the AI checked the payer's policy. If it says source-needed, it did not.
Letting the assistant set the pace of the consultation. It reacts live; it does not need to be answered live. Finish the conversation, then look.