Zum Inhalt springen
Kontakt aufnehmen

Evaluating clinical AI evidence

How to read clinical AI studies: designs and what each establishes, off-policy evaluation, dataset shift, leakage and bedside metrics, with Clinomic examples.

7 Min. Lesezeit


Clinical AI publications range from randomised trials to retrospective analyses of historical databases. Both are legitimate research, and they support very different claims. This article gives technically serious readers a working framework for reading clinical AI and clinical software evidence: which study design answers which question, the technical failure modes that inflate results, the metrics that matter once a tool reaches a unit, and how Clinomic's own studies read through that framework.

Study designs and what each can establish#

Design What happens What it can establish What it cannot establish
Retrospective evaluation A model is trained or tested on historical records; it influenced no treatment Association with recorded outcomes; agreement or disagreement with past clinical actions That using the model changes outcomes
Simulation study Clinicians use the tool on a fictitious patient in a controlled environment Task performance, errors, time, workload and user experience under standardised conditions Behaviour in real clinical work over weeks
Silent prospective evaluation The model runs on live data; its outputs are logged but not shown Performance on live data as it actually arrives Effect on decisions and outcomes
Before-and-after implementation study Process measures are compared before and after a tool or protocol is introduced Change in the measured process in that setting Causation independent of time trends and co-interventions
Randomised controlled trial Patients or units are randomised to care with and without the tool Causal effect on the chosen outcome Transfer to settings unlike the trial sites

Each row is a legitimate step. The error is to take a result from one row and describe it in the language of another.

Reporting guidelines help readers check whether a study has given them what they need: TRIPOD+AI for prediction model studies, DECIDE-AI for early live clinical evaluation, and CONSORT-AI and SPIRIT-AI for trials and trial protocols involving AI.

Retrospective evaluation and off-policy estimates#

Retrospective evaluation is where almost all clinical AI starts, because it is fast, safe and can use large public databases. For reinforcement learning, a learned treatment policy is evaluated off-policy: its value is estimated from historical trajectories generated by clinicians following a different policy, typically with importance sampling or a learned model of outcomes. Such estimates are reliable only where the learned policy stays close to actions clinicians actually took, since rarely observed actions receive large weights and high variance. They also assume that everything which influenced both treatment and outcome was recorded, and in intensive care it seldom is: a clinician's bedside impression of a patient does not appear in the database.

What retrospective results cannot show:

  • That using the model improves outcomes. Patients who historically received the "model-preferred" treatment may have differed in unrecorded ways from those who did not.
  • That the model works under live conditions, where data arrives late, incomplete and out of order, as described in clinical time and data arrival.

Leakage through clinical time#

A frequent source of inflated retrospective results is leakage: the model is given information that would not have been available at the moment of prediction. Clinical records make this easy to get wrong, because every value has several times attached to it. In FHIR terms, an Observation's effectiveDateTime says when the value was clinically true, while the record's own update time says when it became available. A laboratory result effective at 06:00 may be recorded at 07:40. A model trained on effective time "knows" the result at 06:00; the same model in production learns it at 07:40.

Corrections are a second route. If the training set is built from final, corrected values, the model never sees the uncorrected values that clinicians had in front of them at the time. A versioned record addresses both problems. In monaOS every change is kept as a new version of the resource, so an evaluation can reconstruct what the record held at a given moment rather than what it holds today. Values invalidated as entered in error stay in the record and are marked as such, so an evaluation can decide explicitly whether to include them. Corrections and provenance covers this in detail.

Dataset shift#

A model evaluated on one population can degrade on another. Sources of shift include case mix, devices and measurement practice, documentation habits, treatment protocols and the passage of time as standards of care evolve. A model trained on data from one set of hospitals carries their practices inside it.

The defences are external validation on institutions not used for training, temporal validation on later years, and subgroup reporting, all of which need multi-site standardised data such as the federated research infrastructure provides.

Outcome definitions and metrics#

Headline metrics depend on how the outcome was defined: mortality at which horizon, from what starting point, and whether "improvement" is a score built from the model's own inputs.

The metrics that matter at the bedside are often not the ones in the abstract:

Metric What it tells you Why it can mislead
Area under the ROC curve How well the model ranks positives above negatives Says nothing about how many outputs a clinician must review per true finding
Positive predictive value How often an output is right Depends on prevalence, which differs between units
Calibration Whether a predicted 20 % risk means about 20 % Often unreported; poor calibration makes probabilities unusable
Outputs per patient-day Review burden on the team Rarely reported, and decisive for acceptance

Example: a simulation crossover study of documentation#

Clinomic's published documentation study is a randomised crossover study in a simulation setting (Peine et al., JMIR Medical Informatics, e44773). Sixty ICU professionals (physicians, ICU nurses and medical students) completed the same six tasks, covering documentation, retrieval of a laboratory trend and a SOFA score, on three systems: Mona, an established electronic chart and paper. The environment was a noisy simulated ICU with a fictitious patient, and the order of systems was randomised. Mona was tested in its voice-based prototype version.

Measure Mona (voice-based prototype) Established electronic chart Paper
Errors per participant, mean 0.28 0.75 2.37
Time for the six tasks, mean seconds 195 491 764
Raw NASA Task Load Index, mean (lower is better) 3.49 4.17 6.31

Read through the framework, the design is strong for what it measures: every participant used every system on identical tasks, which removes differences between participants from the comparison. Errors and time were significantly lower with Mona than with the electronic chart and with paper. Workload was significantly lower than with paper; against the electronic chart the difference was not significant (P = .06). As a simulation study it establishes task performance under standardised conditions, and the natural next steps are studies in live units.

Example: a before-and-after study of tele-ICU quality indicators#

Unpublished data, in review. Tele-ICU rounds between the Clinomic telemedicine centre and the ICU of a 140-bed primary care hospital moved from unstructured consultations to a protocol of ten telemedical quality indicators derived from the DIVI quality indicators for intensive care. Across 154 patients in 54 rounds (79 before, 75 after the protocol), the mean number of indicators met per patient rose from 3.59 to 6.59 of 10, and 8 of 10 indicators were met more often (P < .001).

This is a before-and-after implementation study of process quality. It shows that the structured protocol changed what was done and documented during rounds in that setting. Process indicators are chosen because they are linked to good intensive care practice; the study measures the process, which is the right endpoint for a protocol of this kind.

Example: VentAI#

VentAI is research investigating reinforcement learning for mechanical ventilation recommendations, specifically positive end-expiratory pressure, inspired oxygen fraction and tidal volume. The policy was learned and evaluated retrospectively on the MIMIC-III and eICU intensive care datasets. Its results describe how the learned policy relates to historical treatment and recorded outcomes in those databases, with the off-policy considerations described above. They do not demonstrate that following the policy prospectively improves outcomes, and no prospective mortality reduction has been shown. Retrospective work of this kind establishes plausibility and defines what a prospective evaluation would need to test.

What this means for deployment#

The evidence framework shapes how AI belongs in clinical workflows. Where evidence is retrospective or simulated, applications assist and a clinician decides: outputs go to a clinician who reviews, edits and remains responsible, as described in human review in assisted workflows. monaOS itself is an assistive tool that does not replace professional judgement; it is not a patient monitoring system, has no critical alarm functions and no diagnostic function.

The platform also provides what a careful live evaluation needs. An AI tool connected through the monaOS AI server reads the record read-only, with no write path in code, under query rules and fixed volume limits per access key, and every access is audited with parameter names and never patient values. That makes a silent evaluation on live data technically straightforward: the model sees data as it actually arrives and cannot change the record. The model also receives correct, time-aware clinical context. The mona AI page describes the platform, and the engineering essay on reading retrospective clinical AI results takes the retrospective question further.

Ihre Frage steht nicht in der Dokumentation?

Fragen Sie das Team, das die Plattform gebaut hat: welche Systeme sich anbinden lassen, welcher Arbeitsablauf betroffen ist, was in der Akte ankommen muss.