Clinical Software and AI (CA)

Clinical Software and AI (CRI-CA), The Patient: a radiograph of a brain outline with a network of nodes.

Taxonomies › Hospitals and medical devices › Clinical Software and AI

The Patient · CRI-CA · suit 08 of 17 · 13 entries

Software as a medical device, decision support, dose calculators, early-warning and sepsis scores, imaging AI, AI scribes and the apps clinicians reach for.

Scope: Software as a medical device, clinical decision support, drug interaction checking, dose calculators, early-warning and sepsis algorithms, imaging and triage AI, model updates, data drift, adversarial inputs, automation bias, unvalidated spreadsheets and apps, ambient scribes writing into the record, consumer apps used clinically.

Entries are ranked Ace (most severe) to 2. In the taxonomy the ID is a stable reference, not a severity: rate each finding in your own system.

Clinical Software and AI: each entry's ID, title, threat, mitigation principle and design measures. The full text follows below.

Text version: every entry is written out below, with its card prompts and mappings. The same text is in the PDF and Excel downloads on the overview page.

The threats

CRI-CA-A Shared clinical knowledge base corrupted

† Wrong or delayed treatment

Threat. Decision support, drug interaction checks and order sets draw on a central rule base or a third-party drug database. One corrupted rule or update gives wrong advice to every prescriber at once, and the advice looks authoritative.

Principle. Clinical rules must be changed and updated like a medical device, with independent verification.

Design measures:

  1. Verify the signature and provenance of every knowledge base update before loading it.
  2. Test each update against a fixed set of clinical cases and block it if any result changes unexpectedly.
  3. Keep the previous version ready to restore and record who approved each change.

Card prompts:

  • Who verifies changes to the clinical rule base before release?
  • How many prescribers would one corrupted rule reach?
  • How are third-party drug database updates checked?

Mappings: STRIDE T · CAPEC-186, CAPEC-184 · CWE-494, CWE-684 · ATT&CK T1565.001, T1195.002 · MDS2 IGAU, CSUP, AUDT · IEC 62304; ISO 14971; IEC 81001-5-1

CRI-CA-K Dose calculators trusted blindly

† Wrong or delayed treatment

Threat. Weight-based and infusion dose calculators accept whatever inputs they are given. A weight in the wrong unit, a moved decimal or a changed concentration produces a tenfold dose that looks correct on screen.

Principle. A calculator that sets a dose must check its inputs against what is plausible for the patient.

Design measures:

  1. Take weight and concentration from verified sources, not free text, and show their origin.
  2. Apply hard and soft limits by age, weight and drug, and require a reason to exceed them.
  3. Show units explicitly at every step and in the final result.

Card prompts:

  • What plausibility checks does the calculator apply to its inputs?
  • Which units does each input expect, and is that shown?
  • Would a tenfold dose look any different on screen?

Mappings: STRIDE T · CAPEC-153 · CWE-1287, CWE-1288 · MDS2 IGAU · IEC 62304; ISO 14971

CRI-CA-Q Early-warning scores fed bad inputs

† Wrong or delayed treatment

Threat. Deterioration and sepsis algorithms score whatever vital signs and results reach them. Missing, delayed or altered inputs, or a raised threshold, mean a deteriorating patient is never flagged.

Principle. An algorithm must know when its inputs are stale, missing or implausible and say so.

Design measures:

  1. Show the age and source of every input beside the score.
  2. Flag a score as incomplete when inputs are missing or late instead of treating them as normal.
  3. Control and audit threshold changes, with clinical sign-off.

Card prompts:

  • How does the algorithm show that inputs are stale or missing?
  • Who can change scoring thresholds, and is it reviewed?
  • What alerts staff when an expected input never arrives?

Mappings: STRIDE T, D · CAPEC-153 · CWE-754, CWE-1288 · ATT&CK T1565.002 · MDS2 IGAU, AUDT · IEC 62304; ISO 14971

CRI-CA-J Imaging AI fooled by crafted images

† Wrong or delayed treatment

Threat. Triage and detection models read images that pass through many systems. Small, deliberate changes to an image can make a model miss a bleed or a tumour, or invent one, with no visible sign to a human reader.

Principle. An imaging model must be tested against manipulated inputs and never be the only reader.

Design measures:

  1. Check image integrity from modality to model before inference.
  2. Test models with adversarial and corrupted inputs during validation and after every update.
  3. Keep a clinician read on every study the model triages as normal for the highest-risk findings.

Card prompts:

  • Has the imaging model been tested against manipulated images?
  • Is the model ever the only reader of an image?
  • How would a missed finding by the model be caught?

Mappings: STRIDE T · CAPEC-153 · CWE-1039 · ATT&CK T1565.001 · MDS2 IGAU · FDA premarket; ISO 14971

CRI-CA-10 Model updates change care unannounced

† Wrong or delayed treatment

Threat. Cloud-hosted software as a medical device can receive a new model version without the hospital knowing. Its performance on local patients shifts, and nobody compares outputs before and after.

Principle. A change to a clinical model must be visible to the hospital and checked on local data.

Design measures:

  1. Require notice and release notes for every model change in the supply contract.
  2. Display the model version with every output and record it in the patient record.
  3. Monitor agreement and alert rates locally, and alarm on a sudden shift.

Card prompts:

  • How does the hospital learn that a model version has changed?
  • Are outputs compared on local patients before and after updates?
  • Who can pause a model whose performance has shifted?

Mappings: STRIDE T, R · CWE-1357, CWE-684 · MDS2 CSUP, SGUD · FDA premarket; IEC 62304

CRI-CA-9 Automation bias hides the failure

† Wrong or delayed treatment

Threat. Interfaces present algorithm outputs as certain, without confidence or evidence. Clinicians stop checking, so any error, drift or compromise in the software flows straight into treatment.

Principle. Clinical software should show how sure it is and invite the clinician to check.

Design measures:

  1. Show confidence, the main inputs and known limits beside every recommendation.
  2. Design workflows so the clinician records a decision, not just an acceptance.
  3. Audit acceptance rates and investigate units where nearly every suggestion is accepted.

Card prompts:

  • Does the interface show how confident the algorithm is?
  • What prompts the clinician to check an output before acting?
  • How would drift or compromise in the software become visible?

Mappings: STRIDE T · CAPEC-148 · CWE-451 · MDS2 IGAU, SGUD · ISO 14971

CRI-CA-8 AI scribes write errors into records

† Wrong or delayed treatment

Threat. Ambient and speech scribes, including agent-driven tools, turn conversations into notes, allergies and medication lists. Mishearing, prompt injection from spoken or pasted text, or invented details enter the record and drive later orders.

Principle. Nothing generated should enter the record until a clinician has checked and accepted it.

Design measures:

  1. Hold generated text as a draft that a named clinician must review and sign.
  2. Highlight generated content that changes allergies, medications or diagnoses for explicit confirmation.
  3. Tell patients when recording and generation are used and let them decline.

Card prompts:

  • Who checks a generated note before it enters the record?
  • Could spoken or pasted text steer what the scribe writes?
  • Which orders could be driven by an unreviewed generated entry?

Mappings: STRIDE T, R · LINDDUN U, DD · CWE-1427, CWE-1434 · MDS2 IGAU, AUDT, MPII · IEC 62304; ISO 14971

CRI-CA-7 Models leak the patients they learned

Threat. Models trained or fine-tuned on hospital data can reveal whether a person was in the training set, or reproduce fragments of their records, to anyone who can query them.

Principle. Patient data used to build a model must not be recoverable from the model.

Design measures:

  1. Train on de-identified data wherever possible and test models for memorisation before release.
  2. Limit who can query models and how often.
  3. Record which datasets trained each model and the legal basis for using them.

Card prompts:

  • Could someone tell from the model whether a patient was in training?
  • Who can query the model, and how often?
  • Was the training data reduced or protected before use?

Mappings: STRIDE I · LINDDUN I, L, DD, Nc · CWE-359 · MDS2 DIDT, MPII · ISO 27799

CRI-CA-6 Unvalidated spreadsheets run clinical decisions

† Wrong or delayed treatment

Threat. Wards and pharmacies build their own calculators, macros and trackers. They have no version control or testing, a changed cell alters every result, and macro-enabled files are an easy way to deliver malware.

Principle. Any tool that informs a clinical decision needs an owner, a version and a test.

Design measures:

  1. Register local clinical tools, name an owner and validate them before use.
  2. Lock formulas and store approved versions in a controlled location.
  3. Block macros from untrusted locations on clinical workstations.

Card prompts:

  • Which local spreadsheets or macros inform clinical decisions?
  • Does each one have an owner, a version and a test?
  • Who could change a formula without anyone noticing?

Mappings: STRIDE T, E · CAPEC-17 · CWE-1236, CWE-15 · ATT&CK T1204.002 · MDS2 IGAU, MLDP · IEC 62304

CRI-CA-5 Consumer apps used for clinical work

Threat. Staff use unregulated apps for doses, wound photos and messaging because they are quick. Patient images and details sync to personal clouds, and app results carry no validation.

Principle. Make the approved tool the easiest one to use for the task.

Design measures:

  1. Provide approved apps for photos, calculations and messaging on managed devices.
  2. Keep clinical photos in a managed container that never syncs to personal storage.
  3. Publish which apps are approved for which clinical tasks.

Card prompts:

  • Which consumer apps do staff use for clinical work, and why?
  • Where do photos and details from those apps sync?
  • Is the approved tool as quick as the app staff prefer?

Mappings: STRIDE I · LINDDUN DD, Nc, U · CWE-359, CWE-922 · ATT&CK T1567.002 · MDS2 STCF, MPII · ISO 27799

CRI-CA-4 Cloud clinical software unreachable

† Wrong or delayed treatment

Threat. Reporting, triage and decision software increasingly runs in a vendor’s cloud. When the link, the vendor or its cloud is down, the pathway has no fallback and clinical work waits.

Principle. A clinical pathway must keep working, at a known reduced level, when its cloud software is gone.

Design measures:

  1. Define and rehearse a manual fallback for each cloud-dependent pathway.
  2. Require service levels and outage notice from the vendor, and monitor availability yourself.
  3. Keep essential reference data available offline.

Card prompts:

  • What happens to the pathway when the vendor’s cloud is down?
  • What reduced level of service is agreed, and rehearsed?
  • How long can clinical work wait for the software to return?

Mappings: STRIDE D · CWE-1357 · ATT&CK T1498, T1489 · MDS2 CONN, DTBK · IEC 80001-1

CRI-CA-3 Alert overrides leave no trail

Threat. Clinicians override drug interaction and allergy alerts many times a day. Without a recorded reason, nobody can tell a justified override from a dangerous one, or prove later what was shown.

Principle. Every override of a safety alert should record who, what and why.

Design measures:

  1. Require a short coded reason for overriding high-severity alerts.
  2. Record the alert text shown, with its version, alongside the override.
  3. Review override patterns and retire alerts that are always overridden.

Card prompts:

  • Is a reason recorded each time a safety alert is overridden?
  • Could you later prove what alert a clinician saw?
  • Who reviews override patterns for dangerous habits?

Mappings: STRIDE R · CWE-778, CWE-223 · MDS2 AUDT · IEC 62304

CRI-CA-2 Algorithms expose sensitive inferences

Threat. Models infer pregnancy, HIV status, mental illness or addiction from routine data and show the inference to anyone viewing the patient. The patient never disclosed it and cannot see or contest it.

Principle. Inferred sensitive information deserves the same protection as information the patient disclosed.

Design measures:

  1. Classify inferred conditions with the same sensitivity as recorded ones and restrict their display.
  2. Label inferences as such and let clinicians suppress them.
  3. Tell patients which inference tools are in use.

Card prompts:

  • Which sensitive conditions can the models infer from routine data?
  • Who can see an inference the patient never disclosed?
  • Can patients see or contest what was inferred about them?

Mappings: STRIDE I · LINDDUN I, DD, U · CWE-359, CWE-1220 · MDS2 AUTH, MPII · ISO 27799

Seen in the wild

No entry in this suit cites a public case yet. The overview page lists every case in the taxonomy.

Mappings

Clinical Software and AI: the mappings of each entry to STRIDE, LINDDUN, CAPEC, CWE, ATT&CK, EMB3D, MDS2 and standards.

Across the 13 entries, the STRIDE counts are Tampering 8, Repudiation 3, Information disclosure 3, Denial of service 2, Elevation of privilege 1 (an entry can carry more than one). 6 map to CAPEC 3.9, 13 to CWE 5.0, 6 to ATT&CK v20.0 and 4 to LINDDUN. None map to MITRE EMB3D. The MDS2 capabilities a buyer should check first are IGAU (7), AUDT (4), MPII (4). 9 carry the amber † for wrong or delayed treatment. A blank means nothing fitted well; a loose tag would mislead.

← Laboratory and Diagnostics · Hospitals and medical devices overview · Records and Patient Data →

© 2026 Jonathan Brett Crawley. Licensed under CC BY-NC-ND 4.0 with additional permissions: see the licence on the hospitals and medical devices overview.