Who's accountable when healthcare AI makes a mistake?

Consider this scenario: a radiologist reviews 200 chest X-rays daily with AI assistance. On scan #147, the AI misses a small tumour. The doctor, trusting the AI's recommendation, moves on. Three months later, the patient returns with stage 3 cancer.

It's hypothetical, but the question it raises is not. Who's liable?

According to Ireland's Medical Council, the answer is clear: the doctor is¹.

The hospital's quarterly audit will not catch it for months. The AI vendor's logs show "functioning normally." Yet the Medical Council's October 2025 position statement states doctors "ultimately remain responsible for their clinical decisions"¹.

The problem is confidence. How can doctors stand behind AI tools they cannot inspect? How do they remain accountable when a failure leaves no signed trail?

The accountability gap

AI is already in clinical workflows across Irish healthcare. Radiology departments use it to flag abnormalities. Pathology labs use it for tissue analysis. Emergency departments use it for triage. Public-sector digital health programmes are expanding that footprint.

The Medical Council says AI should "augment, rather than replace, clinical decision-making"¹. That principle creates a practical gap:

• Doctors are responsible for AI-assisted decisions
• Many clinical AI systems are opaque at the point of use
• Traditional monitoring usually finds failures after harm has already occurred

As Jantze Cotter, Executive Director of Regulatory Policy at the Medical Council, noted: "AI advancements hold great potential for the medical field but also introduce significant ethical, legal, regulatory and professional challenges"¹.

Why traditional monitoring fails

Healthcare organisations still lean on periodic audits and retrospective reviews. Those approaches miss failures that only show up in the moment of care.

Four failure modes that quarterly review is slow to catch:

Loop drift

The model settles into repetitive diagnostic patterns. Clinical impact is repeated miss or overcall on the same class of finding. Aggregate trend reports see it late. A signed receipt on every decision makes the repetition reconstructible case by case.

Bias amplification

Underdiagnosis concentrates in underrepresented populations. Catching it needs demographic review across cases, not a single incident file. Receipts that carry model version, policy, and outcome give that review a starting point without reconstructing vendor logs after the fact.

Model drift

Accuracy degrades quietly as data or deployment conditions change. Gradual shifts rarely trip a quarterly KPI. Signing the model version on every receipt makes "which system ran on that patient" a fact, not a guess.

Integration failures

The model receives incomplete or corrupted patient data. Data-quality faults are rarely logged in a form an auditor can export. Signing the decision inputs into a verifiable receipt means you can show what the system saw when it acted.

The Medical Council acknowledges: "Doctors must have confidence in the standard of the tool they are utilising"¹. Confidence needs contemporaneous visibility. Most healthcare AI still does not leave a signed, independently verifiable record of what it did.

EU AI Act requirements

Clinical AI typically sits in the EU AI Act's high-risk category. Annex III high-risk duties take effect 2 December 2027 under the Digital Omnibus agreement (Council final adoption 29 June 2026, pending OJ entry into force)². Providers and deployers should expect duties that include:

• Risk management with continuous monitoring
• Training-data quality and representativeness
• Technical documentation that supports reliability claims
• Human oversight with meaningful intervention
• Transparency so users can interpret outputs
• Accuracy, robustness, and cybersecurity across the lifecycle

Non-compliance can reach €15 million or 3% of global annual turnover². The harder question for a hospital is not the fine schedule. It is how clinicians keep authority while the system stays fast enough for the ward.

What accountability needs in practice

The Medical Council's principles map to concrete technical requirements.

1. Transparency and auditability

"Patients must be informed when AI tools are used"¹. Organisations need:

• Complete audit trails showing when AI was used, what it processed, and what it recommended
• Explainability clinicians can use with patients
• Version tracking for the model behind each decision

2. Human-in-the-loop

"AI should augment, not replace, clinical decision-making"¹:

• Mandatory review points where clinicians confirm recommendations
• Override paths that preserve clinical judgment
• Escalation when confidence is low or results are ambiguous

3. Bias detection

AI "could reinforce bias, particularly affecting vulnerable groups"¹. A widely used US healthcare risk algorithm assigned Black patients lower risk scores than equally sick white patients, because it used past cost as a proxy for need³. Requirements include demographic monitoring, regular bias review across populations, and scrutiny of training-data coverage.

4. Real-time safeguards

Retrospective review alone is too late for bedside risk. Useful controls include loop detection, anomaly flags before harm accumulates, and circuit breakers that pause a system when safety thresholds are breached.

Ireland's position

Ireland sits inside the EU regime, English-language clinical practice, and a dense health-tech supplier base. That combination makes the Medical Council statement and the AI Act timetable land on the same organisations at once.

Teams that sign clinical AI decisions as they happen can show a doctor, a hospital board, or a regulator the same contemporaneous record. Teams that deploy without that trail inherit an accountability gap that is expensive to reconstruct later.

Closing the accountability gap

If doctors are accountable for AI-assisted decisions, they need visibility into what those systems did in the moment, not a quarterly summary. Mapped to the Medical Council's principles and high-risk AI Act duties, that usually means:

• Loop and budget breakers that stop repetitive or runaway behaviour before it compounds
• Signed audit trails covering timestamp, model, input hash, policy, and outcome
• Human-review paths where overrides land in the same signed trail as the model output
• Exportable evidence a reviewer can verify without trusting the vendor's console
• EU data residency with prompt and response text kept out of the signed receipt, so evidence can move without moving patient content

That is the job Seal is built for: signed receipts for AI decisions, produced before the action executes, verifiable without Aqta.

The accountability gap in healthcare AI is real. The choice is whether you build the trail in from the start or try to invent it after something goes wrong.