Ask AI the Right Way in Healthcare Workflows

Only 3% of Irish consumers aged 35–54 say they use AI regularly, according to the WIN World AI Index 2025. That’s a surprising number in a country packed with tech talent—and it’s a useful warning for healthcare leaders.

Because the real blocker in hospitals and clinics usually isn’t “lack of AI.” It’s lack of confidence: teams don’t know what they can safely delegate, what must stay human-led, and how to avoid the awkward middle where AI produces plausible nonsense and nobody notices.

Ivan Jennings, Tech Sales Leader at Red Hat Ireland, put a sharp edge on this challenge in a recent conversation: “Never ask AI something you don’t already know the answer to.” Taken literally, that sounds restrictive. In healthcare operations, it’s actually a practical operating rule: don’t treat AI as an oracle—treat it as an assistant you supervise.

This post sits in our “AI in Technology and Software Development” series, where we normally talk about automation, software workflows, cloud platforms, and security. Here, we’re applying those same engineering instincts to a healthcare reality: clinical and administrative AI only works when the workflow is designed for verification, auditability, and responsibility.

“Don’t ask AI what you can’t verify” (and why it matters in hospitals)

Answer first: In healthcare, the safest and most effective way to use generative AI is to assign it tasks where you can check the output against known facts, policies, or source documents.

Jennings’ line isn’t about limiting curiosity. It’s about avoiding a common failure mode: a model returns an answer that sounds confident, and a busy team accepts it because it reads well. In software development, that leads to bugs. In healthcare, it can lead to billing errors, privacy breaches, incorrect patient communications, or flawed clinical documentation.

Here’s the stance I take: If your workflow can’t support verification, you’re not “piloting AI,” you’re gambling.

The healthcare version of “known answers”

You don’t need to know the final sentence AI will write. You need a ground truth you can validate against:

• A local policy (infection control, referral rules, discharge letter format)
• A controlled dataset (a de-identified cohort for analytics)
• A trusted source document set (clinical guidelines you already use)
• A defined coding standard (ICD-10, SNOMED CT, procedure codes)
• A measurable output (time saved per claim, reduction in backlog)

If you can’t point to the reference, it’s too early to automate.

Asking better questions: the prompt isn’t the point—the workflow is

Answer first: Better prompts help, but workflow design is what turns AI from a toy into a dependable tool.

Many teams start with “What can AI do for us?” That’s backwards. Start with:

Where are we already doing structured work that’s slowed down by reading, rewriting, triage, or repetitive decisions?

Then design the AI step so it’s bounded, observable, and reversible.

A simple pattern that works: Draft → Check → Commit

This pattern shows up in high-performing software teams and it maps cleanly to healthcare:

1. Draft: AI produces a first pass (summary, letter, categorisation, extraction).
2. Check: A human verifies against sources and applies judgement.
3. Commit: The final output is saved with the right metadata and audit trail.

If you’re using AI in clinical or admin workflows and you don’t have an explicit “check” step, that’s the first fix I’d make.

The most useful healthcare prompts are not “questions”

Instead of “What’s the diagnosis?” (high risk), use instruction prompts that force traceability:

• “Extract the medication list from this discharge note and present it as a table. Include the sentence you extracted each item from.”
• “Draft a patient-friendly explanation of this MRI report at an 8th-grade reading level. Do not add new information. Flag unclear phrases.”
• “Classify these referrals into urgent/soon/routine using our policy rubric. Cite the rubric criterion used.”

Notice what’s happening: you’re asking AI to transform information, not invent it.

Responsible AI adoption in Irish healthcare: what “good” looks like

Answer first: Responsible healthcare AI in Ireland should be built around governance, privacy, and predictable infrastructure—not individual experimentation on sensitive data.

Ireland’s AI conversation is increasingly shaped by real operational concerns: trust, adoption, and readiness (exactly what the WIN World AI Index tries to measure). For healthcare orgs, the bar is higher because you’re operating under patient safety expectations and strict privacy obligations.

So what does “good” look like in practice?

1) Governance that’s practical, not performative

You need a lightweight, repeatable system that answers:

• Who approved this use case? (clinical lead + data protection + IT)
• What data is allowed? (PHI/PII rules, de-identification requirements)
• What’s the failure plan? (how you detect issues and roll back)
• What gets logged? (prompts, outputs, reviewers, timestamps)

A lot of AI governance docs read nicely and change nothing. The useful ones translate into checklists inside the workflow.

2) Privacy by design (especially with generative AI)

If you want adoption, staff must believe the system won’t burn them.

Rules that reduce risk fast:

• Keep patient-identifiable text out of general-purpose public tools.
• Use role-based access control and least privilege.
• Prefer architectures that support data residency and strong audit logs.
• Treat prompts and outputs as records that may require retention controls.

If your organisation can’t clearly state where prompts and outputs are stored, you’re not ready to scale.

3) Infrastructure you can operate at 2 a.m.

Healthcare is an always-on environment. AI features that require fragile integrations or manual babysitting won’t survive winter pressures.

From a technology and software development lens, aim for:

• Standardised APIs for EHR integration
• Monitoring for latency, drift, and error rates
• Clear versioning (model versions, prompt templates, policy rules)
• Security controls aligned with broader cyber hygiene

This is where Irish tech leadership perspectives (like Jennings’ from Red Hat) are helpful: reliable platforms beat clever demos.

Three AI best practices healthcare professionals should adopt now

Answer first: Start with small, verifiable workflows, track measurable outcomes, and enforce human sign-off for anything patient-facing.

These are practical habits that raise quality immediately.

Best practice #1: Use AI where the “right answer” is checkable

Good first targets:

• Clinic letter drafting using an approved template
• Discharge summaries that must match the clinician’s note
• Claims pre-checks against published reimbursement rules
• Triage support where final decision stays human

Avoid starting with tasks that require the model to be a clinician.

Best practice #2: Turn “AI quality” into numbers, not vibes

Pick metrics that matter to operations:

• Minutes saved per document
• Reduction in backlog (e.g., referrals, coding queue)
• Error rate detected in review (before/after)
• % of outputs requiring major rewrite
• Time-to-first-response for patient messages

If nobody is measuring, you can’t defend the programme—or improve it.

Best practice #3: Build a “human-in-the-loop” step that’s explicit

A checkbox buried in a UI isn’t enough. You want an intentional moment where a clinician or admin confirms:

• The output matches source documents
• No new facts were invented
• Tone is appropriate for patient-facing comms
• Sensitive data is handled correctly

Make it visible. Make it auditable. Make it normal.

Common objections (and the answers that hold up)

Answer first: The right response to AI risks isn’t avoidance—it’s bounded use with strong review.

“If we have to review everything, what’s the point?”

Review is cheaper than rework. And drafting is the most time-consuming part of many admin tasks.

When AI reliably produces a 70–80% draft that staff can verify quickly, you’re buying back time without sacrificing control.

“Our staff won’t use it.”

Staff won’t use tools that increase personal risk. Adoption improves when you:

• Publish clear dos and don’ts
• Provide safe, approved tooling
• Offer short training on what AI is good at (and what it’s bad at)
• Protect users with governance that doesn’t blame individuals

“We’ll wait until the tech is more mature.”

Healthcare waiting lists and admin backlogs won’t pause. The organisations that do well won’t be the ones with the most advanced model. They’ll be the ones with the best workflow engineering.

A practical next step: run a 30-day “verifiable AI” pilot

Answer first: A strong pilot proves one workflow end-to-end with governance, metrics, and review built in.

If you’re responsible for operations, digital transformation, or clinical informatics, here’s a straightforward pilot structure:

1. Pick one workflow (example: outpatient clinic letters).
2. Define the ground truth (approved templates + clinician notes + local policy).
3. Set review rules (who signs off, what gets flagged, what’s forbidden).
4. Measure outcomes weekly (time saved, rewrite rate, error catches).
5. Decide scale or stop based on data, not enthusiasm.

That approach aligns with how mature software teams ship features: small releases, tight feedback loops, measurable improvement.

Most companies get this wrong by starting with the model. Start with the workflow. Then the model becomes replaceable—and that’s a good thing.

As Ireland’s AI readiness continues to be debated, healthcare teams have a chance to lead by example: use AI where it’s verifiable, govern it like any other clinical system, and design it so humans stay accountable.

What workflow in your organisation is already structured enough that AI could draft the first pass—and your team could confidently verify it?