All projectsHealthcare

AI Lab Project

Applied AI application

Healthcare Referral Intake Assistant

Structured LLM extraction, rule retrieval and a local FHIR sandbox, with human-approved Task creation and synthetic-data evaluations.

System overview
Referral01
Evidence02
Review03
FHIR Task04
IntakeEvidence-led workflow
Synthetic evaluation set
40 referrals
Lexical, vector and hybrid
3 search modes
Before a Task is saved
Human approval

The challenge

From referral text to an approved task.

Referral information can be incomplete or ambiguous. An assistant must find eligible guidance, expose uncertainty and keep administrative checks and write permission in server code.

Product & engineering

What I delivered

A referral intake application where staff review extracted fields, missing information and rule evidence, then approve an exact FHIR Task draft. Evaluation uses synthetic referrals and administrative rules for a fictional clinic.

  1. 01

    Structured extraction

    Structured LLM extraction with deterministic labeled-field verification, plus distinct Fixture and Connected modes.

  2. 02

    Evidence retrieval

    Lexical, model-vector and hybrid rule retrieval, filtered by scope and effective date before ranking.

  3. 03

    Approval & FHIR

    A local FHIR R4 sandbox for Patient, ServiceRequest and Task, with exact human approval and durable commit idempotency.

  4. 04

    Bounded agent workflow

    A deterministic baseline and bounded agent variant, with permitted read tools, run traces and step, call, token and time limits.

  5. 05

    MCP & evaluation

    A read-only MCP search interface checked by an independent HTTP client, and a 40-case synthetic evaluation set with ten workflow controls.

System design

How the pieces connect

Evidence-led workflow
  1. 01

    Referral

    Synthetic notes and administrative fields.

  2. 02

    Evidence

    Extraction, rule retrieval and coded checks.

  3. 03

    Review

    A person inspects the exact Task payload.

  4. 04

    FHIR Task

    Server approval checks and idempotent commit.

Architecture & decisions

One Spring Boot service and Angular desk use PostgreSQL for rules, vectors, local FHIR resources, runs and committed Tasks. The baseline performs extraction, lookup, retrieval, coded checks and drafting. A bounded agent selects permitted read tools; the server owns Task preparation, approval validation and commit. MCP exposes rule search only.

Outcome & evidence

What the work produced

Delivered outcome

A completed intake workflow with recorded connected evaluations, an independent MCP client check, and approval/idempotency browser checks. Error analysis led to a deterministic extraction safeguard; the agent comparison supported keeping a simpler baseline.

Recorded evaluation

23 September 2026 · 40 synthetic referrals · 22 dev / 18 holdout · corpus clinic-admin-2026.09.1 · dataset referrals-2026.09.2 · gpt-4o-mini, extract-v3 and agent-v3.

Lexical recall@5
0.95
Model-vector and hybrid recall@5 were 1.00 on the same synthetic set, using 64-dimensional text-embedding-3-small vectors.
Connected extraction
40/40
Normalized exact fields after a prompt revision and deterministic checks. This is a regression result, not an untouched holdout score.
Fixture controls
10/10
Coded workflow checks, including approval, duplicate commits and failure paths. Fixture extraction replays gold labels; it does not measure LLM accuracy.
Agent vs baseline
3/3
Identical draft hashes in three live comparisons. The agent used more tokens; this small sample showed no outcome advantage.

An earlier prompt invented a member ID after an instruction embedded in a referral. The final prompt was revised after that holdout case was inspected. Citation checks verify eligible source and snippet membership, not semantic support for every claim. A fresh independent holdout is needed for broader quality claims.

Scope & context

Synthetic data and a local FHIR subset only: no real EHR integration, clinical or coverage decisions, production authentication or durable recovery of interrupted runs. Recorded measurements describe this small regression set and were not rerun on the portfolio site.