Healthcare Referral Intake Assistant
Structured LLM extraction, rule retrieval and a local FHIR sandbox, with human-approved Task creation and synthetic-data evaluations.
- Synthetic evaluation set
- 40 referrals
- Lexical, vector and hybrid
- 3 search modes
- Before a Task is saved
- Human approval
The challenge
From referral text to an approved task.
Referral information can be incomplete or ambiguous. An assistant must find eligible guidance, expose uncertainty and keep administrative checks and write permission in server code.
Product & engineering
What I delivered
A referral intake application where staff review extracted fields, missing information and rule evidence, then approve an exact FHIR Task draft. Evaluation uses synthetic referrals and administrative rules for a fictional clinic.
- 01
Structured extraction
Structured LLM extraction with deterministic labeled-field verification, plus distinct Fixture and Connected modes.
- 02
Evidence retrieval
Lexical, model-vector and hybrid rule retrieval, filtered by scope and effective date before ranking.
- 03
Approval & FHIR
A local FHIR R4 sandbox for Patient, ServiceRequest and Task, with exact human approval and durable commit idempotency.
- 04
Bounded agent workflow
A deterministic baseline and bounded agent variant, with permitted read tools, run traces and step, call, token and time limits.
- 05
MCP & evaluation
A read-only MCP search interface checked by an independent HTTP client, and a 40-case synthetic evaluation set with ten workflow controls.
System design
How the pieces connect
- 01
Referral
Synthetic notes and administrative fields.
- 02
Evidence
Extraction, rule retrieval and coded checks.
- 03
Review
A person inspects the exact Task payload.
- 04
FHIR Task
Server approval checks and idempotent commit.
Product interface
- Angular
Services & interfaces
- Java
- Spring Boot
- FHIR R4
- MCP
Data & search
- PostgreSQL
- pgvector
One Spring Boot service and Angular desk use PostgreSQL for rules, vectors, local FHIR resources, runs and committed Tasks. The baseline performs extraction, lookup, retrieval, coded checks and drafting. A bounded agent selects permitted read tools; the server owns Task preparation, approval validation and commit. MCP exposes rule search only.
Outcome & evidence
What the work produced
Delivered outcome
A completed intake workflow with recorded connected evaluations, an independent MCP client check, and approval/idempotency browser checks. Error analysis led to a deterministic extraction safeguard; the agent comparison supported keeping a simpler baseline.
23 September 2026 · 40 synthetic referrals · 22 dev / 18 holdout · corpus clinic-admin-2026.09.1 · dataset referrals-2026.09.2 · gpt-4o-mini, extract-v3 and agent-v3.
- Lexical recall@5
- 0.95
- Model-vector and hybrid recall@5 were 1.00 on the same synthetic set, using 64-dimensional text-embedding-3-small vectors.
- Connected extraction
- 40/40
- Normalized exact fields after a prompt revision and deterministic checks. This is a regression result, not an untouched holdout score.
- Fixture controls
- 10/10
- Coded workflow checks, including approval, duplicate commits and failure paths. Fixture extraction replays gold labels; it does not measure LLM accuracy.
- Agent vs baseline
- 3/3
- Identical draft hashes in three live comparisons. The agent used more tokens; this small sample showed no outcome advantage.
An earlier prompt invented a member ID after an instruction embedded in a referral. The final prompt was revised after that holdout case was inspected. Citation checks verify eligible source and snippet membership, not semantic support for every claim. A fresh independent holdout is needed for broader quality claims.
Scope & context
Synthetic data and a local FHIR subset only: no real EHR integration, clinical or coverage decisions, production authentication or durable recovery of interrupted runs. Recorded measurements describe this small regression set and were not rerun on the portfolio site.