Internal research assistant
Scoped internal AI product with audit trail, PHI-aware retrieval, and feature flags. Rolled out to research staff in phased cohorts.
- Healthcare analytics
- Platform
- 12 weeks
- Internal product
- Audit trail
- Feature flags
- Phased rollout
Best fit when.
- 01PHI or equivalent regulated data requires entitlement-aware retrieval — the model never sees what the user cannot see.
- 02Every interaction must produce an audit record sufficient for an external review.
- 03Rollout must be reversible per cohort: a single group can return to the previous tooling without affecting others.
What was happening.
A research team wanted an internal assistant that could answer questions across study protocols, results, and operating documentation. The constraint was not capability — the constraint was governance. PHI exposure, audit trails, and a defensible rollout path were the real engineering work.
What we were holding to.
- Retrieval had to be PHI-aware: documents were tagged, and the assistant could not surface PHI to roles without the right entitlements.
- Every model interaction needed an audit record sufficient for an external review.
- Rollout had to be reversible at the cohort level — if a single research group hit a problem, that group came back to the previous tooling without affecting others.
How we built it.
Entitlement-aware retrieval
Documents were tagged with PHI sensitivity and the role classes allowed to see them. Retrieval filtered against the requesting user's entitlements before the model ever saw a candidate. The model could not 'forget' a filter; it never received the redacted material.
Audit trail as a first-class output
Every interaction produced a structured audit record — query, retrieved candidates, model output, role context — written to an append-only store with retention aligned to the client's policy. Audits ran weekly against that store.
Phased cohort rollout behind flags
Feature flags were keyed by research group. Rollout proceeded one group at a time. Each cohort transition was a documented decision with a written rollback procedure, not a deploy event.
What we left with the client.
- Deployed internal assistant with entitlement-aware retrieval.
- Append-only audit store with weekly audit run.
- Feature-flag configuration and a documented cohort rollout plan.
- Joint ownership: platform engineering for the system, data governance for the policies.