Safe AI agents on patient and financial data: a practical checklist

Published 22 October 2026 · dotNew Labs

AI agents become useful when they can look things up and take actions in real systems. In healthcare and finance that is exactly where the risk sits. The controls below are not exotic. They are the same ideas you already apply to people, applied to software that acts on someone's behalf.

Each request passes the same checks a careful employee would, and leaves a record.

1. The agent acts as a real person

Every request should carry the signed-in user's identity, and the agent should get that user's permissions and nothing more. A shared service account that can read every record turns one clever prompt into a data breach.

2. Start read-only, add actions slowly

Begin with lookups and summaries.
Add write actions one at a time, each with a clear name and narrow scope.
Require a person to approve anything that changes a record, moves money or contacts a customer or patient.

3. Send the minimum data

HIPAA's minimum-necessary idea applies well beyond healthcare: give the model only the fields the task needs. Mask identifiers where the task doesn't need them, and keep full records inside your systems.

4. Record everything

Log who asked, which tools the agent called, with what inputs, and what came back. Keep logs where your security team already looks, and sample them every week while the agent is new.

5. Plan for prompt injection

Text inside a document, email or web page can try to instruct the agent. Treat anything the agent reads as data, not orders. Limit what any single action can do, so a tricked agent can't do much harm.

6. Check the paperwork

Confirm your agreement with the cloud or AI provider covers the specific model service you use, including a BAA where patient data is involved.
Know where data is processed and stored, and whether it is used for training. Choose settings and contracts where it isn't.
Add the agent to your risk assessment like any new system.

7. Test before and after go-live

Write a set of realistic questions with known right answers, including ones the agent should refuse. Run them before launch and after every change to the model, prompts or tools.

Not sure where your apps stand?

We offer a free 20-minute scan of one app: what it runs on, what's risky, and the shortest safe path forward. You keep a short written summary whether or not you hire us.

Book a free scan

Or email contact@dotnewlabs.ai