‘Incredibly dangerous’: Woman says the hospital gave her a treatment plan for her condition. A closer look revealed it was written by ChatGPT

A woman who went to the hospital for care said she later discovered her at-home treatment plan and discharge paperwork appeared to come straight from ChatGPT. Tara Rule, who posts on TikTok as @tara_rule, shared the video that has gained hundreds of thousands of views.

In the video, Rule holds up the papers and points out a printed ChatGPT interface. She notes that the printout includes a sidebar of previous searches. Those searches include “emergency consultation for a medical Spanish interpreter,” “translate Spanish phrases,” and “aftercare instruction requests.”

Rule said in the TikTok that she is medically complex and has a rare genetic disorder along with other conditions. She stated that typical treatment plans are not only ineffective for her but can be “incredibly dangerous and sometimes lethal.”

Federal rules on AI medical devices and patient privacy

Rule’s caption on the video states that the discharge instructions, had she followed them blindly, would have likely ended up in bowel perforation given her history, genetic disorder, scar tissue, and previous surgeries. She attributes that assessment to other doctors she asked.

She explained in the video that she could use ChatGPT herself but chooses not to for her care. Rule said it is better for her to see an actual doctor who went to college for 12 or so years to make sure everything is okay and put her on the right course. She described the situation as one where “we’re officially cooked” and called it a dystopian time. Her caution echoes other cases where ChatGPT advice made a car problem worse.

The Department of Health and Human Services Privacy Rule sets standards for the use and disclosure of protected health information by covered entities such as certain health care providers, health plans, and clearinghouses. Covered entities generally must limit uses and disclosures to the minimum necessary to accomplish the intended purpose, with exceptions that include treatment purposes and disclosures to the individual.

Viewers of the video questioned whether the hospital should have used a general chatbot for her paperwork. One commenter asked, “HOW is this legal!??” Another raised privacy concerns and wrote, “Massive HIPAA violation by entering your info into ChatGPT!” A third urged her to seek legal advice, saying, “Begging u to speak with a lawyer.”

Some commenters shared their own experiences with AI in health care settings. One who works in radiology IT stated, “Pretty much all diagnostic imaging reports are AI generated at this point, including findings.” Another noted, “My doctor googled in front of me.”

Rule said that she would take legal action against the hospital. When a commenter asked, “So are you gonna sue or what?” Rule replied, “I always sue.”