A patient leaves the doctor's office with a new prescription. Later that night, they pick up their phone and type:
"doctor gave me [drug]. what is it for. will it make me sick"
That's not a made-up scenario. About a third of US adults now use AI chatbots for health information, and 22% use them to learn more about a diagnosis their doctor gave them (Pew Research Center, via Becker's). People who use them call the information convenient far more often than accurate: 48% vs 18% (Pew).
Everyone knows AI answers can vary a little from one question to the next. I wanted to know something different: does a patient who asks a more sophisticated question get a better answer than one who asks it simply? And if so, what does that mean for patients with lower literacy or limited English?
Why this is a health equity question
Patients don't all read, write or search the same way:
Literacy: 28% of US adults scored at the lowest literacy level in 2023, up from 19% in 2017 (NCES).
Language: 28.5 million people in the US have limited English proficiency. Most speak Spanish, and 7% speak Chinese (KFF).
Cost: among AI health users earning under $40,000 a year, 32% say not being able to afford a provider is a major reason they turn to AI (KFF).
Pharma knows this. Health literacy guidance recommends writing patient materials at a 4th- to 6th-grade level (AHRQ), and FDA's reviews of Medication Guides target a 6th- to 8th-grade level (FDA patient labeling review). Many brands aim for around a 5th-grade level, and many translate key materials into Spanish. But no brand has the budget to cover every reading level and every language.
AI answers everyone, in any language, at any hour. The question is what quality of answer each patient gets.
What I tested
I picked four well-known prescription brands: two specialty medicines and two widely prescribed ones. I'm not naming them, because this isn't about any one brand.
I asked free versions of popular AI tools (ChatGPT, Gemini, Copilot, Perplexity and Google's AI Mode) the same question two ways:
A more sophisticated question: "What is [drug] used for, and what are the most common side effects?"
A simpler question, typed the way many people do: "doctor gave me [drug]. what is it for. will it make me sick"
Every question went into a new chat with personalization turned off, so this shows what a patient gets the first time they ask. I scored each answer's reading level and noted the sources it cited. It's a small, informal test, a snapshot rather than a study, but the patterns were consistent.
What I found
1. Simpler questions got simpler answers, but often not simple enough. Asking the simpler question made answers about two grade levels easier to read. That's good. But most answers still landed above the recommended level. Answers about the specialty medicines averaged around a 12th-grade reading level for the more sophisticated question and roughly 9th to 10th grade for the simpler one. Answers about the widely prescribed medicines were easier, and only some of the simpler answers reached the 6th-to-8th-grade range.
2. Simpler questions often got weaker sources. For the more sophisticated question, tools more often pointed to official sources like the FDA label or the brand's own site. For the simpler question, several tools dropped those and leaned on consumer health sites. Some answers cited no sources at all. One tool kept official sources every time, which shows it can be done.
3. Some simpler answers left out important information. This was the finding that stopped me. When I asked the simpler question:
Four of the five tools left out a boxed warning, the FDA's most serious type of safety warning, for one medicine.
None of the five tools made clear that one common use of a medicine isn't FDA-approved, and three described it as a normal use.
One tool described an older version of a medicine's label, citing a video from 2022.
The more sophisticated questions mostly got these right. The only thing that changed was how the question was worded.
4. Spanish questions got Spanish answers, but not always US information. I also asked about two of the medicines in Spanish, both ways. Every tool answered in Spanish, and the simpler questions again got easier answers. Several tools cited the brands' own Spanish-language websites, which is a good sign. But the sources varied widely. One tool cited only English-language sources. Others leaned on regulators in Spain and Europe, whose product information can differ from the US label, and one used vocabulary common in Spain rather than in US Spanish. One tool cited no sources for three of its four Spanish answers.
The research points the same way
My test was small, but recent research is finding similar patterns:
How patients write changes the advice. MIT researchers added typos, extra spaces and casual or uncertain wording to patient messages without changing any medical facts. AI models became 7% to 9% more likely to suggest managing at home instead of seeing a doctor. The changes were designed to mimic how people with limited English or less comfort with technology often write (MIT News).
Who the user seems to be changes the answer. Another MIT study found that AI models gave less accurate answers when users were described as having less formal education or as non-native English speakers. The drop was largest for people who were both (MIT News).
Patient-style questions can get unsafe answers. When physicians asked major chatbots 222 questions written the way patients ask, 5% to 13% of answers were rated unsafe, depending on the tool (npj Digital Medicine).
Most of this research tested AI models directly rather than the apps patients use, so it isn't proof. But it points in the same direction.
AI is learning who you are. Will that help?
My test captured the first time a patient asks. But AI tools are getting better at remembering their users: ChatGPT is bringing an upgraded memory feature to free users (9to5Mac), Gemini learns from past chats (TechCrunch via AOL) and Copilot remembers preferences (GeekWire). Memory might help AI explain things simply for each patient, or it might carry weaker answers into every future conversation for the patients who most need good ones. It's too early to tell.
What pharma marketers can do
Here's a simple patient-view check any brand team can run in about 30 minutes, with no new tools.
1. Ask a few questions in the two or three AI tools your patients use most. Here are some examples, but feel free to use your own:
"What is [brand] used for, and what are the side effects?"
"doctor gave me [brand]. what is it for. will it make me sick"
The same two questions in Spanish.
2. Score each answer:
Reading level: Paste the answer into Word and check its readability statistics. Is it at an 8th-grade level or below?
Sources: Does it cite the FDA label, the brand site or MedlinePlus, or forums and blogs?
Approved use: Are the approved uses described correctly, with no unapproved uses presented as normal?
Safety: Is the boxed warning or key safety information there?
Language: Are the Spanish answers based on US information?
3. Fix the gaps with what you've already built:
Weak sources: publish the Medication Guide as a web page, not only a PDF, and add an FAQ written the way patients actually ask.
Spanish gaps: make sure your Spanish pages exist and are easy for AI to find.
Safety gaps: flag them to your medical and regulatory teams and track them over time.
As always, changes go through medical, legal and regulatory review.
4. Make it routine. Most AI visibility tools track whether your brand is mentioned, not what each patient is told. Add this check to your AI search reviews every quarter and after any label change.
The bottom line
Health equity matters, and pharma takes it seriously. In the spirit of being patient-centric, brands invest real time and money in understanding their patients: plain language, translated materials, diverse imagery, support programs.
What many teams may not realize is that AI now sits between that work and some patients. If AI isn't drawing on what a brand has built, the full value of that effort doesn't reach everyone. The patients who ask simply, or in another language, can end up at a disadvantage without anyone noticing.
That's why AI search belongs in the analytics mix, alongside web and search data. Tracking and testing what AI tells patients shows where answers fall short and where to focus, so the work reaches every patient, however they ask.
Pharma can't write for every patient. AI answers every patient. The question is what it's working from.
Jennifer Gould, making sense of what's changing in marketing.