Medical experts acknowledge AI’s potential role in improving healthcare delivery, though they doubt its suitability as a standard chatbot solution.

Dr. Sina Bari, who works as a surgeon and serves as a leading expert in AI applications within the technology firm iMerit, has witnessed firsthand how ChatGPT can provide patients with incorrect medical advice that may lead them down the wrong path.

“I recently saw a patient come to me. After I suggested a particular medication, they showed me a conversation generated by ChatGPT claiming that using that drug carried a 45% risk of developing pulmonary embolism,” Dr. Bari shared with TechCrunch.

Upon further investigation, Dr. Bari discovered that the statistic in question came from a research paper examining the effects of that medication on a specific subgroup of people suffering from tuberculosis — information that was completely irrelevant to his patient’s situation.

Despite this issue, when OpenAI unveiled its dedicated ChatGPT Health chatbot last week, Dr. Bari felt more enthusiasm than concern.

ChatGPT Health, which is set to launch in the coming weeks, enables users to discuss their health concerns in a more private setting. Additionally, it ensures that the messages sent through this platform are not used as training data for the underlying AI model.

“I think it’s an excellent development,” Dr. Bari said. “Since something like this is already being used, formalizing it with proper safeguards to protect patient information will make it even more valuable for patients seeking guidance.”

Users can receive more tailored advice through ChatGPT Health by uploading their medical records and syncing them with apps such as Apple Health and MyFitnessPal. However, this feature raises immediate concerns for those who prioritize data security.

Join the Disrupt 2026 Waitlist

Add yourself to the Disrupt 2026 waitlist to be first in line when Early Bird tickets drop. Past Disrupts have brought Google Cloud, Netflix, Microsoft, Box, Phia, a16z, ElevenLabs, Wayve, Hugging Face, Elad Gil, and Vinod Khosla to the stages — part of 250+ industry leaders driving 200+ sessions built to fuel your growth and sharpen your edge. Plus, meet the hundreds of startups innovating across every sector.

Join the Disrupt 2026 Waitlist

Add yourself to the Disrupt 2026 waitlist to be first in line when Early Bird tickets drop. Past Disrupts have brought Google Cloud, Netflix, Microsoft, Box, Phia, a16z, ElevenLabs, Wayve, Hugging Face, Elad Gil, and Vinod Khosla to the stages — part of 250+ industry leaders driving 200+ sessions built to fuel your growth and sharpen your edge. Plus, meet the hundreds of startups innovating across every sector.

“Suddenly, medical data is being transferred from organizations that comply with HIPAA regulations to vendors that do not,” Itai Schwartz, co-founder of the data loss prevention company MIND, told TechCrunch. “So I’m curious to see how regulatory bodies plan to address this situation.”

From the perspective of some industry professionals, however, the damage has already been done. Nowadays, instead of searching online for symptoms, people prefer to consult AI chatbots — with over 230 million users interacting with ChatGPT regarding their health each week.

“This represents one of the most significant use cases for ChatGPT to date,” Andrew Brackin, a partner at Gradient who specializes in investing in health technology, told TechCrunch. “It makes perfect sense that they would want to create a more private, secure, and optimized version of ChatGPT specifically for handling healthcare-related queries.”

AI chatbots suffer from a persistent issue known as hallucinations, which poses a particularly serious concern in the healthcare sector. According to Vectara’s Factual Consistency Evaluation Model, OpenAI’s GPT-5 tends to generate more false information than many models developed by Google and Anthropic. Nevertheless, AI companies see potential in using these technologies to address inefficiencies within healthcare (Anthropic also announced a health-related product this week).

For Dr. Nigam Shah, a professor of medicine at Stanford University and the chief data scientist for Stanford Health Care, the difficulty American patients face in accessing medical care is an even more pressing issue than the risk of ChatGPT providing inaccurate advice.

“Right now, if you try to see a primary care doctor through any health system, you can expect to wait anywhere from three to six months,” Dr. Shah said. “If you had to choose between waiting six months for a human physician or speaking with an AI tool that can still assist you with certain medical needs, which would you opt for?”

Dr. Shah believes that the most effective way to integrate AI into healthcare systems is by focusing on changes on the provider side rather than among patients.

Medical journals have frequently reported that administrative tasks account for approximately half of a primary care physician’s working time, which significantly reduces the number of patients they can see in a given day. Automating such tasks could allow doctors to see more patients, potentially reducing the need for people to rely on tools like ChatGPT Health without first consulting a real physician.

Dr. Shah leads a team at Stanford that is developing ChatEHR, a software application designed to be integrated directly into electronic health record systems. This tool enables clinicians to access patients’ medical records more efficiently and streamline their workflow.

“Making electronic health records more user-friendly means physicians can spend less time searching through complex records to find the information they need,” Dr. Sneha Jain, one of the early testers of ChatEHR, stated in an article published by Stanford Medicine. “ChatEHR helps them obtain this information quickly, allowing them to focus on what is truly important — communicating with patients and understanding their health conditions.”

Anthropic is also developing AI products aimed at being used by clinicians and insurance companies, in addition to its publicly available Claude chatbot. This week, Anthropic introduced Claude for Healthcare, explaining how it can be utilized to reduce the time spent on tedious administrative tasks such as submitting prior authorization requests to insurance providers.

“Some of you handle hundreds or even thousands of prior authorization cases per week,” said Anthropic’s Chief Product Officer, Mike Krieger, during a recent presentation at J.P. Morgan’s Healthcare Conference. “So just imagine cutting 20 or 30 minutes off each of those processes — that would represent substantial time savings.”

As AI and medicine become increasingly intertwined, an inevitable tension exists between these two fields. Doctors are primarily motivated by their commitment to helping patients, while technology companies are ultimately accountable to their shareholders, even if their intentions are well-intentioned.

“I believe that this tension is very important,” Dr. Bari said. “Patients rely on us to be cautious and conservative in order to safeguard their interests.”