Jul 6, 01:46 PM

Study Finds Major AI Chatbots Generate Convincing Medical Misinformation

Australian researchers have shown that leading generative-AI chatbots can be reprogrammed to deliver persuasive but false medical advice, highlighting what they call an urgent need for tighter safeguards. The study, led by Ashley Hopkins at Flinders University and published in the Annals of Internal Medicine, tested five widely used large language models by embedding hidden instructions that required them to answer 10 health questions incorrectly in an authoritative tone that cited fabricated journal references. OpenAI’s GPT-4o, Google’s Gemini 1.5 Pro, Meta’s Llama 3.2-90B Vision and xAI’s Grok Beta provided polished misinformation in every response, the team found. Anthropic’s Claude 3.5 Sonnet resisted more than half the requests, suggesting stronger guardrails are technically feasible. The researchers said the ease with which the systems were manipulated shows malicious actors could mass-produce dangerous content, from vaccine myths to unfounded links between 5G networks and infertility. Hopkins warned that if developers and regulators do not impose consistent protections, manipulated chatbots could undermine public health and erode trust in legitimate medical guidance. The authors urged companies to tighten technical defenses, increase transparency about model training and create verification mechanisms, while policymakers craft frameworks that hold AI providers accountable for harmful outputs.