AI fuels scams while doctors test benefits

AI-powered scams are surging just as AI helps doctors triage patients.

MIT Technology Review’s The Download paints a striking dual picture: a new era where criminals use large language models to craft phishing at scale, generate hyperreal deepfakes, and automate vulnerability scans, all while clinicians start piloting AI tools to summarize notes, sift records, and flag patients who may need extra support. The arc traces back to the late 2022 wave when ChatGPT popularized usable language models, giving bad actors a faster, cheaper playbook for social engineering, fraud, and stealthy intrusions. On the security front, attacks can now be composed, personalized, and launched in volume with far less human toil, turning cybercrime into a factory process rather than a few one off tricks.

In healthcare, AI is already on the floor, not just on the whiteboard. Doctors are using AI to take notes, interpret exam results, and skim through patient records to surface care gaps or trigger alerts for follow‑up. The promise is clear: speed up mundane tasks, reduce cognitive load, and help clinicians spot patterns that humans might miss. But the article leaves a crucial caveat front and center: we don’t yet know if this actually translates into better patient outcomes. The same systems that can surface vital signals may also misinterpret data, amplify biases, or introduce new privacy risks if sensitive health information is mishandled or overexposed.

The unfolding story is a reminder of a familiar paradox in AI: a tool with transformative potential can also widen the attack surface or amplify harm if governance and validation aren’t baked in from day one. The analogy is a two‑edged scalpel. In skilled hands it can save lives; in the hands of criminals or careless operators it can cut just as easily. That metaphor lands especially hard as organizations race to deploy these capabilities at scale, often with external vendors and a patchwork of internal controls.

For engineers and product leaders, a set of practical implications follows clearly. First, security must be built in by default, not bolted on after the fact. Expect to invest in automated detection of AI‑generated content used in phishing, as well as continuous red team exercises to stress test the resilience of workflows that handle patient data and clinician notes. Second, healthcare pilots require rigorous outcome tracking. It is not enough to show faster note taking or more complete records; teams should define measurable patient-centric endpoints and run controlled pilots to see whether AI interventions actually move the needle on care quality, access, or safety. Third, governance matters as much as capability. The same data that fuels helpful AI models is precisely the data that criminals seek; clear data minimization, access controls, and audit trails become competitive differentiators. Fourth, bias and data quality cannot be an afterthought. If AI systems learn from biased or noisy medical data, they risk propagating errors into diagnoses or treatment recommendations, eroding trust and safety.

Looking ahead to the quarter, startups and incumbents alike should plan for two realities: robust, privacy‑preserving security controls that scale with usage, and disciplined evaluation frameworks that distinguish genuine clinical benefit from hype. The takeaway is not to shun AI, but to require evidence, verifiability, and vigilant protection as a condition of adoption.

Sources

The Download: supercharged scams and studying AI healthcare