AI in Healthcare Isn’t as Safe as You Think

FEATURED RESEARCH

AI in Healthcare Isn’t as Safe as You Think

AI is changing healthcare in remarkable ways, but there’s a glaring problem that’s hard to overlook. Out of 429 safety reports submitted to the FDA about AI-enabled medical devices, 25% clearly linked issues to AI, while 34% didn’t provide enough detail to confirm if AI was involved.

That’s a troubling lack of clarity for tools meant to improve care.

Why this matters: Many clinicians can’t easily pinpoint when AI contributes to an error. These systems work quietly in the background, and safety reports aren’t equipped to capture their nuances.

The Biden administration’s 2023 AI Executive Order proposes a national safety program, but relying on existing reporting frameworks like the FDA’s MAUDE database won’t cut it.

Where we go from here: We need better tools to monitor AI systems and capture their failures. Real-time algorithm monitoring, clear reporting standards, and collaborative efforts like AI assurance labs can help.

These steps won’t just identify risks, they’ll build confidence in the technology we’re increasingly relying on to deliver care.

If we’re going to trust AI in healthcare, we have to do better at understanding its failures. Safety isn’t optional, and neither is accountability.

For more details: Full Article

Brain Booster

Select the right answer! (See explanation below)

Opinion and Perspectives

AI DRUG DISCOVERY

AI in Drug Discovery Is Failing to Deliver on Its Promises

2024 has been a tough year for AI in drug discovery. Insilico Medicine’s much-touted Phase 2a trial for its end-to-end AI-designed drug didn’t meet the mark for statistically significant efficacy. Recursion’s clinical trial for one of the first AI-discovered drugs didn’t fare any better.

The fallout: These setbacks are raising hard questions for techbio. Recursion, once an industry darling, has struggled to recover from its own missteps, acquiring Exscientia at a deep discount.

Deep Genomics, another high-profile player, seems to be running out of steam, with its founder publicly lamenting a decade of unmet promises in AI-driven drug discovery.

What’s going wrong? The AI-in-drug-discovery promise has always been ambitious: faster, cheaper, and better results than traditional methods.

But these recent failures highlight a mismatch between expectations and reality. AI models are still limited by the quality of available data and the complexities of human biology, issues that algorithms alone can’t solve.

Moving forward: This isn’t the end for AI in drug discovery. We still see significant funding raised in this sector (see our October’s Report), but it might be the end of uncritical hype.

Investors and researchers must adjust their expectations, focusing on long-term innovation over quick wins. AI may yet transform drug development, but not without patience, better data, and smarter collaborations.

It’s a reality check—one the industry sorely needs.

For more details: Full Article

Top Funded Startups

November’s Monthly Funding Report

How AI Healthcare Startups Raised $681 Million in November

November didn’t quite match October’s adrenaline rush, but it still held its own. The month brought a strong focus on diagnostics and patient monitoring, signaling a shift toward refining precision care.

Access Our November Report: Full Report

Product Pipeline

SPINE SURGERY

Neo Medical Achieves Full EU MDR Certification for Spine Care Solutions

Neo Medical’s innovative spine surgery platform, combining AI-driven augmented reality and force control technologies, is now fully certified under the EU’s stringent MDR standards.

This certification reinforces the platform’s safety and effectiveness in optimizing spine fusion procedures, improving precision, and reducing complications. With tougher EU regulations forcing 70% of manufacturers to remove products from the market, Neo Medical stands out by fully meeting the new standards.

Their certification ensures all their advanced spine surgery solutions remain available without interruption. The approval solidifies Neo’s commitment to safer, more sustainable spine care while positioning the company for growth in Europe and beyond.

For more details: Full Article

Policy and Ethics

FDA REGULATION

What You Need to Know About the FDA’s Long-Awaited AI Guidance

The FDA has finalized guidance on how medical device makers can manage updates for AI-powered tools. This includes clear rules on tracking and revising "predetermined change control plans" (PCCPs), which outline how devices can be safely improved over time.

The guidance was long-anticipated because AI devices evolve quickly, and manufacturers needed clarity on how to plan updates while staying FDA-compliant.

Companies must submit updated plans for review if changes are needed after a product launches. The FDA aims to ensure devices remain safe and effective while giving manufacturers flexibility to innovate responsibly.

This helps balance patient safety with the fast-paced evolution of AI in healthcare.

For more details: Full Article

Byte-Sized Break

📢 Three Things AI Did This Week

• The UK’s National Cyber Security Centre reported a 16% rise in cyber incidents in 2024, with ransomware posing the most immediate threat to critical infrastructure and increasing risks from AI-enabled cyberattacks. [Link]

• OpenAI, valued at $150B, is considering introducing ads to its AI products, amidst rapid revenue growth reaching $4B annually, but faces high costs exceeding $5B annually due to advanced AI model development. [Link]

• Japanese company Science Co. unveiled a prototype AI-powered "human washing machine" that combines automated washing, drying, and biometric analysis, with plans to showcase it at Expo 2025 in Osaka. [Link]

Have a Great Weekend!

Trivia Answer: C) Analyzing indoor air quality and suggesting humidity adjustments

AI systems use sensors to monitor allergens like dust mites, mold, and pet dander in indoor environments. They provide actionable insights, such as adjusting humidity or air filtration, to reduce allergen levels. The other options might sound plausible, but they’re not part of current AI solutions for allergy management—yet!