Google’s generative AI system proved it can answer medical exam questions. But now the company is attempting a bigger leap — infusing its model with medical images such as X-rays and mammograms to help it communicate with doctors about data routinely used in patient care.

The effort, unveiled Wednesday, marks the beginning of a quest to make generative AI a more dynamic tool in medicine. While prior iterations of AI could either analyze images, or respond to questions, the company’s new model, known as Med-Palm 2, aims to put those capabilities together in a single system.

advertisement

“Until this year, this was not on the table,” said Greg Corrado, a senior research scientist at Google. “Now you can build a system — and this is amazing to me, honestly, I did not expect this to happen now — but you can build a system where you give it an image, it writes a report, and then you can ask it follow-up questions.”

Unlock this article by subscribing to STAT+ and enjoy your first 30 days free!