The healthcare industry is undergoing a transformation, driven by the power of artificial intelligence (AI). Last week at the HLTH conference, Microsoft announced advanced healthcare AI models in Azure AI Studio.

Developed in collaboration with Microsoft Research, our strategic partners, and leading healthcare institutions, these AI models are specifically designed for healthcare organizations to rapidly build and deploy AI solutions tailored to their specific needs, all while minimizing the extensive compute and data requirements typically associated with building multimodal models from scratch. With the healthcare AI models, healthcare professionals have the tools they need to harness the full potential of AI to assist patient care.

Modern medicine encompasses various data modalities, including medical imaging, genomics, clinical records, and other structured and unstructured data sources. Understanding the intricacies of this multimodal environment, Azure AI onboards specialized healthcare AI models that go beyond traditional text-based applications, providing robust solutions tailored to healthcare’s unique challenges.

Introducing the New Healthcare AI Models in the Azure AI Model Catalog:

The Azure AI model catalog has a new industry “Health and Life Sciences” filter with a new array of state-of-the-art, open source, healthcare models. This includes Microsoft’s first-party models as well as models by strategic partners:

MedImageInsight (paper) This embedding model enables sophisticated image analysis, including classification and similarity search in medical imaging. Researchers can use the model embeddings and build adapters for their specific tasks, streamlining workflows in radiology, pathology, ophthalmology, dermatology and other modalities. For example, researchers can explore how the model can be used to build tools to automatically route imaging scans to specialists, or flag potential abnormalities for further review, enabling improved efficiencies and patient outcomes. These models must be thoroughly tested, validated, and, in some cases, further fine-tuned to ensure their applicability in specific use cases. Furthermore, the model can be leveraged for responsible AI such as out-of-distribution (OOD) detection and drift monitoring, to maintain stability and reliability of AI tools and data pipelines in dynamic medical imaging environments.

Example Juypter notebooks deployable in Azure Machine Learning:
https://aka.ms/healthcare-ai-examples-mi2-deploy 
https://aka.ms/healthcare-ai-examples-mi2-zero-shot 
https://aka.ms/healthcare-ai-examples-mi2-adapter 
https://aka.ms/healthcare-ai-examples-mi2-exam-parameter

MedImageParse (paper) Designed for precise image segmentation, this model covers various imaging modalities, including X-Rays, CT scans, MRIs, ultrasounds, dermatology images, and pathology slides. It can be fine-tuned for specific applications such as tumor segmentation or organ delineation, allowing developers to test and validate the ability to leverage AI for highly targeted cancer and other disease detection, diagnostics and treatment planning.

Example Juypter notebooks deployable in Azure Machine Learning:

https://aka.ms/healthcare-ai-examples-mip-deploy 
https://aka.ms/healthcare-ai-examples-mip-examples

CXRReportGen (paper) Chest X-rays are the most common radiology procedure globally. They’re crucial because they help doctors diagnose a wide range of conditions—from lung infections to heart problems. These images are often the first step in detecting health issues that affect millions of people. By incorporating current and prior images, along with key patient information, this multimodal AI model generates report findings from chest X-rays, highlighting AI-generated findings directly on the images to align with human-in-the-loop workflows. Researchers can test this capability and the potential to accelerate turnaround times while enhancing the diagnostic precision of radiologists. This model has demonstrated exceptional performance on the industry standard MIMIC-CXR benchmark (paper):

Example Juypter notebooks deployable in Azure Machine Learning:
https://aka.ms/healthcare-ai-examples-cxr-deploy

Partner models: Paige.ai, Providence Healthcare, NVIDIA, and M42 contributed foundational models to the catalog, spanning areas including pathology, 3D medical imaging, biomedical research, and medical knowledge sharing. Developed under a core set of shared AI principles, these models provide a powerful starting point for organizations as they launch their own AI projects, while embedding responsible practices across the industry.

The open access to AI models on the catalog and modular approach allows healthcare organizations to customize solutions, maintain control over their data, and build trust through shared development and oversight. This approach aligns with our commitment to responsible AI, ensuring our technologies meet ethical standards and earn the trust of the medical community.
Microsoft is dedicated to responsibly scaling artificial intelligence and continuously improving our tools by listening and learning. Importantly, Microsoft does not use customer data to train AI models without explicit customer permission or in undisclosed ways. We collaborate with organizations to help them harness their data, enabling the development of predictive and analytical solutions that may drive their competitive advantage.

Azure AI Studio: Empowering Health and Life Sciences with Seamless AI Integration

Azure AI Studio offers healthcare professionals a comprehensive platform to develop, fine-tune, deploy, and continuously monitor AI models tailored to their unique needs. With the introduction of the new healthcare AI models, Azure AI Studio simplifies the integration of AI into healthcare workflows which allows professionals to focus on improving patient outcomes. Here’s how Azure AI Studio delivers value:

Bring your data and fine-tune models: Azure AI Studio and healthcare AI models complement the healthcare data solutions available in Microsoft Fabric, creating a unified environment to bring multimodal proprietary data to enable a wide range of use cases. Healthcare professionals can leverage the models as-is using the playground in Azure AI Studio or fine-tune pre-trained models with their data in Azure Machine Learning to adapt models for their specific clinical needs.
Rapid Development and Deployment: Azure AI Studio provides an intuitive interface and a comprehensive set of generative AI operations (GenAIOps) toolchains that enable professionals to quickly develop, test, and deploy AI applications. This streamlined process can accelerate the adoption of AI in healthcare, empowering organizations to integrate sophisticated diagnostic and analytical tools into their existing workflows. With built-in support for deploying models in cloud, on-premises, or hybrid environments, healthcare professionals can optimize their AI solutions for various clinical settings.
Supporting Safety and Compliance: Trust is crucial in healthcare, where AI can impact patient care. The model cards in the model catalog share details about the training and evaluation datasets used, including fairness testing where applicable. The platform supports hybrid deployment options for enhanced control over sensitive healthcare data. Additionally, Azure AI aligns with healthcare regulations such as HIPAA, helping organizations maintain high standards of data security, patient confidentiality, and overall compliance.

Real-world Impact: Customer Success Stories

Healthcare organizations are already leveraging these models to transform their workflows with Azure AI:

Mass General Brigham: MGB is using Microsoft’s MedImageInsight model to surface additional relevant information during clinical research and streamlining radiologist workflows, alleviating the administrative workload on clinical staff, and enhancing the speed of patient care.
University of Wisconsin-Madison: UW is targeting advanced report generation from medical imaging analysis using Microsoft’s CXRReportGen. With ever-increasing imaging volumes colliding with the ongoing combination of radiologist burnout and shortages, a state-of-the-art medical imaging model can be used to build an application that can transform a medical image into a draft note, supporting better outcomes for patients while helping clinicians focus on the hands-on components of their roles.
Sectra: Sectra is working with Microsoft to build on top of foundational models like MedImageInsight to automate the process of understanding the types of exams coming through the Sectra Vendor Neutral Archive (VNA) system for better routing and display.
Mars PETCARE: Mars PETCARE is exploring the use of the healthcare AI models for veterinary medicine applications, such as data evaluation in radiology and pathology teams. By combining veterinary expertise with advanced AI models, Mars PETCARE is setting new standards in animal health. This collaboration has the potential to transform veterinary diagnostics, improve the quality of care for pets, and showcase the versatility of healthcare AI models in non-human medical contexts.
Paige: In life sciences, Paige is working to combine radiology, pathology, and genomic insights for a more comprehensive approach to disease diagnosis, aimed at accelerating the discovery of new treatments.

Join Us in Shaping the Future of Healthcare

Join us at Microsoft Ignite to witness these models in action and learn how they can transform your healthcare practice. Visit the documentation and AI Studio to explore these cutting-edge healthcare AI models and start your journey toward a data-driven, AI-empowered future.

Medical device disclaimer: Microsoft products and services (1) are not designed, intended or made available as a medical device, and (2) are not designed or intended to be a substitute for professional medical advice, diagnosis, treatment, or judgment and should not be used to replace or as a substitute for professional medical advice, diagnosis, treatment, or judgment. Customers/partners are responsible for ensuring solutions comply with applicable laws and regulations.

Generative AI does not always provide accurate or complete information. AI outputs do not reflect the opinions of Microsoft. Customers/partners will need to thoroughly test and evaluate whether an AI tool is fit for the intended use and identify and mitigate any risks to end users associated with its use. Customers/partners should thoroughly review the product documentation for each tool.

​Microsoft Tech Community – Latest Blogs –Read More