log in  |  register  |  feedback?  |  help  |  web accessibility
Foundation Models and Generative AI for Medical Imaging Segmentation in Ultra-Low Data Regimes
Pengtao Xie
IRB 4105 or https://umd.zoom.us/j/5482450617
Tuesday, May 21, 2024, 11:00 am-12:00 pm Calendar
  • You are subscribed to this talk through .
  • You are watching this talk through .
  • You are subscribed to this talk. (unsubscribe, watch)
  • You are watching this talk. (unwatch, subscribe)
  • You are not subscribed to this talk. (watch, subscribe)

Semantic segmentation of medical images is pivotal in disease diagnosis and treatment planning. While deep learning has excelled in automating this task, a major hurdle is the need for numerous annotated masks, which are resource-intensive to produce due to the required expertise and time. This scenario often leads to ultra-low data regimes where annotated images are scarce, challenging the generalization of deep learning models on test images. To address this, we introduce two complementary approaches. One involves developing foundation models. The other involves generating high-fidelity training data consisting of paired segmentation masks and medical images. In the former, we develop a bi-level optimization based method which can effectively adapt the general-domain Segment Anything Model (SAM) to the medical domain with just a few medical images. In the latter, we propose a multi-level optimization based method which can perform end-to-end generation of high-quality training data from a minimal number of real images. On eight segmentation tasks involving various diseases, organs, and imaging modalities, our methods demonstrate strong generalization performance in both in-domain and out-of-domain settings. Our methods require 8-12 times less training data than baselines to achieve comparable performance.


Pengtao Xie is an assistant professor in the Department of Electrical and Computer Engineering at the University of California San Diego. His research interest lies in machine learning for healthcare. He was recognized with NSF Career Award, Global Top-100 Chinese Young Scholars in AI by Baidu, top-5 finalist for the AMIA Doctoral Dissertation Award, ICLR Notable-Top-5% paper, Amazon AWS Machine Learning Research Award, Tencent AI-Lab Faculty Award, Innovator Award by the Pittsburgh Business Times, among others. He serves as an associate editor for the ACM Transactions on Computing for Healthcare, senior area chair for AAAI, area chairs for ICML and NeurIPS, etc.

This talk is organized by Samuel Malede Zewdu