Loading Events

« All Events

  • This event has passed.

Training procedures and regulatory policies for safe machine learning models in healthcare

February 19, 2020 @ 3:00 pm - 4:00 pm

Presenter: Jean Feng, PhD candidate

Before we can realize the potential of machine learning (ML) in healthcare, we need to ensure the safety of these models for medical decision-making. In this talk, we explore 1) how to learn safer ML models and 2) how to allow modifications to them over time. As a first step to learning safer models, I will discuss the need for fail-safes in ML. Typical models give predictions for all inputs, which can be overconfident or even misleading for out-of-sample inputs. I introduce a penalization method for learning models that can abstain from making predictions and apply it to predict patient outcomes in the intensive care unit. In the second half of the talk, we discuss how the US FDA might regulate modifications to ML-based Software as a Medical Device (SaMD) without hindering innovation. To this end, we consider policies that can automatically evaluate proposed modifications without human intervention. I define a framework for evaluating the error rates of such policies and show that policies without error rate guarantees are prone to “bio-creep.” I then show how to protect against it by combining group sequential and online hypothesis testing methods.

Join Remote:


Telephone:  669 900 6833

Meeting ID: 640 499 330



February 19, 2020
3:00 pm - 4:00 pm
Event Category:


Mission Hall, Room 2700
550 16th Street
San Francisco, CA 94158 United States
+ Google Map