About the Talk
As AI systems increasingly shape critical aspects of society, the need for robust, transparent methods to measure their behavior has never been more urgent. Leveraging her background in particle physics, she has focused her research on designing rigorous, domain informed approaches to measuring AI performance and impact.
In the first half of this talk, Savannah Thais introduced a few of her recent research projects spanning benchmark development for agentic AI systems, novel policy impact analysis frameworks, and technical literacy development. In the second half of the talk, she introduced the exciting new Human Rights Benchmark project which aims to assess language model performance through the lens of internationally recognized human rights. She outlined the methodology, including prompt development and model evaluation strategies, and discuss how this work contributes to more accountable and socially aligned AI systems.
About the Speaker
Dr. Savannah Thais is an Associate Research Scientist in the Columbia Data Science Institute with a focus on machine learning (ML). She is interested in complex system modeling and in understanding what types of information is measurable or modelable and what impacts designing and performing measurements have on systems and societies. This work is informed by her background in high energy particle physics and incorporates traditional scientific experiment design components such as uncertainty quantification, experimental blinding, and decorrelation/de-biasing methods. Her recent work has focused on geometric deep learning, methods to incorporate physics-based inductive biases into ML models, regulation of emerging technology, social determinants of health, and community education.
Dr. Thais is the founder and Research Director of Community Insight and Impact, an non-profit organization focused on data-driven community needs assessments for vulnerable populations and effective resource allocation. She is passionate about the impacts of science and technology on society and is a strong advocate for improving access to scientific education and literacy, community centered technology development, and equitable data practices. She was the ML Knowledge Convener for the CMS Experiment from 2020-2022, currently serves on the Executive Board of Women in Machine Learning and the Executive Committee of the APS Group on Data Science, and is a Founding Editor of the Springer AI Ethics journal.
Dr. Thais received her PhD in Physics from Yale University in 2019 and was a postdoctoral researcher at Princeton University from 2019-2022.