About the Talk
We’re in the process of developing a Human Rights Benchmark for Large Language Models LLMS— a tool designed to assess how well these systems align with core human rights principles.
Why now? As LLMs become embedded in decision-making systems, chatbots, and public services, we need more than accuracy—we need accountability. Our benchmark will explore whether these models treat all identities equally, respond consistently to rights-based questions, and avoid harmful omissions or bias. Led by Savannah Thais, the AI & Equality Human Rights Benchmark for LLM is part of our core commitment: to build AI systems that respect dignity, uphold equality, and serve everyone. In this Pub-Talk, Savannah will present the process and methodology of the benchmark.
About the Speaker
Dr. Savannah Thais is an Associate Research Scientist in the Columbia Data Science Institute with a focus on machine learning (ML). She is interested in complex system modeling and in understanding what types of information is measurable or modelable and what impacts designing and performing measurements have on systems and societies. This work is informed by her background in high energy particle physics and incorporates traditional scientific experiment design components such as uncertainty quantification, experimental blinding, and decorrelation/de-biasing methods. Her recent work has focused on geometric deep learning, methods to incorporate physics-based inductive biases into ML models, regulation of emerging technology, social determinants of health, and community education.
Dr. Thais is the founder and Research Director of Community Insight and Impact, an non-profit organization focused on data-driven community needs assessments for vulnerable populations and effective resource allocation. She is passionate about the impacts of science and technology on society and is a strong advocate for improving access to scientific education and literacy, community centered technology development, and equitable data practices. She was the ML Knowledge Convener for the CMS Experiment from 2020-2022, currently serves on the Executive Board of Women in Machine Learning and the Executive Committee of the APS Group on Data Science, and is a Founding Editor of the Springer AI Ethics journal.
Dr. Thais received her PhD in Physics from Yale University in 2019 and was a postdoctoral researcher at Princeton University from 2019-2022.