BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//AI and Equality - ECPv6.18.4.1//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:AI and Equality
X-ORIGINAL-URL:https://aiequalitytoolbox.com
X-WR-CALDESC:Events for AI and Equality
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:Europe/Zurich
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:20250330T010000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:20251026T010000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:20260329T010000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:20261025T010000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:20270328T010000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:20271031T010000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=Europe/Zurich:20261203T160000
DTEND;TZID=Europe/Zurich:20261203T170000
DTSTAMP:20261103T083853Z
CREATED:20261103T083853Z
LAST-MODIFIED:20261103T083853Z
UID:10000038-1796313600-1796317200@aiequalitytoolbox.com
SUMMARY:Evaluating AI for Human Rights: Lessons from the Field
DESCRIPTION:Pub Talk with Wm. Matthew Kennedy (King’s College London) \n🔗 Access Paper: Toward Human Rights Benchmarking for LLMs: A Pilot Methodology (2026) \n***Register to attend*** \nAbstract: \nLarge language models (LLMs) increasingly mediate legal determinations over what human rights are realized\, and how. Yet\, no evaluation benchmark exists to assess whether they can reason correctly about human rights law. To this end\, we report our efforts to develop a robust and scalable methodology for creating HumRightsBench: the first expert-validated\, scenario-based benchmark for evaluating reasoning grounded in the obligation structure of international human rights law. \nWe adapt the IRAC framework for legal reasoning to better suit the unique reasoning patterns of human rights work (substituting P\, “proposing remedies\,” for C\, “legal conclusion\,” yielding IRAP) to structure our evaluation heuristics. We also produce a pilot series of authentic scenarios designed to implicate the many dimensions of real-world human rights issues and annotated by human rights lawyers and professionals across the world. Ultimately\, we find that model accuracy scores range considerably across legal reasoning tasks (overall model performance ranges from 0.339 to 0.577\, task min-max ranges from 0.025 to 0.774)\, which strongly implies that HumRightsBench is a capable instrument for advancing this emerging subfield of AI evaluations science at a critical moment in its evolution. \nWm. Matthew (Matt) Kennedy is a technical AI governance researcher\, primarily interested in evaluation and red teaming methodologies\, the metadiscipline of evaluations itself\, and AI in certain domains: education\, legal reasoning\, and epistemography or knowledge production. His work is also concerned with the social\, material\, and political situation of technology development\, including but not limited to AI technologies. Prior to joining King’s\, he was a Marie Sklodowksa-Curie Postdoctoral Fellow at the Oxford Internet Institute\, although he also has several years experience in tech as a UX/UI and AI evaluations researcher. He received his PhD in the history of colonialism and international law from the University of Sydney in 2017.
URL:https://aiequalitytoolbox.com/event/evaluating-ai-for-human-rights-lessons-from-the-field/
ATTACH;FMTTYPE=image/png:https://aiequalitytoolbox.com/wp-content/uploads/2026/11/AIE-_-W@TT-Events.-Banners.png
END:VEVENT
END:VCALENDAR