This seminar will focus on how measurement acts as a technology of governance in AI systems. As a foundation, I draw on measurement theory, particularly via educational testing and psychometrics, to discuss the usually-implicit role of measurement modeling in real world AI systems. With that, I will discuss how fairness-oriented conceptualizations of reliability and construct validity can be used to identify, mitigate, and prevent harms arising from such systems. This measurement perspective offers a framework to uncover how often-hidden decisions are enacted, including where existing 'fairness' interventions can fall short; how social categories are made real; and where regulatory interventions can play a role. Understanding measurement as governance gives us leverage into the hidden challenges in designing equitable artificial intelligence systems---and mitigating the harms already emerging from AI. I will also introduce some recent projects extending this work.
Abigail Jacobs is an assistant professor of Information and of Complex Systems at the University of Michigan, where she is also affiliated with the Center for Ethics, Society, and Computing (ESC) and the Michigan Institute for Data Science (MIDAS). Previously she was a postdoctoral fellow at the Haas School of Business at UC Berkeley and a member of the Algorithmic Fairness and Opacity Working Group. She received a PhD in Computer Science at the University of Colorado Boulder and a BA in Mathematical Methods in the Social Sciences and Mathematics from Northwestern University. She previously spent time at Microsoft Research NYC and served on the Board of Directors for Women in Machine Learning, Inc.