Slides: Download here

Video Recording: NeurIPS Website

Abstract

Algorithmic rankings have become increasingly common in the online world. From social media to streaming services and from e-commerce to hiring, ranking has become the primary way online users ingest information. In many cases, recommendations have implications for both the users and items (or creators) being ranked. These systems are also increasingly personalized to viewers, relying on imperfect information to learn and respond to user preferences. In recent years, the machine learning community has become increasingly aware of potential harms to consumers (e.g. echo chambers, addictive design, virality of harmful content) and creators (e.g. access to opportunity, misattribution and appropriation). In this tutorial we will explore the current state of research on responsible recommendations and the primary challenges with designing, evaluating and training these systems for user and content providers. This tutorial additionally presents the primary challenges in applying this research in practice. The perspectives and methods presented in this tutorial will apply to recommendation systems generally and will not include any specific information regarding actual recommendation products. The tutorial is designed to be accessible to a broad audience of machine learning practitioners, some background in predictive systems and ranking is beneficial but not required.

Speakers

Hannah Korevaar is a Research Scientist at Meta on Responsible AI. Hannah got her PhD in Demography and Social Policy from Princeton. Hannah has been working on developing evaluation and mitigation strategies for recommendation systems since starting in August 2020.

Manish Raghavan, is an assistant professor at MIT in the Fall 2022. Manish got his PhD in Computer Science from Cornell University studying algorithmic fairness and its application to job search and recommendation engines, and then worked as a postdoctoral fellow at the Harvard Center for Research on Computation and Society (CRCS).

Ashudeep Singh is an Applied Research Scientist at Pinterest working on developing evaluation methods and algorithms to ensure that Pinterest ML models work well for a diverse set of users and content providers. Ashudeep got his PhD in Computer Science from Cornell University studying fairness in ranking.

Panelists

Chloé Bakalar, Temple University. Chloé is an Assistant Professor of Political Science. She is also a Visiting Research Collaborator at Princeton University’s Center for Information Technology Policy (CITP). She is a political and legal theorist with a background in American politics. Her work focuses on philosophical and legal questions surrounding freedom of speech, especially in relation to liberal democratic citizenship. Additional research interests include: ethics and public policy (esp. technology ethics); normative ethics; democratic theory; contemporary political thought; and the history of ideas.

Fernando Diaz, Google Research. Fernando Diaz is a research scientist at Google Research Montréal. His research spans twenty years and focuses on the design of information access systems, including search engines and recommender systems. He is particularly interested in understanding and addressing the societal implications of artificial intelligence more generally. Fernando has presented his research at multiple academic conferences and served as a Program Co-Chair for the 2019 ACM Conference on Fairness, Accountability, and Transparency. Previously, Fernando was the assistant managing director of Microsoft Research Montréal, where he also led FATE Montréal, and a director of research at Spotify, where he established its research organization on recommendation, search, and personalization.

Alana Shine, Meta. Alana Shine is a research scientist at Meta working in Responsible AI on the ranking and recommendation system fairness team. She is interested in how to design fair recommendation systems within dynamic social networks. Before this, she completed her Computer Science PhD at the University of Southern California on generative graph models.