Building Stereotype Evaluation Resources with Community Engagement

With rapid development and deployment of generative language models in global settings, there is an urgent need to also scale our measurements of harm, not just in the number and types of harms covered, but also how well they account for local cultural contexts, including marginalized identities and the social biases experienced by them. This growth in our evaluation paradigms thus, needs to be enhanced and calibrated by including people from different cultures and societies worldwide. In this work, we demonstrate this socio-culturally aware expansion in the Indian societal context for the harm of stereotyping. We devise a community engaged effort to build a resource which contains stereotypes for axes of disparity that are uniquely present in India. The resultant resource increases the number of stereotypes known for and in the Indian context by many folds and is consequently beneficial for evaluations of generative AI.