Centering Youth Perspectives in Agent Design: Piloting an Activity to Scaffold Discussions on Ethics and Agents

Abstract

In this paper, we share the design and preliminary analysis of student responses to a collaborative whiteboard activity centered around ethical design of a pedagogical agent (CoPA) in a STEM + computing learning environment. We report on qualitative analysis of student perceptions of (1) possible risks and benefits of using AI agents like CoPA; (2) trust & transparency; (3) consent & control; (4) fairness & bias; (5) privacy; and (6) AI learning & literacy. Though several areas of consensus across groups emerged in student responses, further investigation of relationships between agent design and function and the nature and quality of human-AI relationships is needed. We discuss (re)design of this activity, offering this pilot as a useful but imperfect structure for design discussion and seeking student input.