Can the Crowd Tell How I Feel? Trait Empathy and Ethnic Background in a Visual Pain Judgment Task

ABSTRACT Many advocate for artificial agents to be empathic. Crowdsourc- ing could help, by facilitating human-in-the-loop approaches and dataset crea- tion for visual emotion recognition algorithms. Although crowdsourcing has been employed successfully for a range of tasks, it is not clear how effective crowdsourcing is when the task involves subjective rating of emotions. We ex- amined relationships between demographics, empathy and ethnic identity in pain emotion recognition tasks. Amazon MTurkers viewed images of strangers in painful settings, and tagged subjects’ emotions. They rated their level of pain arousal and confidence in their responses, and completed tests to gauge trait empathy and ethnic identity. We found that Caucasian participants were less confident than others, even when viewing other Caucasians in pain. Gender cor- related to word choices for describing images, though not to pain arousal or confidence. The results underscore the need for verified information on crowdworkers, to harness diversity effectively for metadata generation tasks.