Seminar on Novel Measurement Problems in Social Sciences

We would like to invite you to a new PAN-Metrics seminar organized by the Department of Computational Social Sciences (head: Prof. Artur Pokropek). PAN-Metrics seminars cover the topics of methodology and measurement in the broadly defined social sciences.

The next meeting will be held online on Monday, April 27th at 5:00 pm CET (17:00).

The speaker will be Dr. Cameron S. Kay (Climate Cognition Lab. Stanford University ), who will give a presentation entitled: Are your online participants human?

Abstract:

Concerns about online data quality have traditionally focused on the possibility that some participants may not be devoting sufficient effort to their responses. However, with the advent and popularization of large language models (LLMs), researchers now face the unsettling possibility that some of their participants may not even be human. In this talk, I share two recent advances in detecting careless and synthetic respondents. First, I introduce CIFR (the Comprehensive Infrequency/Frequency Item Repository), an online database of 660 items designed to identify inattentive respondents. Second, I present ECLAIR (Exploiting Common Limitations of AI Respondents), a novel framework for creating items capable of detecting LLM-generated responses. In the process, I also present evidence demonstrating why detecting careless and synthetic respondents is essential for drawing valid conclusions from online survey research.

Bio: Cameron S. Kay (he/him) is a postdoctoral scholar in the Climate Cognition Lab at Stanford University. He studies the psychological foundations of antisocial beliefs and behaviours, including conspiracist ideation and climate change denial. To support this work, he develops psychometrically sound scales and tools for improving data quality. Before joining Stanford, Cameron worked as a visiting assistant professor at Union College, completed a PhD in social/personality psychology with a specialization in quantitative research methods at the University of Oregon, and earned a BA in psychology at the University of British Columbia. 

The meeting is scheduled for around one hour.

Please contact us to receive the Zoom link: marek.muszynski[at]ifispan.edu.pl

We would be grateful if you could send the message further to people you think might be interested in the topic.

With warm regards in the name of the whole Department,
Marek Muszyński

classic-editor-remember:

block-editor