This podcast features a conversation with Julian Ashwin, an Assistant Professor in Economics at Maastricht University and a Research Fellow at Ellison Institute of Technology Oxford, about his recent reflective paper on the use of Large Language Models to analyse qualitative data. We discuss the potential of LLMs, but also the interpretive challenges they pose. These arise from the types of data they are trained on, which makes them less able to predict the responses of participants who are not WEIRD (Western, Educated, Industrialized, Rich, and Democratic). Without appropriate precautions, including understanding what good would look like in relation to your coded data, it's hard to see the systematic biases affecting your analyses. We talked about the danger of a future where these programmes are used for qualitative and quantitative analysis by researchers who have not been trained to do this themselves so cannot see what is going wrong. We also noted the importance of remembering that these are commercial products who want to keep you happy, whether by over-coding or by confirming our existing biases. Finally, Julian offered some practical suggestions about how to get the best results, including a downloadable programme that will enable you to train your own mini-LM, exclusively on your data

Listen  Read more about out Task Group on Methods Innovation