Session Information
23 SES 12 B, Politics of International Assessments and Tests
Paper Session
Contribution
The widespread use of comparative quantitative accounts in education makes them an important focus of study. Comparative accounts such as the Programme for International Student Assessment (PISA) have come to be seen as unbiased, neutral and more trustworthy than other types of accounts, and are widely used in policy decisions in many parts of the world. International numeric accounts make their way through the policy world with ease, jumping linguistic and cultural boundaries and displacing native informants (Cullather, 2007). Although some of the findings of such studies, such as the rankings of nations on the ‘league tables’ are widely known, less is known about the models and assumptions underlying such measurements and the means by which such comparisons are made possible, or the tensions and challenges inherent in the production and circulation of such accounts. Numbers tend to shrug off traces of the negotiations and translations that attend their generation.
Drawing upon the resources of the interdisciplinary field of Science and Technology Studies (STS), in this paper I attempt to make visible the negotiations and translations that underpin numeric accounts such as PISA. Understanding the development of international indicators in education as an assemblage, I elaborate the processes through which the chaotic jumble of the world has been gradually categorised, classified, mapped and made measurable. I detail the processes such as developing common nomenclature, standardising entities and preparing a common framework for data collection that were required for the development of indicators which, in turn, have made international comparisons possible. I trace the practices of translation and ‘materialising’ that have rendered abstract and disparate entities calculable and commensurate and enabled their circulation in policy.
Based on in-depth interviews with five former OECD officials and former and current PISA officials and analysts, I discuss the challenges and difficulties inherent in the production of such accounts, and the impact of these assumptions on the way such data circulate in policy. Supported by the narratives of my expert informants, I identify four challenges encountered when making and using such calculations: containment, reification, over-confidence and blind spots. I illustrate each of these challenges with examples from PISA.
Moving away from critiquing the politically expedient use of evidence, the vagaries and ethics of politicians and the limitations and fallibility of research per se, this paper attempts to open a line of critique which focuses on the discursive and material practices that render education measurable. The purpose of this paper is neither to defend numeric accounts nor to argue that ‘numbers are bad;’ rather, the aim is to highlight the limitations and challenges inherent in the production and use of international comparative accounts such as PISA.
Method
Expected Outcomes
References
Barry, A. (2002). The Anti-political Economy. Economy and Society, 31(2), 268-284. Cullather, N. (2007). The Foreign Policy of the Calorie. The American Historical Review, 112(2), 337-364. DeLanda, M. (2006). A New Philosophy of Society - Assemblage Theory and Social Complexity. London: Continuum. Deleuze, G., & Guattari, F. (1988). A thousand plateaus: capitalism and schizophrenia. London: Athlone Press. Jasanoff, S. (2004). States of knowledge: the co-production of science and social order. New York ; London: Routledge. Latour, B. (2005). Reassembling the Social: An Introduction to Actor-Network-Theory. Oxford: Oxford University Press.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.