Session Information
Contribution
The research reported here has been conducted as part of a national three-year evaluation, commissioned by the Qualifications and Curriculum Authority in England of pilot qualifications in mathematics. The pilot qualifications are intended to develop new mathematics learning pathways for 14-19 year olds in England, for extended participation in mathematics at all levels is strongly advocated as part of the drive to develop capabilities in science and technology (Smith, 2004; Roberts 2002; Gago, 2004).
The evaluation is addressing two questions:
• What is the likely impact of the proposed qualifications on take-up of mathematics at all levels, particularly post-16, including candidate engagement and confidence?
• Do the benefits of a new system lead to sufficient gains which justify replacing current provision?
In developing pathways that might better serve a wider population of students, a number of attempts have been made to develop assessment, at all levels, that might be more motivating through a clearer focus on applications. These newly emerging assessments might be expected also to serve as a lever for curriculum change that will prepare students more effectively for the type of mathematical activities expected in international comparative studies such as PISA (2006).
These drivers suggest that we should expect to see a greater degree of authenticity in assessment items than has so far been the case.
As is evident from a literature developed over the last two decades, developing applied mathematical literacy (e.g. Hoyles et al, 2007), and transferring knowledge and skills from the context of the mathematics classroom to elsewhere (Evans, 2000), is not unproblematic. Research into developing ‘Realistic Mathematics Education’ (e.g. Presmeg & van den Heuvel-Panhuizen, 2003) continues to promote means of developing ‘a school mathematics curriculum that is grounded in the experiential reality of the learners.’ p. 1. This paper is about the struggle to embed more realistic contexts also into assessment (e.g. Burkhart, 2007), and the difference between the rhetoric and the reality. It is also about language and how meanings of term/ideas vary.
In this contribution we show that in national examinations both individual items and entire assessments are frequently designed with some implicit or explicit ‘relevant’ application. These items are described by various words, and used seemingly interchangeably across stakeholder groups: functional, real life, realistic, authentic, situated, in context, pseudo-contextualised or artificial; and there may be others. Yet, what is also apparent is that these terms mean different things to different people.
Method
A multi-site team of researchers, all experienced mathematics educators, is conducting a mixed-method evaluation. One layer of this is item-level scrutiny of assessment items in both pilot and current examinations. Each examination is mapped, at item level, to a framework that identifies across key dimensions: structure; content; process skills; task type; resources. To increase reliability, this analysis is conducted independently by two researchers.
The authenticity of mathematics test items emerges from ‘task type’ dimension of the scrutiny. The following criteria were used:
• Pure – the question has no context other than that of mathematics itself.
• Artificial –whilst a context is introduced it is not authentic in that a candidate would not use mathematics to solve the problem in the way suggested.
• Authentic – the context is something that a candidate could possibly engage with in their day-to-day life and use mathematics in the way the question demands.
Expected Outcomes
Scrutiny of items enables us to see several interpretations of authentic assessment, including:
• Something the candidate ‘ought’ to know about, e.g. managing personal finance.
• ‘Real life’, where assessments are set in the context of activities that the candidate might engage with, e.g. reading timetables, or scheduling tasks.
• Problems arising from genuine data, but specifically mathematical in expectation of skills, knowledge and understanding demonstrated.
• Problems that specifically address particular mathematics in order to support students in other areas of study, e.g. Engineering.
• Vocational assessments where mathematics may be implicit, or, at higher levels requiring more explicit skills.
We will present item-level analysis of piloted assessment materials to illustrate challenges faced by examiners constructing 'authentic' assessments. The examples are used to highlight language problems, and varied views of what authenticity is taken to mean. +++continues in References
References
+++This way we also explore the related problem of stimulating the teaching of mathematics in more authentic settings.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.