Session Information
09 SES 11 C, Evaluation Policies, Monitoring and Assessments
Paper Session
Contribution
Structural changes in the Education System have led to a shift from input to output-centered governance in terms of “New Public Management“ which exacerbate accountability pressure on schools. Therefore schools themselves have become important actors in the education system, as governance decisions are increasingly shifted from the system to the organisational level.
Evaluation data offer important control knowledge for educational governance in terms of the accountability approach. How schools make use of evaluative data (e.g. from school inspections, comparative assessment of student achievement, self-evaluation) and how they result in improvement processes on school-level is still a deficiency (Schrader & Helmke 2003; Stamm 2003; Kohler & Schrader 2004; Mandl& Hense 2007). The use of (externally or internally generated) evaluation data is crucial for establishing a self-monitoring system as part of school effectiveness. The current project overcomes mere descriptions of parameters identified by school effectiveness research by identifying organisational key factors on a theoretical basis derived from organisational theory (Henry Mintzberg 1983; Scheerens & Bosker 1997; Luhmann 2000).
Previous research has mainly focused on input from large-scale-assessments (e.g. Kohler 2005), on one specific type of evaluation data or on creating basic theoretical models of Evaluation use (Schrader & Helmke, 2004).
Considering the current project, preconditions and effects of different evaluation policies are to be described and measured. Based on Scheerens and Bosker’s (1997) model of school effectiveness, the project aims at identifying different user-types of evaluation data on school-level and at determining the evaluative capacities of schools.
In addition, the project aims at describing barriers of evaluative knowledge use (e.g. from organisational, professional, usability-related reasons) and effects of different user types on student achievement measured by comparative skill-level-tests. These standard-based tests mainly serve the purpose of instructional development due to national Education Standards.
To measure and describe organisational features of evaluation use an online survey for school principals and faculty leaders of general-educational schools within the German states of Berlin and Brandenburg based on the organisational theoretical framework of Niklas Luhmann will be conducted.
Further theoretical approaches by Scheerens& Bosker (1997), Helmke & Schrader (2004), Groß-Ophoff et al. (2006), Mintrop & Trujillo (2007) have been included in an own comprehensive model for reception of evaluation data.
For an in-depth-study of barriers and resistances (Terhart 2002, Visscher & Coe 2003; Stamm 2003; van Ackeren 2004; Kohler & Schrader 2004; Schrader & Helmke, 2004, Kohler, 2005; Schneewind 2006; Maier, 2010) of data perception and use (Evaluation policy), case studies will be carried out, contrasting high and low performing schools. As a second part of the project, user types and data from standard-based assessment of student achievement will be linked to evaluate long term effects of different evaluation policies. Including different policies, identified barriers of evaluative knowledge use and usability problems, adapted feedback systems of standard-based testing will be created.
At the ECER-conference the results of the online-survey (evaluation policies) as well as an outlook on the organisational features that are to be studied in depth in the case studies will be presented.
Method
Expected Outcomes
References
Groß Ophoff, J., Koch, U., Hosenfeld, I. & Helmke, A. (2006). Ergebnisrückmeldungen und ihre Rezeption im Projekt VERA. In H. Kuper & J. Schneewind (Hrsg.), Rückmeldung und Rezeption von Forschungsergebnissen – Zur Verwendung wissenschaftlichen Wissens im Bildungssystem (S. 19-40). Münster: Waxmann. Kohler, B. (2005). Rezeption internationaler Schulleistungsstudien. Wie gehen Lehrkräfte, Eltern und die Schulaufsicht mit Ergebnissen schulischer Evaluationsstudien um? Münster: Waxmann. Kohler, B. & Schrader, F.-W. (2004). Ergebnisrückmeldung und Rezeption: Von der externen Evaluation zur Entwicklung von Schule und Unterricht. Empirische Pädagogik, 18 (1), 3-17. Luhmann, N. (2000). Organisation und Entscheidung. Wiesbaden. Maier, U. (2010). Accountability policies and teachers’ acceptance and usage of school performance feedback – a comparative study. School Effectiveness and School Improvement 21 (2), 145–165. Mandl, H. & Hense, J. (2007). Lässt sich der Unterricht durch Evaluation verbessern? In W. Schönig (Hrsg.), Spuren der Schulevaluation. Zur Bedeutung und Wirksamkeit von Evaluationskonzepten im Schulalltag (S. 85-99). Bad Heilbrunn: Klinkhardt. Mintrop, H. & Trujillo, T. (2007). The practical relevance of accountability systems for school improvement: a descriptive analysis of california schools. Educational Evaluation and Policy Analysis, 29 (4), 319-352. Mintzberg, H. (1983). Structure in Fives. Englewood Cliffs: Prentice-Hall. Scheerens, J. & Bosker, R. (1997). The foundations of educational effectiveness. Oxford, U.K.: Pergamon. Schneewind, J. (2006). Wie Lehrkräfte mit Ergebnisrückmeldungen aus Schulleistungsstudien umgehen. Berlin: Rau, Schneewind & Partner. Schrader, F.-W. & Helmke, A. (2004). Von der Evaluation zur Innovation? Die Rezeptionsstudie WALZER. Empirische Pädagogik, 18 (1), 140-161. Stamm, M. (2003). Evaluation und ihre Folgen für die Bildung. Münster, New York, München, Berlin: Waxmann. Terhart, E. (2002): Wie können die Ergebnisse von vergleichenden Leistungsstudien systematisch zur Qualitätsverbesserung in Schulen genutzt werden? In: Zeitschrift für Pädagogik, 48, 91-110 Van Ackeren, I. (2004). Evaluation, Rückmeldung und Schulentwicklung. Erfahrungen mit zentralen Tests, Prüfungen und Inspektionen in England, Frankreich und den Niederlanden. Münster u.a.: Waxmann. Visscher, A. J. & Coe, R. (2003). School Performance Feedback Systems: Conceptualisation, Analysis, and Reflection. School Effectiveness and School Improvement, 14(3), 321-349.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.