Session Information
09 SES 13 B, Theoretical and Methodological Issues in Testing and Measurement (Part 2)
Paper Session
Contribution
There have been considerable efforts to develop instruments for a European large scale assessment (LSA) in Vocational Education and Training (VET) since the publication of the Feasibility Study VET LSA by Baethge & Achtenhagen (2006). Political discussions pointed out that those instruments are not only expected to provide estimates of knowledge and cognitive skills. They should also deliver reliable predictions whether or not test takers are able to comply with the requirements of real life working tasks (workplace performance). This paper outlines the validation of a paper and pencil test for competence assessment of VET in the field of car mechatronics and focuses the latter issue.
Competency models for VET currently focus on relatively narrow sections of the entire competence development process (e.g. the outcome of the first of vocational training, see Gschwendtner 2008). As a consequence the measured constructs can be considered to be relatively narrow. The term “narrow construct” refers to a latent trait that enables individuals to show only a limited set of defined performances, in contrast to a “broader” construct that makes persons capable of showing relatively more or qualitatively different performances (see Slomp/Fruite 2005). Since instruments for assessment of “narrow constructs” are usually validated by external criteria which are not linked to workplace performance (e.g. grades) it is neither clear if the measured construct is identical to the capability necessary to act in authentic situations, nor if the attainment of the measured competence is a necessary step on the developmental continuum of such a “workplace performance” capability (Musekamp/Spöttl/Becker 2010).
An exception in the field of VET competence assessment is the simulation-based approach carried out by Nickolaus/Gschwendtner/Abele (2009) where validation has been realized by comparing the outcome of the simulation with the results of a real diagnosis process of a car. The study presented in this paper, however, chooses a different approach. It is assumed that competence measurement with paper and pencil instruments is possible as long as it can be shown that the written test items tap into the same latent trait that also underlies workplace performance. In order to check that assumption apprentices as well as skilled workers with several years of practical experience are included in the test sample.
The competence-assessment instrument is based on a model of “Competence for the Car Service & Repair Domain” which is structured by core work processes in the subdomains of “Service & Maintenance” and “Error Diagnosis”. These work processes have been surveyed by Becker/Spöttl/Hitz/Rauner (2002) by means of work analysis and expert interviews. The levels of competence were hypothetically specified by consideration of the Dreyfus/Dreyfus model of experience development (1987).
Method
Expected Outcomes
References
Baethge, M./ Achtenhagen, F. (2006). PISA-VET. A feasibility-study. Stuttgart: Steiner. Becker, M./ Spöttl, G./ Hitz, H./ Rauner, F. (2002). Wissenschaftliche Begleitung zur Neuordnung der fahrzeugtechnischen Berufe. Aufgabenanalyse für die Neuordnung der Berufe im Kfz-Sektor. Abschlussbericht. Bremen, Flensburg. Online verfügbar: http://www.itb.uni-bremen.de/uploads/media/Kfz-Neuordnung-Bericht4.pdf [21.02.2010]. Dreyfus, H. L./ Dreyfus, S. E. (1987). Künstliche Intelligenz. Von den Grenzen der Denkmaschine und dem Wert der Intuition. Reinbek bei Hamburg: Rowohlt. Gschwendtner, T. (2008). Ein Kompetenzmodell für die Kraftfahrzeugtechnische Grundbildung. In: R. Nickolaus; H. Schanz (Hrsg.): Didaktik der gewerblich-technischen Berufsbildung. Konzeptionelle Entwürfe und empirische Befunde. Baltmannsweiler: Schneider-Verl. Hohengehren, S. 103–119. Musekamp, F./ Spöttl, G./ Becker, M. (2010). Schriftliche Arbeitsaufträge zur Erfassung von Differenzen in der Expertise von Facharbeitern und Auszubildenden. In: Zeitschrift für Berufs- und Wirtschaftspädagogik, Bd. 106, H. 3. Nickolaus, R./ Gschwendtner, T./ Abele, S. (2009). Die Validität von Simulationsaufgaben am Beispiel der Diagnosekompetenz von Kfz-Mechatronikern. Vorstudie zur Validität von Simulationsaufgaben im Rahmen eines VET-LSA. Stuttgart. Online verfügbar: http://www.bmbf.de/pub/Abschluss-Bericht_Druckfassung.pdf [03.11.2009]. Slomp, D. H./ Fruite, J. (2005). Following Phaedrus: Alternate choices in surmounting the reliability/validity dilemma. In: Assessing Writing, Vol. 9, pp. 190–207.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.