Peer review in TVET: An evaluation concept beyond nations and cultures?
This survey investigated the extent to which a concept such as the German evaluation concept of peer review in TVET might be capable of transfer to other national and cultural context. This research question was embedded in the research field of comparative research into TVET and policy transfer (e.g. Steiner-Khamsi/Waldow 2012).
The study surveyed the factors both supporting and inhibiting transfer peer review in TVET into another national and cultural context using the example of China. The host country was selected according to the most-different-approach (Georg 2005), e.g. Germany and China vary in their culture, in the political system and in the TVET structure (e.g. Busemeyer/Trampusch2012). The study was designed to provide a ‘lessons learned’ approach to transfer peer review in TVET to other national and cultural context. The research design used the lesson-drawing approach (Rose 1991).
Peer review in TVET is one model for evaluating quality assurance in TVET schools and works participatively and democratically within a flat organisational hierarchy between the peer team and the TVET school teachers (Gutknecht-Gmeiner 2007). The Chinese evaluation concept follows the opposite approach: for example, the Chinese evaluation concept does not take account of the interests and needs of teachers (Yao 2008). It has a high hierarchical structure. If peer review in TVET could be transferred to Chinese TVET schools, this might overcome recent challenges faced by the Chinese evaluation concept.
The transfer of peer review to Chinese TVET schools requires an open approach, using a flexible pilot. The iterative micro-cycle of the design-based research approach enables policies to be adapted while the pilot study is still under way (Reinking/Bradley 2008). The study therefore used the iterative micro-cycle to implement transfer of the peer review concept.
This study used the environmental system model as a framework for interpretation (O’Connor 1988) and divided the interpretation framework into a macro level, an exo level, and a meso level. At the macro level, interpretation was based on the Chinese cultural model using findings from international management studies research and Chinese cultural studies. At the exo level, China’s national TVET system was used as the reference. At the meso level, the existing quality evaluation structure within Chinese TVET schools was used.
Peer review as a pilot for policy transfer to China was initiated by German researchers. A Chinese research team trained in the peer review concept implemented the pilot at four TVET schools in Shanghai. The implementation and evaluation of the pilot took place in 2014 and 2015. The implementation of peer review in the four schools was supported and evaluated by a German researcher fluent in Chinese.
The methodology at the testing stage involved participatory observation and qualitative interviews. Participatory observation was used during the study to document the process of peer review in each of the TVET school (Taylor/Bogdan/DeVault 2015). However, this observation did not provide insight into the mental attitudes of the participants, which was also relevant for answering the research question fully and comprehensively. The qualitative interview method was therefore used to survey the subjective perceptions and opinions of the participants. In total, four participatory observations and 20 qualitative interviews were carried out. The interviewees were peers, peer review coordinators and principals of TVET schools. The findings were evaluated using the content analysis method.
The findings showed clearly that not every element of the peer review model could be transferred to the Chinese context nor would fit into the organisational structure of Chinese TVET schools. The findings were divided into cultural, structural, program-specific and pilot-specific factors. At the cultural level, for example the strong hierarchical power structure of Chinese culture in particular made transferring the participatory features of the peer review concept difficult. The indirect communication style that characterises Chinese culture was both a benefit and a challenge. At the structural level, the full-time school-based training made the adaption of the peer team necessary. At the program-specific level, peer’s lack of experience in evaluation methods had an inhibiting effect. Furthermore the financial, personnel and time restrictions caused by the pilot had an inhibiting effect as well. However, the findings also indicated that the peer review concept reduces teachers’ fears about participating in quality evaluation and increases their willingness to take on responsibility for quality development within their school.
The paper argues that the peer review concept could be transferred to the Chinese context. But the concept clearly needs to be adapted and therefore requires a support system that fits into the Chinese context. The supporting and inhibiting factors derived from this study enable us to draw lessons about how to support and adapt the transfer of peer review in TVET to different national and cultural context. The summary conclusion is that different cultural and structural contexts make a one-to-one transfer of evaluation concepts like peer review in TVET impossible. Successful transfer of evaluation concepts requires adaptations and a long-term perspective.
Buseymeyer, M. R.& Trampusch, C. (Eds.) (2012). The Political Economy of Collective Skill Formation. Oxford: Oxford University Press.
Georg, W. (2005). Vergleichende Berufsbildungsforschung. In: F. Rauner (Ed.), Handbuch der Berufsbildungsforschung (pp. 186-193). Bielefeld: Bertelsmann.
Gutknecht-Gmeiner, M. (2007). European Peer Review Manual for initial VET. Vienna: öibf.
O’Connor, G. (1988). Case management: System and practice. Social Casework, 69(2), 97-106.
Phillips, D., & Ochs, K. (2003). Processes of policy borrowing in education: Some explanatory and analytical devices. Comparative Education, 39(4), 451-461.
Reinking, D., & Bradley, B.A. (2008). Formative and design experiments. Amsterdam, New York: Teachers College Press.
Rose, R. (1991). What is lesson-drawing? Journal of Public Policy, 11(1), 3-30.
G. Steiner-Khamsi, & F. Waldow (Eds.) (2012), World yearbook of education 2012. Policy borrowing and lending in education (S. 3-17). New York: Routledge.
Taylor, S.; Bogdan, R. & DeVault, M. (2015). Introduction to Qualitative Research Methods: A Guidebook and Resource. New Jersey: Willey.
Yao, A. (2008). 沿革与反思: 我国高等职业教育质量保障制度建设二十年 [20 years quality management in TVET China]. 教育与职业 [Education and Vocation], 23(8), 12-13.