Session Information
12 SES 02, NW 12, Session 2
Paper Session
Time:
2009-09-28
11:15-12:45
Room:
NIG, HS 3E
Chair:
Peter K. P. Meyer
Contribution
In the context of increasingly common and frequent evaluations in educational science, the need to design and operationalize standardized quality indicators in this field is growing.
This paper reports on a project sponsored by the German Research Foundation (DFG) and looks at the project’s results in the light of transnational transferability and general conclusions for the field of European publication reference systems.
The project is exploratory and its objective is not to assess or compare research performance of individual scholars, research groups or university departments, but to develop a methodology for constructing research performance indicators. Initial point for the project was the fact that international instruments for analysing scientific publications include German educational science only marginally, and German databases inadequately support the demands of scientometric analysis. Moreover, the development of new publishing methods renders digital publications relevant. These are not assessed by conventional scientometric methods, which mainly focus on printed journal publications. This starting deficit analysis works also on the European scale.
The project analysed the specific characteristics of the publication and communication culture in the field of educational science. Transferable results of this part of the project will be presented. The project intends to develop a new multi-attributive indicator of scientific relevance, based on two methodological paradigms: the measurement of well-defined characteristics of a publication itself as well as of the way it is used and the measurement of the characteristics of the editorial process it has undergone, i.e. to base the relevance assessment on the affiliation of a publication to a certain journal, publisher or editor. The first way of measuring relevance applies methods of assessing usage of documents, such as log-file analysis, but is aware of the fact, that usage-based metrics are not unproblematic since they indicate merely interest, not user’s reception. In addition, document characteristics such as content, target group or purpose will be used as signals of relevance. The second approach to rate scientific publications is based on the reliability of the source respectively on the evaluation of the editorial process. A key element in this approach is the rating of journals and publishers and their reputation by experts in the field. Preliminary findings and obstacles of this explorative approach are illustrated.
Method
survey of relevance rankings by experts
Expected Outcomes
Results of related projects (EERQI, ERIH) are included to develop a perspective for coordinated European efforts to establish better instruments for monitoring educational research publications.
References
Botte, A. (2007). Scientometric Approaches to Better Visibility of European Educational Research Publications: a state-of-the-art-report. European Educational Research Journal, 6(3), 303-310. Dees, W. (2008). Innovative Scientometric Methods for a Continuous Monitoring of Research Activities in Educational Science. In H. Kretschmer & F. Havemann (Eds.), Fourth International Conference on Webometrics, Informetrics and Scientometrics & Ninth COLLNET Meeting. Berlin: IBI. http://www.collnet.de/Berlin-2008/DeesWIS2008ism.pdf
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.