Session Information
09 SES 06 C, Issues in Developing Tests and Diagnostic Tools
Paper Session
Contribution
With the turn of 2012 the United Nations Literacy Decade expired. The target set was increasing literacy levels and developing literate environments worldwide. In Germany 14.5 % of the adults are functional illiterate[1] (Grotlüschen/Riekmann 2011). This problem is not isolated to Germany: For example in France they have 15 %, in England 21,8 % functional illiterate people. To increase literacy levels, an educational diagnostics approach appropriate for adults is necessary.
In the lea.project, funded by the German Federal Ministry of Education and Research, a hierarchical competence model for the lower rungs[2] was developed. The lea.-diagnostic tool delivers an individual diagnosis in reading, writing and mathematical skills. The diagnostic tool is provided in a paper based form and in an online version. The paper based version is already used by literacy teachers all over Germany. Unfortunately, only 30.000 illiterate people find their own path to literacy courses (Huntemann/Reichart 2011). Often they feel ashamed about their situation and they won`t find their own path to literacy courses. As well, there is a higher level of support required with the paper based version.
One way to tackle these problems is an online tool which can be used anonymously with every internet-capable computer. Challenging the problem of low ICT-Literacy[3] the system uses multimedia cues (audio and video) to help people with low literacy to take a diagnostic test. The system then generates a report for the tool user in easy to understand language as well as a more detailed diagnostic report for adult education counsellors to develop an individual learning path.
To collect valid data through e-assessment for functional illiterate people, it is indispensable to focus on the usability of the interface and the effect that the test method and the low ICT-Literacy might have on the scores.
There are a lot of usability principles and guidelines (Nielsen/Loranger 2006; Shneiderman/Plaisant 2005). To minimize the effect on the test score and to build up an e-assessment for illiterate adults, the usability of the diagnostic tool has to be developed especially for the target group and – of course – has to be evaluated within the target group itself.
Until now there is a big lack of usability guidelines for an online educational diagnostic tool and especially for illiterate people[4].
The research goals face two topics:
1) Evaluation of a computer based diagnostic tool
2) Derivation of general usability guidelines for functional illiterate people
[1] Another common expression is „ people with literacy, language and numeracy skill needs“.
[2] Lower Rungs describes the competence level of the lower steps (Level One) in the formal education system.
[3] Functional illiterate people have low Information and Computer Technology (ICT) -Literacy by trend (Niesyto 2009)
[4] For example Laitusis conducted some research on the effect of item formats with people with learning difficulties or physical disabilities (Laitusis 2010); Yatim developed usability guidelines for children (Yatim 2009). The special needs of illiterate people were not considered.
Method
Expected Outcomes
References
Grotlüschen, Anke/Riekmann, Wibke (2011): leo.-Level-One-Studie. Presseheft. Online: http://blogs.epb.uni-hamburg.de/leo/, zuletzt geprüft am 31.1.2013. Huntemann, Hella/Reichart, Elisabeth (2011): Volkshochschul-Statistik – Arbeitsjahr 2010. Bonn: Deutsches Institut für Erwachsenenbildung. Laitusis, Cara C. (2010): Examining the Impact of Audio Presentation on Tests of Reading Comprehension. In: Applied Measurement in Education, 2, 153-167. Nielson, Jakob/Loranger, Hoa (2006): Web Usability. München: Addison-Wesley. Niesyto, Horst (2009): Digitale Medien, soziale Benachteiligung und soziale Distinktion. In: Medienpädagogik – Zeitschrift Theorie und Praxis der Medienbildung, 1-19. Online: http://www.medienpaed.com/17/niesyto0906.pdf (10.01.2013). Reinmann, Gabi (2005): Innovation ohne Forschung? Ein Plädoyer für den design-Based Research-Ansatz in der Lehr-Lernforschung. Unterrichtswissenschaft, 33(1), 52–69. Snyder, Carolyn (2003): Paper Prototyping. Elsevier: San Francisco. Shneiderman, Ben/Plaisant, Catherine (2005): Designing The User Interface. Boston (u.a.): Pearson Education. Wolf, Karsten/Koppel, Ilka (2010): Paper-Prototyping im Rahmen der Entwicklung von Instrumenten der Förderdiagnostik funktionaler Analphabeten. In: Der Pädagogische Blick, 4, 221-230. Additional References: Bennett, Randy E./Braswell, James/Oranje, Andreas/Sandene, Brent/Kaplan, Bruce/Yan, Fred (2008): Does it Matter if I Take My Mathematics Test on Computer? A Second Empirical Study of Mode Effects in NAEP. In: The Journal of Technology, Learning, and Assessment, H. 6, 1-39. Edelson, Daniel C. (2002): Design Research: What We Learn When We Engage in Design. The Journal of the Learning Sciences, 11(1), 105–121. Lazar, Jonathan (2006): Web Usability : A User-Centered Design Approach. Boston: Pearson Addison Wesley. Plass, Jan L./Moreno, Roxana/Brünken, Roland (2010): Cognitive Load Theory. New York: Cambridge University Press. Wolf, Karsten/Koppel, Ilka/Schwedes, Kai (2011): Potenziale von Rich E-Assessment für die Förderdiagnostik funktionaler Analphabeten. In: Grotlüschen, Anke/Kretschmann, Rudolf/Quante-Brandt, Eva/Wolf, Karsten: Literalitätsentwicklung von Arbeitskräften, Münster: Waxmann, 122-153.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.