Author(s):Funda Savasci-Acikalin (presenting), Emine Adadan (presenting)

Conference:ECER 2011, Urban Education

Network:9. Assessment, Evaluation, Testing and Measurement


Session Information

09 SES 04 A, Assessment of Competencies and Attitudes in Science Classrooms

Paper Session


Room:KL 23/121a,G, 45

Chair:Paulina Korsnakova


Evaluating Secondary Students’ Conceptions of Solution Chemistry Using a Two-tier Diagnostic Instrument

Conceptual Framework, Objectives, and Research Questions


Researchers utilized various methods such as concept maps, interviews, and drawings, to assess students’ conceptions of solution chemistry (Calik, Ayas, & Ebenezer, 2005; Uzuntiryaki & Geban, 2005). However, these methods would not seem to be effectively and properly utilized by practicing teachers due to the overcrowded classes, and limited available class time (Treagust, 1995). Thus, to diagnose students’ conceptions of science, teachers frequently rely on multiple-choice questions. Yet, multiple-choice tests have a number of weaknesses such as an inability to assess the students’ reasoning and the possibility of guessing. To minimize the weaknesses of multiple-choice tests, researchers developed the two-tier multiple-choice diagnostic instruments (e.g., Treagust, 1995). The two-tier multiple-choice items consist of a first tier that require a response for a particular content, and a second tier that request the reason for such a response.  The two-tier instruments were found to be relatively familiar for students to respond to and more practical and valuable for teachers to use (Othman, Treagust, & Chandrasegaran, 2008). Researchers have designed conceptual tests to assess students’ ideas about the solution chemistry (e.g., Pinarbasi, Canpolat, Bayrakceken, & Geban, 2006; Uzuntiryaki & Geban, 2005); however, none of these instruments requested student reasoning; they just asked for the recall of certain pieces of information about the topic.


Thus, the purpose of the study was two-fold: a) to develop a two-tier multiple-choice instrument that included the several associated aspects of solution chemistry, and b) to identify the extent and features of Grade 11 students’ alternative conceptions about the related aspects of solution chemistry (e.g., the nature of solutions and dissolving, factors affecting the solubility of solids and of gases, the types of solutions, the concentration of solutions, colligative properties of solutions, and the electrical conductivity of solutions) by using currently developed two-tier instrument. Thus, the following research questions guided the study:

1.      Is a two-tier multiple-choice instrument, namely The Nature of Solutions and Solubility -Diagnostic Instrument (NSS–DI), able to identify Grade 11 students’ conceptual understandings of solution chemistry concepts?

2.      What is the nature and extent of Grade 11 students’ conceptions about the associated aspects of solution chemistry?



The Nature of Solutions and Solubility - Diagnostic Instrument (NSS–DI) was developed based on the two-tier multiple-choice instrument development procedures proposed by Treagust (1995). First, the scope of the relevant content of the instrument was established. Then, the initial version of the test was generated. Before obtaining the final version of the test, the instrument was pilot tested three times with the three different groups of Grade 11 science-track students. Each time the instrument was modified. The final version of the NSS–DI included 18 items.

A total of 756 Grade 11 science-track students from 14 public high schools were participated in the study. The sample consisted of 45.5% females, and 54.5% males. The test was administered to the entire sample within one class period.

Students’ responses to the content and reasoning tier of each item were scored separately, assigning ‘1’ and ‘0’ for each correct and incorrect response respectively. To obtain an overall score for each item, a score of ‘1’ was assigned, if students’ responses to both tiers of an item were correct, and ‘0’ if otherwise. This scoring is used for calculating the Cronbach’s alpha coefficient, item difficulty, and item discrimination of the test.

Expected Outcomes


The Cronbach alpha reliability coefficients for the content tier and the both tiers were found to be 0.753 and 0.778, respectively. These alpha values are moderately high compared to the values of other two-tier tests in the literature (e.g., Caleon & Subramaniam, 2010; Othman et al., 2008; among others). The difficulty level of the test changed from easy (0.69) to very difficult (0.10). The item discrimination indices of the test for the sample (n = 756) ranged from 0.19 to 0.73.

Twenty-five major alternative conceptions were identified by using the NSS–DI instrument. These conceptions were the ones existed in at least 10% of the participants, and the critical ones included: a) About 23% of the participants failed to identify the homogeneous mixtures of solids in solids or gases in gases as being a solution. b) About 28% of the participants considered that the addition of water increases the solubility of a salt at a certain temperature. c) About 40% of the participants identified a saturated solution with its precipitate as being a supersaturated solution. d) About 11% of the students determined the strength of electrical conductivity with respect to the molar concentration of a solution.



Caleon, I., & Subramaniam, R. (2010). Development and application of a three-tier diagnostic test to assess secondary students’ understanding of waves, International Journal of Science Education, 32(7), 939-961.

Calik, M., Ayas, A., & Ebenezer, J. V. (2005). A review of solution chemistry studies: Insights into students’ conceptions. Journal of Science Education and Technology, 14(1), 29-50.

Devetak, I., Vogrinc, J., & Glaar, S. A. (2009). Assessing 16-year-old students’ understanding of aqueous solution at submicroscopic level. Research in Science Education, 39(2), 157-179.

Ebenezer, J. (2001). A hypermedia environment to explore and negotiate students’ conceptions: Animation of the solution process of table salt. Journal of Science Education andTechnology, 10(1), 73-91.

Othman, J., Treagust, D. F., & Chandrasegaran, A. L. (2008). An investigation into the relationship between students’ conceptions of the particulate nature of matter and their understanding of chemical bonding. International Journal of Science Education, 30(11), 1531-1550.

Pinarbasi, T., Canpolat, N., Bayrakceken, S., & Geban, O. (2006). An investigation of effectiveness of conceptual change text-oriented instruction on students’ understanding of solution concepts. Research in Science Education, 36(4), 313-335.

Treagust, D. F. (1995). Diagnostic assessment of students’ science knowledge. In S. M. Glynn & R. Duit. (Eds.), Learning science in the schools: Research reforming practice (pp. 327-346). Mahwah, NJ: Lawrence Erlbaum Associates.

Uzuntiryaki, E., & Geban, O. (2005). Effect of conceptual change approach accompanied with concept mapping on understanding of solution concepts. Instructional Science, 33, 311-339.

Author Information

Funda Savasci-Acikalin (presenting)
Istanbul University, Turkey
Emine Adadan (presenting)
Bogazici University
Department of Secondary Science and Mathematics Education