RESULT AND DISCUSSION Improvement of Students’ MBT Score

III. RESULT AND DISCUSSION Improvement of Students’ MBT Score

Descriptive statistic of pretest and posttest scores of MBT is summarized in Table 3, and the corresponding box-plots are shown in Figure 1.

Table 3. Descriptive statistics of MBT score by Pretest and Posttest (N = 24)

First quartile

Third quartile

Mean & its std. error

Standard deviation

Skewness & its std. error

Kurtosis & its std. error

Figure 1. Boxplots of MBT Score by pretest and posttest (N = 24)

Mean difference between posttest and pretest is 8.13 (SD = 2.83). Paired samples t-test (using SPSS) yields t = 14.05, df = 23, and p = 0.000 (<0.01, two-tails). It means that the means of posttest and pretest are significantly different. The Cohen’s d-effect size is about 2.50 in the unit of pooled standard deviation. It is calculated using formula d = (M post M pre )/SD pooled

(Morgan et al., 2004: 90; Ellis, 2010: 10), where . = = 3.24. Compare to the expected maximum score (22), students’ achievement on BMT are

about 40% for pretest and 77% for posttest. According to Hestenes and Wells (1992), a score of 60% on the MBT is a kind of conceptual threshold for problem-solving competence. Below this threshold, student’s grasp of Newtonian concepts is too limited for effective problem solving. Another critical score is of about 80%. It is the threshold for mastery of basic Newtonian concepts. They believed that when it is approached, other goals of physics instruction will much easier to attain. According to the criteria, we argue that the representational approach implemented in this study has successfully brought most students about 40% for pretest and 77% for posttest. According to Hestenes and Wells (1992), a score of 60% on the MBT is a kind of conceptual threshold for problem-solving competence. Below this threshold, student’s grasp of Newtonian concepts is too limited for effective problem solving. Another critical score is of about 80%. It is the threshold for mastery of basic Newtonian concepts. They believed that when it is approached, other goals of physics instruction will much easier to attain. According to the criteria, we argue that the representational approach implemented in this study has successfully brought most students

It is interesting to look at the improvement of students’ MBT score. Firstly, look at the score on pretest. This score is comparatively low. The maximum score is only 14 with mean of

8.75 (SD = 3.48). Our previous study (Sutopo, et.al; 2011) with subjects consisted of 35 students had taken Introductory Physics courses plus 24 students had taken both Introductory Physics and Mechanics courses also showed the similar result. PPTS’s MBT score was 7.41 (SD = 3.95) and maximum score was 15. All of students participated in our current study had also taken both Introductory Physics and Mechanics courses. It means that most PPT students had not yet mastered the concepts of mechanics covered in MBT, even though they had learned more mechanics in advance. Secondly, as we mentioned in advance, the learning activity during the treatment did not pay particular attention to the concepts asked in instrument. There was no special time or session to discuss problems asked in test. Therefore, it can be concluded that the representation approach implemented during the treatment indeed gave students such useful learning outcomes that enable students to overcome their difficulty on mechanics.

Improvement of students’ reasoning ability

Figure 2a shows the shift of distribution of technical-reasoning ability levels from pretest to posttest, whereas Figure 2b shows the shift for conceptual validity levels. The figure shows that the representational approach has increased the proportion of the two highest level of technical reasoning from about 30% (pretest) to 74% (posttest). The approach has also increased the proportion of the two highest level of conceptual validity from about 35% (pretest) to 79% (posttest).

Figure 2a. Distribution of technical-reasoning ability levels by pretest and posttest (N = 528 reasoning units for each data set)

Pretest CV

Post test CV

Figure 2b. Distribution of conceptual validity levels of students’ reasoning by pretest and

posttest (N = 528 reasoning units for each data set)

The affectivity of the approach to improve students’ reasoning ability can also be described in term of students’ reasoning score as shown in Table 4.

Table 4. Descriptive statistics of students’ reasoning score by pretest and posttest (N= 24)

Statistics

Technical aspect (scale 0-4)

Conceptual validity (scale 0-3)

Posttest Minimum

3.0 4.0 2.1 3.0 First quartile

1.7 2.8 1.0 2.1 Median

2.1 3.2 1.3 2.4 Third quartile

2.4 3.7 1.8 2.8 Mean & its std. error

2.36 (0.09) Standard deviation

0.46 0.53 0.41 0.44 Skewness & its std.  0.06 (0.47)  0.33 (0.47)

0.09 (0.47)  0.29 (0.47) error

From the values of skewness, it can be concluded that all of the data sets are normally distributed or at least approximately normal (Morgan et. al., 2004). Therefore, it is possible to perform paired sample t-test. The test yields t =12.47, df = 23, and p = 0.000 (<0.01, two-tails) for technical aspect and t = 13.49, df = 23, and p = 0.000 (<0.01, two-tails) for conceptual validity aspect. It means that the means difference between pair of data sets are significant. The d-effect sizes are 2.38 for technical aspect and 2.40 for conceptual validity aspect. In another word, the representational approach implemented during the treatment has indeed improved students’ reasoning ability.