Discussion
The conclusions from this study do not directly address the research question given here because a key assumption, that option homogeneity consistently affects item difficulty, was not true for this dataset. The lack of an effect related to the homogeneity of options was unexpected because four previous research studies had all detected a directional effect, and Guttman and Schlesinger’s argument is logically appealing. Notably, the one dissenting research study sampled an adult, college-educated population like the sample used for this study.
Considering the previous studies, the results of this research suggest at least two possible theories. First, the effect may be age related. Previous studies that used undergraduate or younger examinees all found an effect. The Ascalon et al. (2007) and Smith and Smith (1988) studies both used high-school student responses for item calibrations. The Guttman and Schlesinger (1966) study used responses from students in Grades 7 through 9. Green’s (1984) respondents were undergraduate students, but the specific academic level of the students was not indicated.
The one study that did not find an effect (Downing et al., 1991) was conducted using items from an examination developed by the National Board of Medical Examiners. The sample responses came from medical students who were testing for professional licensure.
Their results along with those from this study suggest that option homogeneity may have a greater effect on younger examinees and that the effect appears to diminish or disappear in the adult population (or by education level). This change could be attributed to the maturity level of the test taker. An older, more mature individual may recognize the significance of high-stakes testing for licensure which in turn could affect preparation for and focus during the test.
