دانلود رایگان مقاله انگلیسی تاثیر همگنی انتخاب در موارد چند گزینه - Sage 2018

Abstract

Previous research has found that option homogeneity in multiple-choice items affects item difficulty when items with homogeneous options are compared to the same items with heterogeneous options. This study conducted an empirical test of the effect of option homogeneity in multiple-choice items on a professional licensure examination to determine the predictability and magnitude of the change. Similarity of options to the key was determined by using subject matter experts and a natural language processing algorithm. Contrary to current research, data analysis revealed no consistent effect on item difficulty, discrimination, fit to the measurement model, or response time associated with the absence or presence of option homogeneity. While the results are negative, they call into question established guidelines in item development. A hypothesis is proposed to explain why this effect is found in some studies but not others.

Discussion

The conclusions from this study do not directly address the research question given here because a key assumption, that option homogeneity consistently affects item difficulty, was not true for this dataset. The lack of an effect related to the homogeneity of options was unexpected because four previous research studies had all detected a directional effect, and Guttman and Schlesinger’s argument is logically appealing. Notably, the one dissenting research study sampled an adult, college-educated population like the sample used for this study.

Considering the previous studies, the results of this research suggest at least two possible theories. First, the effect may be age related. Previous studies that used undergraduate or younger examinees all found an effect. The Ascalon et al. (2007) and Smith and Smith (1988) studies both used high-school student responses for item calibrations. The Guttman and Schlesinger (1966) study used responses from students in Grades 7 through 9. Green’s (1984) respondents were undergraduate students, but the specific academic level of the students was not indicated.

The one study that did not find an effect (Downing et al., 1991) was conducted using items from an examination developed by the National Board of Medical Examiners. The sample responses came from medical students who were testing for professional licensure.

Their results along with those from this study suggest that option homogeneity may have a greater effect on younger examinees and that the effect appears to diminish or disappear in the adult population (or by education level). This change could be attributed to the maturity level of the test taker. An older, more mature individual may recognize the significance of high-stakes testing for licensure which in turn could affect preparation for and focus during the test.