Authors

1. Mr. Tauseef Ahmad

Affiliation

• Research Analyst, College of Medicine, King Saud University, Riyadh, Saudi Arabia

Abstract

Method plays an important role in assessing students in undergraduate medical education. This
study examined multiple-choice question formats and traditional MCQ. Two subject examinations
(Medicine and Obstetrics & Gynecology) were administered to 3rd (n = 80) and 4th (n = 85) year
medical students, Saudi Arabia. They were administered using the three different MCQ formats.
Descriptive statistics were used to evaluate trends. Comparisons between test scores were made
using t-tests, correlations, and analysis of variance.
Student grade point average ranged between 3.00 to 4.74 (out of 5.00). Overall, students scored
lower for National Board relative to the College final examination. Correlations between the College
final examinations with the National Board or USMLE pattern examinations were not significant for
males; however, correlations were significant for females, indicating possible gender differences.
Findings from this study offer support for different MCQ formats that may contribute to understanding
learner performance.

Keywords

MCQ Format, MCQ Guidelines, Learner Outcomes

References

1. Williams, R. G. (1993). Use of NBME and USMLE examinations to evaluate medical education programs. Academic medicine: journal of the Association of American Medical Colleges, 68(10), 748-752.
2. Khan, K. Z., Ramachandran, S., Gaunt, K., & Pushkar, P. (2013). The objective structured clinical examination (OSCE): AMEE guide no. 81. Part I: An historical and theoretical perspective. Medical Teacher, 35(9), e1437-e1446.
3. Walubo, A., Burch, V., Parmar, P., Raidoo, D., Cassimjee, M., Onia, R., & Ofei, F. (2003). A model for selecting assessment methods for evaluating medical students in African medical schoolsAcademic Medicine, 78(9), 899-906..
4. Van Der Vleuten, C. P. (1996). The assessment of professional competence: developments, research and practical implications. Advances in Health Sciences Education, 1(1), 41-67.
5. King, A., & Hoppe, R. B. (2013). “Best practice” for patient-centered communication: a narrative review. Journal of graduate medical education, 5(3), 385-393.
6. Tractenberg, R. E., Gushta, M. M., Mulroney, S. E., & Weissinger, P. A. (2013). Multiple choice questions can be designed or revised to challenge learners
7. Gajjar, S., Sharma, R., Kumar, P., & Rana, M. (2014). Item and test analysis to identify quality multiple choice questions (MCQs) from an assessment of medical students of Ahmedabad, Gujarat. Indian journal of community medicine: official publication of Indian Association of Preventive & Social Medicine, 39(1), 17.
8. Shumway, J. M., & Harden, R. M. (2003). AMEE Guide No. 25: The assessment of learning outcomes for the competent and reflective physician. Medical teacher, 25(6), 569-584.
9. Shugars, D. A. (1991). Healthy America: Practitioners for 2005. An Agenda for Action for US Health Professional Schools.
10. Schuwirth, L. W., & Van Der Vleuten, C. P. (2004). Changing education, changing assessment, changing research?. Medical education, 38(8), 805-812.
11. Sarin, Y. K., Khurana, M., Natu, M. V., Thomas, A. G., & Singh, T. (1998). Item analysis of published MCQs. Indian pediatrics, 35, 1103-1104.
12. Rivo, M. L., Mays, H. L., Katzoff, J., Kindig, D. A., Marylander, S. J., Mays, H., ... & Bryan, G. T. (1995). Managed health care: implications for the physician workforce and medical education. JAMA, 274(9), 712-715.
13. Pinckney, B. A., Borcher, G. M., & Clemens, E. T. (1993). Comparative Studies of True/false, Multiple Choice and Multiple-multiple Choice. NACTA journal, 29, 50-54.
14. Challenges, C. (1995). Revitalizing the health professions for the twenty-first century. The Third Report of the Pew Health Professions Commission. Center for the Health Professions, San Francisco.
15. Pepple, D. J., Young, L. E., & Carroll, R. G. (2010). A comparison of student performance in multiple-choice and long essay questions in the MBBS stage I physiology examination at the University of the West Indies (Mona Campus). Advances in physiology education, 34(2), 86-89.
16. O
17. Newble, D. I., & Jaeger, K. (1983). The effect of assessments and examinations on the learning of medical students. Medical education, 17(3), 165-171.
18. Dillon, G. F., Boulet, J. R., Hawkins, R. E., & Swanson, D. B. (2004). Simulations in the United States medical licensing examination™(USMLE™). BMJ Quality & Safety, 13(suppl 1), i41-i45.
19. Moeen-Uz-Zafar, B. A. (2011). Evaluation of mini-essay questions (MEQ) and multiple choice questions (MCQ) as a tool for assessing the cognitive skills of undergraduate students at the Department of Medicine. International Journal of Health Sciences, 5(2 Suppl 1), 43.
20. Karl, M., Graef, F., Eitner, S., Beck, N., Wichmann, M., & Holst, S. (2007). Comparison between computer-aided testing and traditional multiple choice: an equivalence study. European Journal of Dental Education, 11(1), 38-41.
21. Jones, R. F. (1997). Academic Medicine: Institutions, Programs, & Issues. Association of American Medical Colleges.
22. Hoffman, K. I. (1993). The USMLE, the NBME subject examinations, and assessment of individual academic achievement. Academic medicine: journal of the Association of American Medical Colleges, 68(10), 740-747.
23. Hingorjo, M. R., & Jaleel, F. (2012). Analysis of one-best MCQs: the difficulty index, discrimination index and distractor efficiency. JPMA-Journal of the Pakistan Medical Association, 62(2), 14.
24. Gates, F. R., & Hoyer, W. D. (1986). Measuring miscomprehension: A comparison of alternate formats. ACR North American Advances.
25. Gajjar, S., Sharma, R., Kumar, P., & Rana, M. (2014). Item and test analysis to identify quality multiple choice questions (MCQs) from an assessment of medical students of Ahmedabad, Gujarat. Indian journal of community medicine: official publication of Indian Association of Preventive & Social Medicine, 39(1), 17.
26. Gagne, R. M., Briggs, J. L., & Wager, W. W. (1992). Principal of Instructional Design. America: Harcourt Brace College Publishers
27. Edelstein, R. A., Reid, H. M., Usatine, R., & Wilkes, M. S. (2000). A comparative study of measures to evaluate medical students
28. Crooks, T. J., & Mahalski, P. A. (1985). Relationships among assessment practices, study methods, and grades obtained. Research and development in higher education, 8, 234-240.
29. Collins, J. P., & Gamble, G. D. (1996). A multi-format interdisciplinary final examination. Medical education, 30(4), 259-265.
30. Cohen-Schotanus, J. A. N. K. E. (1999). Student assessment and examination rules. Medical Teacher, 21(3), 318-321.
31. Clauser, B. E., Ripkey, D., Fletcher, B., King, A., Klass, D., & Orr, N. (1993). A comparison of pass/fail classifications made with scores from the NBME standardized-patient examination and part II examination. Academic Medicine, 68(10), S7-9.
32. Case, S. M., & Swanson, D. B. (1998). Constructing written test questions for the basic and clinical sciences (2nd ed., pp. 22-25). Philadelphia, PA: National Board of Medical Examiners.
33. Hingorjo, M. R., & Jaleel, F. (2012). Analysis of one-best MCQs: the difficulty index, discrimination index and distractor efficiency. JPMA-Journal of the Pakistan Medical Association, 62(2), 142.
34. Simbak, N. B., Aung, M. M. T., Ismail, S. B., Jusoh, N. B. M., Ali, T. I., Yassin, W. A. K., ... & Rebuan, H. M. A. (2014). Comparative Study of Different Formats of MCQs: Multiple True-False and SingleBest Answer Test Formats, in a New Medical School of Malaysia. International Medical Journal, 21(6), 562-566
35. Blumenthal, D. (1996). Quality of Care--What is It?-Part One of Six. N Engl J Med, 335(12), 891- 894.
36. Bloom, B. S. (1956). Taxonomy of educational objectives. Vol. 1: Cognitive domain. New York: McKay, 20-24.
37. Blake, R. L., Hosokawa, M. C., & Riley, S. L. (2000). Student performances on Step 1 and Step 2 of the United States Medical Licensing Examination following implementation of a problem-based learning curriculum. Academic Medicine, 75(1), 66-70.
38. Biggs, J. (1979). Individual differences in study processes and the quality of learning outcomes. Higher education, 8(4), 381-394.
39. Berner, E. S., Brooks, C. M., & Erdmann, J. B. (1993). Use of the USMLE to select residents. Academic medicine: journal of the Association of American Medical Colleges, 68(10), 753-759.
40. Begum, T. (2012). A guideline on developing effective multiple choice questions and construction of single best answer format. Journal of Bangladesh College of Physicians and Surgeons, 30(3), 159-166.
41. Bauer, D., Kopp, V., & Fischer, M. R. (2007). Answer changing in multiple choice assessment change that answer when in doubt–and spread the word!. BMC medical education, 7(1), 28.
42. Nelson, M. E., Rejeski, W. J., Blair, S. N., Duncan, P. W., Judge, J. O., King, A. C., ... & Castaneda- Sceppa, C. (2007). Physical activity and public health in older adults: recommendation from the American College of Sports Medicine and the American Heart Association. Circulation, 116(9), 1094.
43. Anderson, M. B. (1993). Proceedings of the AAMC
44. Ananthakrishnan, N., Ananthakrishnan, S., & Oumachigui, A. (1993). MBBS examinations--are we asking the right questions?. Journal of postgraduate medicine, 39(1), 31.