MITE Monthly Tip
October 2020
Eric Brown, MD
How to construct good multiple-choice test questions:
Assessment, or testing, is an important aspect of medical teaching and learning. When done well, testing helps learners meet curricular goals while communicating what the teacher views as important. Multiple choice questions (MCQ) remain a mainstay in testing because they can assess a broad range of knowledge in short period of time, leading to a high degree of testing validity and reliability. When constructed well, MCQs can test higher order learning such as comprehension, application, and analysis. However, as anyone who has ever written a MCQ knows, they can be difficult and time-consuming to construct. Even educators formally trained in the process plan for up to an hour writing each question. For this reason, many poorly written MCQs still abound in medical education and only serve to confuse and frustrate learners. Hopefully these basic concepts and tips will help you demystify the process, avoid pitfalls, and better align learners with your curriculum.
1) Start with your learning objectives:
This probably goes without saying, but a direct relationship between learning objectives and test items must exist. Remember that learning objectives should be written around specific learner behavior and not what the program will “teach”. Following this rule will help avoid simply testing medical trivia.
2) MCQ Terminology:
The Stem
Most MCQs start with the stem, or lead in. The stem should be written as a complete sentence and the item should be answerable without reading all of the answer choices. Instead of a sentence completion (eg Treatment with beta-blockers has been shown to . . . ), the stem should be written as a complete sentence (eg Which of the following is a characteristic of treatment with selective beta-blockers?) Although clinical vignettes are used commonly as the stem, they should not be excessively verbose or contain “red herrings”. Remember that most MCQs should be written so that the learner can read and answer the question in under one minute.
Key point: The stem should be stated so that only one of the options can be substantiated and that option should be indisputably correct. As the question writer, it is wise to document (for later recall) the source of its validity. Above all, avoid negative phrasing.
The Options
These are your test-takers’ answer choices. Between three and five options is ideal. They should follow grammatically from the stem and all be of the same type (all diagnoses, all tests, all therapies). The options should all be similar in length and complexity and should not consistently overlap to avoid ambiguity.
Key point: Avoid options none of the above and all of the above. None of the above is problematic in questions of judgement, and it only informs what the test-taker knows is not correct. All of the above only requires that the test-taker understand two of the options.
The Distractors
These are the answer choices that are not correct. Distractors are essential to write well as they form the basis of an exam’s ability to discriminate (separate those who know from those you do not know).
Key point: The best distractors are statements that are accurate, but do not fully answer the question or incorrect statements that seem right to the test-taker.
3) Types of multiple-choice questions:
True/False question
These questions usually start with a stem and then ask learners to choose all the options that are true (or false). They are usually easy to write but have not been shown to discriminate well between learners that know the content and those that do not.
Key point: True/False type questions should generally be avoided. They are generally either too ambiguous or, when written to remove ambiguity, too easy without good discrimination.
Single best answer questions
This is the most common and best validated MCQ type. In their simplest form they test knowledge, but with some effort, and questions can be written to test comprehension and even higher order learning. This format helps avoid confusion on the part of the learner because they do not have to guess what the writer was “thinking”. Currently, the national board of medical examiners (NBME) uses only this type of question.
Key point: The NBME recommends the following basic rules for crafting an effective single best answer MCQ:
1) Rule 1: Each item should focus on an important concept or testing point.
2) Rule 2: Each item should assess application of knowledge, not recall of an isolated fact.
3) Rule 3: The item lead-in (and stem) should be focused, closed, and clear; the test-taker should be able to answer the item based on the stem and lead-in alone.
4) Rule 4: All options should be homogeneous and plausible, to avoid cueing to the correct option.
5) Rule 5: Always review items to identify and remove technical flaws that add irrelevant difficulty or benefit savvy test-takers.
Additional reading/references:
Download PDF