Writing Good questions

Home Writing Good questions

Identifying Test Worthy Items

Once you have selected readings for the week’s preparation, skim the readings and make a list of critical ideas that student need to get from the preparatory materials. Often after skimming, you can do a slower read, listing important concepts, definitions, and ideas that the student need to get started. You use this list both to develop questions that check students understanding of critical concepts, principles, factual understanding, laws, rules, etc. and to develop preamble/wrapper for reading, so students know what to pay attention to.

Writing Multiple-Choice Questions

The test should be a mix of approximately 30% remembering (did you do the readings?), approximately 30% understanding (did you understand what you read?), and finally, 40% application,. The application questions can be in the form of “which concept applies to this situation” (are you ready to use what you have read?). To use a book analogy, you want to write these tests more at the table-of-contents level then at the index level.

You can include a few simpler questions that just provide simple accountability that the student has completed the readings. Try to ask about topics that students are likely to interpret incorrectly. Test common misconceptions that might undermine students’ ability to successfully begin problem-solving. You can ask which concept applies to a given situation or scenario. You can focus on the relationship between concepts; this is an efficient way to test two concepts at once.

Multiple-Choice questions have two main parts: the question stem or leader, and the options (which include a correct answer). When beginning to construct a multiple-choice question, write the stem of the question first. A well-constructed stem is a stand-alone question that could be answered without examining the options. The wording of the stem and the verbs it contains determines the overall difficulty of the question.

Bloom’s Taxonomy

Remembering

recalling, defining, recognizing, listing, describing, retrieving, naming

Common Question Leaders: How is…? Where is…? When did … happen? How would you describe…? Can you select….? Why did….?

Understanding

explaining ideas or concepts, interpreting, summarizing, paraphrasing, classifying, explaining, locating, identifying, restating

Common Question Leaders: How would you classify…? What facts or ideas show….? Interpret in your own words…? Which statement supports…? How would you summarize…? What is the main idea of…?

Applying

using information in another familiar situation, implementing, carrying out, using, executing , translate, employing, illustrating

Common Question Leaders: What is the best first step? What is the most significant problem? What would be the worst thing to do? Would it be a mistake to…? What is the most common mistake? Which test would you order next? What is the most common diagnosis? How would you use…? How would you solve? What is the most logical order? What approach would you use..? What would result if….? What facts would you select to show…?

This next section highlights some of Bill Roberson’s excellent work.

Writing Questions at different levels

Low-level Questions

What did the text say? (Remembering)

What did the text mean? (Understanding)

How could you apply it? (Recognize an example of a concept)

Low-level Questions that invite discussion

Which statement is most accurate?

Based on the theory that you just read about, what is most likely to happen is we apply X?

Which of these items best represent the qualities/characteristics of X?

Higher-level Question that invite discussion

Based on what you have read about theory A, which of the strategies listed below has the best chance of success, given the specified conditions (X, Y, Z)?

Some Rules for Question Writing

For good question stems, consider following rules:

  • Stems should be stand-alone questions.
  • Stems should be grammatically complete.
  • Negative stems should be used with caution.
  • If a key word appears consistently in the options, try to move it to the stem.
  • Word the stem such that one option is indisputably correct.

For creating good options, consider following rules:

  • Make sure each incorrect option is plausible but clearly incorrect.
  • Make sure that the correct answer (keyed response) is clearly the best.
  • Avoid, if possible, using “all of the above”.
  • Use “none of the above” with caution.
  • Try to keep options similar lengths, since test-wise students will pick the longest option if unsure       (too long to be wrong).
  • Make sure options are grammatically consistent with the stem (question leader) and use parallelism.
  • Make sure that numerical answers are placed in numerical order, either ascending or descending.

Well-constructed multiple-choice questions are not easy to create. But the quality of the multiple-choice questions you use in your Team Test can make or break the tone of your class. Nothing is more uncomfortable than rushing poor questions to the classroom and having to endure the inevitable student backlash. Good questions are absolutely essential to our success, and putting in the effort to write good questions is worth your time and attention.

Spend time reviewing and revising your questions. It can be very helpful to have a colleague look at your questions. When we write them we are often too close to see all the mistakes. Just like good writing is about good editing, good questions are about reflection and revision.