selfquiz/aboutq.html], is about writing multiple choice questions that are fair but hard to guess.
It might be of interest to my students—for example, to warn them away from the usual guessing strategies discussed below—or to visiting educators. Multiple choice questions are widely scorned as "multiple guess" questions.
True/false questions also raise the problem of response bias.
Some people are consistently more inclined to answer True or False.
When teaching auditorium-sized introductory psychology classes based on my textbook (the same one now online) I used the study questions ("quickcheck" questions) that were in the margins of each page, next to the relevant material, in the print version.
These study questions covered virtually every important concept from the chapter.One must also design a test so that answers are not obvious to the student who has merely skimmed the assignment or studied only highlighted words or read only summaries.To encourage high-quality studying, one must defeat the common rules of thumb students use to guess correct answers.The criterion of success in writing a fair test item is simple.A student examining the item, while the book is open and turned to the relevant page, should agree the item is fair.The more a person knows about a subject matter, the easier it is to make arguments in favor of answers somebody else might regard as wrong. Often the truth value of an isolated statement is quite debatable!It all depends on how it is interpreted, the definition of a key term, or complexities of context.As for the complaint that giving students study questions results in "teaching the test", the solution is to offer study questions for all the most important ideas in an assignment. Teaching the Test (so to speak) was actually part of the system in Keller Plans, an independent study method popular in the 1970s.Students were given lots of specific objectives or study questions, and they received many opportunities to take quizzes covering that material. In fact, Keller Plans were one of the few educational innovations in the entire 20th Century that produced better results, consistently, than traditional lecture/discussion methods of instruction. Over the years I noticed that my best students disliked them the most.When these are eliminated, learning the material becomes the easiest way to pass a quiz.Rule of thumb: "Do not pick the same alternative more than twice in a row." To defeat this strategy: use randomly generated answer positions.