Writing Classroom Exams

Experienced faculty members know that it takes a considerable amount of time and effort to write classroom exams that provide a reliable and valid assessment of student learning. “Reliable” means that your exam consistently measures student performance. “Valid” means the test results provide appropriate information for drawing fair and justifiable conclusions about what your students know and can do. Here are some suggestions for improving both the validity and the reliability of your classroom assessments.

Deciding the type of exam

If you are teaching a large number of students, if you need to test more content than could be covered in a couple of short-essay questions, or if you have more time to prepare the questions (pre-test) than you do to score them (post-test), you may prefer to give a multiple-choice exam than an essay/problem-solving exam. But remember that some students perform better by writing out essays or solutions than by taking multiple-choice exams and vice-versa. It's important that you provide an opportunity for all students to do their best by giving a variety of types of exams during the semester or by including both types of questions on the same exam.

Choosing between objective and subjective test items

There are two general categories of test items: (1) objective items which require students to select the correct response from several alternatives or to supply a word or short phrase to answer a question or complete a statement; and (2) subjective or essay items which permit the student to organize and present an original answer. Objective items include multiple-choice, true-false, matching and completion, while subjective items include short-answer essay, extended-response essay, problem solving and performance test items. For some instructional purposes one or the other item types may prove more efficient and appropriate. To begin out discussion of the relative merits of each type of test item, test your knowledge of these two item types by answering the following questions.

Test Item Quiz

(circle the correct answer)

1.

Essay exams are easier to construct than are objective exams.

T

F

?

2.

Essay exams require more thorough student preparation and study time than objective exams.

T

F

?

3.

Essay exams require writing skills where objective exams do not.

T

F

?

4.

Essay exams teach a person how to write.

T

F

?

5.

Essay exams are more subjective in nature than are objective exams.

T

F

?

6.

Objective exams encourage guessing more so than essay exams.

T

F

?

7.

Essay exams limit the extent of content covered.

T

F

?

8.

Essay and objective exams can be used to measure the same content or ability.

T

F

?

9.

Essay and objective exams are both good ways to evaluate a student's level of knowledge.

T

F

?

Quiz Answers

1.

TRUE

Essay items are generally easier and less time consuming to construct than are most objective test items. Technically correct and content appropriate multiple-choice and true-false test items require an extensive amount of time to write and revise. For example, a professional item writer produces only 9-10 good multiple-choice items in a day's time.

2.

?

According to research findings it is still undetermined whether or not essay tests require or facilitate more thorough (or even different) student study preparation.

3.

TRUE

Writing skills do affect a student's ability to communicate the correct "factual" information through an essay response. Consequently, students with good writing skills have an advantage over students who have difficulty expressing themselves through writing.

4.

FALSE

Essays do not teach a student how to write but they can emphasize the importance of being able to communicate through writing. constant use of essay tests may encourage the knowledgeable but poor writing student to improve his/her writing ability in order to improve performance.

5.

TRUE

Essays are more subjective in nature due to their susceptibility to scoring influences. Different readers can rate identical responses differently, the same reader can rate the same paper differently over time, the handwriting, neatness or punctuation can unintentionally affect a paper's grade and the lack of anonymity can affect the grading process. While impossible to eliminate, scoring influences or biases can be minimized through procedures discussed later in this booklet.

6.

?

Both item types encourage some form of guessing. Multiple-choice, true-false and matching items can be correctly answered through blind guessing, yet essay items can be responded to satisfactorily through well written bluffing.

7.

TRUE

Due to the extent of time required by the student to respond to an essay question, only a few essay questions can be included on a classroom exam. Consequently, a larger number of objective items can be tested in the same amount of time, thus enabling the test to cover more content.

8.

TRUE

Both item types can measure similar content or learning objectives. Research has shown that students respond almost identically to essay and objective test items covering the same content. Studies1 by Sax & Collet (1968) and Paterson (1926) conducted forty-two years apart reached the same conclusion:
"...there seems to be no escape from the conclusions that the two types of exams are measuring identical things." (Paterson, p. 246)
This conclusion should not be surprising; after all, a well written essay item requires that the student (1) have a store of knowledge, (2) be able to relate facts and principles, and (3) be able to organize such information into a coherent and logical written expression, whereas an objective test item requires that the student (1) have a store of knowledge, (2) be able to relate facts and principles, and (3) be able to organize such information into a coherent and logical choice among several alternatives.

9.

TRUE

Both objective and essay test items are good devices for measuring student achievement. However, as seen in the previous quiz answers, there are particular measurement situations where one item type is more appropriate than the other. Following is a set of recommendations for using either objective or essay test items: (Adapted from Robert L. Ebel, Essentials of Educational Measurement, 1972, p. 144).

1Gilbert Sax and LeVerne S. Collet, "An Empirical Comparison of the Effects of Recall and Multiple-Choice Tests on Student Achievement," Journal of Educational Measurement, vol. 5 (1968), 169-73.
Donald G. Paterson, "Do New and Old Type Examinations Measure Different Mental Functions?" School and Society, vol. 24. (August 21, 1926), 246-48.

Choosing when to use essay and when to use objective tests

Essay tests are especially appropriate when:

  • the group to be tested is small and the test is not to be reused.
  • you wish to encourage and reward the development of student skill in writing.
  • you are more interested in exploring the student's attitudes than in measuring his/her achievement.
  • you are more confident of your ability as a critical and fair reader than as an imaginative writer of good objective test items.

Objective tests are especially appropriate when:

  • the group to be tested is large and the test may be reused.
  • highly reliable test scores must be obtained as efficiently as possible.
  • impartiality of evaluation, absolute fairness, and freedom from possible test scoring influences (e.g., fatigue, lack of anonymity) are essential.
  • you are more confident of your ability to express objective test items clearly than of your ability to judge essay test answers correctly.
  • there is more pressure for speedy reporting of scores than for speedy test preparation.

Either essay or objective tests can be used to:

  • measure almost any important educational achievement a written test can measure.
  • test understanding and ability to apply principles.
  • test ability to think critically.
  • test ability to solve problems.
  • test ability to select relevant facts and principles and to integrate them toward the solution of complex problems.

In addition to the preceding suggestions, it is important to realize that certain item types are better suited than others for measuring particular learning objectives. For example, learning objectives requiring the student to demonstrate or to show may be better measured by performance test items, whereas objectives requiring the student to explain or to describe may be better measured by essay test items.

The matching of learning objective expectations with certain item types can help you select an appropriate kind of test item for your classroom exam as well as provide a higher degree of test validity (i.e., testing what is supposed to be tested). To further illustrate, several sample learning objectives and appropriate test items are provided below:

Learning Objectives

Most Suitable Test Item

The student will be able to categorize and name the parts of the human skeletal system.

Objective Test Item (M-C, T-F, Matching)

The student will be able to critique and appraise another student's English composition on the basis of its organization.

Essay Test Item (Extended-Response)

The student will demonstrate safe laboratory skills.

Performance Test Item

The student will be able to cite four examples of satire that Twain uses in Huckleberry Finn.

Essay Test Item (Short-Answer)

Planning the number of exams

The more exams administered during the term, the more reliable and valid the final course grades. This is because students have had more opportunities to demonstrate what they have learned. Besides, frequent quizzes early in the semester may facilitate students' learning, especially in introductory-level courses.

Determining length

The more items on an exam, the greater the reliability and validity of its intended use. With more items, students are asked about more material and have more chances to communicate their learning to you. However, if you have too many items, you are doing more to assess the quickness of student response than the extent of student knowledge. To help you judge the length of your exams, you can respond to your own test and then multiply the time it took you to complete it by two and one-half or three. For example, for an exam given in a typical 50-minute class period, it should take you no longer than 15 minutes to fill out the exam as if you were a student (from writing your name and reading the instructions, to bubbling in each multiple-choice answer or hand-writing the essay or the solution to the problem). If it takes you 15 minutes, it will probably take the students approximately 38-45 minutes.

Choosing content

Avoid testing an uneven amount of content in one or two particular topics, material that was not covered in either the lectures or required readings, and trivial information. Exams should reflect course material in approximately the same proportions that you covered it. Students should not encounter surprises.

Typing the exam

Students consider testing to be a serious exercise. Demonstrate your respect for the process by producing a professional-looking, easy-to-read, and error-free test.

  • Carefully proofread the exam for typographical errors.
  • Include a complete set of directions; tell students, if feasible, to write comments on the exam about how they interpreted your instructions or test questions.
  • Group together similar items (e.g., the items covering Chapters 1-3, the multiple-choice items, the items relating to a particular graph or image).
  • Provide point values for each item or set of items as well as for the test.
  • Make alternate forms of a multiple-choice exam by scrambling the order of the items and/or the response alternatives; this helps to reduce cheating.
  • If possible, have someone such as a TA, colleague, or student who previously took the course, review/take the exam to catch areas of confusion.

Administering the exam

You can provide a setting that enhances performance and discourages distraction by minimizing interruptions and by communicating announcements or corrections on a blackboard. Reduce students’ temptation to cheat by asking students to sit at least one seat apart from each other and by having a sufficient number of proctors to help distribute and collect exams quickly.

Handing back results

Make testing a learning experience by grading and returning the exams as soon as possible when student knowledge is still “fresh.” Take class time to go over the exam, item by item, once it is returned. If you are concerned about exam security, review each item and its answer using PowerPoint. Allow sufficient time for discussion, but terminate lengthy dialogues by inviting those students to see you after class or during your office hours.

Revising the exam

Use comments written on the exam or made during discussion to modify exam items for future use. Statistical analyses of student responses to each item and to each response alternative (an “item analysis”) can be obtained by using machine-scorable answer sheets in cooperation with CITL’s Measurement and Evaluation area.

Links

  • “Quizzes, Tests, and Exams”, from Tools for Teaching by Barbara Gross Davis

Resources

  • Ory, J., & Ryan, K. (1993). Tips for improving testing and grading. Newbury Park, CA: Sage Publishing.
Contact Exam Services

E-mail us at exams@illinois.edu or call 217-244-3839 if you have questions or would like a consultation.