Teaching Healing Searching Serving
Ongoing Services  
Faculty & Staff
Teaching Scholars
Virtual Posters
Resources
  Educational Research Resources
  OED Library
UAMS eLearning
  Teaching Scholars Program
  OED Fact Sheets
  PBL Resources
Student Services
eLearning
UAMS Home
OED Home

Exam Writing Guidelines

For more information contact:

Item Writing

 

From: NBME’s Constructing Written Test Questions for the Basic and Clinical Sciences. Available at: http://www.nbme.org/PDF/ItemWriting_2003/2003IWGwhole.pdf

What Should be Tested?

  • Content should match course objectives
  • Important topics should be weighted more heavily than less important topics
  • Testing time devoted to each topic should reflect the relative importance of the topic
  • Sample of items should be representative of instructional goals

    • General Guidelines for Item Construction

  • Make sure the item can be answered without looking at the options OR that the options are 100% true or false.
  • Include as much of the item as possible in the stem; the stems should be long and the options short.
  • Avoid superfluous information.
  • Avoid "tricky" and overly complex items.
  • Write options that are grammatically consistent and logically compatible with the stem; list them in logical or alphabetical order.
  • Write distracters that are plausible and the same relative length as the answer.
  • Avoid using absolutes such as always, never, and all in the options; also avoid using vague terms such as usually and frequently.
  • Avoid negatively phrased items (i.e., those with except or not in the lead-in). If you must use a negative stem, use only short (preferably single word) options.

    • And most important of all: Focus on important concepts; don’t waste time testing trivial facts.

    Flaws that Tip Off Students to the Correct Answer

  • Grammatical cues - one or more distracters don’t follow grammatically from the stem.
  • Logical cues - a subset of the options is collectively exhaustive.
  • Absolute terms - terms such as "always" or "never" are in some options.
  • Long correct answer - correct answer is longer, more specific, or more complete than other options.
  • Word repeats - a word or phrase is included in the stem and in the correct answer.
  • Convergence strategy - the correct answer includes the most elements in common with the other options.

    • Flaws that Make Items Unnecessarily Difficult

  • Options are long or complicated, or contain duplicated portions.
  • Numeric data are not stated consistently.
  • Terms in the options are vague (examples--"rarely," "usually").
  • Language in the options is not parallel.
  • Options are in an illogical order.
  • "None of the above" is used as an option.
  • Stems are tricky or unnecessarily complicated.
  • The answer to an item is "hinged" to the answer of a related item.

    • Back to Top

    Time Guidelines

    Rule-of-Thumb Guidelines

      Type of Question

      		 Time to Complete
      True-False ˝ minute
      Multiple-Choice 1 minute
      Short-Answer  2 minutes
      Limited Essay 10-15 minutes
      Broad Essay 30 minutes

       

     

     

     

     

    • Allow another 5 minutes for students to review their exams.
    • Allow time for distributing and collecting exams.
    • Another rule-of-thumb: Allow students 4 times the amount of time that it takes you to complete the exam.

    Back to Top

    Test Blueprint

    To ensure that all major topics covered in a course are represented on the exam, use a two-way grid called a test blueprint.

    Simple Example:

     

    Levels of Item Complexity
      Knowledge Comprehension Application Analysis Synthesis Evaluation
    Objective 1            
    Objective 2            
    Objective 3            
    Objective 4            

    Guidelines for Number of Items per Objective

  • Weight assigned to content area
  • Emphasis given to levels of complexity in Bloom’s Taxonomy
  • Amount of class time spent on topic

    • Back to Top

    Take-Home Exams

    Pros

  • Students have an opportunity to integrate and synthesize material and apply knowledge to varying contexts.

  • Students can work at their own pace; no time constraints usually result in better written responses.

  • Allow longer and more involved questions.

  • More thorough exploration of ideas and material can take place.

  • More resources are available to the student.

  • Less test anxiety.

  • More a test of thinking and writing than of memory.

  • Easier for you to read, if you require students to type.

    • Cons

  • Take longer to grade.

  • More difficult to grade objectively than multiple-choice exams.

  • Students can be unsure about the time and effort to put into completing exam.

  • Students may not do their own work.

  • Students may spend less time preparing for exam.

    • Appropriate types of questions

  • Problem sets

  • Short answers

  • Essays

    • Cautions

  • Do not make the exam too difficult.

  • Make limits on length or time. Students may find it difficult to know when to stop writing and/or when to stop researching and start writing.

  • Give students instructions on what they can and cannot do—what resources they can use, who they can consult, etc.

    • Consider a compromise

  • Include both an in-class and take-home component to the exam.

  • Distribute possible essay questions a week or two before the exam.

    • Back to Top

    Group Exams

    Benefits

    • Students learn work related skills.

      • Cooperation

      • Negotiation

      • Valuing differences

     

    Guidelines for take-home group exam

  • Students need to understand the purpose of writing a group exam.
  • Are they developing particular skills?
  • Are they learning to do research?
  • What is the level of analysis that you will require of them?
  • The instructor needs to outline a schedule.
  • All requirements should be specified.
  • Length of written products
  • Typestyle
  • Prohibitions
  • Stylistic issues
  • Use of graphics
  • Method of documentation
  • The grading procedure should be stated.
  • One grade can be given for the exam, another for the individual’s part in the exam.
  • Students can be allowed a voice in the grade.
  • Students can be allowed to assign the final grade.
  • Final grading can be postponed until students have a chance to make revisions.

    • Guidelines for in-class group exams

  • Allow students to practice working in groups throughout the semester.
  • When using multiple-choice exams, use higher-level questions and allow about 50 minutes to do about 25 items.
  • Have students discuss each item, not just vote on the answer.
  • Try dividing students into groups of five.
  • Try having students complete the exam by themselves, pass it in, and then complete the exam in a group.

    • Back to Top

    Sources

    • …….(2002). Special Issue: Facilitating Students' Collaborative Writing, ASHE-ERIC Higher Education Report, Vol. 28, Issue 5. Available at: http://0-www3.interscience.wiley.com.www.libcatalog.uams.edu/cgi-bin/fulltext?ID=93516823&PLACEBO=IE.pdf (no longer available).
    • Brinkley, A. et al. (1999). The Chicago Handbook for Teachers. The University of Chicago Press: Chicago.
    • Davis, B.G. (2001). Tools for Teaching. Jossey-Bass: San Francisco.
    • Fry, H., Ketteridge, S., & Marshall, S. (1999). A Handbook for Teaching & Learning in Higher Education: Enhancing Academic Practice. Kogan Page: London.
    • Jacobs, L.C. & Chase, C.I. (1992). Developing and using tests effectively: A handbook for faculty. Jossey-Bass: San Francisco.
    • Sax, G. (1980). Principles of educational and psychological measurement and evaluation. Wadsworth Publishing Company: Belmont, California.

    Back to Top

     

     

    If you have questions or comments about this page please .

    Office of Educational Development, eLearning
    University of Arkansas for Medical Sciences
    4301 W. Markham St., #595
    Little Rock, AR 72205

    Contact UAMS     Copyright Statement