Teaching Healing Searching Serving
Ongoing Services
Faculty & Staff
Newsletter
Teaching Scholars
Resources
  Educational Research Resources
  Team-Based Learning
  OED Library
UAMS Web-based Learning
  Teaching Scholars Program
  OED Fact Sheets
  PBL Resources
UAMS Home
OED Home

Exam Writing Guidelines

For more information contact:

Item Writing

 

From: NBME’s Constructing Written Test Questions for the Basic and Clinical Sciences. Available at: http://www.nbme.org/PDF/ItemWriting_2003/2003IWGwhole.pdf

What Should be Tested?

  • Content should match course objectives
  • Important topics should be weighted more heavily than less important topics
  • Testing time devoted to each topic should reflect the relative importance of the topic
  • Sample of items should be representative of instructional goals

    • General Guidelines for Item Construction

  • Make sure the item can be answered without looking at the options OR that the options are 100% true or false.
  • Include as much of the item as possible in the stem; the stems should be long and the options short.
  • Avoid superfluous information.
  • Avoid "tricky" and overly complex items.
  • Write options that are grammatically consistent and logically compatible with the stem; list them in logical or alphabetical order.
  • Write distracters that are plausible and the same relative length as the answer.
  • Avoid using absolutes such as always, never, and all in the options; also avoid using vague terms such as usually and frequently.
  • Avoid negatively phrased items (i.e., those with except or not in the lead-in). If you must use a negative stem, use only short (preferably single word) options.

    • And most important of all: Focus on important concepts; don’t waste time testing trivial facts.

    Flaws that Tip Off Students to the Correct Answer

  • Grammatical cues - one or more distracters don’t follow grammatically from the stem.
  • Logical cues - a subset of the options is collectively exhaustive.
  • Absolute terms - terms such as "always" or "never" are in some options.
  • Long correct answer - correct answer is longer, more specific, or more complete than other options.
  • Word repeats - a word or phrase is included in the stem and in the correct answer.
  • Convergence strategy - the correct answer includes the most elements in common with the other options.

    • Flaws that Make Items Unnecessarily Difficult

  • Options are long or complicated, or contain duplicated portions.
  • Numeric data are not stated consistently.
  • Terms in the options are vague (examples--"rarely," "usually").
  • Language in the options is not parallel.
  • Options are in an illogical order.
  • "None of the above" is used as an option.
  • Stems are tricky or unnecessarily complicated.
  • The answer to an item is "hinged" to the answer of a related item.

    • Back to Top

    Time Guidelines

    Rule-of-Thumb Guidelines

      Type of Question

      		 Time to Complete
      True-False ˝ minute
      Multiple-Choice 1 minute
      Short-Answer  2 minutes
      Limited Essay 10-15 minutes
      Broad Essay 30 minutes

       

     

     

     

     

    • Allow another 5 minutes for students to review their exams.
    • Allow time for distributing and collecting exams.
    • Another rule-of-thumb: Allow students 4 times the amount of time that it takes you to complete the exam.

    Back to Top

    Test Blueprint

    To ensure that all major topics covered in a course are represented on the exam, use a two-way grid called a test blueprint.

    Simple Example:

     

    Levels of Item Complexity
      Knowledge Comprehension Application Analysis Synthesis Evaluation
    Objective 1            
    Objective 2            
    Objective 3            
    Objective 4            

    Guidelines for Number of Items per Objective

  • Weight assigned to content area
  • Emphasis given to levels of complexity in Bloom’s Taxonomy
  • Amount of class time spent on topic

    • Back to Top

    Take-Home Exams

    Pros

  • Students have an opportunity to integrate and synthesize material and apply knowledge to varying contexts.

  • Students can work at their own pace; no time constraints usually result in better written responses.

  • Allow longer and more involved questions.

  • More thorough exploration of ideas and material can take place.

  • More resources are available to the student.

  • Less test anxiety.

  • More a test of thinking and writing than of memory.

  • Easier for you to read, if you require students to type.

    • Cons

  • Take longer to grade.

  • More difficult to grade objectively than multiple-choice exams.

  • Students can be unsure about the time and effort to put into completing exam.

  • Students may not do their own work.

  • Students may spend less time preparing for exam.

    • Appropriate types of questions

  • Problem sets

  • Short answers

  • Essays

    • Cautions

  • Do not make the exam too difficult.

  • Make limits on length or time. Students may find it difficult to know when to stop writing and/or when to stop researching and start writing.

  • Give students instructions on what they can and cannot do—what resources they can use, who they can consult, etc.

    • Consider a compromise

  • Include both an in-class and take-home component to the exam.

  • Distribute possible essay questions a week or two before the exam.

    • Back to Top

    Group Exams

    Benefits

    • Students learn work related skills.

      • Cooperation

      • Negotiation

      • Valuing differences

     

    Guidelines for take-home group exam

  • Students need to understand the purpose of writing a group exam.
  • Are they developing particular skills?
  • Are they learning to do research?
  • What is the level of analysis that you will require of them?
  • The instructor needs to outline a schedule.
  • All requirements should be specified.
  • Length of written products
  • Typestyle
  • Prohibitions
  • Stylistic issues
  • Use of graphics
  • Method of documentation
  • The grading procedure should be stated.
  • One grade can be given for the exam, another for the individual’s part in the exam.
  • Students can be allowed a voice in the grade.
  • Students can be allowed to assign the final grade.
  • Final grading can be postponed until students have a chance to make revisions.

    • Guidelines for in-class group exams

  • Allow students to practice working in groups throughout the semester.
  • When using multiple-choice exams, use higher-level questions and allow about 50 minutes to do about 25 items.
  • Have students discuss each item, not just vote on the answer.
  • Try dividing students into groups of five.
  • Try having students complete the exam by themselves, pass it in, and then complete the exam in a group.

    • Back to Top

    Sources

    • …….(2002). Special Issue: Facilitating Students' Collaborative Writing, ASHE-ERIC Higher Education Report, Vol. 28, Issue 5. Available at: http://0-www3.interscience.wiley.com.www.libcatalog.uams.edu/cgi-bin/fulltext?ID=93516823&PLACEBO=IE.pdf (no longer available).
    • Brinkley, A. et al. (1999). The Chicago Handbook for Teachers. The University of Chicago Press: Chicago.
    • Davis, B.G. (2001). Tools for Teaching. Jossey-Bass: San Francisco.
    • Fry, H., Ketteridge, S., & Marshall, S. (1999). A Handbook for Teaching & Learning in Higher Education: Enhancing Academic Practice. Kogan Page: London.
    • Jacobs, L.C. & Chase, C.I. (1992). Developing and using tests effectively: A handbook for faculty. Jossey-Bass: San Francisco.
    • Sax, G. (1980). Principles of educational and psychological measurement and evaluation. Wadsworth Publishing Company: Belmont, California.

    Back to Top

     

    If you have questions or comments about this page please .

    Office of Educational Development
    University of Arkansas for Medical Sciences
    4301 W. Markham St., #595
    Little Rock, AR 72205

     

    UAMS Online     Copyright Statement     Privacy Statement