Planning an Effective Test

Writing an effective test starts with good planning. The most important goal is to match your test items with your course objectives. Your course objectives are most likely aligned with Bloom’s Taxonomy. Your expectation of the level of student learning will help determine the types of test items to be used.

Knowledge

· Remembering previously learned material, such as terms, facts, trends and sequences, methodology, principles and theories.

Comprehension

· Grasping the meaning of material

· Converting from one form to another

· Explaining or summarizing material

· Extending the meaning beyond the data

Application

· Using information in concrete situations

Analysis

· Breaking down material into its parts

· Identifying the parts and relationships

· Identifying the organization

Synthesis

· Putting parts together into a whole

· Production of a plan or proposed set of operations

· Derivation of a set of abstract relations

Evaluation

· Judging the value of a thing for a given purpose using definite criteria

Outline your unit to be tested into major categories based on your objectives. Determine how many questions should be asked in each category and at what level the student should be tested. Remember that testing is a matter of sampling. Select a representative sample of learning tasks and include an adequate number of items on your test. The size of the domain of information covered by the objectives should help determine the length of the test. For reliability, create parallel test items that assess the same objective.

You may want to construct a table to help you either construct a new test or evaluate an existing test. Note in the following sample planning chart that there are four major objectives to be tested. Within each category, different levels of learning will be examined. The numbers indicate either how many questions the instructor wishes to ask for each outcome and at each cognitive level on a new test or how many are present on an existing test.

Table of Specifications for a Summative Test

Content/Outcomes	Knows			Comprehends Principles	Applies Principles	Total Number of Items
	Terms	Facts	Procedures
Role of Tests in Instruction	4	4		2		10
Principles of Testing	4	3	2	6	5	20
Norm Referenced vs. Criterion Referenced	4	3	3			10
Planning the Test	3	5	5	2	5	20
Total Number of Items	15	15	10	10	10	60

Once you have constructed this chart, you will know how many questions to ask about the various course objectives, and you will know the level of learning you wish to test. The next step is to determine the type of test questions that will best measure student learning at the appropriate level.

The following list includes commonly used types of test items. For each of these test item types, we’ll look at the kind of content for which it is best suited, the level of Bloom’s Taxonomy that can be assessed by that type, and the advantages and disadvantages of each type.

TRUE/FALSE

Best suited for: Naturally dichotomous content, or content that clearly has only two possible choices.

Bloom levels: Knowledge, comprehension, and application (however, almost always written at knowledge level.

Advantages: Typically easier to write than MC (usually because they are written at knowledge level), easily scored, can be submitted to statistical item analysis.

Disadvantages: Test takers have 50/50 chance at correct answer by guessing; more difficult to write at comprehension and application levels.

MATCHING

Best suited for: Understanding of homogeneous content (use of tools, types of illnesses, etc.)

Bloom levels: Knowledge and comprehension (seldom written beyond knowledge level).

Advantages: Easy to write, easily scored, can be submitted to statistical item analysis.

Disadvantages: Limited to lowest levels of Bloom’s Taxonomy; if phrases listed in Column A are not homogeneous, matching items becomes relatively easy; test takers use process of elimination.

MULTIPLE CHOICE

Best suited for: Any type of content.

Bloom levels: Knowledge, comprehension, application, analysis (synthesis and evaluation require original responses).

Advantages: Can assess content at a variety of levels, probability of guessing correctly is lower than with T/F questions, useful for diagnostic testing using appropriate distractors, easily scored, can be submitted to statistical item analysis.

Disadvantages: Difficult and time-consuming to write, can’t assess objectives that require the test taker to recall information unassisted, can’t assess at synthesis and evaluation levels.

FILL-IN ITEMS

Best suited for: Recalling or creating a correct answer rather than recognizing it.

Bloom levels: Knowledge, comprehension, application (usually written at knowledge level).

Advantages: Easy to write, good for recalling information.

Disadvantages: Suitable only for answers that require a word or short phrase, correctness of an answer can be debated so judgment of scorer is called upon.

SHORT ANSWER

Best suited for: Recalling information unassisted or creating original responses of relatively short length.

Bloom levels: All levels except possibly evaluation (evaluation questions might need to be longer).

Advantages: Able to elicit original responses from test takers, lower level questions are easier to write than MC questions, good for assessing objectives that can’t be assessed with closed-ended questions.

Disadvantages: Very difficult to score reliably; evaluation prone to errors due to halo effects, scorer fatigue, and handwriting; scoring is time consuming; more time consuming for test takers.

ESSAY

Best suited for: Objectives that require original, lengthy responses; good for assessing writing skills.

Bloom levels: All levels of Bloom’s Taxonomy.

Advantages: Capacity to assess the highest cognitive levels.

Disadvantages: Very difficult to score reliably; evaluation prone to errors due to halo effects, scorer fatigue, and handwriting; scoring is time consuming; more time consuming for test takers. Use only when the cognitive level of the objective requires it.

QUANTITATIVE PROBLEM-SOLVING

Best suited for: Fields such as mathematics, the sciences, and engineering.

Bloom levels: Application, analysis, synthesis, evaluation.

Advantages: Capacity to assess at all cognitive levels, low probability of guessing

Disadvantages: Can be difficult and time consuming to score/grade, evaluation prone to errors due to halo effects, scorer fatigue, handwriting, more time consuming for test takers.

Erwin, T. Dary, Assessing Student Learning and Development, Jossey-Bass Publishers, 1991.

Gronlund, Norman E., How to Construct Achievement Tests, 4^th ed., Prentice-Hall, Inc., 1988.

Shrock, Sharon A., and William C.C. Coscarelli, Criterion-Referenced Test Development, Addison-Wesley Publishing Company Inc., 1989.