The functional and cognition-oriented approach to items of Rigorous Test Development offers a different view of stimuli than the more page-layout-oriented view that which sees item parts simply in terms of where they appear on a page or on the screen—but not how they impact the cognitive paths that test takers use to generate their responses. RTD focuses on how every part of an item contributes to those cognitive paths, including stimuli.
Layout View of Items
Yes, items have many parts that can take up visual space, but they are more than just visual layout. While stems are important, they are just one part of an item. Some items have instructions to lead them off, or are inserted somewhere around the stem. Items have some sort of work space, be it a word processing window, and a set of answer options or some other interaction space. Of course, test takers have somewhere to offer their response, be it the section of answer options, a blank to fill in, that word processing window, a complex technology enhanced final state or even a less structured space (e.g., a piece of paper). Often, the stimulus is the largest part of an item.
A more functional view considers the parts of the item through the lens of item validity—an item’s ability to elicit evidence of the targeted cognition for the range of typical test takers. Thus, the two basic functions of the parts of an item are i) to make clear what the intended task is and (usually) ii) to provide something to perform the cognitive actions of that task upon. In this view, the stimulus is the material upon which for the KSAs (knowledge, skills and/or abilities) that make up the targeted cognition are applied, the context in which they are used. It is what test takers manipulate, analyze or work upon to demonstrate their proficiencies, regardless of where it appears in the item—or even if it appears explicitly at all!
The RTD Cross-Content Stimulus Evaluation Framework (C2SEF) offers 11 dimensions to consider when evaluating stimulus. It may be used to examine stimulus in hand for their appropriateness and value, or to specify what is needed in stimulus to be developed.
Requirement Whether the targeted cognition of an item requires a stimulus.
Explicit/ Implicit The explicit presence or absence in what is presented to test takers of the material to be analyzed or manipulated.
Structure The number and type of components (e.g., graph + passage) in a stimulus.
Testable Points Specific elements of a stimulus that can elicit evidence of proficiency through items developed around them.
Density Closeness of testable points within stimulus.
Complexity The variety of testable points (and KSAs required) to understand a stimulus.
Authenticity How much the stimulus resembles real-world contexts or classroom learning activities.
Familiarity Extent to which stimulus content—or even form—is known to test takers.
Time-Length Functional time needed to process (i.e., not word count or visual size).
Fairness Risk The potential of a stimulus to create bias or sensitivity issues across the test taking population of an assessment.
Permissions The licensing and/or editability of the stimulus.
These dimensions play out differently in different contexts. Each content area has its own needs and preferences for stimuli, and they can vary across alignment references in a single content area. Of course, particular assessments have their needs and constraints on items and on stimuli. Therefore, the C2SEF is not itself a value-based framework and does not offer scores for suitability of stimuli. Rather, its 11 dimensions should be considered in light of the needs a particular content area, alignment reference and assessment project. The leader of each project and the expert CDPs who do the work of content development need to make those decision, meets those needs and work with those obstacles. The C2SEF helps them to do just that.