Cognitive Complexity
What is cognitive complexity?
Cognitive complexity is one tool that can be used to think about whether test items (or even complete tests) are aligned with their standards, assessment targets and/or targeted content domain. Essentially, it pushes evaluators to consider whether items are just literally aligned, but still lacking the depth or complexity that they really ought to get at. Or, do the items prompt the complexity of thought that they are supposed to? Or…
Actually, cognitive complexity can mean any number of things and there are various schemas and operationalizations of cognitive complexity. Some people think that it should focus on how many ideas must be brought together. Some think of it as being tightly related to item difficulty. There are many many ideas about what cognitive complexity could or should mean.
However, the most common typology for cognitive complexity started with Norman Webb’s Depth of Knowledge (DOK). We refer to Webb’s DOK as wDOK. The industry has implemented DOK in ways that differ from what he wrote over the years, and we call that now-standard practice iDOK. iDOK is much decried and lamented, often being treated as largely meaningless hoops to jump through.
We believe that there is value in DOK, but that Webb’s explanations were a bit confusing, at times. We have offered a revised Depth of Knowledge (rDOK) that captures his central idea and focuses on it more narrowly. This central thrust of wDOK and rDOK is the range of more automatic cognition to more deliberative cognition. Some standards and some tasks call for simple recall of facts or formulas. Other standards and other tasks call for more analysis and care, perhaps even planning and/or explication of thought process.
Why bother with cognitive complexity?
Many educators and others who look at the contents of tests see just the most reductive and simplistic versions of the content matter and learning objectives. They worry about the higher order thinking, 21st century skills and critical and analytical thinking skills that are transferable across content areas and useful throughout life — as opposed to the sorts of content and lessons that are most easily forgotten.
Those people — including us — believe that it is important to include such content on standardized assessment. In fact, because this is often the most important knowledge and skills to those who most dearly love their content areas, we think that this is the most important stuff to include on tests.
Cognitive complexity — DOK — is a tool for recognizing when test items fail to get at that sort of content.
Why focus on Webb’s Depth of Knowledge (wDOK)?
wDOK is the basis for almost all examination of cognitive complexity in assessment. Regardless of whether it is the operationalization of the idea that we would start with or we would prefer, its terminology is widely known and is almost universally accepted.
Most educators know Bloom’s Taxonomy far better than they know DOK — if they know DOK at all. But Bloom’s fails at this purpose, for reasons we explain in An RTD Approach to DOK (downloadable from the sidebar to the left).
This leaves DOK as the standard, and therefore the best chance of improving item validity via cognitive complexity.
What is rDOK?
rDOK is revised Depth of Knowledge. It builds on wDOK’s central thrust of the automaticity-deliberation continuum. It is a conservative approach to recover and reinvigorate Depth of Knowledge so that it can be used as a rigorous tool to improve item and test validity.
We offer an overview of rDOK in An RTD Approach to DOK and Reinforcing Webb’s Depth of Knowledge: Laterally Extending DOK by Acknowledging Proficiency’s Impact on Cognitive Demand. We explain how to apply rDOK in math, in science and in ELA in Reinvigorating Webb’s Depth of Knowledge in Three Content Areas and the other papers downloadable from the sidebar to the left.