The National Research Council has observed that of all the changes made to Title I in 1994, “perhaps the most far-reaching changes were in the assessment arena.”[15] This is because prior to 1994, states and school districts – driven by federal mandates – relied almost exclusively on norm-referenced tests of basic skills both to evaluate the effectiveness of the program and to select eligible students for services.
The 1994 amendments recognized that norm-referenced tests simply compared students to one another rather than measuring their attainment of standards. The new law sought to stimulate states to create and use assessments that measured the higher-order skills needed to live and work in an increasingly complex society. The expectation of the new law was that assessments would be rich and varied, and would rely less on multiple-choice “fill-in-the-bubble” items. Instead, many advocates (including some states like Maryland and Kentucky that were experimenting with new measures) envisioned state assessment systems that included student writing, constructed responses, portfolios, and other measures.
Section 1111 (reprinted in Appendix A) spells out all the specific requirements for state assessments, including, most importantly, that they be aligned with the state’s own standards. Other requirements address: grade levels assessed, full inclusion of all students, disaggregation and reporting of results, and valid and reliable uses of the assessment results. Related sections of the law (sections 1114, 1115 and 1119) call on schools to deliver instruction aligned with the standards and to assist individual students who are having difficulty mastering them.
If standards are to mean anything, particularly to children in high poverty schools, then teachers must be trained to teach to them and schools must provide the instructional and other resources needed for all students to master them. In short, the curriculum must to be aligned with the state’s official determinations about what students should know and be able to do. Moreover, the state assessments must provide an accurate measure of whether and to what extent schools and districts are succeeding in delivering the standards-based curriculum. Only when all three components of this structure – standards, curriculum and assessment – are aligned with each other will accountability measures and consequences be perceived in the community as fair and equitable.
The Citizens’ Commission has identified three major concerns with the use of norm-referenced assessments as a sole or primary measure for Title I accountability:
1. Norm-referenced tests are not aligned with state standards. While there may indeed be substantial overlap among items included in these tests and states’ own standards, they are not designed to match the varying breadth, emphases, and other dimensions of each state’s content and performance standards.
2. Teachers will teach to the test with the highest stakes. When tests used for accountability purposes (e.g., for rewards, school improvement or corrective action) are standards-based, teaching can focus comfortably on the standards. When there is a mismatch, however, between what is tested and what is written in standards documents, the test will often trump. As a consequence, teachers and whole schools may resort to “test prep” (e.g., subjecting students to a mind-numbing series of multiple choice and other worksheets) at the expense of a rich and varied curriculum.
3. The scoring of norm-referenced tests is antithetical to the notion that all children can learn at high levels. Norm-referenced tests are scored on a bell curve, which compares students only to each other (not to a standard) and always tags half of the test-takers as “below average.” Test questions are not based on a determination that they measure attainment of a standard but are based on whether they will result in a bell-curved distribution of scores. Tests are constructed by including or eliminating questions in such a manner that most students will score somewhere in the vast middle of the distribution, with a few outliers at the very high and very low ends of the achievement spread. In contrast, a standards-based assessment system seeks to measure whether students know and are able to do what the state has determined is important for children of their age and grade in each subject area.
How have the states fallen short of the requirement?
In its review over the past year of states’ final assessment plans, the Department withheld full approval both to states that had incomplete assessment systems and to those that had adopted assessments but could not demonstrate that they were aligned with standards.
Incomplete assessments. There are many states whose systems were missing a required element of the standards-based system described in the statute. For example:
· Colorado’s system, for example has incomplete performance standards for elementary math and for the 10-12-grade span.
· Department officials found that the evidence submitted from North Dakota did not clearly indicate whether the state will use its current norm referenced test and authentic skills assessments in English/literature and math, noting that there is “no system fully developed and documented, or assured of annual implementation.”
· Nebraska’s “standards and assessments will not be completed and reviewed until at least the summer 2001, beyond the deadline for meeting Title I requirements.”
· Nevada has no standards-based assessment component for the middle grades.
· South Dakota was asked to submit evidence that it has final standards-based assessment in math and approved performance standards in reading and math.
Mismatch with standards. Perhaps the most important issue the Department grappled with was the use of norm-referenced tests for Title I accountability purposes. While the Department apparently concluded that the use of norm-referenced tests (NRTs) as one component of a system using multiple measures could be countenanced, it flatly denied submissions of three states where NRTs were the exclusive measure of adequate progress.
· California, West Virginia, and Wisconsin were all deemed not in compliance because of reliance on norm-referenced tests as the sole achievement measure for accountability purposes.
· The Department told West Virginia that it had failed “to establish alignment between the SAT-9 and content standards.”
· In Wisconsin, the Department found:
“The State relies exclusively on the Wisconsin Knowledge and Concepts Exam, which is the Terra Nova, for Title I and State accountability purposes. The State’s own alignment studies show that the Terra Nova addresses only 41.2% to 64.7% of the state’s content standards, depending on the subject area and grade level. In addition, a significant number of the standards are assessed by only one test item, and a number of test items are counted as addressing multiple standards. Consequently, Wisconsin’s assessment system does not reflect the breadth and depth of the State’s academic standards, and fails to measure one-third to more than one-half of what Wisconsin students are expected to learn. It can not provide parents, teachers and students with valid information on the progress students are making toward meeting state standards.”
· The Department wrote to California:
“Many of the specific instances of noncompliance discussed below result from California's failure to complete development and implementation of assessments that are aligned to State content and performance standards, and use the results of these tests to hold schools accountable for the performance of all students. Consequently, California can comply with many of these requirements if it follows through on its plan to develop and implement a standards-based assessment in a timely fashion”
· The Department also found that the Comprehensive Test of Basic Skills (CTBS), used in North Dakota, provided “useful information on how North Dakota students …compared to a national sample,” but was “not fully aligned” with the state’s content standards.
· In contrast, in states including Missouri and Delaware, the Department approved state assessments that included elements of norm-referenced tests along with other measures.
[15] National Research Council, Testing, Teaching, and Learning, p. 8 (National Academy of Sciences, 1999).
| Foreword | Summary | Introduction | Chapter I |
| Chapter II | Chapter III | Chapter IV | Chapter V |
| Chapter VI | Chapter VII | Conclusion | Acknowledgements |