by Donald R. Moore & Matthew Hanson, Designs for Change

CHICAGO, April 30, 2001 -- On the first day of Iowa reading and math testing in the Chicago Public Schools, nine nationally recognized test experts issued a statement concluding that Chicago's use of the Iowa Test to make critical decisions about students and schools is fundamentally flawed.  Their statement concluded that this spring's Iowa Test results cannot be used fairly to make critical decisions about (1) whether students should be promoted or sent to summer school and (2) whether year-to-year variations in school test averages should be used for sanctioning schools (see Attachment B). 

            While Chicago's Mayor has called for larger gains on the Iowa reading test as evidence that Chicago's children are learning to read, the test experts called the Iowa Test, as it is now being used in Chicago, "a broken thermometer."

            The nine experts on how tests are constructed and how they impact schools and students are Gene Glass of Arizona State University; Clifford Hill of Columbia University; Richard Figeuroa of University of California; Robert Sternberg of Yale University; Walter Haney of Boston College; Gary Orfield of Harvard University; Ernest House of University of Colorado; Gerald Bracey, the former director of testing for the state of Virginia; and Donald Moore of Designs for Change in Chicago. The experts' statement cites two major related reasons why this spring's Iowa testing procedure in Chicago is fundamentally flawed:

 (1) The version of the Iowa test being used in Chicago in spring 2001 (CPS 94 or Form L) is the same identical test that has been administered six times before in the last six years in spring, summer, and mid-year testings (see Attachment A). This version of the test has the same reading passages and math problems that teachers have seen six times before. These repeated opportunities for school staff to learn the exact questions on the test while administering the test in past years fundamentally undermine the accuracy of important decisions about students or schools that rely primarily or significantly on spring 2001 Iowa reading and math test results.

(2)   The Iowa Test is not systematically focused on the learning standards required by the Illinois State Board of Education and the Chicago Public Schools in such subjects as reading.  Thus, school or student scores on this spring's Iowa Test are not appropriate measures for judging school or student progress in order to make "high stakes" decisions.

            For example, in the critical area of third grade reading ability, the Iowa Test reading passages are too limited to judge whether third graders can read competently, argued Donald Moore of Designs for Change, one of the statement's signers.   The third grade Iowa Test in reading consists of reading passages 150 to 200 words long, followed by an average of four questions (see Attachment C).  

            However, the level of reading that is needed to meet state and national standards for third graders requires that children understand the plot of a story, understand how characters in stories change, and understand texts that describe complex processes, such as "how dolphins communicate." 

            In contrast to the Iowa reading test, the reading passages for third graders on the Illinois reading test average about 500-600 words, and each passage is followed by about 17 questions (see Attachment D).  The reading passages on the Illinois reading test are two to three pages long and include pictures and diagrams with captions.  Such reading passages require the more complex reading skills that a third grader needs to become a competent reader, Moore argued.

            Chicago's third graders are being drilled to pass the Iowa reading test, because their Iowa score will be decisive in determining whether they are promoted to the next grade, what job rating their principal gets, and whether their school is put on a list of sanctioned schools whose tests scores have dropped a few percentage points over one or two years (so-called "B" and "C" schools).  Thus, Chicago's principals and teachers are being sent two contradictory messages, Moore argued.  They are told to teach students to meet high reading standards, but the real pressure is to raise scores on the Iowa reading test.

            This tremendous pressure to improve Iowa Test scores has, according to extensive classroom observations by the Consortium on Chicago School Research, turned many Chicago schools into Iowa Test preparation factories from March until testing begins. The Consortium researchers, who observed a cross section of Chicago classrooms, concluded that:

Students often spend a considerable chunk of their day working in test preparation workbooks and taking practice tests [that reflect the types of questions on the Iowa test].  They are often moved back into single row seating to get comfortable with how they will sit during the tests.

 

            The American Educational Research Association recently highlighted the dangers of exactly the testing practices that Chicago is carrying out this spring:

Because high-stakes testing inevitably creates incentives for inappropriate methods of test preparation, multiple test forms should be used or new test forms should be introduced on an regular basis, to avoid a narrowing of the curriculum toward the content sampled on a particular form (Position Statement Concerning High Stakes Testing in PreK-12 Education).

 

           Ironically, another recent study by the Consortium on Chicago School Research showed that Chicago students who are taught more challenging lessons and given less test preparation score higher on the Iowa Test.   Further, this pattern holds true for both low-scoring and high-scoring students (Fred Newman. Authentic Intellectual Work and Standardized Tests:  Conflict or Coexistence? January 2001). 

           The statement by the national testing experts concerning Chicago's testing program that is being released today underscores the fact that that one minimum requirement for the type of "high stakes" testing that Chicago is carrying out is test security.  For example, the ACT college admission testing organization constantly changes the specific questions on their tests.  No student competing for college admission would believe it was fair if they were judged by a test that had been given six times before. 

             Because of fundamental flaws in Chicago's Iowa testing program, the national experts concluded that this spring's Iowa Test results are inappropriate as "the primary factor or as a significant factor" in determining (1) "whether students are promoted to the next grade or retained after summer school and required to repeat a grade" and (2) "whether schools will be sanctioned, based  on average school-wide gains or losses on the Iowa Test over one or two years" (for example, whether they are classified as "B" or "C" schools.

              The experts' conclusions echo the consensus statement made by the Steering Committee of the Consortium on Chicago School Research in March 1998:

The Consortium Steering Committee concludes that CPS needs a new standardized testing and reporting system in reading and mathematics, in order to have a more reliable basis for judging school and system improvement in these key areas (Anthony Bryk.  Academic Productivity of Chicago Elementary Schools.  March 1998).

 

One proposed alternative to the current high stakes Iowa testing process is the "New ERA Plan," prepared by a coalition of Chicago educators and school reform groups (the LSC Summit Coalition).  Under the New ERA plan, the Iowa Test would be eliminated, and the Illinois achievement tests (which are focused on the Illinois learning standards) would become one of a set of tools for assessing the progress of students and schools.  However, the Illinois tests themselves would not play the same high stakes role that the Iowa Test now plays, and a key to success would be to successfully implement classroom-based assessments in addition to standardized tests (see Attachment F).

           

 # # # #

 

Calendar Year	Mid-Year (January)	Spring (April)	Summer (August)
1990	——	CPS90 (J)	——
1991	——	CPS91 (G)	——
1992	——	CPS92 (H)	——
1993	——	CPS93 (K)	——
1994	——	CPS94 (L)	——
1995	——	CPS93 (K)	——
1996	——	CPS94 (L)	——
1997	——	CPS97 (M)	CPS93 (K)
1998	CPS94 (L)	CPS94 (L)	CPS93 (K)
1999	CPS94 (L)	CPS97 (M)	CPS93 (K)
2000	CPS94 (L)	CPS93 (K)	CPS 94 (L)
2001		CPS94 (L)

 

Source:  Consortium on Chicago School Research, The Chicago Public Schools, Office of Accountability, Division of Student Assessment, and School Principals.

 

---

The Chicago Public Schools is administering a particular form of the Iowa Tests of Basic Skills, beginning on April 30, that has been employed by Chicago six previous times in the last six years.  This form of the test has had exactly the same test questions each time it has been used.   As experts in the field of test development and test use, we agree that the version of the Iowa reading and math tests that is being administered in Chicago in spring 2001:

 

n   Should not be used as the primary factor or as a significant factor in determining whether students are promoted to the next grade or retained after summer school and required to repeat a grade. 

 

n   Should not be used as the primary factor or as a significant factor in determining whether schools will be sanctioned, based on average school-wide gains or losses on the Iowa Test over one or two years.

 

We base this position on the following specific ways in which test use by the Chicago Public Schools does not meet appropriate test use standards for high stakes testing:

 

n   The version of the Iowa test that is being administered in Chicago in spring 2001 is the same identical test that has been administered six times before in the last six years in spring, summer, and mid-year testings.  Thus, repeated opportunities for school staff to learn the exact questions on the test while administering the test in past years, coupled with the fact that the copies of the test that is to be administered beginning on April 30 were delivered to the schools more than two weeks before the scheduled administration date, fundamentally undermine the accuracy of important decisions about students or schools that rely primarily or significantly on spring 2001 Iowa reading and math test results.

 

n   The Iowa Test is not aligned with the learning standards mandated by the Illinois State Board of Education or the Chicago Public Schools.  Thus, school or student scores on the Iowa Test being administered in spring 2001 are not an appropriate measure for judging student and school progress in light of these mandated learning standards.

 

For the purposes of making major decisions about students and schools, the Iowa Test, as it is being used by the Chicago Public Schools in spring 2001, is a broken thermometer.

 

Some signers of this statement may or may not believe that Chicago’s current use of the Iowa Test violates testing standards in other ways or may disagree about the best approach for using standardized tests to improve the quality of teaching and learning.   However, such views should not be assumed unless they are separately and affirmatively stated by an individual signer.

---

Dr. Gerald Bracey is the author of the recently-released Put to the Test: An Educator's and Consumer's Guide to Standardized Testing, which analyzes the range of standardized achievement tests that are used in education, and the ways such tests are commonly used and misused. Dr. Bracey has been a research psychologist for the Educational Testing Service in Princeton, New Jersey and served for nine years as Director of Research, Evaluation, and Testing for the Virginia Department of Education.

 

Dr. Richard Figueroa, Professor of Education at University of California-Davis, has conducted research concerning special education for bilingual learners, psychological assessment, and school psychology. Dr. Figueroa has authored more than thirty articles and books on the subjects of bilingual education and the education of children from culturally diverse backgrounds. His recent publications include Best Practices in the Assessment of Bilingual Children and Bilingualism and Testing: A Special Case of Bias.

 

Dr. Gene V. Glass is Professor of Education Policy Studies and Associate Dean for Research at Arizona State University. Dr. Glass is former President of the American Educational Research Association.  He is author or editor of eleven books and more than 100 articles on such subjects as educational evaluation and the use of statistics in educational research.  He has been an editorial board member of thirteen scholarly journals.

 

Dr. Walter Haney is Professor of Education at Boston College and Senior Research Associate with the Center for the Study of Testing, Evaluation, and Educational Policy (CSTEEP). He is the author of numerous articles and books on testing and evaluation, including The Fractured Marketplace for Standardized Testing, which analyzes the testing industry in the United States.

 

Dr. Clifford Hill is Professor of Language and Education at Teachers College, Columbia University.  He also directs the Program in African Languages at the Institute of African Studies in the School of International and Public Affairs. Dr. Hill has authored a number of books and articles that analyze language and literacy assessment, which include Children and Reading Tests and From Testing to Assessment: English as an International Language.

 

Dr. Ernest House is Professor of Education at the University of Colorado at Boulder and an internationally recognized expert on evaluating educational programs.  He has authored such books as Professional Evaluation:  Social Impact and Political Consequences.  In the early 1980's, Professor House conducted an evaluation audit of the impact on student achievement of grade retention programs in the New York City Public Schools. (New York's program was similar to Chicago's current initiative to "end social promotion.") He recently served as Senior Fellow at the Center for Advanced Study in the Behavioral Sciences in Stanford, California.

Dr. Donald Moore is Executive Director of Designs for Change, an educational research and reform organization in Chicago.  One important focus of his research, which has been supported by the Ford Foundation, Spencer Foundation, and U.S. Department of Education, is the role of standardized tests in tracking, grade promotion, admission to selective magnet schools, and placement in special education. He is one of 15 graduates of the Harvard University Graduate School of Education to have received its Outstanding Contribution in Education award.

 

Dr. Gary Orfield is Professor of Education and Social Policy at Harvard University Graduate School of Education and Co-Director of the Civil Rights Project at Harvard University. Dr. Orfield's central interest has been the development and implementation of social policy, with a focus on it impact on equal opportunity for success in American society. One of a number of civil rights related projects that he has recently directed is a research conference on "The Civil Rights Implications of High Stakes K-12 Testing," which will result in a book on this issue. Dr. Orfield was also Professor of Political Science and Education at the University of Chicago for nine years.

 

Dr. Robert Sternberg is a Professor in Yale Universitys Department of Psychology, who is particularly well known for his research and writings that analyze the nature of human intelligence. His publications include Beyond IQ:  A Triarchic Theory of Human Intelligence and Metaphors of Mind:  Conceptions of the Nature of Human Intelligence.  Dr. Sternberg is Director of the PACE Center of Yale University, a research group that investigates human abilities, competencies, and expertise.