March 1998

State statute requires the Department of Public Instruction to adopt or approve standardized tests for statewide use in the fourth, eighth, and tenth grades. The tests, which are known as Knowledge and Concepts Examinations, measure student achievement in reading, mathematics, science, social studies, and language arts. Since the testing program was established in 1992, the Department has contracted with three different vendors to provide the tests and related materials; score student answers; and report results for individual students, schools, school districts, and the state. The third vendor—CTB/McGraw-Hill—first provided services in the 1996-97 school year at a total cost of $1.425 million in general purpose revenue.

Numerous Problems Were Encountered During the 1996-97 School Year

Although school officials generally give high ratings to the contents of TerraNova, the testing instrument used, there has been dissatisfaction with administration of the Knowledge and Concepts Examinations program. During the 1996-97 school year, problems with the testing process led to significant inconvenience and frustration for school districts and classroom teachers. Many of the problems were ultimately of little consequence to the testing program, but others contributed to delays in the test scoring and reporting process. For example, because of a delay in the distribution of testing materials, Milwaukee Public Schools extended its testing window and delayed the return of tests for scoring. This, in turn, delayed compilation of test results for the entire state.

The delays in the reporting of test scores were significant. In past years, score reports for students, schools, and school districts had been provided in January and early February, which allowed districts to use them for mid-year curriculum adjustments and guidance counseling. In the 1996-97 school year, however, all test results were not distributed until mid-April, and the statewide report was not distributed until July 1997. In addition, some reports that had been provided in previous years, such as a summary of each school's results by test question, were not provided.

The Testing Contract Lacked Specificity

A primary cause of the delays and other problems encountered in 1996-97 was the lack of specificity in the Department's contract with CTB/McGraw-Hill. The Department had agreed to use the vendor's standard score reports but had envisioned a variety of adjustments to them. These expectations were not, however, adequately clarified in the contract. Consequently, negotiations regarding score reporting were required late in the process.

Despite the problems encountered in 1996-97, the Department renewed its contract with CTB/McGraw-Hill for the current school year. Steps have been taken to improve contract management and program operations, including the development of a detailed project schedule and the inclusion of clearer expectations. As a result, test administration proceeded smoothly in 1997-98, and score reports were available with minor delays.

A Longer Contract Cycle May Be Appropriate

Not only did the change in testing contractors initially result in problems with test administration and score reporting, the new contractor's use of a new testing instrument also impaired the Department's and school districts' ability to identify trends in student achievement. Therefore, questions have been raised as to why the change in contractors was made. The change was precipitated by state procurement guidelines, which require that contracts for most professional services be rebid every three years. Given the unique nature of the testing program, including the need to ensure program continuity, the appropriateness of rebidding the contract every three years can be questioned.

To address concerns about program continuity, the Department sought and received approval from the Governor to extend its contract with CTB/McGraw-Hill until 2001-02. Longer contracting cycles will allow the Department to secure the benefits of program continuity; however, without the benefit of price comparisons required by the bidding process, the Department will need to take steps when renegotiating contract costs to ensure it is paying a competitive price for the services it is purchasing.

Program Continuity May Be Difficult to Achieve

Continuity within the Knowledge and Concepts Examinations program may be difficult to achieve because changing policy and program goals continue to affect administration of the program and the comparability of test results. For example, the current school year is the first in which students' performance is evaluated based on proficiency standards adopted by the Department. In the past, test scores had compared students in Wisconsin only with one another and with students nationwide. Concerns about the new method of reporting have led to a legislative proposal that would allow school districts to exempt themselves from the tests. If the proposal is adopted, program continuity would be diminished and federal aid under Title I of the Improving America's Schools Act, which is expected to total $128 million in fiscal year 1997-98, may be jeopardized. Moreover, tenth graders will no longer be subject to the Knowledge and Concepts Examinations after the implementation of a high school graduation test beginning in 2000-01, and the Knowledge and Concepts Examinations will be given in the spring rather than the fall semester beginning in the 1998-99 school year.