MEAP IRREGULARITIES - S.B. 562: FIRST ANALYSIS
Senate Bill 562 (as passed by the Senate)
Sponsor: Senator Valde Garcia
Committee: Education
Date Completed: 11-19-01
RATIONALE
In the spring of 2001, the Department of Treasury held a press conference at which the State Treasurer reported on the Department's investigation into standardized testing irregularities in public school districts. According to the Department, the conference was intended to head off a leak to the media about suspicious test answers on the Michigan Educational Assessment Program (MEAP) test. Measurement, Inc., the outside contractor responsible for scoring the MEAP's open-ended response questions, had reported the irregularities, along with the names of the schools and districts associated with them, to the Department.
In response, some newspapers published a list of the districts under investigation before the schools could examine the cause of the irregularities. Some districts were mistakenly identified, and others were embarrassed by the public perception that they had allowed or encouraged cheating. In some cases, if the districts had been notified first of the specific problem, they presumably could have identified themselves and been cleared of wrongdoing. Some people believe that school districts flagged for irregular test answers should be notified before the press is informed.
CONTENT
The bill would amend the Revised School Code to require the Department of Treasury to notify a school district or public school academy in the event the Department suspected irregularities in the district's or school's administration of, or preparation of pupils for, a Michigan Educational Assessment Program test. The Department would have to notify the school district or public school academy of the suspected irregularities, and give it an opportunity to respond, at least 24 hours before reporting the suspected irregularities to any person or entity not involved in scoring or administering the test.
Proposed MCL 380.1279a
BACKGROUND
The term "testing irregularities" is an official, technical term used by the Measurement, Inc. Experts who read open-ended tests have identified the following indications of irregularity:
-- Answers that are virtual copies.
-- Answers with different handwriting.
-- Answers with the same general outline.
-- Answers with vocabulary not consistent with the grade level of the test taker.
-- Answers with handwriting not consistent with the grade level of the test taker.
-- Similar or identical phrasing of answers.
-- Response patterns that differ significantly from the norm.
In the case of the spring 2001 MEAP tests, a number of steps preceded the identification of schools with irregular test answers. First, a scorer (usually a current or former Michigan teacher) identified a questionable answer and passed the test packet to the table leader, a more experienced scorer. (Neither the scorer nor the table leader had access to student, school, or district information as he or she reviewed the test responses.) If the table leader also believed that an irregularity existed, the packet went to the scoring director, and then to the project manager, both of whom were full-time employees of the Measurement, Inc. If the project manager agreed the answer was questionable, the manger passed on that information, and the questionable test packets, to the State's MEAP office.
At the MEAP office, the packets in question were reviewed by subject-matter experts in each testing area. Next, the packets were given to the MEAP Supervisor and the Michigan Merit Award Director for review. Finally, a team of nine school officials reviewed the information one more time. The team consisted of school superintendents, principals, curriculum directors, and teachers who regularly reviewed the test responses. These officials also had no access to student, school, or district information as they read the tests.
ARGUMENTS
(Please note: The arguments contained in this analysis originate from sources outside the Senate Fiscal Agency. The Senate Fiscal Agency neither supports nor opposes legislation.)
Supporting Argument
Requiring the Department of Treasury to give advance notice to school districts would restore stability to the MEAP testing process and credibility to its results. In the case of Hornung Elementary School in Brighton, 24-hour notice would have given the school time to alert the Department about a coding error that had mistakenly identified it as a school in Detroit. Even though the newspapers later printed a corrected list of schools, the damage to Hornung Elementary had been done. In other districts, the newspaper reports were published just a few days before voters were asked to approve a bond proposal; in Garden City, for example, the voters were deciding whether to approve a $36 million bond. When the district investigated the irregularities, it found that they were confined to five responses (out of 4,000) to a single question.
Sometimes the irregularities are built into the design of the test. In the Lansing School District, two schools were cited for several students' similar responses on the fifth-grade writing MEAP. That test, however, allowed students to brainstorm in small groups before they drafted a response to the open-ended question. It is possible that students discussed examples before they began writing, and that this resulted in similar phrasing across essays.
If the public believes that schools are encouraging or condoning cheating on the MEAP tests, the test results become meaningless. It is important that students believe in the merit of these tests and not become disillusioned. Since it is difficult to change initial impressions, especially damaging ones, school districts should have the chance to exonerate or explain themselves before negative stories are published.
Opposing Argument
The Department of Treasury is currently revising its internal policies to include a requirement that districts be notified of irregularities before the press. When this is accomplished, the Department will seek to make that policy part of its administrative rules and procedures. Writing this policy and possible rule into State statute would be duplicative and unnecessary.
Response: While advance notification is now policy, it should be codified. School districts cannot afford to stake their reputations on policy alone.
Opposing Argument
It seems unlikely that 24 hours would be adequate time for a school to investigate the cause of either a very particular problem or a very vague issue. This proposed legislation should give school districts a longer period of time to respond to reported testing irregularities.
- Legislative Analyst: C. Layman
FISCAL IMPACT
The bill would have no fiscal impact on State or local government.
- Fiscal Analyst: J. RunnelsA0102\s562a
This analysis was prepared by nonpartisan Senate staff for use by the Senate in its deliberations and does not constitute an official statement of legislative intent.