Experts Challenge State-By-State Education Rankings

A widely-publicized report that ranks state education systems by comparing how much each one spends on schools with student performance on selective tests is flawed in its methodology and findings, education experts say. The state-by-state analysis was released on April 17 by the American Legislative Exchange Council (ALEC), a conservative organization that numbers 2,400 state legislators among its members.

“This is a very unsophisticated type of research that provides evidence for a set of beliefs,” said Craig Jerald, former director of Education Week’s annual Quality Counts survey of the state of U.S.education. “It is disingenuous and naive,” adds Jerald who is now a senior analyst with Education Trust, a non-partisan, non-profit research group.

ALEC’s major finding is the same one the organization’s seven previous annual reports have reached: that public education doesn’t need more money to succeed. According to the report, the US has increased spending on schools by 22 percent since 1977, yet SAT and ACT test scores have remained stagnant.

“If only the debate were so simple as to whether or not money matters,” said Andrew Rotherham, Director of the 21st Century Schools Project, a moderate education think tank. “It is much more complex than that. Looking at aggregate spending and comparing it to inaccurate measures such as SAT and ACT is frankly irresponsible” he said.

Andy T. LeFevre, the report’s author, said his study gives lawmakers information they need to make policy decisions. “The way that state legislators are accustomed to talking about education is about money going into the system. How big are our classes? How much do we pay teachers? They are hearing that if we pay teachers another ,000 we will get better scores. If we lower class-sizes schools will improve, but our report shows that isn’t the case,” he said

Governors and lawmakers have been trying to improve U.S. public education since A Nation At Risk was published in 1983. That report by a presidential commission, which stated that American schools were drowning in a “rising tide of mediocrity,” inspired reforms that have established standards of learning and require more accountability

ALEC is trying to reopen a debate educators resolved long ago, Jerald said. “More dollars can make a difference, if they are used to purchase things that matter – higher qualified teachers for instance. When you spend money on things that matter for student learning you can produce more student learning,” he told Stateline.org

“There is a huge problem with using SAT/ACT scores because the sample of students who take them is self-selected. You never ever use self-selected samples when you are measuring the impact on student achievement, “Jerald said.

The report includes a disclaimer for using SAT/ACT numbers. “Such self-selection makes state-by-state comparisons of educational achievement, based on either test alone, somewhat misleading,” it says on page 62.

Experts said ALEC’s report was also misleading because its figures on growth of spending from the mid-1970s through 2000 make no allowance for inflation. “Growth in education budgets has been just keeping up with inflation, even when there was lots of extra money in state budgets,” school finance expert John Augenblick said.

“Not being economists, we didn’t look at the rate of inflation throughout the time period. We looked at benchmark years, 98-99, 88-89 and 78-79,” LeFevre responded.

Jerald points out that the study also didn’t adjust per pupil spending from district to district. These numbers can be downloaded from the Department of Education website.

“That really undermines their contention that they have conducted a reliable analysis of whether money matters. State averages really fail to capture all of the nuances and variations within an individual state. There is such a wide variation across New York of all the indicators the ALEC report uses: student achievement, spending per pupil, teacher-student ratios to average these up to the state level and try to do a sophisticated analysis is like trying to build a skyscraper with Legos. That is why people don’t do it,” Jerald said.

ALEC’s study finds that Maine and Connecticut increased spending on schools more than any other state over the period studied. Those two states rank 15th and 13th respectively in the study, but Maine 8th graders rank first in the nation and Connecticut 8th graders second on the National Assessment of Education Progress reading assessment test.

The ALEC report ranks Iowa first in the nation in education because it spends less on teachers and students yet yields high SAT/ACT scores. The report also contends that spending more per pupil, shrinking class size and paying teachers more does not improve student performance.

“We have looked at a lot of different factors and we looked at each individual factor on its own. The report has a lot of information in it and we are trying to show that there really is no correlation between increasing funding and getting better scores,”said LeFevre.

Rotherham says ALEC comes up with the same report every year. “There is nothing new here. It is a compendium of data with an ideological veneer put on it,” he said.