This is a period of unrivaled education reform. Charter schools, teacher merit pay, diversity policies - all efforts to improve what many see as a failing public school system. But every effort needs evidence as to whether it is working. Much of it comes from standardized testing. That puts even more emphasis on the most effective way to analyze the growing amount of data. The information is being used in all kinds of ways, from what classes a student takes to teacher merit pay and, in some cases, termination.
In the early 1980s, Bill Sanders was a well-respected stats professor at the University of Tennessee. He had worked at Oak Ridge Labs and done some work in agriculture. One day, he was walking to class and picked up a newspaper. Inside was an article detailing then-Governor Lamar Alexander’s effort to hold teachers more accountable for student performance. A line in the article from a critic stated that statistical models could not be used to evaluate a teacher’s effectiveness. There were too many human and sociological factors, the critic said.
That did not sit well with Sanders.
Bill Sanders: "And when I read that article I said there may be good reasons for not doing that but those are not good reasons."
Thirty years later, the education reform movement has finally arrived at the same place Sanders did, way back then. Now, the statistical models he has carefully cultivated and tweaked over three decades have become the Education Value Added Assessment System, or EVAAS. It’s sold by SAS Institute, and it’s the established leader in a multi-billion dollar industry that is poised to grow exponentially.
Nancy Allen is the principal at East Wake Middle school. She’s sitting at her desk after another busy school day and signing on to EVAAS. She says she uses it every day.
Allen says this year EVAAS indicated that dozens more students could handle Algebra I than were being put there by teacher evaluations. Many of the students are minority males.
Nancy Allen: "So having this data is a great tool because it gives us a starting point. We need to make data-informed decisions. "
Allen says EVAAS helps her determine what students should be taking Algebra 1, a key course that is a predictor of how students will fare in high school. Generally, those who take it in middle school do well. Those who wait until high school or don’t take it at all often drop out.
Allen pulls up a student’s data on EVAAS. He’s a Latino eighth-grader who always did better on standardized tests than he did in the classroom. EVAAS said he could handle Algebra I. Teachers said he couldn’t. Allen put him in it and it changed how he saw himself as a student. He’s done well in Algebra and his grades picked up significantly in other classes, too.
Allen: "And it’s because we said, no, you are smart. You may be the first person in your family ever to go to college, get a scholarship. This is how smart you really are."
The North Carolina Department of Public Instruction is paying SAS $2 million for EVAAS this year to make it available to every teacher and principal. EVAAS is sophisticated, but what it basically does is this: every end-of-grade or end-of-class test score for every student is considered. Using a complicated algorithm that is, in part, proprietary, EVAAS then can predict how that student will do on future tests.
Principals like Allen can also use EVAAS for something else. It can show her how teachers perform. If students in a teacher’s class do better than EVAAS says they should, a teacher gets a positive score… students who do worse than predicted show up as a negative score.
Allen can then identify which teachers do well with certain types of students… and which teachers are performing poorly over several years with all students.
Allen: "This data enables us to see that. It’s hard sometimes, but you know we’ve got to do it and become better educators because of it."
Right now, only principals and administrators can see teacher performance in North Carolina. That’s not true everywhere. The Los Angeles Times recently published teacher performance scores. New York City schools also made teacher performance data public.
Closer to home, Halifax County has fired ineffective teachers. Governor Bev Perdue says it’s the right thing to do.
Perdue placed a big bet on EVAAS, including it in the state’s successful Race to the Top application. So did many of the other winners - 9 of the 12 states that won Race to the Top grants use SAS’s EVAAS.
Gov. Bev Perdue: "There is no excuse for keeping a low performing teacher or a low performing principal in a school. And we have student outcomes that measure that capacity but there are also other things that need to go into the equation. I believe North Carolina is moving forward on this. Not fast enough. Nothing’s ever fast enough for any of us, I think, around kids."
But one major district in North Carolina isn’t using it. Charlotte Mecklenburg Schools has created its own system, thanks to a $1.4 million grant from the Gates Foundation.
Peter Gorman is the superintendent in Charlotte. He says the proprietary nature of EVAAS and its sole focus on test scores isn’t what he’s looking for in evaluating teachers…
The creator of EVAAS says it’s a mistake to introduce factors other than student test performance. Bill Sanders claims that gives a skewed result.
Peter Gorman: "We’ve created our own value-added system which we’re using as an open system which we will share the algorithm with anyone, it’s not a black box hidden. And then the other piece we think there are multiple measures cause it’s not just increasing student achievement on a test, it’s how you contribute to improving other teachers’ work. We think there’s multiple measures to measure greatness for teachers."
Other aspects of making decisions based on data aren’t as hotly debated. Both Sanders and Gorman agree, for instance, that more data will help analysts paint a clearer picture.
Sanders: "You really want to hold educators accountable over things for which they have control. You don’t want to hold people accountable for things over which they do not have control."
Currently, end-of-grade testing begins in the third grade in North Carolina.
Last week, Charlotte-Mecklenburg launched a pilot program that adds 52 more standardized tests across the elementary grades, including an exam for kindergartners.
It’s all part of an effort to get more data into the hands of educators, even if they’re not in agreement on what to do with it.
