Gates Foundation: Test scores not enough for teacher reviews
SEATTLE (AP) — After three years of research on measuring teacher performance, the Bill & Melinda Gates Foundation announced Tuesday that it takes multiple measures to most accurately judge educators.
The Seattle foundation concluded in its final report on its Measures of Effective Teaching research that test scores or principal evaluations are not enough on their own. The findings mirror what teachers unions have been saying.
The federal government has been pushing states through incentive grants and waivers to update their teacher evaluation systems because it felt existing systems were inadequate. At the same time, the Gates Foundation was studying these issues, saying it wanted to add to the discussion. Most states and big city districts have adopted some elements of the recommendations.
Foundation officials say the more reliable systems include a balanced mix of evaluation methods: student test scores, lesson observation and student surveys.
"If you do it right, you can generate measures that will help identify teachers that are having a bigger impact. That's a really big deal," said lead investigator Thomas J. Kane, professor of education and economics at the Harvard Graduate School of Education.
The foundation studied 3,000 teachers across the country. The research included classroom videos of 13,000 lessons, interviews with students and administrators, test scores and experiments to test theories.
Classrooms were studied in Charlotte-Mecklenburg Schools, the Dallas Independent School District, Denver Public Schools, Hillsborough County Public Schools in Tampa and St. Petersburg, Fla., Memphis City Schools, The New York City Department of Education and Pittsburgh Public Schools.
One of the new conclusions of the report is that having a second person, other than the principal, evaluate a teacher greatly enhances reliability.
The researchers also established a baseline for how much influence test scores should have on teacher evaluations, saying tests should not represent more than half the total teacher evaluation score, unless the district is just trying to determine future test scores.
Vicki Phillips, director of the foundation's K-12 education program, said the focus of teacher evaluation systems should be on giving feedback to help teachers improve.
Several districts involved in the research acknowledged that student surveys were the most controversial part of the process, and some, like Hillsboro County Public Schools in Florida, have opted to leave them out of the mix when scoring teachers.
Jean Clements, president of the Hillsboro Classroom Teachers Association, said her district decided the results of student surveys, which ask questions like "do you feel challenged to do your best work," may not be trusted by teachers.
The researchers found, however, that student surveys help teachers improve their practice because those results evoke the most emotions.
Test scores and principal evaluations don't bring tears to many teachers' eyes, Kane said. "Getting these student surveys back ... hits you where your heart is."
The Seattle foundation concluded in its final report on its Measures of Effective Teaching research that test scores or principal evaluations are not enough on their own. The findings mirror what teachers unions have been saying.
The federal government has been pushing states through incentive grants and waivers to update their teacher evaluation systems because it felt existing systems were inadequate. At the same time, the Gates Foundation was studying these issues, saying it wanted to add to the discussion. Most states and big city districts have adopted some elements of the recommendations.
Foundation officials say the more reliable systems include a balanced mix of evaluation methods: student test scores, lesson observation and student surveys.
"If you do it right, you can generate measures that will help identify teachers that are having a bigger impact. That's a really big deal," said lead investigator Thomas J. Kane, professor of education and economics at the Harvard Graduate School of Education.
The foundation studied 3,000 teachers across the country. The research included classroom videos of 13,000 lessons, interviews with students and administrators, test scores and experiments to test theories.
Classrooms were studied in Charlotte-Mecklenburg Schools, the Dallas Independent School District, Denver Public Schools, Hillsborough County Public Schools in Tampa and St. Petersburg, Fla., Memphis City Schools, The New York City Department of Education and Pittsburgh Public Schools.
One of the new conclusions of the report is that having a second person, other than the principal, evaluate a teacher greatly enhances reliability.
The researchers also established a baseline for how much influence test scores should have on teacher evaluations, saying tests should not represent more than half the total teacher evaluation score, unless the district is just trying to determine future test scores.
Vicki Phillips, director of the foundation's K-12 education program, said the focus of teacher evaluation systems should be on giving feedback to help teachers improve.
Several districts involved in the research acknowledged that student surveys were the most controversial part of the process, and some, like Hillsboro County Public Schools in Florida, have opted to leave them out of the mix when scoring teachers.
Jean Clements, president of the Hillsboro Classroom Teachers Association, said her district decided the results of student surveys, which ask questions like "do you feel challenged to do your best work," may not be trusted by teachers.
The researchers found, however, that student surveys help teachers improve their practice because those results evoke the most emotions.
Test scores and principal evaluations don't bring tears to many teachers' eyes, Kane said. "Getting these student surveys back ... hits you where your heart is."
Get rid of the antiquated notion of tenure/lifetime-employment-no-matter-what and create a collection of elements for evaluation that are modern and make sense together to get an accurate assessment of how teachers perform in their role. Tenure combined with the quirks of academic culture are one part of what is eroding our education process in this country.
Could you please explain to me what tenure means as related to the K-12 teachers of Washington State? Iâve been at this profession for nearly two decades and have yet to experience this concept of tenure. For the first three years a teacher can be ânon-renewedâ (fired) for any reason without cause. After that it just takes a little effort from the principal to remove an incompetent teacher. Step one: Observe and document that the teacher is not meeting the Teaching for Learning Standards. This can be done in as little as a one hour formal observation. Step two: Teacher is placed on a Plan of Improvement. Here the teacher meets with a mentor in the field to see if there is a chance for improvement.  This often lasts 60-90 days. Step three:  An outside expert, chosen by the District Administration, offers their opinion after review and observation. Based on this the district will choose to fire or retain said teacher. The Union supports this process and assists in removing a great many poor teachers each year. This entire process is skipped if a teacher is accused of physical or sexual misconduct. They are placed on paid leave until the accusations are confirmed or disproved. Teachers are considered guilty until proven innocent.
Â
Here is an excellent analysis of what the data really said, rather than the Gates Spin.
Â
http://jaypgreene.com/2013/01/09/understanding-the-gates-foundations-measuring-effective-teachers-project/
 ....Â
... the âbest mix?â Best for what? Itâs best for the political packaging of a centrally imposed, mechanistic teacher evaluation system, which is what this is all really about.........
Â
So, rather than having âfigured out what makes a good teacherâ the Gates Foundation has learned very little in this project about effective teaching practices. The project was an expensive flop. Letâs not compound the error by adopting this expensive flop as the basis for centrally imposed, mechanistic teacher evaluation systems nationwide.
start firing the teachers that can not teach they already work a part time job for full time pay and they keep bitching how over worked they are with three months off in the summer and every holiday ever invented most are lazy
 @armor Wow, your ignorance is homeric.
 truth hurt  hey its wed why are you not teaching to much time on your hands or surfing the net on the tax payers dime
 @armor Typical response. Check out my past posts and you will find me quite moderate. Did your "spell check" miss the lowercase "s" in Seattle?
 @thebigteacher spoken like a true seattle liberal
 @armor  @thebigteacher Your ideas are as Imprudent as your English.
 @thebigteacher its about ideas and not english
 @armor Ignorant, no ( I worked shifts and did so while teaching for my first 5 years). Fishing yes. You attempted to get a rise out of me and I simply turned the onus back on you. By the way, your English is atrocious.
 @lakeview no auto correct failed on my cell phone
do you always make fun of people to try to make your self feel superior
@lakeview @armor Maybe armor wasn't paying attention in class as is the case with a lot of kids failing these days. Their lack of discipline and their "I can do what ever I want" attitude is what is failing them. Teachers don't stand a chance these days with the holy than thou attitudes of some of these kids.
 @thebigteacher what an ignorant statement coming from a teacher obviously you do not realize that appx 25% of the american work force works off shift or weekends and as a casino manager for a major casino i work nights and weekend when most of the business is but hey we all know what assume means
 @armor  Your English teacher failed you.Â
 @armor I assume you are unemployed. As a class we check the news twice a day. Now I'm enjoying my 20 minute lunch.
Seattle should bring in Michelle Rhee and try some off her ideas on education reform of schools and teachers..
 @Windowseat Rhee???  ....  Wasn't Goodloe-Johnson was bad enough?  (both pushed Everyday Math).  Notice the WA DC math scores on NAEP for grade 4 in 2009:
Â
200 Detroit  213 Cleveland Ohio  219 Fresno
220 WA DC Â 220 Milwaukee
222 Chicago  222 Philadelphia  222 Los Angeles
223 Jefferson County KY Â 225 Atlanta Â
236 Huston  236 San Diego  236 Boston
236 Miami-Dade County FL Â 237 New York City
240 Austin  245 Charlotte
Â
large city average was 231
national average was 239
 @Windowseat She was discredited and left in shame.
Everyone who has been in school knows this. You can't help what the teacher last year taught the kids and you certainly can't help it if some kids have a bad homelife. That particularly why kids in poorer areas tend to do worse. When you are constantly worried about your parents or if shots are going to come through your house or when your next meal is going to come, you won't do well.Â
Â
The only possible testing I could see working would be test the kid at the beginning of the year to evaluate their skills (the test would have everything on it they will learn during the year) and then retest them at the end to see if they've learned most of the concepts. Even then you can't account for a percentage of kids who have horrible home lifes, untreated ADD, or the million other things that can be wrong.Â
It took them 3 years and how much money to figure that one out? Â Teacher evaluation is extremely hard to do because of the wide range of skill levels in the classroom. Â A teacher one year may get a classroom full of brats whose parents could care less and the next year have only one or two. Â
@cyclops Wouldn't a high performing teacher be able to adjust and accommodate these differences? It's easy to teach bright, interested students with engaged parents. The real test of a teacher is connecting and making impact to the lowest common denominator when it comes to the student population....AND how to provide enrichment to the HIGHLY CAPABLE students. the kids in the middle are the easiest. Students at both ends of the learning spectrum are what challenges teachers and the education system. After one looks at the learning spectrum the next thing a teacher needs to be able to address is the students home environment. DOes the student have adequate support in the home to aid in their learning? This includes nutrition, nurturing environment, proper parental oversight etc etc. Teachers should be able to recognize where the challenges are stemming from and take actions to address at whatever level they need. This is key in the first 8 years of education.Â
 @AmFuzzy Yes that is exactly what a great teacher does and it cannot be measured by test scores.
Â
You can't fix stupid....and it usually begins with the parents.
 @AmFuzzy  @cyclops Sounds great in theory. I can see that if they have a class size of 10 or less. In reality the lowest common denominator sucks up most of the teacher's time and leave little to students who are doing ok or are under-challenged. And how is teacher suppose to take action to address home challenge? It is hard enough when parents take the side of their little angels and lay blame to the teacher instead. (no the teacher has 25 other kids and you are not special enough to him/her to pick on individually)
 @AmFuzzy Too many parents expect the teacher to be a baby sitter as well as a teacher and it is degrading the entire system.
@AmFuzzy If schools were to split kids up these days they would call it discrimination and segregation. Lawsuits and lawyers is all it would turn in to be.
@1000 @cyclops This is why I am a strong advocate of segmenting students by ability so teachers are able to teach to the masses at the same ability level. When I was growing up in Ohio our school system had "tracks". Track 1 were the students that were able to learn quickly and were more advanced on a particular subject. Track 2 were the average students and Track 3 were the students that needed additional assistance/support. I felt this allowed the teachers to give the students the appropriate level of educational support. Many say this level of segmentation "harms" a child's self-esteem. I say BS. In the real world we all have varying levels of abilities. We need to educate to the best of our ability first...build self-esteem along the way vs. worrying first about self-esteem. For home related issues it's the teachers responsibility to "recognize" the issues first...then try to figure out how best to address the home issue impact on the child's education then come up with a plan to work it out at school if the parents aren't open to address. Most home related issues are likely due to the parents so expecting the parents to "fix" them is not realistic. The teachers need to be trained on how to address the impact home issues have on a child's education without expecting support from the parent. If the teacher was able to get a parents support it would be a bonus but never should be EXPECTED. Â
I find it strange that Bill gates is posting him self out there on education. I mean the guy was a Havard drop out. Yeah he may have made a Multi-Billion dollar company. But he really just kind of stole most of his stuff. I just don't think that his success. Shows the best examples of how someone would want their kids to get ahead. I just find it sad that the most successful people we choose to idolize. Actualy do things that most of us find very criminal to get ahead. Kind of sick for us to allow this of our self's. I guess that's why our government system is so messed up. Its all about the person in office and not the people anymore.
@Bizquick Wow you basically called BIll Gates a criminal. Why such anger toward Bill Gates and MS? Hidden issue here maybe? Who would you hold up in the business world as a good example? Bill cares about kids....what are 90+% of the world business leaders doing that even comes close to his efforts?   I guess most of these people on this list should not be considered as good examples for kids growing up? http://www.retireat21.com/blog/the-most-successful-college-dropouts-in-history It's not about how much education one gets beyond high school its more important what they do with their education that matters. You make it sound as if a college education is what determines a persons worth as an individual or their value to a business or society. I guess you are one of those that believes everyone should go to college.
 @Bizquick There's also the fact that plenty of entrepreneurs have dropped out of school because they had the idea and the cash backing already to start their company.Â
 @Bizquick That was incredibly incoherent but I can at least address the Bill Gates Harvard issue - Bill Gates got his 10,000 hours in at a time when it was almost impossible to get 10,000 hours due to his parents sending him to one of the best private schools in the state. They had a computer in their school before almost anyone else did.  Also, because he had his parents backing - therefor not needing to work while he was starting his company like some of his partners did, he was able to successfully kick off Microsoft.
 @Bizquick Now that was incoherent. Â
Kids don't read enough. The line between rich and not doesn't just grow because of taxes and stuff. It also has to do with the growth between smart and stupid. There are too many obstacles today in life for a kid to commit towards education. reading won't solve it, but it'll help.
 @sirgavin7 That's where early childhood education would help. Alot of people don't realize how important it is to get kids reading at least by 2-3 years old (Reading to them and teaching them). They wait until kindergarten rolls around and I feel like that is really too late to kindle the spark of the enjoyment of books.Â
Gates Foundation always have an agenda behind their grant. I wonder what it is?
Â
While I do agree with Gates Foundation findings, I do not believe that Gates Foundation should have any right to get involved with education. The way that Microsoft is run, we need to ask ourselves, "Do we want kids to suffer emotionally (aka suicidal, angry, depression, bullying, bipolar, etc) in the educational system?" Most likely your answer is "no".  Then how do you help these kids at such a young age with all these problems? Gates Foundation does not cover this area so that means "pop a pill into the kids" is the solution in today's society.
 @komosux I do NOT believe in the Easter Bunny.
I do not believe in the Gates Foundation study findings because an analysis of the data does NOT support the conclusions being pushed by GF.
Â
Value Added Measures (using student test results for teacher evaluation) were found invalid as
VAM varied tremendously from year to year. Â The Sample Size of a teacher's students is too small for reliability.
Â
This balanced approach is the latest Gates attempt to push some form of VAM.
It is a lot more expensive and according to Jay Greene's analysis just as unreliable.
http://jaypgreene.com/2013/01/09/understanding-the-gates-foundations-measuring-effective-teachers-project/
Â
 @komosux Read again. The only grant money is federal, not microsoft.
@komoispropaganda Do you believe everything in the article and ONLY in this article? Have you heard from their employees or other propaganda (as you put it)?
 @komosux I;m debating the article. I never believe the media.
 @komosux You agree with the findings...... Okay
Â
Â
Who says microsoft is running anything, it's a study.
Â
How do we get from a study to kids suffering?
Â
Where does the study say kids need to take drugs? It's a study about teacher evaluations.
Â
I'm sorry, I don't see an agenda. I see a study that was done outside of government influence. Maybe the information has been biased by the media, but I believe a study to shed light into education is at least a start.
@komoispropaganda Gates Foundation does not give money away for free out of the goodness of their hearts. They give money in order to get something back from their grant. What is that?
The way that Microsoft and Gates Foundation's treats its employees is enough to indicate that the educational system may be heading that way. If you have kids, do you want your kids to have mental issue in the educational system if they can't keep up with the "grades" with their peers and there is no help to deal with these mental issues? Do you want your kids to slit their wrists cause their grades are not align with their performance review peers? Do you want your kids to come to school with a gun cause of poor performance reviews? I don't cause they're not adults and ready to deal with these kind of emotions, nor is popping a pill (eg Ritalin) the solution.
Gates Foundation and Microsoft have no business getting involve in the education system until they can fix their own employees morale and also get Microsoft stock back up. And Gates is still running Microsoft from the Board of Directors seat.
 @komosux Do you have any factual proof or is this just you talking out your arse? Give us proof such as data, links to factual websites (not wikipedia).Â
 @komosux Every scenario you've described would surely require stats. This about teacher evaluation studies. What is wrong with keeping teacher quality by way of evaluation? Microsoft is not doing the teacher evals, nor are they offering any grant money.
Â
Now if you were actually be able to connect a large government contract or something else substantial. Until then I will consider any push to change our lagging education systam as a good thing.
 @komosux  Where does the article say there is any "microsoft grant"?
Grant money comes at too high a price. Â It would be nice if our districts would have the wisdom to forgo such largese from oligarchs. Â
Our excellent teachers are being harangued by an increasing number of hoops and hurdles; being robbed of much of the joy that their calling would otherwise supply.
Whenever a voice is saying that there's a 'problem', red flags should go up, as another hegelian dialectic is most likely being foisted upon us (problem, reaction, solution)
 @cheekygesturton Lot's of big words there hoss. Do actions that would see the truth about teachers/teachers unions/efficient evaluations bother you?
Â
If you reply, don't use your thesaurus, it makes you look snooty.
 @komoispropaganda I suspect that someone is an English teacher/instructor/professor but I do agree with the snooty part. :)
 @PrairieDawn  @komoispropaganda  I hope not considering that he doesn't know how to use a semi-colon.Â
Waiting for the union response.............
 @Grumpa the teacher Unions have been saying the same thing for years. but the 'gubmint is in bed with test and textbook publishers. to them, they want more and more tests.