Does Teach First boost results?
Education Business, In The News, Research, Teach First Tuesday, November 16th, 2010
Beeb education journos are again showing a true lack of ability when it comes to appraising ‘research’.
Perhaps it isn’t their job to check the quality of research but they wouldn’t write about a medical research paper based on bad research so why is education research any different.
This time the Beeb tell us that Teach First teachers are supposed to be making a difference to school results.
This view is based on research conducted by University of Manchester researchers. Yet even without reading it in full, it is clear that this research study and its findings should be viewed with suspicion:
1. The research was commissioned by the very same people (i.e. Teach First) that have a vested interest in Teach First having some effect. To think that conflicts of interest have no impact on either the way research is conducted (e.g. types of answers given in interviews) or the way the findings are presented is being very, very naive.
Corrupt research practices are known to be rife in education. We don’t hear about it because researchers, who face losing access to future research funding for rocking the boat and embarrassing the many business/political/individual interests involved with a study, will choose their careers over the truth every time. And who’d blame them with food to put on the table. How many whistle-blowers can you name in education?
2. Reactivity is a widely-recognised phenomenon in social research. That is to say, the mere fact that the spotlight is on schools with Teach First teachers will have a positive effect on performance. This has nothing to do with Teach First methods, the teacher’s abilities or degree classifications etc. It results from nothing more than the fact the students feel special with so much new attention on them and they respond accordingly.
Better to study the same teachers over a longer period with the same class of students and compare this with other non-Teach First teachers. Of course, this isn’t possible given the fact Teach First teachers only do it for 2 years before getting a job outside teaching. Why wouldn’t they? They know by then that teaching is a battle against-the-odds and realise that a life in business is the golden ticket. To make significant long-term improvements in student performance requires more than fresh blood, and fanfares only produce short-term results.
3. Our particular favourite: correlation DOES NOT equal causation. How many times is this confused in journalism and social research?!
4. “They analysed data and conducted interviews in 87 Teach First schools and 87 comparable secondary schools, and with 848 teachers across England.” Comparable to what? Aren’t all schools different?
5. Interviews are incredibly unreliable. The wording of questions can never be precise enough to guarantee reliabilty. Interviewees change their minds, lie or tell the interviewer what they want to hear.
6. “Researchers used a complicated points system to measure this and concluded the average improvement equated to a third of a GCSE per pupil per subject. ” So what? Just because it’s complicated doesn’t then mean it’s true. Also, the researchers concluded that the average improvement equated to a third of a GCSE per pupil per subject. Based on what? Their instinct or the fact that they were under pressure to show improvement? All very arbitrary to us.
7. “the charity is hoping to expand from the 552 teachers it recruited this year to 1,140 per year by 2013.” There we go. This why they commissioned the research. This ‘charity’ wants to expand. And what better way to show how great it is than a bit of ‘research’.
This is not true, bona fide research. This is cynical marketing dressed up in the scientific language of research. Will somebody at the Beeb understand this eventually, we wonder. We can only hope.