Saturday, October 16, 2010

Media Matters Tries but Fails to Refute the School Choice Evidence

Yesterday, Media Matters tried to refute a blog post in which I point out, among other things, that the impact of voucher use in the D.C. Opportunity Scholarship Program resulted in a 91 percent graduation rate compared to 70 percent in the control group. The findings are from the U.S. Department of Education’s final evaluation of the voucher program, authored by Dr. Patrick Wolf.

Walid Zafar writes via Media Matters: "Where does Burke get the 91 percent figure from? Well, not this [the Department of Education’s] report. It’s hard not concluding that she made that statistic up. The report puts the graduation rate for students receiving vouchers at 82 percent."

Not so fast. The report does in fact find that the use of voucher resulted in a 91 percent graduation rate. On page 20 of the report’s executive summary, Wolf writes: "The offer of an OSP scholarship raised students’ probability of completing high school by 12 percentage points overall. The graduation rate based on parent-provided information was 82 percent for the treatment group compared to 70 percent for the control group. There was a 21 percent difference (impact) for using a scholarship to attend a participating private school.

The 21 percentage point difference for impact means the typical student who received a voucher and actually used it to attend a private school had a graduation rate of 91 percent, compared to 70 percent for non-voucher students. Here’s exactly how the graduation rates break down:

* D.C. Public Schools graduation rate: 49 percent.

* Control group (those students who applied for a voucher but did not receive one) graduation rate: 70 percent.

* Voucher recipient group (students who applied for a voucher, won the lottery to receive one, but did not necessarily use it) graduation rate: 82 percent.

* Impact of voucher use: (students who applied for, received, and actually used the voucher to attend a private school) graduation rate: 91 percent.

Zafar also argues that the results of the study are minimized due to the increased motivation of parents who applied for a scholarship: "You can’t compare the graduation rate at DC Public Schools (which take in all who apply, regardless of learning disabilities and level of parental involvement) to a lottery based voucher system to which only the most highly motivated students (and parents) choose to apply."

First, the D.C. Opportunity Scholarship Program does have to take all students who apply. When applications exceed scholarships, officials use a lottery to determine which students receive vouchers. In fact, because evaluators anticipated objections like Zafar’s, they controlled for the students who applied for a voucher but were ultimately not offered one.

These presumably highly motivated students were evenly distributed across the treatment and control group, which is probably why the control group graduation rate of 70 percent was higher than the overall DCPS rate of 49 percent. The voucher students significantly outperformed the control group on the crucial measure of high school graduation even though the lottery ensured that both groups were equally stocked with motivated students and parents.

While it’s true that parents have to have a certain level of interest in the educational opportunities of their children in order to apply for a voucher, thousands of low-income families in the District jumped at the opportunity to do so when given the chance. In fact, there were four applicants for every available scholarship.

Finally, Zafar argues that the DCOSP had no impact on academic achievement: "In the area of student achievement, the report concludes, “Overall reading and math test scores were not significantly affected by the Program, based on our main analysis approach.” Most crucially, the report notes that “No significant impacts on achievement were detected for students” who “were lower performing academically when they applied.” In other words, the students who did well on the voucher program were those who were already doing well in public school".

While the final evaluation did not find a statistically significant impact on academic achievement (which was not the main point of our argument), it did find that the scholarships had a positive impact on academic outcomes for some subgroups of students. Moreover, Dr. Wolf, the lead researcher on the OSP study, explains in a statement from the University of Arkansas that the significant positive impact on graduation rates is more important than the impact on academic achievement:
These results are important because high school graduation is strongly associated with a large number of important life outcomes such as lifetime earnings, longevity, avoiding prison and out-of-wedlock births, and marital stability. Academic achievement, in contrast, is only weakly associated with most of those outcomes.

In the area of education, how far you go is more important than how much you know, and D.C. students went farther with the assistance of a school voucher.

Facts matter, and we hope we’ve stated them clearly enough so that even Media Matters can’t deny them.


Regular exams boost your brain power

Regular testing actually improves your brains ability to learn, scientists find, in a study that is likely to reopen the debate over the effectiveness of exams.

Researchers found that preparing for tests actually improved memory by making the brain come up with more efficient ways to store and recall facts. In particular the brain comes up with mental keywords – called mediators – which trigger memories which they would not do when studying only. That means they remember more facts, for longer.

Dr Katherine Rawson, a psychologist at Kent State State University, said: "Taking practice tests – particularly ones that involve attempting to recall something from memory – can drastically increase the likelihood that you'll be able to remember that information again later.

"Given that hundreds of experiments have been conducted to establish the effects of testing on learning, it's surprising that we know very little about why testing improves memory."

Dr Rawson and former Kent State graduate student Mary Pyc reported an experiment indicating that at least one reason why testing is good for memory is that testing supports the use of more effective encoding strategies.

Dr Rawson said: "Suppose you were trying to learn a foreign language vocabulary. "In our research, we typically use Swahili-English word pairs, such as 'wingu – cloud'. "To learn this item, you could just repeat it over and over to yourself each time you studied it, but it turns out that's not a particularly effective strategy for committing something to memory. "A more effective strategy is to develop a keyword that connects the foreign language word with the English word. 'Wingu' sounds like 'wing', birds have wings and fly in the 'clouds'.

"Of course, this works only as well as the keyword you come up with. "For a keyword to be any good, you have to be able to remember your keyword when you're given the foreign word later. "Also, for a keyword to be good, you have to be able to remember the English word once you remember the keyword."

The findings are due to be published in the journal Science.


Australian State wants to get rid of dummy teachers

But you would have to be a dummy to take up teaching in their schools these days. Some new teaching graduates walk out after a week when they encounter the reality of it

QUEENSLAND'S teaching profession is facing a crackdown on university entrance standards in a bid to boost quality in the classroom.

Students will have to attain an OP score of 12 or better to gain entry to a teaching degree under a proposal being considered by the State Government. OP cut-offs have been as low as OP19 at some Queensland universities in recent years, fuelling concerns over the quality of newly graduate teachers. Students will also have to obtain a minimum standard in English, mathematics and science.

The proposals are among 21 recommendations put to the State Government in a review of teacher education and induction, part of the Flying Start project.

The review, currently being considered by Queensland education stakeholders, follows consultation on proposed national entry requirements for undergraduate pre-service teacher programs. Proposed national standards require a minimum level of mathematics and English for Year 12 students, but not science, as suggested in Queensland.

As of next year, Queensland primary school teaching graduates will be required to sit a test in English, maths and science to become a registered teacher.

Queensland Teachers Union president Steve Ryan said while he did not oppose the idea of having an OP12 cut-off for a Bachelor of Education, moves including paying teachers more were needed. "It is quite clear that we need to try to attract the better students in terms of OP scores into the teaching profession but the problem we have got is that the OP score is demand driven," he said.

Mr Ryan said many students who didn't achieve an OP12 could be wonderful teachers. The Christian Heritage College's Colette Alexander agreed, warning some universities which catered for lower-scoring OP students might be badly affected. "Academic performance during school does not guarantee quality teaching," she said. "What makes a difference with a teacher is whether a person wants to teach."

Professor Peter Renshaw, head of the School of Education at the University of Queensland where the OP cut-off was 11 this year, agreed that a student's OP didn't always reflect their capability. But he said the OP cut-off would be good for the perception of teaching. He also said the OP requirement did not apply to many Bachelor of Education graduates at his university, with many coming from other degrees rather than straight from school.

Queensland Deans of Education Forum chair Professor Wendy Patton said contingencies built into the proposal meant most universities supported the cut-off. Under the proposal, students with lower OPs can be granted entry in exceptional circumstances. "It provides the opportunity for individuals to say 'I can put forward a case' and for institutions to say 'well, let's have a look at this case'," Professor Patton said.

The teacher training review follows an investigation into the state's education system last year, when Professor Geoff Masters raised concerns about the competency of beginning teachers.


No comments: