Wednesday, November 26, 2008

SAT usage improves results

For some years now, many elite American colleges have been downgrading the role of standardized tests like the SAT in deciding which applicants are admitted, or have even discarded their use altogether. While some institutions justify this move primarily as a way to enroll a more diverse group of students, an increasing number claim that the SAT is a poor predictor of academic success in college, especially compared with high school grade-point averages.

Are they correct? To get an answer, we need to first decide on a good measure of "academic success." Given inconsistent grading standards for college courses, the most easily comparable metric is the graduation rate. Students' families and society both want college entrants to graduate, and we all know that having a college degree translates into higher income. Further, graduation rates among students and institutions vary much more widely than do college grades, making them a clearer indicator of how students are faring.

So, here is the question: do SATs predict graduation rates more accurately than high school grade-point averages? If we look merely at studies that statistically correlate SAT scores and high school grades with graduation rates, we find that, indeed, the two standards are roughly equivalent, meaning that the better that applicants do on either of these indicators the more likely they are to graduate from college. However, since students with high SAT scores tend to have better high school grade-point averages, this data doesn't tell us which of the indicators - independent of the other - is a better predictor of college success.

Instead, we need to look at the two factors separately. And we can, thanks to the recent experience of the State University of New York, America's largest comprehensive university system, where I was provost from 1997 to 2006. SUNY is blessed with many different types of campuses, mirroring most of the collegiate options (other than small elite private institutions) that characterize contemporary higher education. The university also collects a gold mine of student data, including statistics on pre-admission academic profiles and graduation rates.

In the 1990s, several SUNY campuses chose to raise their admissions standards by requiring higher SAT scores, while others opted to keep them unchanged. With respect to high school grades, all SUNY campuses consider applicants' grade-point averages in decisions, but among the total pool of applicants across the state system, those averages have remained fairly consistent over time.

Thus, by comparing graduation rates at SUNY campuses that raised the SAT admissions bar with those that didn't, we have a controlled experiment of sorts that can fairly conclusively tell us whether SAT scores were accurate predictors of whether a student would get a degree.

The short answer is: yes, they were. Consider the changes in admissions profiles and six-year graduation rates of the classes entering in 1997 and 2001 at SUNY's 16 baccalaureate institutions. Among this group, nine campuses raised the emphasis they put on the SAT after 1997. This group included two prestigious research universities (Buffalo and Stony Brook) and seven smaller, regional colleges (Brockport, Cortland, New Paltz, Old Westbury, Oneonta, Potsdam and Purchase).

Among the campuses that raised selectivity, the average incoming student's SAT score increased 4.5 percent (at Cortland) to 13.3 percent (Old Westbury), while high school grade-point averages increased only 2.4 percent to 3.7 percent - a gain in grades almost identical to that at campuses that did not raise their SAT cutoff.

Yet when we look at the graduation rates of those incoming classes, we find remarkable improvements at the increasingly selective campuses. These ranged from 10 percent (at Stony Brook, where the six-year graduation rate went to 59.2 percent from 53.8 percent) to 95 percent (at Old Westbury, which went to 35.9 percent from 18.4 percent).

Most revealingly, graduation rates actually declined at the seven SUNY campuses that did not raise their cutoffs and whose entering students' SAT scores from 1997 to 2001 were stable or rose only modestly. Even at Binghamton, always the most selective of SUNY's research universities, the graduation rate declined by 2.8 percent.

The change is even more striking if we compare experiences of three pairs of similar SUNY campuses that, from 1997 to 2001, took sharply divergent paths. First, Stony Brook and Albany, both research universities: over four years, at Stony Brook the average entering freshman SAT score went up 7.9 percent, to 1164, and the graduation rate rose by 10 percent; meanwhile, Albany's average freshman SAT score increased by only 1.3 percent and its graduation rate fell by 2.7 percent, to 64 percent.

Next, Brockport and Oswego, two urban colleges with about 8,000 students each: Brockport's average freshman SAT score rose 5.7 percent to 1080, and its graduation rate increased by 18.7 percent, to 58.5 percent. At the same time, Oswego's freshman SAT average rose by only 3 percent and its graduation rate fell by 1.9 percent, to 52.6 percent.

Finally, Oneonta and Plattsburgh, two small liberal arts colleges with 5,000 students each: Oneonta's freshman SAT score increased by 6.2 percent, to 1069, and its graduation rate rose 25.3 percent, to 58.9 percent. Plattsburgh's average freshman SAT score increased by 1.3 percent and its graduation rate fell sharply, by 6.3 percent, to 55.1 percent.

Clearly, we find that among a group of SUNY campuses with very different missions and admissions standards, and at which the high school grade-point averages of enrolling freshmen improved by the same modest amount (about 2 percent to 4 percent), only those campuses whose incoming students' SAT scores improved substantially saw gains in graduation rates.

Demeaning the SAT has become fashionable at campuses across the country. But college administrators who really seek to understand the value of the test based on good empirical evidence would do well to learn from the varied experiences of New York's state university campuses.


More than a third of schools failing pupils, British regulator warns

More than a third of schools are not giving pupils a good education, inspectors warned today. One in ten 11-year-olds are still leaving primary school without reaching the level expected of their age group in English and maths, Ofsted's annual report found. And more than half of England's teenagers are still leaving school without five good GCSEs, including English and maths.

In her third annual report, Chief Inspector of Schools Christine Gilbert said England must do better if it is to compare favourably with the rest of the world. She said she was concerned that there was still too much variation in achievement between different areas of the country. Poor quality services existed across the education and care sectors, for those from disadvantaged backgrounds. Poorer children, such as those who qualify for free schools meals, were still less likely to achieve five good GCSEs, including English and maths, than their peers. In 2007, only 21 per cent of children on free school meals achieved this benchmark, compared with 49 per cent of other pupils.

Ms Gilbert said there was a strong link across every sector between deprivation and poor quality services. She said: "This means that children and families already experiencing relative deprivation face further inequity in the quality of care and support for their welfare, learning and development. "In short, if you are poor you are more likely to receive poor services: disadvantage compounds disadvantage." But Ms Gilbert added it was possible to "buck this trend" and there were examples of places that were outstanding. She said: "Typically the provision that really makes a difference is ambitious. It does not believe that anyone's past or present circumstances should define their future."

Today's report covers the first full year of Ofsted's new wider remit - they now inspect and regulate social care, children's services, adult learning and skills, as well as schools and childcare. It found improvements in school standards, with 15 per cent of schools judged to be outstanding, up slightly from 14 per cent last year. In primaries that figure was 13 per cent while in secondaries it was 17 per cent. But more than a third of schools (37 per cent) were found to be not good enough - given a rating of "satisfactory" or "inadequate". More than four in ten (43 per cent) secondary schools were rated no better than satisfactory, although this was down from 49 per cent in 2006/07. In primaries this figure was 37 per cent. Nursery schools had some of the best ratings, with 39 per cent judged to be outstanding and 58 per cent rated good. Just 3 per cent were rated satisfactory and there were none that were inadequate.

A higher proportion of childcare and early education was good or outstanding this year. But the quality of provision varies, and it is not as good in areas with high deprivation. The report said that teaching literacy and numeracy skills must "remain a priority" and while there was evidence of improvements in these areas, in some progress was still too slow. And it warned that more needed to be done to raise standards at GCSE level. "A decade ago, two-thirds of secondary age pupils left compulsory education with five good GCSEs, including English and maths - it is still more than half."


Australia: Lazy teachers "forget" toddler

An 18-month-old toddler has been left abandoned inside a locked childcare centre in Sydney's west - the second such case in just months. Uriah Vollmer, son of Daily Telegraph reporter Tim Vollmer, was left sleeping in a cot inside Penrith's Nepean Pre-School when staff went home early. His mother Michelle arrived 10 minutes before closing time to find the centre already locked and empty - apart from baby Uriah. It was only when a centre staff member drove by and spotted a distraught Mrs Vollmer that Uriah was discovered asleep inside.

The two incidents have prompted calls for a State Government review of centre lock-up procedures. "(This) is proof that a serious overhaul of the procedures is urgently needed," Mrs Vollmer said. The first case, in May, resulted in the baby being left alone for more than an hour before police broke in.

Mrs Vollmers' concerns have been backed by the body that represents childcare centres around the country. Childcare Associations Australia conceded yesterday that it was timely for the State Government to re-examine centre training programs in the wake of two incidents in just six months. "The fact that it has happened twice might mean there needs to be a training program and a review of lock-up procedures," executive director Helen Kenneally said. "It does astound me people have these things happen."

A spokeswoman for Community Services Minister Linda Burney said yesterday a total review of the regulations was under way but changes, if any, would not come into effect until 2010. The current regulation states that two primary contact staff must inspect the premises to ensure no children are left behind.

Mrs Vollmer said the delay was "completely unacceptable and it is putting the safety of kids at risk". She also criticised the fact she was still yet to be contacted by DOCS investigators despite making a complaint on Friday. Ms Burney's spokeswoman said investigators had already interviewed staff at the centre and checked its records and would move on to the family.


No comments: