Tuesday, December 21, 2004


Post lifted from Mahalanobis

High-stakes testing, like school choice, has become an increasingly prominent feature of the educational landscape. Every state in the country, except Iowa, currently administers state-wide assessment tests to students in elementary and secondary school. Federal legislation requires states to test students annually in third through eighth grade and to judge the performance of schools based on student achievement scores.

The debate over high-stakes testing traditionally has pitted proponents arguing that such tests increase incentives for learning and hold schools accountable for their students' performance against opponents who argue that the emphasis on testing will lead teachers to substitute away from teaching other skills or topics not directly tested on the exam. Along with Brian Jacob, I have written two papers that explore a very different concern regarding high-stakes testing -- cheating on the part of teachers and administrators. As incentives for high test scores increase, unscrupulous teachers may be more likely to engage in a range of illicit activities, such as changing student responses on answer sheets, or filling in the blanks when a student fails to complete a section. Our work in this area represents the first systematic attempt to identify empirically the overall prevalence of teacher cheating and to analyze the factors that predict cheating.

To address these questions, we once again turn to data from the Chicago Public Schools, for which we have the question-by-question answers given by every student in grades 3-7 taking the Iowa Test of Basic Skills (ITBS) over an eight year period. In the first paper,(4) we develop and test an algorithm for detecting cheating. Our approach uses two types of cheating indicators: unexpected test score fluctuations and unusual patterns of answers for students within a classroom. Teacher cheating increases the likelihood that students in a classroom will experience large, unexpected increases in test scores one year, followed by very small test score gains (or even declines) the following year. Teacher cheating, especially if done in an unsophisticated manner, is also likely to leave tell-tale signs in the form of blocks of identical answers, unusual patterns of correlations across student answers within the classroom, or unusual response patterns within a student's exam (for example, a student who answers a number of very difficult questions correctly while missing many simple questions).

Empirically, we find evidence of cheating in approximately 4 to 5 percent of the classes in our sample. For two reasons, this estimate is likely to be a lower bound on the true incidence of cheating. First, we focus only on the most egregious type of cheating, where teachers systematically alter student test forms. There are other more subtle ways in which teachers can cheat, such as providing extra time to students, that our algorithm is unlikely to detect. Second, even when test forms are altered, our approach is only partially successful in detecting illicit behavior. We then demonstrate that the prevalence of cheating responds to relatively minor changes in teacher incentives. The importance of standardized tests in the ChiPS increased substantially with a change in leadership in 1996. Schools that scored low on reading tests were placed on probation and faced the threat of reconstitution. Following the introduction of this policy, the prevalence of cheating rose sharply in classrooms with large numbers of low-achieving students. In contrast, schools with average or higher-achieving students, which were at low risk for probation, showed no increase in cheating.

Our second paper on this topic(5) reports on the results of an unusual policy implementation of our cheating detection tools. We were invited by ChiPS to design and implement auditing and retesting procedures implementing our methods. Using that cheating detection algorithm, we selected roughly 120 classrooms to be retested on the Spring 2002 ITBS. The classrooms retested include not only cases suspected of cheating, but also classrooms that had achieved large gains but were not suspected of cheating, as well as a randomly selected control group. As a consequence, the implementation also allowed a prospective test of the validity of the tools we developed in our first paper on the subject.

The results of the retesting provided strong support for the effectiveness of the cheating detection algorithm. Classrooms suspected of cheating experienced large declines in test scores (on average about one grade equivalent, although in some cases the fall in mean classroom test scores was over three grade equivalents) when retested under controlled conditions. In contrast, classrooms not suspected of cheating a priori maintained virtually all of their gains on the retest. As a consequence of these audits and subsequent investigations, disciplinary action was brought against a substantial number of teachers, test administrators, and principals.


Note that nobody could point out where the usefulness in the criticized courses might lie

Charles Clarke, the education secretary, has continued his assault on the great subjects of academe by revealing that he regards medieval history as "ornamental" and a waste of public money. Not long after expressing the view that he didn't think much of classics and regarded the idea of education for its own sake as "a bit dodgy", Mr Clarke, who read maths and economics at King's College, Cambridge, went one further. "I don't mind there being some medievalists around for ornamental purposes, but there is no reason for the state to pay for them," he said on a visit to University College, Worcester. He only wanted the state to pay for subjects of "clear usefulness", according to today's Times Higher Educational Supplement.

Michael Biddiss, professor of medieval history at Reading University and a former president of the Historical Association, said: "Perhaps Mr Clarke and his spinners at the DfES are hoping to inspire the band of political yahoos who, in making New Labour ever more illiberal, must feel increasingly tempted to parrot Khrushchev's lament that 'historians are dangerous people - capable of upsetting everything'." Gillian Evans, a Cambridge University medievalist, said: "With a philistine thug like that in charge ... we need to protect the jobs of all the historians of thought and all the wordsmiths we can."

A spokesman for the Department for Education and Skills said: "The secretary of state was basically getting at the fact that universities exist to enable the British economy and society to deal with the challenges posed by the increasingly rapid process of global change."

Jane McAdoo, president of the Association of University Teachers, said: "I cannot believe that a secretary of state for education can ... have such a terribly narrow view of what education is."



For greatest efficiency, lowest cost and maximum choice, ALL schools should be privately owned and run -- with government-paid vouchers for the poor and minimal regulation.

Comments? Email me here. For times when blogger.com is playing up, there is a mirror of this site (viewable even in China!) here


No comments: