Tuesday, December 14, 2010

What if the tests are wrong?

A very sad story below. Even sadder is the fact that the teacher writing it has a Master's degree and yet still does not know the difference between an adjective and an adverb (as in "wrong" versus "wrongly"). And "me included" is also a bit rough. The reflexive "myself included" would be the formal usage

In today’s climate of high stakes testing, I haven’t heard too many people talking about the tests themselves. For example, who is making these tests, and what are some problems we’re having with them here at the ground level?

I’ll tell you some concerns I have. As dire as some of them may seem, there is a silver lining.

Last year, an essay question on the STAR test asked something like this, “What is the biggest thing you would change if you were given the key to your city?” The problem a lot of students had with this question, aside from being really bad at writing, was they didn’t have any idea what having a key to a city meant? They didn’t have the cultural capital, language proficiency, or life experience to know what that expression meant. So this prompt immediately relegated a large group of students to failure because they didn’t understand what the question was asking of them.

Detractors might say something like, “Well, that’s ridiculous. Those kids obviously aren’t even attempting to learn. How can they not know what a key to a city is? They have never had an interest in their own education.” Once again, it seems that society’s perceptions at large, and even those of the test makers, assumes these kids are a lot smarter than they are.

Let me put it this way. When the Chilean miners were being rescued, I showed my students some of CNN’s footage from my computer. Guess what? Over half of the students had no idea who the Chilean miners were. We sometimes forget how young and inexperienced these students are. We assume everyone has 900 channels, but you have to remember, many of these kids can’t afford cable. Hardly any of them have computers at home, and even less that have one hooked up to a printer that has ink. Internet? Forget about it.

We forget that while corporations and the rich are becoming more technologically advanced, the poor are lagging very far behind.

Now, I’m not arguing for dumbing-down test questions to their level, so to speak. Just because some of them spell their names wrong doesn’t mean our assessments should assess nothing more than getting the date right. I like the idea of thinking about what you would change in your city; I think that is a good writing prompt, I just think you need to make sure low-income students, ELL students, and students who can’t afford computers, the internet, or own a television that is simply plugged into a wall, understand the idioms and expressions you’re using.

Last month, at my school, we gave the same assessments in every high school. So every 9th grader in my English classes took the same assessment for House on Mango Street as every other 9th grader in the other two high schools in my city. Now I’m not even going to go into the fact that some teachers didn’t even teach that book, or that some classes don’t even have teachers yet (yes, still going on today on December 8th at my school). I’m just going to talk about the fact that after we assessed them, and uploaded their scores into the District database, a group of English teachers in my department got together to assess how they did on certain questions. What we found was a bit depressing. In our opinion, 8 of the questions were wrong. Either they were worded wrong, there was more than one answer, the answer given was wrong, there was a better answer- you get the picture. Now, on a test with less than 30 questions, how can we possibly assess anything if 8 of them are wrong?

So who is making these tests? Well, in our case, the tests were thought up by actual teachers at multiple sites, along with some district officials. Teachers (me included) took the tests beforehand and gave feedback as to which ones were wrong or needed work. The problem there was many of us differ about what is wrong with the tests and what needs to be reworded. Yet still, the finished product was still a test with many flaws.

What that tells me is that we need a greater focus in this profession about writing test questions, prompts, and we need to examine whether questions are really assessing what we want them to.

Here’s the kicker. They also need to be age appropriate and culturally sensitive.

So with all the things it takes to write a good prompt, or design a decent test, I just don’t think we have enough experts out there who are good at this.

Here’s a quick, but pathetic anecdote. A friend of mine got a job working for the education company in charge of monitoring my high school when we were taken over by NCLB. He was hired because of his business background, and his background in sales. He told me that at one point HE was helping write assessments- a man with absolutely no background in education. We both laughed about it. Then I went to work and put my head in my hands.

I am skeptical about every single assessment I give out, whether it’s a STAR test for NCLB or a District Assessment. I have serious reservations about what exactly we are assessing, and I am worried about how this will reflect on students and teachers.

I am all for increased teacher evaluation and accountability. But it is things like this that hurt our fight to show evaluations can work. How can you evaluate a teacher on a 28-question test when 8 of them are wrong? Instead of the class average being a 70%, it is now a 60%. If you compare that with how your students did last year in 8th grade, all that’s going to show is that in 9th grade (when they had YOU as a teacher), their scores dropped from Proficient to Basic. And five years from now, when we have REAL evaluations, those kinds of mistakes in the assessment itself will be forgotten, they’ll just see how large chunks of kids lost headway under your watch.

Okay, here’s the good news.

As far as I can see, even though a lot of these assessments are horrendous, they are pretty consistent. That is, they are consistently horrible, they always have many, many mistakes, so we can just hope all of the tests we are giving have a quarter of the questions wrong. In the end, the smartest kids still score the highest, and the struggling learners still struggle. These irregularities are not completely destroying our assessments of these kids. The danger is when the little problems result in bigger classifications- going from Proficient to Basic. That is a huge jump, even if it might only be a few points.

So our next big assessment is coming up. I just took the proposed exam and scored around a 78%, or a C+. I have a Master’s Degree. But that’s okay, because all the tests the students have been taking every year are this messed up, so hopefully it won’t reflect on what I’m teaching them.


Academic achievement ignored in Boston

But jocks highly praised. Why? One guess: The overwhelming whiteness of academic high achievers

Today, The Boston Globe published the latest in a long series of special “All-Scholastics” 14-page (12x22-inch) supplements on good local high school athletes from a variety of sports. These celebrations are produced three times a year (42 pages) with lots of pictures and little bios and lists of all-stars from the Boston area.

Again this Fall, there was no room for any mention by The Boston Globe of any noteworthy academic achievement by local students at the high school level. Christiane Henrich of Marblehead HS, in Marblehead, Massachusetts, wrote a 7,360-word Emerson-prize-winning history research paper on the quality (good for the day) of U.S. Civil War medicine. It was published in the only journal in the world for the academic papers of secondary students...No room in The Boston Globe for that to be mentioned. She is now at Stanford and doesn’t mind, but I mind about all the Boston-area students who are fed a constant diet of praise for athletic achievement by their peers and at the same time are starved of any and all news of the academic achievements of their peers.

In fact, over the years I have published a good number of exemplary history papers by high school students from the Boston area and they did not and do not get mentioned in The Boston Globe, nor do the academic achievements of our high school students in foreign languages (e.g. National Latin Exam, etc.), AP subject tests in Calculus, Chemistry, European history or in any other field, receive any notice from the Globe.

International competitions reveal that we are below average in Reading, Math and Science. Perhaps we should just explain that we don’t care about that stuff as much as we do about swimming, soccer, cross-country, football, golf, field hockey, and volleyball, because achievement by our high school students in those efforts are what we really like to pay attention to, (not that academic stuff), at least when it comes to The Boston Globe.

The Boston Globe (and its subscribers) are, in this way, sending a constant stream of clear messages (42 pages at a time in supplements, not to mention regular daily columns on HS sports) that in Boston (The Athens of America) what we care about is kids doing well in sports. If they do well in academics we don’t think that is worth mentioning. Sick, sad, and self-destructive, but there we are.


Don't touch pupils' fingers, British music teachers are told

Music teachers are being told not to touch the fingers of pupils learning to play instruments. The Musicians' Union has produced a video telling teachers: "It isn't necessary to touch children in order to demonstrate: there's always a better way."

But the video has provoked a storm of protest from teachers and campaigners who attacked the guidance as "madness" and said the video – which features a man teaching a child the violin – as a "grossly caricatured version of teacher-pupil contact".

The video, called "Inappropriate Demonstration", shows a lesson in which a pupil fails to play the right notes. The teacher first explains the technique by placing a hand on the pupil's shoulder and holding his fingers in the right position on the violin. He then explains it a second time by demonstrating on his own violin the correct position. The pupil then immediately plays the correct notes.

A voice-over on the video says: "When you're teaching instruments, there are times when you need to demonstrate particular techniques. "In the past, this has often been done by touching students, but this can make students feel uncomfortable and leave teachers open to accusations of inappropriate behaviour." The narrator adds: "You should never need to touch a student for demonstration. Use your creativity to find other equally effective ways to demonstrate."

The union said the video, produced with the NSPCC, MusicLeader (a charity-funded organisation to help music leaders) and the Associated Board of the Royal Schools of Music, was aimed at helping music teachers "gain a better understanding of their child protection responsibilities and avoid situations that could lead to accusations of misconduct".

But teachers criticised the video and the "no-touching" policy. One music teacher, writing on the Associated Board of the Royal Schools of Music's online forum under the name "Banjogirl", said: "It's all madness. I can't help touching children occasionally. "It's bringing children up to think that there is something dirty about touch and to be suspicious of other people."

Seer Green, another music teacher, said the union and the NSPCC had "missed the point". "What is most important in all this is common sense. Building a good working relationship between teacher, pupil and parent is essential. "A sense of trust needs to be built up and then when any issues around 'touch' arise, they can be handled sensibly and with the minimum of fuss."

Henry Fagg, from The Tutor Pages, an independent educational services company based in North West London, said the video depicted "a grossly caricatured version of teacher-pupil contact.

He said the "no-touching" policy was "hysterical" and interfered with day-to-day music teaching. "It also fails to distinguish between appropriate and inappropriate touch, and hence the real issue of child abuse is completely ignored."

Josie Middleton, of the Manifesto Club which campaigns against excessive regulation, said: "The video is absurd. Teachers need to be able to straighten backs, reposition fingers, or shake out stiff hands. "The assumption of this video is that all touch is potentially suspicious. This turns normal behaviour into something very seedy, and encourages decent people to be anxious all the time. "It also blurs the boundary between abusive touch, and caring or instructive touch – and makes it harder to distinguish genuine abuse."

Diane Widdison, spokesman for the union, said: "It's a difficult area but we are here to protect children and to protect our members' careers. "When allegations are made against music teachers they are suspended immediately while an investigation is carried out and their careers are damaged or ruined even if they are declared innocent."

In one recent case the parents of a child learning the guitar complained that the teacher had touched their child's finger to pluck a guitar string.

"A lot of children don't like to be touched by adults," she added. "You don't need to touch children to teach them how to play an instrument. We live in a culture where children know their rights and touching can be misinterpreted."


No comments: