Tuesday, April 11, 2017
Academia is f*cked-up. So why isn't anyone doing something about it?
Comments from feisty German physicist Sabine Hossenfelder:
A week or so ago, a list of perverse incentives in academia made rounds. It offers examples like "rewarding an increased number of citations" that - instead of encouraging work of high quality and impact - results in inflated citation lists, an academic tit-for-tat which has become standard practice. Likewise, rewarding a high number of publications doesn't produce more good science, but merely finer slices of the same science.
It's not like perverse incentives in academia is news. I wrote about this problem ten years ago, referring to it as the confusion of primary goals (good science) with secondary criteria (like, for example, the number of publications). I later learned that Steven Pinker made the same distinction for evolutionary goals, referring to it as `proximate' vs `ultimate' causes.
The difference can be illustrated in a simple diagram (see below). A primary goal is a local optimum in some fitness landscape - it's where you want to go. A secondary criterion is the first approximation for the direction towards the local optimum. But once you're on the way, higher-order corrections must be taken into account, otherwise the secondary criterion will miss the goal - often badly.
The number of publications, to come back to this example, is a good first-order approximation. Publications demonstrate that a scientist is alive and working, is able to think up and finish research projects, and - provided the paper are published in peer reviewed journals - that their research meets the quality standard of the field.
To second approximation, however, increasing the number of publications does not necessarily also lead to more good science. Two short papers don't fit as much research as do two long ones. Thus, to second approximation we could take into account the length of papers. Then again, the length of a paper is only meaningful if it's published in a journal that has a policy of cutting superfluous content. Hence, you have to further refine the measure. And so on.
This type of refinement isn't specific to science. You can see in many other areas of our lives that, as time passes, the means to reach desired goals must be more carefully defined to make sure they still lead where we want to go.
Take sports as example. As new technologies arise, the Olympic committee has added many additional criteria on what shoes or clothes athletes are admitted to wear, which drugs make for an unfair advantage, and they've had to rethink what distinguishes a man from a woman.
Or tax laws. The Bible left it at "When the crop comes in, give a fifth of it to Pharaoh." Today we have books full of ifs and thens and whatnots so incomprehensible I suspect it's no coincidence suicide rates peak during tax season.
It's debatable of course whether current tax laws indeed serve a desirable goal, but I don't want to stray into politics. Relevant here is only the trend: Collective human behavior is difficult to organize, and it's normal that secondary criteria to reach primary goals must be refined as time passes.
The need to quantify academic success is a recent development. It's a consequence of changes in our societies, of globalization, increased mobility and connectivity, and is driven by the increased total number of people in academic research.
Academia has reached a size where accountability is both important and increasingly difficult. Unless you work in a tiny subfield, you almost certainly don't know everyone in your community and can't read every single publication. At the same time, people are more mobile than ever, and applying for positions has never been easier.
This means academics need ways to judge colleagues and their work quickly and accurately. It's not optional - it's necessary. Our society changes, and academia has to change with it. It's either adapt or die.
But what has been academics' reaction to this challenge?
The most prevalent reaction I witness is nostalgia: The wish to return to the good old times. Back then, you know, when everyone on the committee had the time to actually read all the application documents and was familiar with all the applicants' work anyway. Back then when nobody asked us to explain the impact of our work and when we didn't have to come up with 5-year plans. Back then, when they recommended that pregnant women smoke.
Well, there's no going back in time, and I'm glad the past has passed. I therefore have little patience for such romantic talk: It's not going to happen, period. Good measures for scientific success are necessary - there's no way around it.
Another common reaction is the claim that quality isn't measurable - more romantic nonsense. Everything is measurable, at least in principle. In practice, many things are difficult to measure. That's exactly why measures have to be improved constantly.
Then, inevitably, someone will bring up Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure." But that is clearly wrong. Sorry, Goodhard. If you want to indeed optimize the measure, you get exactly what you asked for. The problem is that often the measure wasn't what you wanted to begin with.
With use of the terminology introduced above, Goodhard's Law can be reformulated as: "When people optimize a secondary criterion, they will eventually reach a point where further optimization diverts from the main goal." But our reaction to this should be to improve the measure, not throw the towel and complain "It's not possible."
This stubborn denial of reality, however, has an unfortunate consequence: Academia has gotten stuck with the simple-but-bad secondary criteria that are currently in use: number of publications, the infamous h-index, the journal impact factor, renown co-authors, positions held at prestigious places, and so on.
We all know they're bad measures. But we use them anyway because we simply don't have anything better. If your director/dean/head/board is asked to demonstrate how great your place is, they'll fall back on the familiar number of publications, and as a bonus point out who has recently published in Nature. I've seen it happen. I just had to fill in a form for the institute's board in which I was asked for my h-index and my paper count.
Last week, someone asked me if I'd changed my mind in the ten years since I wrote about this problem first. Needless to say, I still think bad measures are bad for science. But I think that I was very, very na‹ve to believe just drawing attention to the problem would make any difference. Did I really think that scientists would see the risk to their discipline and do something about it? Apparently that's exactly what I did believe.
Of course nothing like this happened. And it's not just because I'm a nobody who nobody's listening to. Similar concerns like mine have been raised with increasing frequency by more widely known people in more popular outlets, like Nature and Wired. But nothing's changed.
The biggest obstacle to progress is that academics don't want to admit the problem is of their own making. Instead, they blame others: policy makers, university administrators, funding agencies. But these merely use measures that academics themselves are using.
The result has been lots of talk and little action. But what we really need is a practical solution. And of course I have one on offer: An open-source software that allows every researcher to customize their own measure for what they think is "good science" based on the available data. That would include the number of publications and their citations. But there is much more information in the data which currently isn't used.
You might want to know whether someone's research connects areas that are only loosely connected. Or how many single-authored papers they have. You might want to know how well their keyword-cloud overlaps with that of your institute. You might want to develop a measure for how "deep" and "broad" someone's research is - two terms that are often used in recommendation letters but that are extremely vague.
Such individualized measures wouldn't only automatically update as people revise criteria, but they would also counteract the streamlining of global research and encourage local variety.
Why isn't this happening? Well, besides me there's no one to do it. And I have given up trying to get funding for interdisciplinary research. The inevitable response I get is that I'm not qualified. Of course it's correct - I'm not qualified to code and design a user-interface. But I'm totally qualified to hire some people and kick their asses. Trust me, I have experience kicking ass. Price tag to save academia: An estimated 2 million Euro for 5 years.
What else has changed in the last ten years? I've found out that it's possible to get paid for writing. My freelance work has been going well. The main obstacle I've faced is lack of time, not lack of opportunity. And so, when I look at academia now, I do it with one leg outside. What I see is that academia needs me more than I need academia.
The current incentives are extremely inefficient and waste a lot of money. But nothing is going to change until we admit that solving the problem is our own responsibility.
Maybe, when I write about this again, ten years from now, I'll not refer to academics as "us" but as "they."
Rahm Emanuel’s College Proposal Is Everything Wrong With Democratic Education Policy
Emanuel’s idea is the reductio ad absurdum of the “college solves poverty” idea…
On Wednesday, Chicago Mayor Rahm Emanuel announced a new educational proposal: starting with this year’s freshman class, every student in the Chicago public school system will be required to show an acceptance letter from a college, a trade school or apprenticeship, or a branch of the military in order to graduate. “We live in a period of time when you earn what you learn,” Mayor Emanuel said. (Democratic politicians’ attempts at folksiness are always pretty grim.) “We want to make 14th grade universal,” he also said. The proposed measure is almost certainly a publicity stunt which will have little effect in practice. But Emanuel has made it clear how he thinks educational problems should be solved.
The Emanuel plan is perhaps the stupidest idea a nationally prominent politician has publicly endorsed in the past decade. I hesitate to even explain why it’s stupid lest I insult my readers’ intelligence by belaboring the obvious. But it’s worth spelling out what’s wrong with this, because the fact that a major Obama-aligned Democratic politician is attempting to do this says a great deal about the worldview of the establishment Democratic Party. So here goes.
In Mayor Emanuel’s opinion, working-class kids are too stupid to recognize their own interests. They’re simply unaware that people who go to college earn more than people who don’t, which is why (silly them) they don’t go to college. If you just force them to go to college by flunking them out of high school unless they promise to go to college, they’ll all become highly compensated white-collar workers and America will be a wealthier place.
Allow me to propose an alternative model: working-class kids are not stupid. They’re aware that college grads earn more money on average than they ever will. They’re also aware that not all college degrees are created equal, and that a degree from a community college or some fly-by-night for-profit—the kind of school most working-class kids from Chicago might actually get into—is dramatically less valuable than one from Sarah Lawrence, where Rahm got his BA. They’re aware that college degrees aren’t what they once were, partly because so many degrees are from mediocre institutions; perhaps they’ve seen family members work hard to get that University of Phoenix diploma only to wind up little better off than they’d have been otherwise.
They’re also aware that college costs money, not only money for tuition but all the money you won’t be able to earn while you’re in school, and that people whose parents can’t support them, people who may in fact need to help support their families themselves, can’t afford to just not work for two to four years. Finally, they’re aware that college is hard, particularly for working-class kids with less academic preparation than their middle-class peers who also have less social support and need to work while their peers are studying, and that working-class kids are at a high risk of dropping out. They know that going into debt to attend a college and then dropping out with no degree can be financially catastrophic.
In other words, they know, unlike their mayor, that what happens to the average kid who goes to college—a middle-class kid from the suburbs with white-collar parents who can afford to subsidize his textbooks and partying for four years—is a very poor indicator of what will happen to them, personally, if they decide to go to college. Knowing all this, they make their choice; 62% of Chicago’s high school students decide to have a crack at college after they graduate, 38% don’t.
Now, it may well be that there are a few kids in that 38% who are making the wrong choice, just as there are a few in that 62% (very possibly more than a few) who are making the wrong choice and will just end up dropping out with debt or graduating with a worthless degree and more debt. It might be that a better school guidance program would push some kids into college for whom it’s the right decision. But Rahm isn’t proposing to nudge a few more kids into college; he’s proposing to hold the high school degree of every student in the system hostage until they all go to college, or sign up for the army, or enter an apprenticeship.
What’s likely to happen if his proposal passes? Well, trade schools and apprenticeship programs are bright enough to know that the world only needs so many plumbers, so not a lot of students are going to manage to go that route. Some will join the army, at which stage Mr. Emanuel can congratulate himself for having forced some working-class kids to die for their country on pain of facing the stigma of the high school dropout for the rest of their lives. Some will simply decide to leave high school without graduating. But many will be forced into a choice they know is the wrong one, and have a crack at whatever community college or awful open-admissions for-profit college they can get an acceptance letter from. Expect to see the already overburdened and underfunded community college system pushed to the wall. Expect to see a small boom in the for-profit college industry and the exploitative student loan industry that feeds it. Expect to see many, many students drop out of school with nothing to show for it but un-bankruptable education debt that will haunt them for years.
And finally, perhaps most importantly, expect to see those students who do manage to graduate from whatever bottom-tier school is willing to accept them quickly discover that the degree Rahm Emanuel forced them to earn at great personal expense isn’t worth the paper it’s printed on. First, because college-educated workers, like any other commodity, are subject to the law of supply and demand, and Rahm’s plot to dump hundreds of thousands more of them onto the Chicago labor market will cause supply to greatly outpace demand and prices to crater. Second, because employers will recognize that people who got a college degree from a bottom-tier school that slashed admissions standards to take advantage of the Rahm-and-debt-fueled bonanza don’t have the same skill set or qualifications as the college students they now pay higher wages. In other words, producing a genuinely more educated workforce is a lot harder than Rahm’s plan to print a whole bunch more college diplomas, but even if you could produce a genuinely more educated workforce it wouldn’t raise wages; you’d just have more people competing for the same number of white-collar jobs., and wages would go down.
(Of course, middle-class kids who went to Sarah Lawrence would still do just fine.)
Emanuel’s plan, in other words, will be a disaster if implemented. But if the plan were just his own idiosyncratic idiocy, it would be beneath refutation. Unfortunately, it’s not. The mayor of Chicago is an utterly characteristic representative of the dominant wing of the Democratic Party, and his “you earn what you learn” claptrap reflects what has been a core element of its messaging and policy for decades: the notion that we can solve poverty through education. For most of my lifetime, the Democratic Party’s answer to the apparently permanent stagnation of working-class wages has been to advise the electorate that it’s a knowledge economy and only a better-educated workforce can hope to earn more.
This is terrible policy based on obviously shoddy reasoning: while it’s true that highly educated computer programmers make a lot of money, the notion that if everyone were a highly educated computer programmer everyone would make more money is absurd, first because not everyone can become a highly educated computer programmer and second because if everyone could then computer programmers would no longer make a lot of money.
It should be emphasized, though, that on top of being terrible policy this is also terrible messaging. When voters hear that your analysis of the economy is that it simply has no place anymore for uneducated workers, and that your plan to increase working-class wages is “educate people better for the knowledge economy,” they get three messages: first, that if you’re a low-income thirty-year-old high school graduate with a family who can’t go to school, the Democrats’ plan for you is that you’ll die poor, because hey, it’s a knowledge economy, what can they do? It’s a knowledge economy. Second, that Democrats think your poverty is pretty much your fault for not doing better in school. And third, that Democrats are so completely out of touch that they genuinely believe that becoming a high-tech worker is a serious option for your working-class kids. In other words, what you hear is that Democrats don’t know you, don’t care about you, look down on you, and have no plan to help you. Is it any wonder that you don’t bother to vote, or that if you do you vote for someone who promises to bring the jobs back?
Claremont University Students Shut Down Conservative Speaker
Students at the Claremont Colleges, a consortium of undergraduate and graduate liberal arts colleges in Claremont, California, blocked entrances to the building that Mac Donald was scheduled to speak in.
Mac Donald, a senior fellow at the Manhattan Institute, a conservative think tank, was giving a speech titled “The War on Police,” a reference to her 2016 book about how political rhetoric against police makes America less safe.
Mac Donald was forced to livestream her speech after protesters blocked students and professors from entering the building.
According to the school’s newspaper, The Forum, 250 students watched Mac Donald speak. Students had to submit questions via email.
“Among other chants, protesters yelled ‘Black Lives Matter’ and ‘How do you spell fascism? CMC!’ while banging on windows of the [Athenaeum],” according to The Forum.
Mac Donald was a guest of the Rose Institute of State and Local Government, a research institute on the campus of Claremont McKenna College, a member of the Claremont Consortium.
Peter Uvin, vice president of academic affairs for Claremont McKenna College, said in an email to students after the incident: “I fully understand that people have strong opinions and different—often painful—experiences with the issues Heather Mac Donald discusses. I also understand that words can hurt.”
Uvin went on to condemn the students’ behavior, saying, “What we face here is not an attempt to demonstrate, or to ask tough questions of our speaker, all of which are both protected and cherished on this campus, but rather to make it impossible for her to speak, for you to listen, and for all of us to debate. This we could not accept.”
Many conservative speakers have been protested on college campuses in recent months.
In March, students protested Charles Murray, a fellow at the American Enterprise Institute, while he was giving a speech at Middlebury College. Milo Yiannopoulos, a former editor at Breitbart, was also violently protested when he attempted to speak at Berkeley this past February.
Murray was able to give his speech, talking over screaming demonstrators, while Yiannopoulos was forced to cancel his speech and leave campus, thanks to a police escort.
Mary Clare Reim, an education policy analyst at The Heritage Foundation, cited a pattern, telling The Daily Signal in an email:
We can now add Heather Mac Donald to the long list of experts whose voices have been shut down on college campuses. From Middlebury to Berkeley, college students and administrators continue to treat conservative viewpoints with hostility and sometimes violence.
Reim went on to encourage institutions to re-evaluate their purpose, writing, “Universities have an obligation to protect First Amendment rights on campus. Unfortunately, recent events suggest that many universities no longer take that obligation seriously.”
Posted by jonjayray at 12:48 AM