Never mind magic, miracles, witchcraft, the previous topics of this week. Many people don’t believe in science either, what with all the contradictory findings and and newly debunked studies that ruled the field for decades.

Coffee is bad for you. Coffee is good for you. A glass of wine with dinner will see you happily grow old. A glass of wine with dinner will be your demise! Exercise 20 hours a week and you will live forever. Exercise 20 minutes a week and you reap the same benefits. How often do we read that some previously touted finding is now completely reversed…. why should we ever trust scientific findings?

Note that my examples, the typical examples found in popular science writing you grab in the newspaper or listen to on NPR, do not concern physics, or chemistry, where it would be technically far harder to report something and far less interesting from a human interest point of view. It would also be far more consistent and thus of little interest to announce. The wild swings we read about come from epidemiological studies, research with large groups of people about something affecting their lives based in their life styles.

This is extremely hard science to do since it does not lend itself to one of the basic demands of the scientific method: random assignment of the participants. I cannot divide a random group of 1000 or more people into two, commanding and controlling one half to forgo alcohol for 10 years, while the other is allowed to indulge, and then test for outcomes. Instead, I have to rely on a group that happens to be non-drinkers to compare to those who are. This introduces the possibility that there is something else going on with one or both of the groups that feeds into my results.

Maybe non-drinkers are poorer and thus not buying wine, so other aspects of poverty affect the (negative) outcome. Maybe abstinent people are more health-conscious, reflected in additional healthy behavior, that is really at the core of the (positive) outcome. Maybe drinkers are in general a more social bunch, and the social aspects of their lives influence (positive) outcome. Maybe drinkers have underlying depression for which they self-medicate with alcohol and that affects the (negative) outcome. You get the idea. The conclusions – alcohol is good/bad for you – might be influenced by factors that have nothing to do with your beverage of choice after all and vary from study to study since these factors crop up in different populations. I might hope to control for all of these variables, but it is more or less impossible with self-selected populations.

*

Then there is the furious debate over the non-replicability of standard scientific studies and the discovery of scientific data massaging, or in worse cases, scientific fraud. I was reminded of that when I learned that a study I had taught for decades to great theatrical effect is now partially debunked. On being Sane in Insane Places by David Rosenhan came out in 1973 in the highly respected journal Science. Eight healthy participants went to local psychiatric hospitals, claiming they heard voices saying: “Hollow, empty, thud.” (Imagine my dramatic re-enactment in front of the college classroom.) The study reported that all of them were diagnosed with mental illness, in many cases schizophrenia, based on this symptom alone, and kept in hospital, up to 52 days, leaving eventually against the advisement of the doctors. A total of 2,100 pills — serious psychiatric drugs — were claimed to be prescribed to these otherwise pseudo patients.

A new book by Susan Callahan, The Great Pretender, unravels all that went wrong with the set-up and the representation of the results of this study, while at the same time examining the pitfalls of a broken mental health-care system, and acknowledging that mistaken labeling in psychiatric care is all too real. However, inventing data about it is not going to help the cause. The same seems to be the case for Phil Zimbardo’s famous Stanford Prison Study, with criticism delivered here.

And here is one last example, a study of gender bias, “Orchestrating Impartiality: The Impact of ‘Blind’ Auditions on Female Musicians,” by Claudia Goldin and Cecilia Rouse, that has been cited over 1500 times and even referenced by Justice Bader-Ginsburg in one or another dissent.

Two subsequent critics have taken apart the statistics used to arrive at these conclusion. Here is a summary report (short video clip) by Christina Hoff Sommers, who wrote on this for the Wall Street Journal implying that our mistaken efforts towards political correctness propelled this supposedly flawed paper into mainstream discourse.

My immediate reaction: why on earth would I trust an author from the conservative think tank American Enterprise Institute, known for her critique of contemporary feminism in her diatribes on The Factual Feminist? More on that tomorrow when Distrust in Science continues!

Photographs today echo public sentiment of science as feather weight.

Music today with a female conductor who broke the all too real glass ceiling: Marin Alsop.