I think people in general need to have a firmer grasp of basic statistics as it crops very often in decisions of extreme importance. Those in ignorance are too easily fooled and manipulated by persons with an agenda to push and who know how to misrepresent statistical techniques. A newspaper article meant to be shocking claimed one third of all deaths occuring on the weekend! Well, duh! Friday evening to Sunday night is one third of a week. Likewise, in Scott Adams’ Dilbert, the boss gets angry and suspicious when told that 40% of sick days are taken on a Monday or a Friday.

With even more damaging consequences, in 1968 a couple was found guilty of a robbery because the jury was given bad instruction on the calculation of probability. The jury thought the probability that an innocent couple sharing all 6 attributes as those who committed the crime was 1 in 12,000,000 and convicted on that basis. When conditional probabilities were taken into account, the chance of coincidence increased to about 1 in 4 — far from beyond reasonable doubt.

More recently in the UK, a solicitor was convicted of killing her two children on the basis of an expert witness who testified that the chances of two Sudden Infant Deaths in the same affluent, non-smoking family were 1 in 73 million. To arrive at this figure he had to assume that two SID deaths among siblings are independent events but provided no evidence of this. The calculation is rubbish if SID has a common cause that is specific to the family, e.g. is genetic or traced to a pollutant in the home.

In the Intelligent Design vs. Evolution battle over education, the pro-creationists held that the chance that all the evolutionary mutations necessary to bring human beings into existence occured completely at random is so vanishingly small as to be impossible. This is the fallacy of equating the improbable with the impossible. I could shuffle a deck of cards and deal out all 52 face up, noting each one in order. When I work out the probability that such an order will come up, I will get a very tiny number (smaller than 1 in 8 followed by 69 zeros). And yet the event happened, because I just dealt out the cards in that very same sequence!

Global warming denialists play even more sophisticated statistical tricks, examples of which can be seen at ClimateAudit — a website devoted to debunking (mostly unsuccessfully) of one 8 year old scientific paper. They have gathered quite a following, mostly people not familiar with any of the statistical techniques involved. The website founders have published entire books disputing whether average temperature has a physical meaning while committing some very basic errors that are easily spotted by those with a little Excel experience.

Other myths, illusions and fantasies of a statistical nature include Bible Codes and homeopathy — all disproven with proper application of statistics.

This past week the Lancet published a study that employed the statistical technique of clustered sampling to estimate that the war in Iraq had resulted in 650,000 additional deaths. Immediately, this was disputed by many warmongers, many of whom had little understanding of the statistical methods involved, and those with a little knowledge only confirming the adage that a little learning is a dangerous thing. Tim Lambert’s blog provides a good summary of the attacks on this paper and includes counter arguments defending the research. There is little need to reiterate them here.

“There are three kinds of lies: lies, damned lies, and statistics.”, by Disraeli (according to Mark Twain).

4 November, 2006 at 10:28 am

“And yet the event happened, because I just dealt out the cards in that very same sequence!”

For readers not convinced of the cards thing, consider that whatever you dealt, even if you’d asked for it beforehand, the prior probability of that sequence occurring is the same as the simplest ordering quoted.

Evolution has its likelihood significantly augmented by the feedback involved. Once you have replication with mutation in a challenging environment, if the replicating generations survive, then the population as a whole will represent an organism more suited to the environment. When they die out, you don’t see them. So, evolution can fail. Cod is being overfished, and I think it’s unlikely they’ll evolve to stop that happening.