The Positive Result Bias
There is a simple way to prove anything. All you need to do is filter out disproving evidence and describe anything that would seem to support your theory, and to do it with conviction and enthusiasm. It is a subject I studied at university: the courses I attended on the Philosophy of Science dealt with empirical research, its methods, consequences and justification. And we studied mechanisms of human reasoning, like the positive result bias, confirmation bias, and falsification, about which I recently wrote.
Let me start today’s discourse with a statement: I am, I believe, a very talented scammer. I have come up with countless tricks and deceptions to fool people—like the famous Guess prank or the Higgens Ploy. Mind you I have not ever made a penny out of any of the schemes—they were always perpetrated just for fun. Or to test the foibles of human thinking.
Take the Lotto Scam, which I will describe now. I came up with it in my student years and described it in a paper for a seminar on confirmation bias. It consisted of claiming to have a method to improve your chances of winning in the weekly lottery. In the German Lotto you pick six numbers from 49, before Friday evening. A day later, on Saturday night, there is a drawing of six numbers. Betting on your numbers cost you one German Mark, and getting all six right could earn you a million. But the chances of winning that is one in fifteen and a half million. In other words: you need to play the game for around 300,000 years in order to win a jackpot, spending 15½ million Marks during this lengthy endeavour.
There are smaller prizes: five correct numbers correctly guessed could earn you around DM 5000, typically, since the 15% of the prize pool that is allocated to this will be shared among all winning predictions. The odds of getting five right are 1 in 60,223. So once again it is not a really good deal, considering you needed to play for over a thousand years to get five right in a weekly drawing. There are smaller prizes for four correct numbers, and even a token amount for three correctly predicted numbers.
So how can we shorten the odds? At the time there were plenty of schemes that were supposed to increase your chances of winning money. Some of them are still around, like those given in the section “How to WIN Germany National Lotto” in this article. But the only genuine ones were pretty disheartening. For instance it is good to bet on numbers like 1,2,3,4,5,6 or 44,45,46,47,48,49. Or even just exactly the same numbers that were drawn in the previous week. Why that? Well, the chances of these numbers being drawn was exactly the same as any other set, but: the number of people choosing them will be — correct me if I am wrong* — much smaller than any random scattering of numbers. These combinations would be actively avoided by players, while having exactly the same chances of a pay-off as any other set.
*Addendum: My friend Prof Christian Hesse, a mathematician who is extremely knowledgable on such subjects, tells me I am indeed wrong: the fact that this strategy should increase your chances of winning is widely assumed, and so particularly often used. So it actually diminishes your winnings. He tells me one way of increasing them is to avoid the number 19, and since a few years 20. This is because so many people use numbers that reflect their birthdays, and these two numbers are most often part of them.
My scheme, however, which I described did not aim at improving your mathematical chances of winning, or increasing the pay-off if you do. It aimed at participating in the play of others. And this is how it worked: you advertised that you could improve the chances of anyone winning by giving them numbers to use. You claim that you could do this on the basis of extensive statistical analysis of past drawings, on some mystical numerology, on astrological calculations, or any other nonsensical method that sounded vaguely plausible to a mathematically challenged audience.
That is not yet the main body of the scam. It was the following: you tell your customers that you could in fact guarantee that your method would work. This was so certain that they would not be charged for the service, not a penny, if the predictions failed. Only if four, five or six of the numbers you provided actually came up in the drawing would there be a charge of 10% of the win. That sounded quite reasonable: the system was guaranteed to work, and you would lose absolutely nothing if it didn’t.
There was one more important aspect to the scam: the flyer offering the service had testimonials of earlier customers confirming that the system had succeeded for them: “Already the second time I used the method I got five numbers right and earned 6800 Marks. Thanks for the splendid service, which really works! — Hans Meyer, Munich.” There would be dozens and dozens on the flyer, all very convincing. They would be genuine — some with pictures of the winners — and they would encourage new customers to try their luck with the system.
Of course the scam was that you would be participating in the winnings of lucky Lotto players — or to put it another way: you would have them pay for many thousands of bets that you would place, and let hem keep 90% of the prizes. It is as insidious as blessing people entering a casino in return for 10% or their winnings — if there are any. Or participating in the earnings of stock market investors in return for giving them random unresearched advice.
The point I was making in the Philosophy of Science seminar is that the scheme is using the Positive Result Bias in order to succeed. Testimonials like the one given above would start pouring in, while there would be very few people saying: “I got numbers from you a dozen times now, and did not win anything.” If they were actually written such letters would be simply ignored.
I explained that a similar method was employed in many other areas, e.g. in pseudo medicine. There you can take some random substance — ground peacock feathers, very thin slices of rhino horn, bile extracted from living bears — and claim it cured some random ailment: acute migraine or back pain, cancer, impotence, arthritis, anything. The point is that people who try it and feel better, due to other random circumstances, will sing the praises of the “miracle cure,” while the vast majority on which it doesn’t work do not write testimonials. Positive Result Bias.
There is one more thing I need to mention. A colleague attending the seminar, Helmut, told a friend about my scheme. This person ran a chain of Lotto stations, and he invited me to a lunch meeting to discuss the subject. I basically held my talk again for him, and saw that he was taking mental notes. It seemed like he was going to actually implement the scheme. Some weeks later Helmut showed me some flyers this man had designed, just like the ones I had described. I met him again and told him that the scheme was dishonest — and that anyway it needed tens of thousands of customers in order to work. That was the last time I spoke to him, and do not know if he went ahead with the scam. I sincerely hope he didn’t.
There is a lot to say about one area of human gullibility I mentioned above: pseudo medicine. There is no cure for ailments or diseases that has been too absurd to be practised for centuries or millennia. And I have seen many cases of the consequences of this scheme. But we will leave that for another time and a different Bio Blog article.