Bernoulli's Fallacy

Statistical Illogic and the Crisis of Modern Science

De: Aubrey Clayton

Narrado por: Tim H. Dixon

There is a logical flaw in the statistical methods used across experimental science. This fault is not a minor academic quibble: It underlies a reproducibility crisis now threatening entire disciplines. In an increasingly statistics-reliant society, this same deeply rooted error shapes decisions in medicine, law, and public policy, with profound consequences. The foundation of the problem is a misunderstanding of probability and its role in making inferences from observations.

Aubrey Clayton traces the history of how statistics went astray, beginning with the groundbreaking work of the 17th-century mathematician Jacob Bernoulli and winding through gambling, astronomy, and genetics. Clayton recounts the feuds among rival schools of statistics, exploring the surprisingly human problems that gave rise to the discipline and the all-too-human shortcomings that derailed it. He highlights how influential 19th- and 20th-century figures developed a statistical methodology they claimed was purely objective in order to silence critics of their political agendas, including eugenics.

Clayton provides a clear account of the mathematics and logic of probability, conveying complex concepts accessibly for listeners interested in the statistical methods that frame our understanding of the world. He contends that we need to take a Bayesian approach - that is, to incorporate prior knowledge when reasoning with incomplete information - in order to resolve the crisis. Ranging across math, philosophy, and culture, Bernoulli’s Fallacy explains why something has gone wrong with how we use data - and how to fix it.

PLEASE NOTE: When you purchase this title, the accompanying PDF will be available in your Audible Library along with the audio.

©2021 Aubrey Clayton (P)2021 Audible, Inc.

Narration of equations is awkward, forced false dichotomy and strawman

Why couldn’t they revise it slightly to avoid reading out so many equations?

Also, the main point is interesting: a lot of science and implementation of probability is based on an invalid assumption. An example of this invalid assumption is the false equivalence of the two probabilities:
1. the probability of having a disease given a positive test result
2. the accuracy of the test

that is, given a 97% accurate test for a disease, it is (or was, since this problem has been known for some time) often assumed that if the patient tests positive, they are 97% likely to have the disease. Most of you who are listening to this book or reading this review will be aware of the author’s point about this, which is that the assumed equivalence fails to take into account the base likelihood of having the disease, which can result in huge error in confidence for rare diseases.

The author makes the point that this kind of problem and fallacy actually applies to all implementations of statistics, not just disease tests. This seems interesting and likely to be true, and he makes some good arguments!

However, a lot of the book is caught up on making some very niche, circular, and confusing points that are less convincing. Often, the author argues against a strawman that isn’t really representative of how people use probability today. The author assumes that because probability and some of the important assumptions were created by racists, that makes those assumptions and methods invalid. Some of that may be true, but the author doesn’t do a good job arguing why they should be discarded on a technical basis. Anyone can misinterpret or misapply statistical tests, even their inventors, but that doesn’t mean the tests are inherently useless.

A strong case for Bayes

Good intro to Bayesian statistics but the descriptions of equations and graphs were distracting. I bought the book for those.

Statistical method based upon Racist Justification

Immersed in the labyrinthine realms of statistical theory, I found myself captivated by the nuanced debate between the frequentist and Bayesian schools of thought. In the book I had the pleasure of reviewing, Clayton masterfully illuminates the stark incompatibilities that lie at the heart of these two methodologies. His adept critique of frequentist assertions, which he then artfully deconstructs, proved both enlightening and accessible, demanding no more than a foundational understanding of undergraduate statistics.

My intellectual voyage through this domain was profoundly enriched by Clayton's work, which bestowed upon me the essential historical context of the Bayesian versus frequentist discourse, underscoring Jaynes' work as a pivotal intellectual achievement.

Entitled "Bernoulli’s Fallacy," the book adeptly traces the trajectory of statistical thought, journeying from Bernoulli's pioneering efforts to the unsettling application of statistics in the pursuit of eugenic agendas. It also confronts the contemporary "crisis of replication" afflicting various research fields, a crisis stemming from an excessive dependence on statistical significance and p-values in hypothesis evaluation.

In its initial chapters, the book articulates its core concepts, which, though not revolutionary, remain critical and frequently misunderstood in modern discussions. These concepts pivot around the idea of probability as a subjective belief informed by available knowledge, the imperative of articulating assumptions in probability statements, and the transformation of prior probabilities into posterior probabilities via observation. The book underscores that data alone cannot yield inferences; rather, it reshapes our existing narratives based on their plausibility.

A pivotal insight from the book is the acknowledgment that improbable events do indeed transpire. This realization challenges the practice of deducing the veracity or fallacy of hypotheses solely based on the likelihood of observations. Instead, it advocates for adjusting our subjective belief in the plausibility of a hypothesis in relation to other competing hypotheses.

Moreover, the book elucidates a critical distinction: Bayesian and frequentist methods are not merely two different perspectives but rather, the Bayesian approach forms the bedrock of probability understanding, with the frequentist method emerging as a historical aberration, a specific instance within the expansive Bayesian paradigm.

It was particularly enlightening to learn how a small cadre of British mathematics professors, namely Galton, Fisher, and Pearson, engineered an entire statistical school of thought. This school, founded on flawed and convenient principles, served to justify and rationalize their eugenic and racist viewpoints, reinforcing the Victorian-era racial supremacy of the British upper class through a veneer of mathematical rationalization. This review offered a fascinating glimpse into a quasi-scientific method employed by researchers who, standing on shaky ground, resort to limited group sampling and mathematical subterfuge to lend false precision and authority to their biased models and probability findings.

Amazing book

Great read and must have for everyone in risk management community. Yet another wake up call to the flaws in many traditional risk analysis techniques.

Changes World Views

Occasionally one finds a book or audio presentation that challenges the roots, the rock, on which all you thought you and beliefs are based is dissolves and is taken away. For me, “Bernoulli’s Fallacy: Statistical Illogical and the Crisis of Modern Science” is that kind of book.

As a child, I always wanted to be a scientist when I grew up, even though I never worked as a scientist, science was my passion, the ability to use numerical analysis to aid in understanding the world, business, finance, production control, and scientific research and publications was the rock I based my view of reality on. From the earliest learning to graduate school in philosophy, it was what could be counted on and trusted. Logic, Mathematics, and Philosophy could be used to solve any problem. Then I read both text and digital versions and listened to the audio rendition once, twice, and now many more times.

Slowly, with the precision of a surgeons knife, Aubrey Clayton has cut the roots of my knowing and smashed the rock on which they were anchored.

Coming to see the logical fallacy upon which much of modern statics (the orthodox Frequentist methods) has deceived me in a since that many of my key beliefs and understanding are built on / based on errors, logical errors, that, under some conditions approximate what is correct or valid. However, when applied in general as the prescribed method of analysis, criterion for publication, and the preferred method of analysis, above all others, one finds that these methods lead to many issues and often bogus or even silly conclusions.

Even worse, the methods are all that has been taught at all levels of education in the statistics departments. The result of starting with logical errors, all that follows results in asking the wrong questions, designing the wrong experiments, analyzing incorrectly and getting result for the orthodox methods that lend themselves to easy manipulation, uncertainty, and the ability to cleverly wave the hands of complex methods and conclude the most absurd of all possible outcomes that may result in millions of deaths.

Hopefully more will read and study the text and ideas and arrive at conclusions that aid them in doing better science, living more wholesome life’s, and having a deeper appreciation for clear and accurate thinking.

No punches pulled!

There has been some effort to make frequentist and Bayesian approaches seem compatible in the last few years. But they really aren’t compatible. Clayton gives a full explanation of why this is the case. The reader should know introductory statistics at the undergraduate level well to appreciate the arguments, but more advanced understanding beyond that is not required. Clayton is very generous in recapping basic claims in frequentist statistics before turning them upside down and demonstrating their absurdity.

Excellent and persuasive

I read the book along with listening to the Audible narration. I'm a big Edwin Jaynes fan, so this was preaching to the choir. In particular, a Presbyterian sermon from Probability Theory, driving home its themes thoroughly.

The best introduction to Bayesian stats I’ve read

The walk through the history of stats was very enlightening, and the discussion around frequency and probability explain why I’ve always had a hard time with stats in the past.

Eye-Opening

Don’t worry about the equations. Math books are usually ill-suited to audio, but there is enough explanation provided that if you have a handle on basic statistics you can get the gist even if, like me, your eyes glaze over when the reader is referring you to the accompanying pdf.

Unhinged and thought provoking.

The author calls his own book "propaganda" and rails against frequentist methods. He calls Fisher and Nayman "racist" and raves about a perceived conspiracy against Bayesian methods. My degree taughr Bayesian methods, classical methods and machine learning theory in equal parts, so the conspiracy theories felt over the top. He raises some valid points and bungles some. Overall, the book is thoroughly entertaining. If you are a practitioner of statistics or data science, this book will make you think. The author's intense feeling and bias adds to this because you have to think about each example and decide for yourself if the point being made should be accepted.