Nice piece, Vance. It's pretty amazing how much bad science is out there masquerading as gospel, mostly made so by either profit or political motives. The politicization of science is my pet peeve. But profit-driven junk science is certainly a contender for 'Bad News for Mankind' as well.
Thanks for the insights.
Medical research is a disaster. All kinds of tricks are used to give the impression that medications work, when in fact they don't. How can this be the case? Well, there are many reasons, not the least of which is the pathetic methods used to evaluate the methodological quality of medical studies.
The Jadad score is a rating scale that is often used to evaluate the methodological quality of randomized clinical trials. Some very poor trials might, despite their glaring flaws, still obtain a perfect score of five, be accepted and heralded as rigorous, and go on to inform medical policy, even though the results are completely unreliable.
To understand just how deficient the Jadad score is, one might consider as an analogy a Jadad score of chess. Putting aside Internet chess and looking at chess played over the board, we see immediately that chess players require the arm strength that will enable them to lift the pieces. They also need the visual acuity to recognize their own pieces and distinguish them from those of their opponent. Color blind chess players might be at a distinct disadvantage. Chess players need the eye-hand coordination that will enable them to place pieces on the proper square without knocking down other pieces. Finally, they need quickness if they are to avoid losing on time.
Putting this all together, a training regimen for chess players might involve lifting weights for arm strength, eye exercises for visual acuity, coordination exercises, and plyometrics for quickness. The evaluation of a chess player might similarly be based on a five-point Jadad scale that measures 1) the ability of the player to lift a one pound weight 200 times (not many games go beyond 200 moves, and not many pieces weigh more than one pound each); 2) the ability of a chess player to distinguish white from black; 3) the speed with which a player can place a piece on a square in heavy traffic (there are pieces on all adjacent squares) without knocking any other pieces down.
Specifically, a player gets one point for being able to lift a one pound weight 200 times, another point for picking out all 16 white pieces on a randomly arranged chess board with the 32 pieces (16 of each color) arranged in random positions, and a third point for taking less than two seconds to place a piece on a square in heavy traffic without knocking down any other pieces. Add an extra point if the visual acuity test can be passed without glasses, because this ability confers robustness against such eventualities as heavy fog (outdoor tournaments) or glasses falling off and becoming broken. Add a fifth point if the piece can be placed in heavy traffic in less than one second without knocking down any others. The new rating system is to replace the existing one, and shall serve as the basis for selection onto chess teams all over the world. Players need three or more points to be considered good chess players, and to qualify.
In fact this brave new world exists today. Medical studies need score only three of five on the real Jadad score to be considered high quality. You want to keep the invaders out of the building, so you pick five of the many doors, and ask if at least three of them are locked? If so, then you are content?