Failure to deliver factual information is more a problem in morals than in science or politics. Institutions that fail morally must be rebuilt with people who believe in liberty informed by morals grounded in reverence for life. Corrupt people in law enforcement is no proof against corrupt people in science.
My worry is that because of predictable mission creep, instead of merely being banned on Facebook or losing credentials, scientists will face imprisonment for conducting unpopular studies or reaching unpopular conclusions. People might then ask, "Who are the real criminals?"
My favorite section of this post is the middle section in which you discussed your thoughts about “fraud fraud.” And my reaction was: “Gasp! and OMG!”
I ended up skimming through your AI Claude quotes (I bet most of your readers did too). It presented a few interesting points, but what would have happened if you the writer had done a typical Glenn Reynolds pithy and critical summary instead. (I guess we are all just experimenting with our chatbots these days!)
My two cents! Good job!
And, is there any way to get rid of those revolting Crohn’s disease ads on Instapundit?
“…typical Glenn Reynolds pithy and critical summary”
I originally became a regular reader of Instapundit over two decades ago because of the “typical Glenn Reynolds pithy and critical summary.” And that is the reason I continue to be an avid and continuous reader of Instapundit and a paid subscriber of this Substack. I also appreciate the biting humor, often intertwined within the pithy and critical summary, and the pertinent comments!
"That would slow down the connection between research and public policy, but would that really be such a bad thing?"
Not only would it not be a bad thing, it would be a good thing. There is no reason to think that public policy--and especially federal policy--should be treated differently than writing new laws. Going slow is mostly better than going fast.
Then again, it would take new public policy to accomplish that, and that's where I contradict myself.
I am not a lawyer, but scientific (research) fraud is already a criminal offense. Specifically, if you engage in research fraud during an FDA regulated clinical trial and you are found out, the penalty is a felony conviction (making false statements - lying - to the government) and jail time. See, for example, the case of Dr. Robert Fiddes in the late 1990's. Unfortunately, it is pretty easy to cover up research fraud.
Excellent presentation. My favorite of the suggestions is making the raw data, including software modeling, public. To take one example, all of anthropogenic warming climate models would be immediately cratered by this. All the worst offenders, like Professor Hockey Stick, have refused for decades to release raw data and the software. If the methodology is secret, it can’t be reproduced or studied, and is therefore not science.
One way to dramatically reduce scientific fraud is to eliminate government funding of science. Ideally, a proper government exists only to protect individual rights. It does not "do" things, rather, it acts only as a retaliatory force. We are nowhere near that and I'm aware that the likelihood of getting the government out of research funding (or legislating the economy) is virtually nil. Nonetheless, reducing it as much as possible would shift focus and emphasis to the most important or promising research. Fewer government-funded research projects would also make them easier to scrutinize.
would ending government funding be a solution? This assumes the private sector could take it over and it also assumes the private sector would insist on better results because they would be investing dollars that would want some sort of return on investment.
I posted my comment at about the same time you posted yours and only saw it after-the-fact. Private sector funding would not eliminate fraud, but it greatly reduces the number of people who pay for fraudulent results.
Peer reviewers should demand that each submitted paper include a section explaining how the paper's results could in fact be wrong. Any paper not including this section is rejected automatically. Among other benefits, it may cause the original researcher to be more circumspect about their work.
"And – and this was suggested by a commenter to an earlier blog post – not relying on scientific research for public policy purposes until it has been successfully replicated by someone else is not a bad idea." Sounds like something I'd say, and maybe I did! But the point is broader than that. The broader point is based on the observation that the vast majority of published science has no impact at all (not on public policy, not on textbook writers, not on reporters in the media, and not on other scientists). One approach to the replication crisis--which is popular--is to make sure that virtually all published findings (no matter how obscure and irrelevant) will replicate, even at great expense to the taxpayer. Another approach is to instead focus limited resources on published findings that gain traction, not just in terms of public policy but also in terms of media attention, attention from other scientists, attention from textbook writers, etc. NSF or NIH could fund proposals that make the case that the to-be-replicated results are gaining traction, so precious taxpayer dollars should be devoted to independently replicating those findings in a large-N study. That's a good reason to spend taxpayer dollars. Spending that money to instead ensure that everything replicates, not so much. Moreover, even reforms that seem free are more costly than they appear to be (e.g., in terms of a scientist's limited time). Just fund proposed replications of published findings that are gaining traction. It would solve so much of the problem. Unfortunately, NSF and NIH are both reluctant to do that. They'd rather fund "transformative" (often non-replicable) experiments. Fine, but put aside 5-10% of the federal science budget to replicate published results that matter. Why cheat if NSF is going to quickly fund an independent replication of the findings that are (momentarily) making you famous?
I think the public, including myself, are too uninformed regarding statics to effectively evaluate scientific papers. Perhaps integrating some education in statistics during high school and college math & science courses could help. I recognize that my training was inadequate regarding stats. But then again, these institutions are performing poorly regarding simple reading & arithmetic.
Failure to deliver factual information is more a problem in morals than in science or politics. Institutions that fail morally must be rebuilt with people who believe in liberty informed by morals grounded in reverence for life. Corrupt people in law enforcement is no proof against corrupt people in science.
My worry is that because of predictable mission creep, instead of merely being banned on Facebook or losing credentials, scientists will face imprisonment for conducting unpopular studies or reaching unpopular conclusions. People might then ask, "Who are the real criminals?"
My favorite section of this post is the middle section in which you discussed your thoughts about “fraud fraud.” And my reaction was: “Gasp! and OMG!”
I ended up skimming through your AI Claude quotes (I bet most of your readers did too). It presented a few interesting points, but what would have happened if you the writer had done a typical Glenn Reynolds pithy and critical summary instead. (I guess we are all just experimenting with our chatbots these days!)
My two cents! Good job!
And, is there any way to get rid of those revolting Crohn’s disease ads on Instapundit?
Roger von Oech
Paid Subscriber (from day 1)
“…typical Glenn Reynolds pithy and critical summary”
I originally became a regular reader of Instapundit over two decades ago because of the “typical Glenn Reynolds pithy and critical summary.” And that is the reason I continue to be an avid and continuous reader of Instapundit and a paid subscriber of this Substack. I also appreciate the biting humor, often intertwined within the pithy and critical summary, and the pertinent comments!
And then there is climate science destroying entire economies
"That would slow down the connection between research and public policy, but would that really be such a bad thing?"
Not only would it not be a bad thing, it would be a good thing. There is no reason to think that public policy--and especially federal policy--should be treated differently than writing new laws. Going slow is mostly better than going fast.
Then again, it would take new public policy to accomplish that, and that's where I contradict myself.
I am not a lawyer, but scientific (research) fraud is already a criminal offense. Specifically, if you engage in research fraud during an FDA regulated clinical trial and you are found out, the penalty is a felony conviction (making false statements - lying - to the government) and jail time. See, for example, the case of Dr. Robert Fiddes in the late 1990's. Unfortunately, it is pretty easy to cover up research fraud.
How many scientists tailor their research and results so they can keep getting grants?
I don't know but one is too many, I think.
Excellent presentation. My favorite of the suggestions is making the raw data, including software modeling, public. To take one example, all of anthropogenic warming climate models would be immediately cratered by this. All the worst offenders, like Professor Hockey Stick, have refused for decades to release raw data and the software. If the methodology is secret, it can’t be reproduced or studied, and is therefore not science.
One way to dramatically reduce scientific fraud is to eliminate government funding of science. Ideally, a proper government exists only to protect individual rights. It does not "do" things, rather, it acts only as a retaliatory force. We are nowhere near that and I'm aware that the likelihood of getting the government out of research funding (or legislating the economy) is virtually nil. Nonetheless, reducing it as much as possible would shift focus and emphasis to the most important or promising research. Fewer government-funded research projects would also make them easier to scrutinize.
would ending government funding be a solution? This assumes the private sector could take it over and it also assumes the private sector would insist on better results because they would be investing dollars that would want some sort of return on investment.
I posted my comment at about the same time you posted yours and only saw it after-the-fact. Private sector funding would not eliminate fraud, but it greatly reduces the number of people who pay for fraudulent results.
require a result to reproduced by someone not party to the initial research before publication.
Peer reviewers should demand that each submitted paper include a section explaining how the paper's results could in fact be wrong. Any paper not including this section is rejected automatically. Among other benefits, it may cause the original researcher to be more circumspect about their work.
Despite the problems with fraud, I don't see how criminalizing fraud improves things.
Then why do we criminalize non-scientific fraud?
Criminalizing scientific fraud will lead only to criminalizing scientific dissent. Or should I call scientific dissent by its true name: heresy.
"And – and this was suggested by a commenter to an earlier blog post – not relying on scientific research for public policy purposes until it has been successfully replicated by someone else is not a bad idea." Sounds like something I'd say, and maybe I did! But the point is broader than that. The broader point is based on the observation that the vast majority of published science has no impact at all (not on public policy, not on textbook writers, not on reporters in the media, and not on other scientists). One approach to the replication crisis--which is popular--is to make sure that virtually all published findings (no matter how obscure and irrelevant) will replicate, even at great expense to the taxpayer. Another approach is to instead focus limited resources on published findings that gain traction, not just in terms of public policy but also in terms of media attention, attention from other scientists, attention from textbook writers, etc. NSF or NIH could fund proposals that make the case that the to-be-replicated results are gaining traction, so precious taxpayer dollars should be devoted to independently replicating those findings in a large-N study. That's a good reason to spend taxpayer dollars. Spending that money to instead ensure that everything replicates, not so much. Moreover, even reforms that seem free are more costly than they appear to be (e.g., in terms of a scientist's limited time). Just fund proposed replications of published findings that are gaining traction. It would solve so much of the problem. Unfortunately, NSF and NIH are both reluctant to do that. They'd rather fund "transformative" (often non-replicable) experiments. Fine, but put aside 5-10% of the federal science budget to replicate published results that matter. Why cheat if NSF is going to quickly fund an independent replication of the findings that are (momentarily) making you famous?
Tom Siegfried, formerly with Science News, has done some great work in this area
Odds Are, It’s Wrong
https://www.sciencenews.org/article/odds-are-its-wrong
To make science better, watch out for statistical flaws
https://www.sciencenews.org/blog/context/make-science-better-watch-out-statistical-flaws
I think the public, including myself, are too uninformed regarding statics to effectively evaluate scientific papers. Perhaps integrating some education in statistics during high school and college math & science courses could help. I recognize that my training was inadequate regarding stats. But then again, these institutions are performing poorly regarding simple reading & arithmetic.