CrossCheck Blog

CrossCheck Blog

Check Processing & Payments Information

Federal Reserve Questions Economists' Conclusions

Posted by Andrew Donahey | Tue, Oct 20, 2015 @ 07:30 AM

Any business owner who reads the news knows how much influence economists have over corporate America, Wall Street and the government.

economists-try-to-be-scientists.jpgBut how did they become so influential? Well, most of the talking heads we see on screen have published at least one or two peer-reviewed papers, which substantiates them in the eyes of other economists and gets them a seat in front of the camera.

But according to a recent Federal Reserve study, a lot of those papers reach conclusions that can't be independently verified.

Economists like to think their profession is a science, and the essence of science is to conduct an experiment and to publish the results in such a way that other scientists can run the same experiment to confirm them. Whenever scientific discoveries are overturned, its because other scientists didn't get the same result from the same experiment.

Following in those footsteps, major economics publications often require authors to include all of their data and even the computer code they used to analyze it right along with their paper for publication. The idea is that other economists can then use the data and the tools to confirm the author's conclusions.

The Federal Reserve found out that this is rarely possible.

They analyzed sixty-seven different papers and tried to repeat the analysis that the author claimed to have done to reach his conclusions. They succeeded for only a third of them.

Then, for the ones they couldn't crack, they contacted the author for help. With a little coaching, they were able to verify the conclusion of 49% of the papers. "Because we are able to replicate less than half of the papers in our sample, even with help from the authors," they concluded, "we assert that economics research is usually not replicable."

Not replicable means not reliable, and the authors suggest that the bar ought to be a lot higher for such an influential field. They recommended, for starters, that submitting data and code should be mandatory.

Any businessperson who makes decisions or plans budgets based upon economic forecasts might want to keep this paper in mind: Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say "Usually Not."

Check Guarantee Insider's Guide


Written by Andrew Donahey

Marketing Director Andrew Donahey leads a dedicated group of digital marketing specialists that generates relevant content for the financial services sector. He also collects vintage “anything” and has a Golden Retriever kennel.