I won’t repeat the numerous points that have been made in the debate of the past couple of days over the Reinhart and Rogoff paper “Growth in a time of debt”. A useful overview is provided by Bruegel here. I will emphasise a few points based, not on the original paper nor on what recent critics have written, but largely with reference to a journalistic article written by the authors themselves..
The statistical errors and shortcomings (excel mistakes, exclusion of countries, weighting issues etc.), that have dominated the debate are not the key issue. In fact there is a certain irony that computational errors have finally drawn much-needed attention to much more fundamental problems with the whole approach. The crucial point is that even on the basis of the data reported and discussed prior to the recent “shitstorm”, everyone should have been highly suspicious about the results and the claim made – specifically the idea that there is a threshold around 90% for the debt-GDP ratio beyond which countries subsequently experience substantially slower growth. There are two main reasons for this. The first is the known fact that there are a very small number of cases of countries with debt-to-GDP ratios above that threshold. The second is that there is very obviously a question as to whether, even if the statistical correlation were strong, it is correct to read the causation from high debt ratios to slow growth rather than the other way around.
Now, part of the debate in the blogosphere has turned on what the authors (universally abridged to “R-R”) themselves claimed and what other, perhaps interested, parties made of those claims. In their response to their critics R-R have insisted that they always spoke only of correlation rather than causation.
Well, a couple of minutes on the internet turns up this piece by R-R for Bloomberg.
Note first that it appeared in July 2011, well after the initial AER paper and after they had filled the data gaps on countries like New Zealand, which was part of their defence against their critics. Then there is the small matter of the title: Too Much Debt Means the Economy Can’t Grow. I find it hard to imagine a balder statement of causation running from debt to growth than that. But perhaps that was due to an over-eager Bloomberg editor? Well, consider three quotes from the text itself:
Our empirical research on the history of financial crises and the relationship between growth and public liabilities supports the view that current debt trajectories are a risk to long-term growth and stability, with many advanced economies already reaching or exceeding the important marker of 90 percent of GDP.
The biggest risk (to advanced countries – AW) is that debt will accumulate until the overhang weighs on growth.
They also suggest a causal mechanism (albeit in the form of a rhetorical question):
Or, more likely, is it because at some point, even advanced economies hit a ceiling where the pressure of rising borrowing costs forces policy makers to increase tax rates and cut government spending, sometimes precipitously…
If it was indeed the editor that wrote the title, he or she had a good sense of the point that R-R wanted to get across.
But the piece is also interesting because of what it reveals to even the statistically unsophisticated about the robustness of the findings (point one above). Early on the authors underline the large number of data points in their survey:
Our results are based on a data set of public debt covering 44 countries for up to 200 years. The annual data set incorporates more than 3,700 observations spanning a wide range of political and historical circumstances, legal structures and monetary regimes.
Pretty robust, huh? Yet later, in attempt to derive a causal explanation from the correlation, they pose, under the sub-heading “extremely rare”, the rhetorical question:
Those who remain unconvinced that rising debt levels pose a risk to growth should ask themselves why, historically, levels of debt of more than 90 percent of GDP are relatively rare and those exceeding 120 percent are extremely rare…
So in attempting to add causal force to their argument (insinuating something like: governments are smart and usually they don’t allow themselves to get ensnared in a debt trap, and if they do then they suffer…) , they actually reveal its statistical weakness, namely that it is based on “extremely rare” cases. (And that is precisely why the excel boob and other problems had such a significant effect on the result.) But if you have extremely rare cases underpinning your central claim, you soft-pedal the findings: at least that is what you do if you are a serious researcher. The onus is on you, particularly if you are using data that is not generally available, to look at the extremely rare cases in turn and see if they back up your story. And as observers pointed out early on, they didn’t, post-1945 demobilisation in the USA being a case in point. The authors knew this, and still they continued over-egging their results.
And what about the second point, reverse causation? As many others have pointed out (and R-R accept in principle) it is at least as likely that causation runs from slow growth to high deficits. In fact it is quite easy to test for this. A simple econometric study has been doing the rounds, knocked up it seems within 24 hours or so. The intuition is simple. Check whether high debt-to-GDP ratios are a better predictor of past or of future growth rates. And what do you know: high debt-to-GDP ratios are a poor predictor of what growth will be like in the next three years, but a very good predictor (in a statistical sense) of what growth performance had been like in the three previous years. This bit of econometrics might itself not be the argument to end all arguments. But at the very least a serious researcher would be extremely cautious in making any causal claims at all.
Reinhart and Rogoff are renowned economists with, I imagine, access to several research assistants and doctoral students each. The matters under debate are of utmost importance for economic policy and thus the welfare of millions of people, a fact of which the authors were well aware. Given these evident weaknesses, the fact that the authors were so gung-ho in over-egging work that is clearly highly problematic even without excel errors is simply unjustifiable. Perhaps they were flattered by the attention from the media and “serious” politicians. Whatever other good work they may have done, their professional reputations are damaged. Occasionally shitstorms are fair, and by and large this one has been. Millions of ordinary people have suffered much worse. Yes, over-eager austerity hawks jumped uncritically on the results which confirmed their prejudices and used them in a political fight. The authors are clearly not entirely to blame for that. However, a look at their Bloomberg article convinces me that they must take a share of the responsibility even here.