The authors of the new paper use responses in the Keynesian beauty contest game to construct a measure of sophistication for each individual in the study. Their key assumption is that sophistication is an individual-level property, one that is not strongly dependent on the decision-making context. Under that assumption, it seems reasonable that the sophistication of play in this laboratory game might serve as a proxy for an individual's innate sophistication level.
But my understanding of the experimental economics literature is that the extent of strategic-skills transfer across games is still very much an open question. Hence my answer to the puzzle: I fear that policy elites' reasoning abilities within their own areas may have little to do with their success in a few rounds of one unfamiliar laboratory game. It's likely that we all vary in our proficiency with different strategic settings, perhaps but not necessarily due to prior experience.
The authors say their key assumption "has been supported by recent studies that show how subjects' level of strategic reasoning frequently persists across different types of strategic interactions." I'm not sure that this is a full representation of the literature. First, let's look at the three papers the authors cite in support of this representation:
- De Sousa, Hallard, and Terracol 2012: This paper finds that those who play unsophisticated strategies in the beauty-contest game also tend to play unsophisticated strategies in another laboratory game. But the entire subject pool is drawn from participants at a major international chess tournament. That means a sizable number of strong chess players seem to be strategically unsophisticated when judged on their performance in two laboratory games. The authors conclude, "The distinction between strategic and non-strategic players does not correspond to field behaviour. Even strong chess players may end-up being classified as non-strategic in our experiment. This suggests that strategic ability is not an individual characteristic (p. 18)." We can rephrase the Friday Puzzler in light of this research by asking, "Why would any international chess tournament invite advanced chess players whose strategic skills were so poor?"
- Agranov, Caplin, and Tergiman 2013: This paper is about the ability of the "SCP protocol" to identify naive players across different games. The SCP protocol, however, is meant to get around a shortcoming of the typical beauty contest approach to measuring sophistication (p. 2).
- Bhui and Camerer 2011: I couldn't find this manuscript but, a bit unfairly, I'll talk about another work by these researchers instead. Camerer and his team find that chimpanzees play strategies closer to equilibrium predictions than do humans in the experimental games under examination. Does that mean chimps are innately "more strategic" than humans across disparate settings?
Second, the paper understandably doesn't mention a recent but directly relevant study. The abstract of Georganas et al. 2014 reads as follows:
We examine whether the ‘Level-k’ model of strategic behavior generates reliable cross-game predictions within an individual. We find no correlation in subjects’ estimated levels of reasoning across two families of games. Furthermore, estimating a higher level for Ann than Bob in one family of games does not predict their ranking in the other. Direct tests of strategic reasoning generally do not predict estimated levels. Within families of games, we find that levels are fairly consistent within one family, but not the other. Our results suggest that the use of Level-k reasoning varies by game, making prediction difficult.