Book Review: Field Experiments in Economics: History and Methodology. Judith Favereau and Michiru Nagatsu. Publisher: Routledge 2025
Field Experiments in Economics: History and Methodology by Judith Favereau and Michiru Nagatsu is a concise book that does two things. First, it provides an overview and analysis of field experimental practices in economics, both from a philosophical and historical perspective. Second, it proposes ways to improve said practices by combining their existing methodologies and importing resources from non-experimental and participatory approaches. The book as a whole is an essential read for those wishing to familiarize themselves with field experiments and their methodological and philosophical questions, while the original proposals invite those already well versed in the subject matter to consider how field experiments could better serve the epistemic goals of researchers and the political goals of stakeholders alike.
After an introduction, the book kicks things off with a general overview of experimental economics. This second chapter is filled with illustrative examples of different traditions and paradigms – from theory-testers to institution-builders and from the lab to the field – and it explores their distinctive characters from a philosophy of science perspective. The chapter is very comprehensive as such, and the numerous references provide an excellent bibliography for those looking for an even more thorough understanding of the topic.
In the third chapter, a distinction is introduced between two strands of field experiments, which much of the book is organized around. The first strand, “lab-in-the-field” experiments (LFEs), originate from traditional laboratory experiments, and they are characterized by attempts to systematically relax the experimenters’ control over some contextual features to make them more faithful to economic behavior as it occurs outside the lab. The second strand, “Randomized Field Experiments” (RFEs), have their origins in the evaluation of large-scale development programs and public policies, and their main goal is to produce evidence about which interventions are successful in alleviating policy-relevant social issues such as poverty.
LFEs and RFEs differ from each other on multiple dimensions, but the most central notion used to characterize their differences is that of control. According to the authors, LFEs are associated with direct control. This refers to the control of the laboratory, where the experimenter chooses the location, writes the instructions to elicit a certain framing of the situation, establishes the incentives, and so on, in order to exclude the influence of disturbing factors. RFEs, in contrast, are characterized by indirect control. As the name suggests, this is obtained most notably by randomizing the subjects into treatment and control groups, so that the observed difference in the outcome variable can be attributed to the intervention.
The fourth chapter ties these two strands and different forms of control to the issue of external validity. This too, it is argued, comes in two forms. The worry of artificiality is associated with direct control exerted in LFEs, where the threat is that the results are confined to the relatively “unnatural” context created by the experimenter. RFEs, on the other hand, have to deal with generalizability. Because RFEs purposefully level the influence of contextual factors in order to identify the effect of the policy under evaluation, it may be difficult to make the further inference that similar effects will be obtained when the policy is implemented in a different context. Both strands thus face different issues of external validity, and in many ways, the history of these strands is the history of trying to address these issues. In LFEs, the purposeful loosening of direct control represents an effort to reintroduce some of the natural contextual variation of the economic phenomena outside the lab. As for RFEs, the authors review different suggested as well as actual steps taken to improve the generalizability of their results, ranging from the use of machine learning algorithms to the theoretical modelling of hypothesized mechanisms.
The last two chapters are devoted to the authors’ own proposals for improving the generalizability and thus policy relevance of field experiments in economics. The first suggestion is to combine RFEs with LFEs to benefit from the high degree of direct control associated with the latter. Because LFEs are able to reproduce theoretically interesting results, such as cooperation in collective action problems or hyperbolic discounting, their experimental setups can serve as portable ways to measure the related constructs, such as levels of trust or present bias. Accordingly, the authors envision that when RFEs first identify a policy-relevant problem, such as underinvestment in preventive health care products, LFEs could then be conducted to investigate the causal mechanisms at play, e.g. present biasness of the individuals, and subsequent RFEs would then test whether a proposed solution based on this knowledge is effective in practice. The authors also argue that field experiments could benefit from utilizing non-experimental and qualitative methods to collect facts about the population, which could guide the design of experiments and interventions to better address heterogeneity and population-specific features. Given the increased awareness of the need to address heterogeneity in applied behavioral science more generally, this suggestion is particularly compelling (see Bryan et al. 2021).
However, the perhaps more radical suggestion is that field experiments in economics should lean more into the subjects themselves as authors of relevant knowledge by incorporating participatory methods. Epistemically, involving the subjects in designing and implementing experiments is useful because they are often in a privileged position to know which resources they need and which obstacles they face trying to access them, as well as why an intervention may have failed to have the intended effect on their behavior. Participatory involvement serves also the political goal of empowering the subjects: when evaluating policies intended to improve people’s lives, those people should have a say in what counts as an improvement to their lives in the first place.
On a more practical level, the authors draw from Sabina Alkire’s (2005) work among others and envision participatory field experiments as a joint effort between the researcher, local field partners, and members of the target population. The design of the experiment would center around public discussion between the participants, facilitated by the field partner and observed by the experimenter, where the relevant needs and obstacles, as well as possible solutions, would be identified. Rather than being someone who asks the questions and answers them too, the researcher would thus be one who listens and uses their methodological expertise to help implement the experiment designed jointly with the participants.
At this point, an objection may come to the mind of many readers, which is that in allowing the subjects and other stakeholders to take part in planning and implementing the experiment, the researcher sacrifices their control over it, compromising the validity of the inferences that can be made. However, the authors elegantly argue that this is not the case, precisely because what is relevant is not control in the agential sense as the exercise of power. For the authors, experimental control is at its core an inferential notion, “an epistemic ideal and achievement”, and direct and indirect control are both ultimately just different ways to make causal inferences sound and secure. The question of how much control the experimenter has is thus different from whether they are able to manipulate the “data-generating processes” at will, or whether they have to isolate the relevant influences through statistical means instead.Even in the most directly controlled experimental setups, the subjects may bring interpretations and framings that differ from what the experimenter intended; at the same time, statistical methods can sometimes yield valid causal inferences even in naturally occurring data with no intervention at all by the researcher.
Overall, Field Experiments in Economics: History and Methodology offers a comprehensive look at field experiments in economics, their methodological differences and historical origins. It is intended for a wide audience, and I agree that many different readers should find it worthwhile. The earlier chapters are admittedly more on the educational side and quite densely packed with various typologies and concepts, but given the book’s short length, this extra attention to detail can hardly be objected. For those already familiar with the topic, however, the forward-looking discussion of the last two chapters should be especially exciting.
At a conference this summer, I asked a behavioral economist if they ever gather qualitative data about subjects’ experiences in lab experiments. Referring to a broader sentiment in the field, the colleague replied half-jokingly that “we economists don’t seem to really trust our subjects”. Now, while prioritizing incentivized behavioral measures can arguably often be justified in a lab setting, I take the present book to make a convincing case that people’s experiences, opinions, and beliefs as expressed by themselves constitute evidence that should not be ignored, at least in field experiments aiming to improve the lives of those people. In fact, I was delighted to note that in two subsequent conferences, many presentations discussed participatory methods from both practical and theoretical perspectives, including one which directly referenced this very book by Judith Favereau and Michiru Nagatsu. In conclusion, it is safe to say that Field Experiments in Economics: History and Methodology not only offers an excellent analysis of the past and present of field experimental practices in economics, but also contributes to their future development.
The book is available in print, and an open access version can be read online through its publisher Routledge.
Tatu Nuotio is a doctoral researcher at the University of Helsinki and a member of TINT – Centre for Philosophy of Social Science. His PhD research focuses on theories of social norms and how they are applied in behavior change interventions and behavioral public policy. The project is funded by the Kone Foundation.
References
Alkire, S. (2005). Valuing freedoms: Sen’s capability approach and poverty reduction. Oxford and New York: Oxford University Press.
Bryan, C. J., Tipton, E., & Yeager, D. S. (2021). Behavioural science is unlikely to change the world without a heterogeneity revolution. Nature Human Behaviour, 5, 980–989.