In a recent contribution to this journal, Stefano Pagiola, Jordi Honey-Rosés and Jaume Freire-González (Pagiola et al., 2020) assess the permanence of restoration-oriented silvopastoral land use changes in relation to a short-term Payment for Environmental Services (PES) flagship project: the Regional Integrated Ecosystem Management Project (RISEMP) funded by the GEF and implemented by the World Bank between 2003 and 2008 in Colombia, Costa Rica and Nicaragua. They revisit the Nicaraguan project site in Matiguás-Río Blanco in 2012, i.e. four years after project completion, and assess how payments have induced farmers to adopt and sustain the promoted silvopastoral practices. By surveying a subset of former project participants, the authors narrow down 5 hypotheses that, as we describe below, are disengaged from actual grounded evidence of this particular project. In addition to a priori establishing core assumptions of human behaviour and ecological relationships, they selectively analyse data to assemble evidence claiming that payments were successful in inducing both short-term and longer-term land use changes and to “rule out alternative explanations” (p. 1). As a group of scholars who have been conducting extensive research on PES policies and projects, including on the above-mentioned project, we have serious concerns about the findings of the article, as well as the broader implications of evaluations of conservation projects oriented to showcase “success stories” for replication and scale-up to other contexts.

In the following sections we reassess the authors’ evidence and conclusions in view of alternative explanations and findings that shed a different light on the project’s short and longer-term results. We thereby highlight two dangers of the original assessment, which range from structural and historical patterns of land-use change in the region that have been largely ignored to methodological flaws of the research. We argue that being transparent about underlying knowledge and value frameworks upheld by researchers and actors implicated in conservation projects is a fundamental aspect of conservation science and practice and therefore should be a crucial element in any attempt to evaluate PES projects. We further caution against attempts to showcase success stories through analyses that tend to disregard many relevant contextual elements and contrasting evidence. The failure to be transparent about diverging value frameworks that underpin conservation research directly impacts how success gets defined and evaluated (Bromley, 2012; Joslin, 2019; Svarstad & Benjaminsen, 2017). As Nelson and Bigger (2021) recently argue, interventions like PES that encourage tropical conservation do not necessarily supplant extractivist projects that are otherwise counterproductive to conservation; instead, they may simply build on underlying drivers for intensive commodity production like timber, or securitize the ecological soil conditions for sustaining intensive and ecologically-destructive forms of agriculture. In this sense, generating success stories is akin to a site of extraction where specific evidence is literally mined, sometimes unconsciously, to reflect a particular epistemological framing that is blind to context and which reinserts decontextualized evidence to justify and expand a global conservation value chain. This value chain emerges when conservation success stories beget the endless growth of bankable conservation projects worthy of investment, while lacking the capacity to confront the longer term historical drivers of biodiversity loss and even work to encourage the political and economic systems that exploit both humans and non-human natures.

Mining for ‘Success’ in PES Evaluation

The primary claim of Pagiola et al. (2020) is that the provision of payments is the core reason for greening land uses. In other words, ‘success’ (and permanence of this ‘success’) hinges on establishing a linear causal relationship between financial incentives and ensuing land use changes. However, we show that in the RISEMP case, project payments alone are an insufficient explanation of the underlying socio-cultural and economic drivers that were actually influencing land use changes.

To corroborate the potential link between the PES incentives and desired land use change, we have reassessed the project’s results (Vaessen & Van Hecken, 2009; Van Hecken et al., 2015; Van Hecken & Bastiaensen, 2010) and contextualized them within a broader series of (ongoing) studies in the region (Bastiaensen et al., 2015; Polvorosa, 2013). This included an analysis of the original project surveys, as well as the results of its randomized control trial (RCT) experiment, complemented with a two-month qualitative field study in 2009 encompassing in-depth interviews with thirty-five participant and non-participant farmers and with local project staff. Additionally, in 2013 follow-up interviews were conducted with thirty-two project participants, which provided insights in the multiple motivational underpinnings of land-use changes and social–ecological dynamics in the region since project termination. The main findings of these studies were also validated with former local project staff during workshops in 2013 and 2014. Finally, our analysis is also complemented by in-depth studies into processes of agrarian change in the context of cattle-based agricultural frontier expansion in the broader region, encompassing the area in which the RISEMP project took place (Bastiaensen et al., 2015, 2021; Van Hecken et al., 2019). We draw upon these previous studies to illustrate the diverging ways in which success can be interpreted from the RISEMP project, and in doing so highlight why context matters for conservation projects.

Methodological Mismatches

In addition to the above arguments, the methodology adopted by Pagiola et al. (2020) to evaluate the program’s “success” raises further questions regarding the rigor of the analysis. In order to attribute desired land-use changes to the induced payment, the project initially designed a randomised control trial (RCT) in which 123 farmers were subdivided into three different experimental groups, receiving payments only (28 farmers), payments and technical assistance (TA) (70 farmers), or neither of these in serving as a control group (25 farmers). This setup was intended to separate the effect of PES and TA from other (undefined) contextual variables in order to assess their relative effectiveness. Not surprisingly, however, and as explained extensively elsewhere (Vaessen & Van Hecken, 2009; Van Hecken & Bastiaensen, 2010), the RCT design proved to be problematic and far from random in practice, making it impossible to distinguish between the different ‘treatment’ and control groups as if the project took place in an experimental laboratory instead of the real world. The follow-up survey, with a subset of the original participants that Pagiola et al. (2020) refer to, and the reference to land use maps (which are absent for the reader to verify) attempt to make claims about the short-term and long-term additionality of payments-induced land use changes. However, without any meaningful ‘counterfactual’ or contextualized analysis, Pagiola et al. (2020) claim that project payments, rather than TA or any other possible motivation, were most likely to have accounted for the observed land use changes. In order to back-up the claim of project success, they rely on “casual observations” that indicate that neighbouring areas not included in the project observed “very few land-use changes” (p.8). We are left wondering how “casual observations” can ever rigorously justify such an important claim, given the context that we have described in the previous sections.

The fact that the authors decided to initially use an RCT¹ to assess the program’s success and then subsequently proceed to selectively disregard elements of the RCT (when the expected outcomes were not forthcoming, as noted below), demonstrates how they carefully cherry-pick the evidence that supports their claims of success. Our comparison of the different treatment/control groups (Tables 1 and 2) indicates that dismissing the control group from the analysis, as the authors have done with the argument that the control group was “highly suspect” (p. 7), denotes a strong selection bias to support their claims and to dismiss other explanations that fall outside of their interpretive framework. Closer examination of the farmers in the control group reveals that these individuals were among the wealthiest and largest land-owners, having on average more land and greater stocking of cattle, and a smaller household size (Table 1). These factors alone invalidate the control group as a reliable counterfactual for the RCT. Yet, it remains relevant to observe that these land-owners had among the most substantial outcomes in terms of enhanced silvopastoral land-use (Table 2), suggesting once again that the authors’ claim that observed land-use changes are attributable to the payment incentive alone is problematic.

Table 1.

Project Participants by Treatment Group, 2002 (Mean and Standard Deviation).*

Table 2.

Changes in Land Use, per Treatment Group (*), 2003–2007.

Table 2 gives an overview of land use changes across the two treatment and the control groups, showing that degraded pastures have decreased in all groups, with the highest reduction paradoxically in the control group (−23 per cent). Furthermore, the implementation of living fences have also increased most in the control group (eight times more than in the PES group). Fodder banks show a very similar pattern across all groups, while the highest increase in natural pastures, with and without trees, was also in the control group. Due to the outstanding results of the control group, it is baffling to us as to why Pagiola et al. (2020) failed to mention these surprising outcomes and then proceeded to dismiss any kind of engagement with the findings of this control group by arguing that its inclusion in the analysis would bias the supposed objectivity of the RCT experiment. This makes their argument a circular one in which the authors disregard or even purposefully dismiss² the relevant information these cases provide reflecting the political and economic dynamics that are operating in this landscape.

Implications for Conservation: Missing the Forest for the Trees

Tropical conservation depends on regenerating the intimate ecological functions and relations that sustain topsoils, groundwater, biodiversity, and which regulate the climate. These should be among the core aspects that count in assessing the additionality of PES projects. In this sense, the putative purpose of PES interventions is to reduce land degradation in the long-term by generating the conditions that attend to these dynamic relations and motivations and which crucially involve both human-social and non-human-biophysical entanglements. However, in our view, focusing only on the adoption of particular land-uses on the farm plot disregards the political-economic systems that permit or obstruct restoration and regeneration of ecological functions at a landscape level. In the case of the RISEMP project, this absence of analytical precision on the part of determining “success” is resulting in a PES program that piggybacks on underlying land use dynamics that encourage increased cattle-stocking (and the ecological consequences these entrain at the project site) as well as the expansion of the agricultural frontier elsewhere, further exacerbating tropical deforestation.

Moreover, reducing phenomena to the sum of their parts by concentrating only on the effect of payments on land-use change, in our perspective, ultimately circumscribes both human behaviour and ecological relations in such a narrow way as to effectively render futile and even counterproductive any intervention that claims to be about conservation. To avoid the risk of simplifying complexity to justify donor spending by selectively engaging with the social and political context of conservation interventions, we underscore the importance of inserting alternative knowledge framings to bring rigour to the core of tropical conservation science. Barring these considerations, the PES recipe becomes a vacuous exercise of mining for evidence of success with minimal or even counterproductive implications for conservation. We therefore echo others who have warned against the fetishization of PES blueprints (e.g. Ervine, 2010) through the deployment of new bankable PES projects in other locations (e.g. Colombia, Brazil, and elsewhere in Nicaragua, as the authors also hint at in the conclusion of the article). Instead, attentiveness to methodological plurality and conceptual openness is crucial to more responsibly inquire into the multifaceted nature of project outcomes and to respect the legitimate perceptions and experiences of a diverse range of actors involved in these interventions.

The implications of the authors’ study amplifies the critique of many other scholars across different vantage points within the field of conservation, ranging from the natural to the social sciences. The conclusion of conservation success from the evidence provided by Pagiola et al. (2020) echoes deep-seated concerns in terms of how communities and ecological relations become caricatured to match the simplistic relation of ecosystem services (Barnaud & Antona, 2014; Tadaki et al., 2015). The authors’ decontextualized and reductionist assessment of conservation intervention outcomes strikes a chord with critiques raised by interdisciplinary scholars on the social asymmetries of power that shape environmental governance (Büscher, 2014; Ferdinand, 2019; Myers et al., 2018) as well as the ecological inconsistencies of overly simplified study designs (e.g. Christie et al., 2019). The lack of engagement with existing studies that offer conflicting scientific evidence is itself contrary to the process of doing rigorous conservation science (e.g. Sutherland et al., 2020). Lastly, scholars have noted that conservation interventions may also securitize against potential social and ecological risks to production schedules that ultimately serve to worsen social and ecological degradation in the long-term (e.g., Battistoni, 2017; Nelson & Bigger, 2021; Pasgaard et al., 2017).

The concerns we have raised are by no means limited to PES implementation in Nicaragua, but also reflect the ways that adverse ecological outcomes result from conservation interventions more generally when attention to land-use drivers, power asymmetries, human-nature relations, and conceptual framing and research methods are poorly considered. In another example of payments for forest conservation in China’s Sloping Land Conversion Program for instance, Li et al. (2015) question the long-term ecological outcomes of rural depopulation and urban migration as new forms of cheap labour to produce manufactured goods to global markets. Such transnational forms of ‘leakage’ are deeply embedded in the broader supply chains of resources and labour that ultimately exacerbate ecological breakdown that conservation is meant to address. Elsewhere in Canada, Kolinjivadi et al. (2020) highlight how the implementation of buffer zone conservation strips ultimately induce farmers to acquire more land to bring into cultivation in order to meet ever tightening quotas for industrial french-fry production within a growing fast-food industry. Failure to generate specified quotas has led to a form of corporate blackmail, obliging producers, and indeed governments across scales, to ensure production is met or risk large-scale potato processors picking up and leaving for greener, more lucrative pastures. Once again, conservation that ignores the political economy of production -paradoxically then- serves to hasten land-use degradation.

There is a greater need for transparency on the value and knowledge frameworks that define how criteria for conservation success are determined and interpreted. Such reflections are particularly relevant when research is carried out by project implementers themselves. The World Bank in particular plays a commanding role in funding PES projects around the world, yet simultaneously encourages outside investors to turn so-called “underutilized” land into value-added industrial-scale production for integration into global commodity circuits (e.g. Borras et al., 2007; Li, 2014). As such, more reflection is needed by researchers and project evaluators as to the political and economic interests that underpin their positions and affiliations when carrying out these kinds of studies that directly intervene in the everyday lives and life choices of peasants and smallholders around the world. Great care is required so as to be responsible and attentive to the unintended ripple effects that such projects might generate.

It is these points that we hope will serve as a springboard for more open, horizontal, and long-term dialogue and reflection between participants, project promoters, scholars, activists, and all other directly and indirectly involved actors in PES projects around the world. To this end, we call for a deliberative platform to unpack PES interventions that are interpolated by plural knowledge and value frameworks in order to examine how socio-political and ecological dynamics influence long-term outcomes and their implications for conservation. In short, greater attention to “ePEStemology” is required in exposing how uneven power relations shape the ways PES interventions unfold in practice. In this way, it becomes possible to better understand how conservation “success” is defined and by whom, while fostering much needed dialogue between diverse social actors where diverging positions exist. We believe improved transparency on conservation values will enable a stronger collaborative dimension in identifying the long-term outcomes of conservation interventions.Given the continued and substantial interest in PES for responding to social and ecological challenges around tropical conservation, such an initiative is urgently needed.