Agent-Based Modelling of Social-Ecological Systems: Achievements, Challenges, and a Way Forward

Understanding social-ecological systems (SES) is crucial to supporting the sustainable management of resources. Agent-based modelling is a valuable tool to achieve this because it can represent the behaviour and interactions of organisms, human actors and institutions. Agent-based models (ABMs) have therefore already been widely used to study SES. However, ABMs of SES are by their very nature complex. They are therefore di icult to parameterize andanalyse, which can limit their usefulness. It is time to critically reflect upon the current state-of-the-art to evaluate to what degree the potential of agent-based modelling for gaining general insights and supporting specific decision-making has already been utilized. We reviewed achievements and challenges by building upon developments in goodmodelling practice in the field of ecological modelling with its longer history. As a reference, we used the TRACE framework, which encompasses elements of model development, testing and analysis. We firstly reviewed achievements and challenges with regard to the elements of the TRACE framework addressed in reviews and method papers of social-ecological ABMs. Secondly, in a mini-review, we evaluated whether and to what degree the elements of the TRACE framework were addressed in publications on specific ABMs. We identified substantial gaps with regard to (1) communicating whether the models represented real systemswell enough for their intended purpose and (2) analysing themodels in a systematic and transparent way so thatmodel output is not only observed but also understood. To fill these gaps, a joint e ort of themodelling community is needed to foster the advancement anduse of strategies such as participatory approaches, standard protocols for communication, sharing of source code, and tools and strategies formodel design and analysis. Throughout our analyses, we provide specific recommendations and references for improving the state-of-the-art. We thereby hope to contribute to the establishment of a new advanced culture of agent-basedmodelling of SES thatwill allowus to better develop general theory andpractical solutions.


Introduction
. Social-ecological systems (SES) describe the tight coupling of social and ecological systems: ecosystems are a ected by humans and in turn provide societies with ecosystem services and are thus the basis of human well-being (Berkes & Folke ; Folke et al. ). These interactions are continuously changing due to feedbacks and internal or external factors, and they take place across di erent temporal and spatial scales, making ). Grimm et al. ( ) included the "evaludation" steps within the TRACE framework (Schmolke et al. ) to document the full modelling cycle (Table ). TRACE documents are designed to document, using a standardized format and terminology, that a model was "thoughtfully designed, correctly implemented, thoroughly tested, well understood, and appropriately used for its intended purpose" (Grimm et al. , p. ). .
Here, we use the TRACE framework including the evaludation scheme (Table ) as the basis for our attempt to answer the key question: where does social-ecological agent-based modelling currently stand regarding the di erent elements of model evaludation? Specifically, we address the following questions about the use of ABMs as a scientific method: What is the model's purpose? Is its design ad hoc or based on generic principles? How is human decision-making represented? Are qualitative and/or quantitative data used to develop and parameterize the model? Has it and its rationale been thoroughly documented? Is it reproducible? Has its implementation been tested? Is its behaviour well understood? How much calibration does it include? Does it deliver any testable new predictions? .
To answer these questions, we first checked whether and how these issues are addressed in reviews and method papers of social-ecological ABMs. Then, in a mini-review of published ABMs of SES, we evaluated whether and to what degree the elements of model evaludation were addressed in publications on specific ABMs. The purpose of these two reviews is not to distinguish ABMs in this field into "good" or "bad" but to document the current awareness and practice regarding important elements of model development and use. The purpose of our review is to identify poorly developed elements in the current culture of agent-based modelling of SES, suggest improvements, and thereby contribute to the maturation of agent-based modelling of SES.

Methods
Review of reviews . We assessed reviews and method papers on social-ecological ABMs and extracted information on achievements, open challenges and ways ahead along the following categories suggested by the TRACE framework (Grimm et al. ): problem formulation/purpose, model description, data evaluation, conceptual model evaluation, implementation verification, model output verification, model analysis, model output corroboration, iteration of the modelling cycle, and upscaling/transferability (Table ). In addition to the TRACE categories (Grimm et al. ), we added the category upscaling/transferability because it is currently being discussed as one the major open challenges for social-ecological ABMs (Arneth et al. ; Verburg et al. ). The final category "iteration of the modelling cycle" illustrates the cyclic character of the TRACE framework.

Problem formulation
This TRACE element provides supporting information on: The decisionmaking context in which the model will be used; the types of model clients or stakeholders addressed; a precise specification of the question(s) that should be answered with the model, including a specification of necessary model outputs; and a statement of the domain of applicability of the model, including the extent of acceptable extrapolations.

Model description
This TRACE element provides supporting information on: The model. Provide a detailed written model description. For individual/agent-based and other simulation models, the ODD protocol is recommended as the standard format. For complex submodels, it should include concise explanations of the underlying rationale. Model users should learn what the model is, how it works, and what guided its design.

Data evaluation
This TRACE element provides supporting information on: The quality and sources of numerical and qualitative data used to parameterize the model, both directly and inversely via calibration, and of the observed patterns that were used to design the overall model structure. This critical evaluation will allow model users to assess the scope and uncertainty of the data and knowledge on which the model is based.

Conceptual model evaluation
This TRACE element provides supporting information on: The simplifying assumptions underlying a model's design, both with regard to empirical knowledge and general, basic principles. This critical evaluation allows model users to understand that the model design was not ad hoc but based on carefully scrutinized considerations.

Implementation verification
This TRACE element provides supporting information on: ( ) whether the computer code implementing the model has been thoroughly tested for programming errors, ( ) whether the implemented model performs as indicated by the model description, and ( ) how the so ware has been designed and documented to provide necessary usability tools (interfaces, automation of experiments, etc.) and to facilitate future installation, modification, and maintenance.

Model output verification
This TRACE element provides supporting information on: ( ) how well the model output matches observations and ( ) how much calibration and the e ects of environmental drivers were involved in obtaining good fits of the model output and data.

Model analysis
This TRACE element provides supporting information on: ( ) how sensitive model output is to changes in model parameters (sensitivity analysis) and ( ) how well the emergence of model output has been understood.

Model output corroboration
This TRACE element provides supporting information on: How model predictions compare to independent data and patterns that were not used, and preferably not even known, while the model was developed, parameterized, and verified. By documenting model output corroboration, model users learn about evidence that, in addition to model output verification, indicates that the model is structurally realistic so that its predictions can be trusted to some degree.

Upscaling/Transferability
This category encompasses the application of social-ecological ABMs to large geographical areas as well as the transfer of ABMs to di erent case studies. Iteration of the modelling cycle The development of models is an iterative process, and model versions continually improve based on knowledge gained in previous model versions. Therefore, the categories above should ideally be visited several times during the model development. ; Grimm et al. ). The last two elements were added for the purpose of this review.

Mini-review of ABMs
.
We conducted a Web of Science Topic Search (TS) with the search term TS = (((agent AND based AND model*) OR (multi AND agent)) AND social-ecological). We restricted our research to articles published in the last years ( -). A er cursory scanning of the resulting articles, we excluded articles because they did not describe ABMs and retained for our mini-review. In this review, we scanned for the more technical elements of Table , i.e., model purpose, model description, data evaluation, implementation and model output verification, model analysis, and model output corroboration (Table ). The other elements are covered in the "review of reviews".

Results and Discussion
Review of reviews . The list of the reviews and method papers we evaluated is provided in Supplementary Material S . Although we aimed to assess current achievements, we included articles from the last two decades because only a few recent reviews exist and because change to modelling cultures within specific fields is generally slow (Grimm & Berger b), usually at the time scale of decades. At some points, we included modelling examples to complement the reviews and method papers by highlighting examples of recent developments in the field ( papers). One purpose of our additional mini-review of ABMs of SES was to also systematically cover more recent publications. For the following, it is important to have in mind the specific definition of the corresponding categories (Table ) to avoid misunderstandings.

Problem formulation/model purpose .
A key question is whether social-ecological ABMs should focus on demonstrating and exploring ideas and testing hypotheses, or whether they can also provide decision and management support (Matthews et al. ). The boundaries between these model purposes are porous, and both are important. Nevertheless, these different model purposes call for di erent model types, ranging from toy models, which focus on specific features relevant for the research question under study, to structurally more-realistic models (Schlüter et al. ). While, in principal, models from both ends of this spectrum can be valuable for management support, ABMs aimed at supporting the appropriate design of a management strategy in a specific case study should have greater structural realism compared to ABMs designed for improving system understanding of general principles. .
In an extensive review of land use ABMs, (Groeneveld et al. ) showed that the overwhelming majority of ABMs are used for system understanding. So far, ABMs have been used to understand processes in di erent contexts, such as land use change, natural resource management, and urban processes (Heckbert et al. ), or they have been used to study processes of systemic change in coupled SES, such as regime shi s (Filatova et al. ; Polhill et al. ). The use of ABMs has already improved our understanding of collective actions and the governance of common pool resources (Janssen & Ostrom ; for examples, see Agrawal et al. , Zellner et al. or Schill et al. ) and of human-environmental interactions in the distant past (for examples, see Crabtree & Kohler , Barton et al. or Kohler et al. ). Despite this huge potential of ABMs to understand SES, there are multiple processes relevant for these systems that have so far only rarely been addressed in ABMs, such as the role of institutions, hierarchical interactions, multi-level decision-making (Hare & Deadman ; Rounsevell et al. ; Groeneveld et al. ), interaction network structures (Janssen & Ostrom ), and iterative influences between attitude and behaviour (Edwards-Jones ).
. We want to highlight work that has been done in recent years in some of these directions. For example, (Agrawal et al. ) use an ABM to study the impact that informal norms in the form of social networks and formal organizations have on forest consumption. Wang et al. ( ) used an ABM to analyse the role of institutions, such as sedentary grazing or pasture rental markets, for climate change adaptation in Mongolian grasslands. Finally, Manson et al. ( ) developed an ABM of social network dynamics and showed the importance of social networks for the adoption of rotational grazing in dairy farming systems in the northern United States. As these are only examples, new reviews are needed on specific topics such as the state-of-the-art in the incorporation of institutions or, more specifically, the representation of social networks in ABMs.

.
While ABMs have been frequently used to support system understanding, there seems to be a substantial gap in their use for solving real-world problems by providing guidance for the design of management and policy strategies in specific case studies. There is still a lack of predictive power of ABMs, and it is an open challenge to demonstrate the value of ABMs to solve real-world problems and for operational decision support (Matthews et al. ; Schlüter et al. ). In the same line, Verburg et al. ( ) stress the limited application of ABMs as policy tools.

.
There are di erent approaches to making progress in this field. For example, the realism of social-ecological ABMs has been improved by recent developments in the combination of ABMs with GIS studies (Heckbert et al. ). Furthermore, the value of ABMs for decision support can be pushed forward through stakeholder participation (Matthews et al. ), transdisciplinary project design (Levin et al. ), or through testing the impacts of policies over di erent futures and evaluating the probability of a policy to perform well (Lempert ). Still, experience from other fields of research suggests that simulation modelling may rather be useful to derive rules of thumb instead of being used directly by the end user (Matthews et al. ).
. One particular strength of social-ecological ABMs is that they enable the incorporation of stakeholders and decision makers through the use of participatory modelling approaches (Schlüter et al. ). The number of studies using such participatory approaches has been increasing considerably over the last decade, and ABMs are one of the main modelling techniques to support it (Seidl ). In contrast, in a review of land use ABMs, O'Sullivan et al. ( ) find that not many models have a participatory focus; they conclude that the potential of ABMs in the field of participation is currently underexplored. This potential encompasses di erent functions, such as collecting specific knowledge, informing decision makers, and social learning (Seidl ). Participatory modelling facilitates trans-and interdisciplinary communication, education and outreach (Matthews et al. ) and developing a shared representation in a companion modelling process (Bousquet & Le Page ). The latter can be used to develop and test social-ecological ABMs and has been practiced in a number of case studies (Bousquet et al. ; Castella et al. ). One method for participatory model development that has gained more attention over the last decade is the use of role-playing games to identify the decision-making processes of stakeholders and to include them in ABMs (Voinov & Bousquet ; Seidl ). While the current achievements in this area are promising, there are still multiple open challenges such as the subjectivity of stakeholders, conflicts between parties or the representative selection of stakeholders (Doole & Pannell ). Additionally, so far, there is no understanding of how accumulated knowledge derived in one situation can be generalised (Rouchier ). Here, Voinov & Bousquet ( ) predict that it will still take some time for a theory to be established. Until then, there will be only a few generalisations beyond the local applications of participatory modelling. Finally, standards for reporting on participatory modelling projects are currently missing (Seidl ). .

Voinov & Bousquet (
) gathered several valuable general principles to foster e icient participatory modelling such as early involvement of stakeholders or the selection of a diverse group of stakeholders. Seidl ( ) present a general template to structure transdisciplinary modelling projects, including the definition of the functions of di erent stakeholders during the project phases.

Model description .
In recent years, great e ort has been put into improving and standardizing how to describe ABMs. Examples include protocols such as the ODD (short for Overview, Design and Details; Grimm et al.
; Grimm et al. ); its extension for ABMs, including human decision-making ODD+D (Müller et al. ); the TRACE framework for documenting the full modelling cycle (Schmolke et al. ; Grimm et al. ); and ontologies describing entities and their relationships (Parker et al. a; Polhill & Gotts ). Müller et al. ( ) review di erent types of model descriptions and propose as a minimum standard the provision of natural language descriptions, for example, in the form of the ODD, combined with the provision of source code. ODD is widely used in ecology, and its use for social-ecological modelling seems to be increasing. Also, it is recommended by one of the main outlets for ABMs on SES, the journal JASSS, and by the OpenABM initiative (www.openabm.org). In any case, an established standard for model descriptions reduces the e ort required by model developers, reviewers and peers to describe and understand a model because the same kind of information is always to be found in the same part of the model description. Moreover, standardised model descriptions also facilitate and harmonize model development because the categories of, e.g., the ODD framework can be used as a checklist for the design decisions a modeller has to make. This also implies that for ABMs of SES, ODD+D might be more advisable, as representing human decision-making is the key element of these models.

Data evaluation .
ABMs profit from the availability of quantitative and qualitative data to directly or indirectly parameterize the model. In particular, collecting micro-level and interaction data can improve the representation of decisionmaking in ABMs (Filatova et al. ). Here, multiple approaches toward empirically informing social-ecological ABMs have emerged in recent years, including surveys, interviews, laboratory experiments, participatory and companion modelling, stylized facts (also referred to as "patterns"; Grimm & Railsback ), GIS and remotely sensed data (Rouchier ; Janssen ; Robinson et al. ; Heckbert et al. ; Voinov & Bousquet ). Also, "citizen observatories or crowd sourcing have the potential to contribute information on individual decision making" (Verburg et al. , p. ). .
Despite this large number of approaches and the progress made in empirically informing ABMs (Robinson et al. ; Windrum et al. ; Smajgl et al. ), ABMs still face multiple challenges with regard to parameterization and calibration (Robinson et al. ; Parker et al. b; Nolan et al. ). These challenges are especially pronounced for empirically informing human decision-making in ABMs, as "data collection is a very expensive task in [social science] and in most cases it is even impossible to generate long time series for individual or group behaviour" (Troitzsch , p. ). For example, methods to collect data on temporal processes in decision-making, such as frequency, duration, order or changes over time, are lacking (Gilbert ; Robinson et al. ). A further example of a gap in data collection is that survey data o en "treats individuals as isolated 'atoms' and pays little attention to the impact of people's interactions with others" (Gilbert , p. ).
. However, promising approaches exist that can help overcome challenges in parameterizing human decisionmaking in ABMs. In the following paragraphs, we will discuss some of these. One promising approach to model calibration established in ecological modelling is pattern-oriented modelling (Wiegand et al. ; Grimm et al. ), where di erent patterns/stylized facts are combined and used to calibrate the model. For social-ecological ABMs, such stylized facts or patterns can be used to reject agents' decisions rules that fail to reproduce patterns (Heckbert et al. ). As a further approach, Smajgl et al. ( ) provide a framework for the parameterization of human decision-making in ABMs: based on the modelling context, a combination of empirical methods for di erent steps in model building is proposed. .
Finally, like in other modelling fields, for social-ecological ABMs, the "balance between fitting the data and generalizability remains another problem" (Janssen ). In automated optimization procedures, this might be tackled by including penalties for model complexity due to, for example, the number of parameters (Janssen ). Reducing complexity will increase the tractability of models (Verburg et al. ). However, we would like to note that model complexity per se is not a bad thing but rather that the right level of complexity depends on the model purpose (Sun et al. ).

Conceptual model evaluation .
In social-ecological ABMs, the human decision-making sub-models are key elements, so their conceptual design is crucial for model evaluation. The adequate representation of human decision-making is a prerequisite for models to provide reliable policy recommendations (Milner-Gulland ; Bank ). There are already a variety of multiple decision models and architectures that are partly already used in ABMs. For example, Balke & Gilbert ( ) review architectures, including belief-desire-intention, norms, cognition, and learning. An ( ) di erentiate among nine categories: microeconomic models, space theory-based models, psychological and cognitive models, institution-based models, experience-or preference-based models, participatory models, empirical or heuristic rules, evolutionary programming, and assumption-based models. However, this large number of di erent techniques and theories also hampers advancement in the field (Parker et al. ). .
Despite this variety of potential decision models, the representation of decision-making in ABMs is o en ad hoc and only seldom based on established theory. Decision-making is most o en based on simple heuristics (Hare & Deadman ) and merely on psychological theories (Groeneveld et al. ). To advance the use of complex decision theories in social-ecological models, Schlüter et al. ( ) proposed the MoHuB framework for mapping and comparing behavioural theories. The MoHuB framework (Modelling Human Behaviour) specifies the necessary elements of the decision process (e.g., perception or selection process) and provides a guideline to map decision theories onto those elements. It aims to support informed selection of the appropriate decision theory and its formalisation in models. In contrast to this rather theoretical starting point, Bayesian belief networks present a promising approach to include empirically observed patterns of human decision-making in ABMs (e.g., Sun & MüLler ). .
Interestingly, very few ABMs of SES compare alternative models of decision-making and check how well they are able to lead to system-level behaviour that is realistic. One example is Janssen & Baggio ( ), who compare di erent behavioural theories to experimental data on irrigation games. This "pattern-oriented theory development" (Grimm & Railsback ; Railsback & Grimm ), which is increasingly implemented in ecology, is nothing more than following the scientific method. It was explicitly formulated by Platt ( ) and dubbed "strong inference". This basic principle, which is commonly used in natural sciences, seems to be less acknowledged in social-ecological research. For example, Schlüter & Pahl-Wostl ( ) state the obvious in saying that "The choice of how to represent the behavior of human actors in an agent-based model has a strong influence on model results". They refer to Hare & Pahl-Wostl ( ) in saying, "They showed that the sensitivity of model results to structural uncertainties in the social model largely exceeded the e ect of parameter uncertainties in the natural system." However, the potentially high sensitivity of model results to how a certain sub-model is formulated is inherent to any modelling discipline and does not necessarily prevent progress. Rather, corresponding to quantifying the sensitivity to changes in model parameters, modellers also need to systematically explore the sensitivity to di erent formulations of a certain sub-model. Basically, the modeller has to define a set of patterns or stylized facts characterizing the system of interest and its behaviour and see how well the ABM, with the di erent alternative formulations of the decision model, is able to reproduce these patterns simultaneously. In this way, the most appropriate representation of behaviour, in the context of the question addressed and the system characteristics considered, is filtered from the candidate representations. .
A particular challenge regarding the conceptual development of social-ecological ABMs lies in the needed integration of social and ecological systems: the majority of ABMs tackles only one-way linkages and not two-way feedbacks between the social and the ecological system ( ). .
Further e ort is needed to represent ( ) social networks (An ), ( ) spatial or scale (e.g., harvest decisions lead to biodiversity loss at a higher scale) mismatch between decisions and impacts [e.g.,][] (Parker et al. ) and ( ) adaptive behaviour and learning (Bousquet & Le Page ; Filatova et al. ). For the latter, "laboratory and field experiments are perhaps best suited to identifying the structure of the learning process where the repeated decisions of actors are recorded (e.g., Evans et al. )" (Robinson et al. , p. ).

Implementation verification .
So ware development for the implementation of ABMs has advanced in recent years, with so ware packages becoming available that no longer require extensive programming skills (Railsback et al. ; Heckbert et al. ) and thereby also facilitate the checking of model implementations. However, due to their complex nature, verification of ABM implementations is challenging (Nolan et al. ), and the detection of errors and artefacts is hampered (Galán et al. ). To avoid errors and artefacts, Galán et al. ( ) propose di erent strategies such as the "repetition of experiments in di erent platforms, reimplementation of the code in di erent programming languages, reformulation of the conceptual model using di erent modelling paradigms, and mathematical analyses of simplified versions or particular cases of the model". Independent replication of models has also been suggested as a general strategy not only for implementation verification but also for theory development, because modellers will less o en start from scratch, preferring to give more time to new analyses and identify the processes that actually control model behaviour (Thiele & Grimm ). Doole & Pannell ( ) suggests identifying the errors of each model component separately instead of testing the fully integrated model, a strategy also advised in an ABM textbook by Railsback & Grimm ( ), who demonstrate how re-implementing key sub-models in, e.g., spreadsheets can help identify even subtle implementation errors. A further approach is to make source code publicly available (Parker et al. ), which can be realized via online platforms such as OpenABM (http://www.openabm.org; for a discussion of experiences with OpenABM, see Janssen et al. , Rollins et al. or Janssen ). Further platforms include github (http://www.github.com) and Modelling All (http://m.modelling all.org/).

Model output verification
.
Verification of social-ecological ABMs in the sense of comparing model results to real-world data or patterns is still in its infancy. For example, Balbi & Giupponi ( ) reviewed ABMs in the field of climate change adaptation and found that half of the reviewed studies did not address validation and verification due to the level of model abstraction, which hampers model testing. Following Ahrweiler & Gilbert ( ), there are two ways to verify model quality: a) the constructivist approach, where the observer compares the results of a constructed simulation model and constructed real world observations, and b) the user community approach, where model outcomes are compared to expert expectations (also in line with Troitzsch ). In a similar vein, Polhill et al. ( ) argue that validation methods appropriate for ABMs could be expert validation (Smajgl & Bohensky ) or pattern-oriented modelling (Grimm et al. ). For participatory modelling endeavours, the quality of developed models is related to the degree of stakeholders' agreement (Voinov & Bousquet ). Verburg et al. ( ) state that instead of predicting how an SES will develop in the future, ABMs could explain why SES "produce currently observed behaviour" (Verburg et al. , p. ). .
A particular challenge for verifying the output of ABMs is knowing how to choose the appropriate level of testing, as di erent model structures at the micro-scale can lead to the same emergent patterns at the macroscale (Gilbert ). Hence, Takadama et al. ( ) propose conducting both micro-and macro-level validation of ABMs, and they provide an example using a simulation of a bargaining game. Bert et al. ( ) o er the same suggestion and distinguish between iterative conceptual validation, together with experts and stakeholders, and iterative empirical validation via calibration. They demonstrate their protocol for validation using an ecological-economic land use model that includes many practical examples of useful model analysis techniques. It should be noted, however, that Augusiak et al. ( ) suggest a more refined framework for model evaluation and validation; in particular, they distinguish between "output verification" and "output corroboration". "Output verification" includes model tweaking via calibrations and pattern-oriented theory development. "Output corroboration" refers to predictions of patterns in the structure and behaviour of the modelled system that were not used or, preferably, not even known during model development and calibration (see corresponding section below). Moreover, with output verification, environmental drivers should be reported together with model output because sometimes these drivers largely determine output, while details of model structure do not matter much. Janssen ( ) reports a case where the seemingly impressive match of model output and data was almost entirely driven by the imposed time series of the carrying capacity of the system. This does not necessarily render a model useless, but it should be communicated to avoid over-interpretations and to stimulate critical discussion and further development.

Model analysis .
Model analysis comprises assessing the impact of model parameters and assumptions on model output and deriving an understanding of how model results emerge (Augusiak et al.
). Due to their flexibility in representing processes at the micro-scale in great detail, as well as their stochastic nature, ABMs deliver "a high volume of output data rendering the identification of salient and relevant results (such as trends) and the assessment of model sensitivities to varying experimental conditions a challenging problem" (Lee et al. , par. . ). Moreover, "verification of model output" usually includes calibration, which implies running the model for the whole range of inputs, which is computationally infeasible for complex models, since "real-life systems have too many di erent kinds of inputs, resulting in a combinatorial explosion of test cases" (Cole ; a er Ahrweiler & Gilbert ). Additionally, the field currently lacks guidelines for appropriate analysis and presentation of complex ABM results (Lee et al. ).
. Therefore, the ABMs of SES o en focus on scenario comparison (as discussed for ABMs on regime shi s by Filatova et al. ; see also the mini-review below). Usually, the sensitivity of a few highly aggregated model outputs, such as average income or years staying in business, to di erent management, policy, or institutional scenarios is tested. The results may be interesting and important, but without understanding how the di erent responses of the system actually emerge, we have to trust the model blindly and we will not gain much in terms of how, why, and when certain mechanisms are dominant. Consequently, the "insights" from the model remain limited and cannot easily be transferred to other systems, scenarios, and questions.

.
There are a few examples of social-ecological ABMs for which at least an extensive sensitivity analysis has been performed (Filatova et al. ). However, these are exceptions, and "multidimensional parameter sweeps across several scenarios of change, and accompanying large-scale data analysis techniques and visualisations" are urgently needed (Filatova et al. , p. , O'Sullivan et al. ). Lee et al. ( ) discuss several available techniques for ABM analysis following three themes: an appropriate number of runs, sensitivity analysis and processing spatial and temporal output data. The authors also conclude that user-friendly so ware products for performing available analysis techniques are in high demand. First attempts in this direction include the BehaviorSpace of Netlogo (Wilensky ). BehaviorSearch (http://www.behaviorsearch.org) is a tool for calibrating models implemented in NetLogo. Many approaches for model calibration and sensitivity analysis that are widely used in other modelling disciplines are implemented in R (www.r-project.org). Using the R-package RNetLogo (Thiele et al. ), they can be easily used for ABMs implemented in NetLogo or, via file exchange of parameters and outputs, in any programming language. Thiele et al. ( ) provide a "cookbook" for calibration and sensitivity analysis based on R packages. In ecological modelling, local or one-at-a-time sensitivity analyses are included in most model analyses, but increasingly more-comprehensive techniques are being used. An example is the so-called "Morris screening" to identify the most-sensitive parameters, followed by global sensitivity analyses using, e.g., the Sobol of FAST methods (Thiele et al. ; Ayllón et al. ).

Model output corroboration
.
The gold standard for model validation, or for theory in general, is to make independent or secondary predictions. Dependent predictions are those that the model was more or less forced to make by choosing model structure and parameter values. However, this tweaking, which is inherent to modelling but does not yet often occur with ABMs of SES, might lead to the right outcome by combining the wrong mechanisms. "Testing" a theory in physics means making predictions that can be tested with existing but not yet used data or new experiments, for example, the gravitation waves predicted by Einstein, which were only confirmed in . .
In ABMs, such predictions are di icult but not impossible. The beech forest model BEEFORE (Rademacher et al. ) predicted a certain tree age structure in the canopy and a certain spatial distribution of very old trees; neither model feature was considered or even known during model development, but they were later confirmed by analysing historical data. .
For ABMs of SES, this means thoroughly analysing the model using more than just a few aggregated variables as model output and being alert to unusual patterns in model structure and behaviour. Similar to the discussion of "model output verification" above, expert assessment can help advance "model output corroboration". Especially in participatory modelling projects, the final step is to disseminate results beyond the stakeholder group that was originally involved in the model development process (Voinov & Bousquet ). This can help advance "model output corroboration" by the feedback of a larger audience.

Iteration of modelling cycle .
Modelling has to start from simple versions, which are analysed to be understood and then refined to add only necessary detail and to develop understanding incrementally. In principle, iterative model development in this "modelling cycle" is never finished, but it should ideally be run through at least several times. This is of special value for theory development: the inability of a model to reproduce empirical findings can indicate flaws in the theoretical foundation of the model and lead to an update of the underlying theory (Schutte ). This is especially interesting for advancing the representation of human decision-making in social-ecological ABMs and the understanding of how humans make decisions in general (see comments on "pattern-oriented theory development" above).

Upscaling and transferability .
A recently discussed open question is how to upscale social-ecological ABMs to larger geographical areas. So far, an upscaling theory is missing (Parker et al. ; Rounsevell et al. ; Arneth et al. ; Verburg et al. ). This would enable the coupling of ABMs with environmental and vegetation models at di erent spatial scales (Rounsevell et al. ) and would thereby help realize hybrid approaches that couple or tightly integrate di erent models (O'Sullivan et al. ). In particular, it is an open question of "how to scale up processes of interactions of a few agents to interactions between many agents" (Janssen ). Upscaling of information on societies "demands substantive methodological development beyond simple statistical aggregation" (Verburg et al. , p. ). Here, Verburg et al. ( ) discuss two methodological approaches: ( ) outscaling to represent all individuals in a large geographic area by the use of increased computational power and ( ) upscaling based on observed response patterns at an aggregated level (i.e., instead of individuals, the behaviour of the entire community is represented). .
Another recently discussed approach to tackling these challenges is using agent functional types, which group individual agents by type (Arneth et al. ), comparable to global vegetation models, where a small number of plant functional types are used (e.g., LPJ GUESShttp://iis4.nateko.lu.se/lpj-guess/). We could also learn from political science, where ABMs have been applied at national and state levels (Cederman ).
. A related challenge is to develop models that facilitate the transferability and generalizability of ABMs over different case studies while still being applicable for specific case studies (Janssen ; Schlueter et al. ). In this context, O'Sullivan et al. ( ) warn against the tendency of the ABM field to develop increasingly more independent ABMs for specific case studies (the "YAAWN syndrome -Yet Another Agent-Based Model. . . Whatever. . . Nevermind. . . ") and propose that researchers working on specific case studies should "more fully articulate how they contribute to theoretical and/or methodological debates and concerns" (O'Sullivan et al. , p. ).

Mini-review .
Of the articles found in our scan of the Web of Science database (see Supplementary Material S ), we excluded eight articles because they did not include a particular model but presented modelling frameworks (e.g., Schlüter et al. ) or larger projects of which ABMs are just a part and are not described in more detail (e.g., Astier et al. ; Forrester et al. ), or because so far only the conceptual model exists but not its implementation as a computer program (Spies et al. ). The fact that approximately % of the articles we found discuss ABMs and their role instead of actually presenting and analysing one confirms how young the field of ABMs of SES still is. Also, in ecology, there was initially a large number of review, methodological, or framework papers, but nowadays, ABMs are accepted while lengthy discussions of their potentials and limitations are no longer needed, or wanted, in research articles.

.
In the following, we summarize and briefly discuss the main findings of our mini-review ( Figure ). Most models were designed to mimic a specific real system about which data or observations exist. Only in four cases, however, were the models designed to directly support decision-making in a specific context (Carpenter & Brock ; Parrott et al. ; Chion et al. ; Pizzitutti et al. ). In cases, the more or less realistic model was used to address a generic or theoretical question, whereas in seven cases, the real system was represented in a stylized way. The di erence between these two categories is that "specific but theoretical" models aimed at a more or less realistic representation of a specific system, for example, by including GIS data, but then they explored theoretical questions, for example, how di erent types of governance a ect resilience. In contrast, "specific but stylized" still refers to a specific system, but without aiming for a realistic but more or less stylized representation, with the benefit of being less complex and, therefore, easier to analyse. Only one model (Pérez & Janssen ) was a toy model, as it did not relate to any specific system.
. This distribution of model types may reflect the fact that the SES addressed with ABMs are complex systems. Representing them in generic models usually leaves too many degrees of freedom in the model structure and parameters. Confinement to real systems limits the degrees of freedom but at the same time limits the general insights that can be gained. Stylized representations of classes of systems seem to be a way to combine realism with the potential for general insights, but so far, it has not been tried to define classes of systems, in contrast to ecology, where only a limited number of ecosystem types exist. .
Model description turned out to be a major issue. It is impossible to say just by reading a model description whether it is complete enough to be the basis of an independent re-implementation. We might therefore err on the positive side, but it seems that of the model descriptions were more or less complete. Of the remainder, were incomplete and five were presented in earlier publications. Five presumably full model descriptions were in online supplements that were proprietary to journals and therefore were not generally accessible. On the other hand, online model descriptions were accessible, eight of them on OpenABM. The use of ODD (Grimm et al. ) and ODD+D (Müller et al. ) has increased in recent years. Nine models used these formats, but two were incomplete (Guzy et al. ; Huber et al. ), one used a modified format (Janssen ), and one existed on OpenABM but was not mentioned in the publication (Heckbert et al. ). .
Reading the model descriptions revealed the necessity for substantial improvements in this area. Even though the ODD descriptions were not all entirely consistent, at least the reader knew where to expect what kind of information; o en, they were complemented by the corresponding computer program. In contrast, the non-ODD descriptions did not have any consistent structure across the publications. Some were overly short and incomplete, while some were very long but did not follow a consistent structure so that despite the e orts of the authors, it remained unclear whether they were complete. Usage of the ODD format is obviously not perfect in practice, similar to the fact that being a native speaker in, e.g., English does not mean one is a good writer. However, even a suboptimal ODD/+D is much better than most free-format model descriptions. .
Consistently useful ODDs can be achieved by taking the requirement to follow a standard format seriously, using the template provided by Grimm et al. ( ) and Müller et al. ( ), reading a few existing ODDs and checking in what way they are good or incomplete, and, most importantly, by factually describing what the model actually does, describing the program code, not any meta-representation of the model.

.
Still, even the best ODD will contain the ambiguities of any verbal description. Therefore, one should always also provide the very program code that was used to produce the results presented, preferably on an openaccess online platform such as OpenABM (cf. Müller et al. ). The link between ODD and program code can be improved by using hyperlinks, or by numbering model equations and algorithms and using the same numbers, as comments, in the code. Referring to full model descriptions in earlier papers is also a cumbersome practice, as readers might not have access to that journal. Again, platforms like OpenABM are a much better solution and should be used even if earlier model versions have already been published elsewhere. .
As stated above, except for one single model, all models reviewed referred to specific systems. However, only publications mentioned a comparison of model output to observation and data, and a strong case for model realism was not presented in any of these models. Model corroboration with independent data was tried only once (Drake & Mandrak ), perhaps because this model was published in an applied ecology journal where providing evidence for su icient realism is required. .
It seems that this seemingly limited e ort of model output verification is mainly a matter of communication.
Modellers who have a specific system in mind actually do use criteria by which they decide whether or not their model o ers a su iciently realistic representation for their purposes (Rykiel ; Augusiak et al. ), but in social-ecological modelling, no culture has yet been established to communicate these criteria. However, even if the criteria are qualitative or categorical, they can be good indicators of a model's realism and generality, and they should be listed in the methods section and their tests briefly presented in the results section. .
The greatest issue we identified in our mini-review is model analysis. We found very few cases, e.g., Wilson et al. ( ), where the major e ort of the modeller was model analysis in order to understand how model results emerged. In cases, model analysis was non-existent, extremely limited, or confined to scenario analysis. Only in seven cases were some elements of the sensitivity analysis presented, and none of them in a systematic way. Only eight publications included elements of model analysis that went beyond sensitivity analysis and provided at least some mechanistic understanding, e.g., sensitivity experiments (varying individual parameters over a larger range: Wang et al. ; Pérez & Janssen ), global sensitivity analysis (Rasch et al. ), comparison of model output with multiple observed patterns (Heckbert et al. ; Parrott et al. ), extensive calibration (Janssen ), and unrealistic scenarios (Wilson et al. ). .
It seems that modellers of ABMs for SES focus too much on the question of how to represent SES and too little on how they could actually learn from their models. Guzy et al. ( ) admit this very openly: .
"We arrived at a very costly, complex model configuration that is di icult to explain, interpret, and generalize from. Because we had already exhausted the resources available to us, we could not explore the consequences of alternative agent representations, including interactive role-playing, dynamic generation of agent preferences, mechanisms for specifying agent behavior, and other advances in the core mechanism, such as those from game theory and behavioral psychology that involve agent interactions." .
To improve this situation, we recommend iterative model development where early oversimplified model versions are thoroughly analysed, with all relevant model outputs and testing methods implemented. Only in this way can we already learn from early versions how to incrementally refine the model and actually understand how model behaviours emerge.

Conclusions
. With this review, we recorded achievements made in the field of agent-based modelling of SES. However, we also identified poorly developed elements in the current modelling culture and suggested improvements to overcome these. To conclude, we would like to highlight promising ways forward in advancing the field of socialecological ABMs.
. Improving the representation of human decision-making: Here, interdisciplinary collaboration among behavioural economics, social psychology and agent-based modelling should be fostered (Groeneveld et al. ). This can improve the availability of empirical data on decision processes, our understanding of how humans make decisions and how these decisions can be formalized in models. In this context, participatory modelling approaches also represent a valuable tool. .
Sharing/reusing models: In a concerted action, we should advance the reuse and sharing of social-ecological ABMs. This can be achieved by the provision of source codes of full models as well as of model modules (cf. Janssen and Bell et al. ; see also Thiele & Grimm on the benefits of model replication). This would enable the pooling of energies, as modellers would not begin from scratch but would learn from each other. Furthermore, the sharing of models could foster the transferability of models to other research contexts. .
Model development and analysis: Substantial improvements are needed to improve the current culture of model analysis in the field of agent-based modelling of SES. Method compilations for an adequate, transparent and systematic model analysis in combination with iterative model development need to be provided. This is the only way trust in social-ecological ABMs can be increased and their full potential -for example, for decision and management support -can be realized. .
In particular, tools and strategies such as the evaludation framework, standard protocols for model communication such as ODD and ODD+D, and sharing code on online platforms such as OpenABM are simple but e icient means to learn more from our models and to make modelling in this field a community e ort. We believe that following these routes and the suggestions provided throughout our review will help advance the modelling culture and thereby make agent-based modelling of SES more e icient and more coherent in the future.