Models Within Models – Agent-Based Modelling and Simulation in Energy Systems Analysis

This paper tries to show the various roles agent-based modeling and simulation (ABMS) can play in technology and policy assessment of energy systems. We examine the advantages of ABMS methods using three case studies of electricity market models as example (AMIRIS, EMLab-Generation and PowerACE). In particular, we argue why ABMS might serve as framework for many future energy system models that integrate many different algorithms. We then discuss practical and theoretical problems in the development, validation and assessment of energy-system-analytical ABMS and conclude with an outlook and recommendations for energy system modellers who consider incorporating ABMS into their modelling toolbox.


Introduction
. The climate impact of fossil fuels emissions makes it necessary to decarbonise the energy system. Energy systems analysis deals with the investigation of the structural elements of the energy system (Möst & Fichter ). One major topic is how to shi energy production towards a sustainable energy supply, which in turn is crucial for reducing carbon emissions. Especially the electricity sector can be made technically largely climate-neutral (see, for example, Scholz and Williams et al. ). .
The transformation of the energy system requires complex, multi-criterial assessments, taking into account environmental concerns while moving towards global sustainability (Pfenninger et al. ). At the core of energy systems analysis are thus various energy models that analyse how to reduce emissions, to secure energy supply and to minimize costs. .
A large variety of modelling approaches exists -a review counts energy system models since (Hall & Buckley ). These approaches can be classified along various dimensions (Hall & Buckley ; Pfenninger et al. ). The two most important dimensions to classify models are perhaps the analytical approach, i.e. whether models are top-down or bottom-up, and the methodology, i.e. whether models use optimisation, simulation or equilibrium techniques. Other di erences include the kind of mathematical approach, the extent of technological and sectoral coverage and the time horizon (Hall & Buckley ). These models have to trade-o various demands, for example a high resolution in time and space, transparency, how to address the complexity of energy systems in enough detail and how to integrate human behaviour (Pfenninger et al. ; Keles et al. ). .
The vast majority of models in energy systems analysis are bottom-up optimisation methods (Fernández-Blanco Carramolino et al. ). Validation of these types of models is not o en seen as important, mostly because they construct system-optimal futures that are inherently self-consistent. However, if one validates optimization models retrospectively, i.e. by trying to replicate real world electricity system transitions of the past, the results are mixed (Trutnevyte ), indicating that long-term equilibria might not manifest themselves in reality, and that current tools to construct energy scenarios need amendments.
. The paper is organized as follows. First, we will give an overview of the usage of ABMS in energy systems analysis. The Methods section describes three ABMS case studies: AMIRIS, EMLab and PowerACE, which are used as primary examples for assessing the benefits of ABMS electricity market models in more detail. The Results section presents a summary of the advantages of ABMS based on these case studies. Finally, the Discussion section focuses both on the disadvantages and limits as well as the challenges that lie ahead for ABMS for energy systems analysis. We conclude with an outlook and recommendations for energy system modellers.

Use of ABMS in Energy Systems Analysis
. Agent-based modelling has been applied to a wide variety of research areas (Helbing ), among them energy systems analysis (Deissenroth et al. ). Abstract descriptions of ABMS are available (Bandini et al. ) and various agent decision making architectures have been used (Balke & Gilbert ). .
Given the predominant analytical tools -optimisation and simulation -our research question is what methodological advantages does the ABMS-approach o er in the specific case of energy systems analysis? .
We will come back later in detail to these advantages when discussing the case studies results. In contrast, several disadvantages have also been associated with ABMS: • there are few rigorously validated complex models (Fagiolo et al. ), • correct specification of realistic behaviour of agents is very hard, • correct interpretation of emergent behaviour of agents is problematic, • generalisation of very context-specific simulations is di icult.
. ABMS is thus particularly suitable for the simulation of networked and complex systems with adaptive agents (Holland & Miller ; Macal & North ; Tesfatsion ). Agents describe encapsulated system entities that are equipped with attributes and rules of behaviour. Complex interrelationships of system entities can be described by explicitly modelling the properties and rules of behaviour of actors depending on interactions and a changing environment (Macal & North ). The ABMS approach therefore allows the integration of complex decision rules into system models. The model behaviour of the system is not given "centrally". On the contrary, with su icient knowledge of decision rules and possible interactions between actors, it is possible to simulate the system bottom up, understand dynamics and investigate possible control and regulation approaches (Chappin ). In addition, ABMS are able to capture emergent phenomena (Bonabeau ), they are particularly flexible (Bonabeau ), and can cope with attributes of complex systems like phase transitions, breaking points, perturbations etc. (Helbing ).
. Especially in the context of energy transition, many policies deal with the problem of how to incentivize new investments in low-carbon technologies and flexibility options at low costs. But when policies are analysed and tested -or even newly designed -it is essential to not only look at costs but also to actors' income, stranded assets and the risks they are exposed to, if they need to refinance their investments (Gross et al. ). Therefore, it is indispensable to take the actors' perspective into account for evaluating the e ectiveness and e iciency of policies.
. ABMS can contribute to the analysis of energy systems with di erent foci. It is not the intention of this article to provide a comprehensive overview of all works of ABMS in energy sciences. For an extensive review, the reader might refer e.g. to Weidlich & Veit ( )  .
Another application of ABMS in energy systems analysis is the simulation of energy trading. Actors estimate the profit and place corresponding bids on a market. Usually, the actors learn to adjust the bids in such a way that the profit is maximized. Examples of energy trading models are shown by Wehinger et al. ( ) and Yousefi et al. ( ).
. Furthermore, ABMS is used to depict the demand side of the energy system, how price-elastic consumers could benefit from demand response (Thimmapuram & Kim ), and how that price-elasticity can be determined in a bottom-up way (Narasimhan et al. ). Perhaps the most important sub-field in energy systems analysis are ABMS electricity market models which examine the development of electricity systems as a whole, taking into account various markets and policy framework conditions, regarding the energy system as a complex adaptive system (Bale et al. ). During a simulation period of up to several decades, agents invest in power plants or flexibility options and try to maximize profits. The design of markets and regulatory framework influences the behaviour of agents and thus the development of the entire electricity system. Examples are discussed in the case studies below. Table presents a short overview of di erences between optimisation -the current most dominant modelling paradigm in energy systems analysis -and agent-based modelling approaches.

Selection of case studies
. In order to assess the benefits and challenges of ABMS electricity market models in more detail, we present several case studies. Selection criteria for the case studies were that the model under investigation had to cover "whole system" elements as agents (not only parts, i.e. they include a representation of conventional power plants and renewable energy generators, the energy exchange and of grid and/or system operators). Furthermore, there had to be an explicit determination of market prices and the consideration of market fundamentals like gas, oil, coal, or CO prices. To be selected, case studies had to have cra ed energy scenarios in a similarly holistic way as other tools used in energy systems analysis (Fernández-Blanco Carramolino et al. ). Thus, like other complete electricity system models, they had to evaluate the cost of energy procurement, to analyze the composition of the energy generation mix, or to derive policy recommendations on an actor or system level. .
Given these selection criteria, the case studies selected are the models PowerACE (KIT/Fraunhofer ISI, see for example Weidlich & Veit , Genoese , Keles et al. and Sensfußet al. ), EMLab-Generation (TU Del , see for example Chappin et al. and Richstein et al. ), and AMIRIS (DLR, see for example Deissenroth et al. and Reeg et al. ).

Case studies .
To clearly structure the comparison, we start with a general classification (Hall & Buckley ) and contrast these models with an optimisation model E M (Sun et al. ; Reeg et al. a).  Table : Comparison of attributes of the three agent-based models discussed as case studies compared to an optimisation model
The German Aerospace Center (DLR) has developed and has been using an agent-based model of the German electricity market (AMIRIS) to analyse and evaluate energy policy instruments and their impact on the actors involved in the simulation context (Deissenroth et al. ; Reeg et al. b). AMIRIS maps the electricity system as a complex socio-technical system -in particular the interdependencies between regulatory framework, actors and markets ( Figure ). . Figure shows the conceptual framework of the AMIRIS model and illustrates the relationships addressed by the simulation. Actors such as plant operators of renewable energy sources (RES) or their direct marketers make decisions constrained by the respective techno-economic regime which encompasses all available technologies and their costs. Coordination mechanisms such as spot market and control energy market determine interactions between actors. These system elements are influenced by the regulatory framework, which in turn influences e.g. the apparent costs of technologies by instruments such as feed-in tari s. .
Within the modelling process, prototypical groups of actors are mapped as individual agents in AMIRIS. Different agent specifications -determined by expert interviews -make it possible to determine the degree of uncertainty and the limited rationality of the actors and to reflect their heterogeneity. In this way, renewable energy marketers can be defined in detail, for example by defining their di erent portfolios of contractually linked renewable energy plants, their respective cost structures, price and performance forecast qualities and capital stocks.
. AMIRIS particularly excels in modelling the micro-behaviour and economic success of heterogeneous actor classes in a system and policy context. The interactions between agents, such as those between marketers and plant operators or between marketers and the electricity exchange determine the success of the actors at micro level. They also have an impact on the macro level of the energy system. Thus, it can be assessed whether the political renewable energy expansion goals can be achieved with certain policy and market designs. One example is the adjustment of premiums that direct marketers grant to renewables plant operators in order to market their electricity (Deissenroth et al. ). As a central variable, the development of electricity prices in the simulation has been validated against historical prices, showing a good fit (Reeg ). If this premium is too low, the renewables plant operators may switch to another direct marketer, who can thus expand their portfolio with larger balancing e ects and with lower specific balancing energy payments. Therefore, the economic success of these agents can be monitored in detail.
The Energy Modelling Laboratory (EMLab) is being developed at the TU Del at the faculty of Technology, Policy and Management (Chappin et al. ; Chappin ). .
It has been developed to study transition pathways that could lead to a decarbonisation of the power sector. The model part described here, EMLab-Generation, covers one or two interconnected market zones which are cleared via market splitting, and a conjoint CO market. Bidding is done by the power generators, who make decisions on di erent time-scales. In the short-term, power producers bid on the electricity market and buy emission certificates (including emission certificate banking). The spot market clears the joint electricity and CO market, assuming that the short-term actions provide equilibrium in electricity and CO prices within the year (Chappin et al. ).
. EMLab particularly excels in explicit policy modeling and exploratory modeling of transition pathways capturing uncertainties. Considering the long-term development of the power sector, the energy producers employ a model within the model to assess if investments in new plant capacities paid o from their business perspective.
They do so by forecasting the merit order of future years and perform a net present value analysis for each generation type. Contrary to optimisation or (partial) equilibrium models, it is not assumed that the power sector is in long-term equilibrium. Agents can make bad investments, errors and do not have perfect foresight.
The system is thus in constant flow. Within this framework, the power plant capacity mix evolves over the course of simulation.
. On a technical level, the model consists of an engine, a data and an agent layer and is based on the AgentSpring agent-based modelling framework. The engine layer is the simulation controller and user interface. The data layer contains the system state as a graph database and is parameterized using XML files to initialize the model, for example policy options, and to determine which agents are present. The agent layer contains all agent behavior and decision rules. The di erent behaviors are separated into roles; agents can have di erent roles which makes the model modular and flexible. Each time step, the agents access the data layer and change the system state through their actions. These changes are stored in the database.
PowerACE has been conjointly developed at the Karlsruhe Institute of Technology (KIT), Fraunhofer ISI and the University of Mannheim (see e.g. Genoese ). The model depicts several electricity markets (day ahead, futures, reserve, capacity), and the key decision makers that operate and trade on these markets, like generation companies, system operators and regulators. .
Power producing agents make supply bids that include variable cost for generation (fuel and emission costs, operation and maintenance), potentially start-up costs, and additional markups if there is market scarcity. The power market is cleared in the short-term on an hourly basis (Keles et al. ).
. Similar to EMLab-generation, supply agents can extend their power plant portfolio once a year. Several decision criteria ensure that no agent can deviate too much from their current market share per time step, and that infrequent price peaks will not lead to an overestimation of potential investment profitability. These agents, too, are bounded in their rationality insofar as they a) evaluate their investment options by forecasting uncertain future potential revenues, and b) only know which power plants exist and which investments and decommissions at the time of decision are known. This is di erent from a perfect foresight "central planner".
. PowerACE particularly excels in representing the interaction between di erent power and capacity markets. Unlike in bottom-up optimisation energy system models, no constraint has to be fulfilled that demand is always met. Therefore, the model does not need to ensure that generation adequacy is always fulfilled, and in principle market failure can be represented (Keles et al. ).

Results
. Here, we present the main advantages of ABMS outlined in the list in Section by means of three case studies. For each of the main methodological challenges, ways to model energy systems as a whole are compared. More specifically, we present the main disadvantages of status-quo modelling approaches and how these can be addressed by ABMS. .
In Table we show as a summary which of the explicit advantages of ABMS are being covered by the three ABMS case studies compared to a classical optimisation approach. In the following, we then give an example on how each one of the advantages is modelled explicitly in one of the ABMS under investigation. We start with agent-decision making, or in fact, with the absence of it. To our knowledge, the individual level of decision-making is not represented in optimisation or equilibrium energy system models on a national or supra-national level. At the moment, national time series like load development or capacity investments are not broken down to individual decisions. As a consequence, causes of di erent development or trends cannot be traced to their roots.
In contrast, ABMS allow modelling the decision-making processes in a fundamental way, as realistically and detailed as feasible. In AMIRIS, agents like plant operators of RES are represented by di erent types and the results of their cumulative decision-making processes determine the total of RESS energy sold at the wholesale power market (Deissenroth et al. ). These decision-making processes can be modelled based on individual investment decisions and aggregated to state level. Thus, factors determining investment decisions like demographic, regional, economic or social drivers are taken into account. .
To illustrate the importance of individual decision making on the micro level compared to a perfect foresight approach on the macro-level used in optimisation modelling approaches, we show the di erences in model outcomes of the AMIRIS and E M optimisation model.
. Figure demonstrates the results of model runs (AMIRIS vs. E M ) with di erent perspectives of storage operation strategies on the electricity price. With activated "system optimal" (SysOpt) strategy in AMIRIS, the plant operators dispatch is almost congruent with the dispatch of the optimisation model E M leading to almost exactly the same electricity prices. But with the "maximizing profits" (MaxProfit) strategy, which is generally used by actors in the real world, the storage operator agent does not smooth the demand curve perfectly as it profits from higher price spreads themselves, resulting in high and lower electricity prices respectively as in the "system optimal" or perfect market case of E M . Furthermore, di erent electricity prices have explicit impacts on revenues of all other market participants as well. . Moreover, due to capacity withheld by the storage agent in order to increase price spreads, peak prices can occur in the "MaxProfit" case leading to load shedding if the modelled system configuration is calculated at the margin which optimisation approaches usually do. This in turn leads to important implications for the security of supply in energy systems under transition.

.
On a macro-level, there are decision-making models that are able to represent prototypical features. However, (in energy systems analysis) there are no sophisticated decision-making models available on an individual level. This is still a limitation for ABMS, but this might be improved in the near future, e.g. by the integration of social practice theory into energy models (Narasimhan et al. ).

Models within models -ABMS as multi-paradigm method .
A second advantage of ABMS is its ability to incorporate or host many di erent models, i.e. to provide a framework for models within models. An ABMS is defined by a set of agents, decision rules and an environment, but no statement is made about the degree of abstraction to be assumed. This is most obvious in its decision rules: O en these consist of simple logics and heuristics such as if-then relationships or case distinctions. However, there is no necessity to make them that simple. .
On the contrary, decision rules can also be based on complex learning algorithms or other representations. This is a central feature of ABMS -it explicitly allows the implementation of entire models in the model itself. Therefore, ABMS o ers the possibility of linking and nesting models. Since an agent's decision logic can be as complex as needed, algorithms and models of any complexity can be integrated, and these behavioural models may obey any paradigm (optimisation, stochastics, and simulation). In general, model couplings can therefore be abstracted as ABMS. Thus, ABMS can be understood as a multi-paradigm method. Multidisciplinary fields of science such as the field of energy systems analysis can therefore use ABMS in many ways and application fields.
. For example, AMIRIS calculates hourly wholesale prices for electricity and the optimal use of power plants using a merit order model -a model within the model (Deissenroth et al. ). Other agents in the system, such as power marketers, access model results and base their operational decisions on it. Hosting more than one paradigm is no problem as mentioned above, since e. g. some time series like the prices on the balance energy market are calculated using a multiple linear regression model, while another model part for investment decisions is derived by machine learning algorithms (e.g. gradient boosting or neural networks, see Frey et al. ). .
Another example is using decision models from psychology when modelling investment behaviour of solar home owners (Klein & Deissenroth ). When trying to replicate the monthly deployment rates of photovoltaics (PV) installations in Germany from -, neo-classical approaches using the exponential utility function u(t) from rational-choice-theory result in average model fits (see Figure , le ). This can be a problem in optimisation and equilibrium models as they usually rely on this theory but do not readily reproduce the stylized features of the investment behaviour of real-world actors. Instead, in ABMS like AMIRIS we can easily integrate decision models from other disciplines like prospect theory from Kahneman & Tversky ( ) and Tversky & Kahneman ( ). Figure : Replication of the monthly deployment rate of PV-Home installations with exponential utility theory (le ) and prospect utility theory (right) (own calculations). On the right, Utility: u(t) = e Kπ(t) with: π(t) = IRR(t) − ρ(t), Deployment rate: d(t) = cxu(t). On the le , with: IRR − internal rate of return, K − life span (here: a), c − conversion factor. .
In prospect theory, expected utilities are not rated in absolute values but relative to the status quo. Potential losses are rated higher than potential gains replicating that preferences of decisions makers are not linear. Integrating this knowledge into a PV-home ownersâĂŹ investment model produces a deployment rate with a much better fit to real-world data (see Figure , right). However, in neo-classical utility theory -being the basis for equilibrium and optimisation models linear preferences of decision makers are one of the core concepts. So, ABMS using models-within-models demonstrates that in order to improve replication real-world behaviour of actors', insights from other scientific disciplines like psychology should play a much more important role in energy system analysis models.

ABMS as a bridge between scientific fields .
A third advantage of ABMS is its inherent interdisciplinary character. ABMS can represent a bridge between di erent scientific disciplines (Axelrod ). Analysing methodologies within the field of ABMS itself is a relatively small area of research, since the specific design of decision rules or internal models, which make up a major part of the validity of an ABMS, depends very much on context. However, the method is used in almost all scientific disciplines (Macal ). A great merit of the ABMS framework is that it provides an approach to system description and modelling that is useful in many areas. Agents (objects) are described with internal properties (attributes) and decision rules (methods); a concept that is strongly related to object-oriented programming. In this sense, ABMS can describe system entities as objects with helpful abstract properties such as inheritance or encapsulation. Since any decision rules can be formulated and interfaces between objects can be defined, any scientific discipline can also exchange information within this framework.

.
For AMIRIS this is also true. Researchers working together on AMIRIS come from several disciplines. In addition to the Department of Energy Systems Analysis at DLR (analysists have a background in physics or engineering science), computer scientists, economists and sociologists have been involved in the development of AMIRIS (Reeg et al. b). .
For the model formulation itself, well-founded actor analyses on the basis of theoretical assumptions derived from market sociological neo-institutionalism have been carried out in order to adequately illustrate qualitative di erences of the actors in terms of strategies, motives and goals (Wassermann et al. ). Via semi-structured expert interviews and document analyses di erent market players in institutions like markets and organisations were questioned about their perspectives. The assumptions of the AMIRIS-model and the classification of prototyped actor groups were then tested and evaluated in expert workshops with representatives of the most important stakeholder groups (Reeg et al. b; Wassermann et al. ).
. Finally, this information was incorporated into the design of the so ware agents in order to investigate the e ect of di erent marketing strategies for RES in the German electricity markets. Empirical validation in the year showed that prospective statements from for di erent RES power plant operator types and their associated (direct) marketers have been quite exact (Reeg ). With the socio-technical di erentiation of actor types in ABMS, research questions about market structure developments or a possible evolution of actors with market power can thus be investigated in contrast to optimisation or equilibrium models.

Policy modelling .
Since ABMS can incorporate individual decision making as well as social processes, such models can be helpful in the design and evaluation process of policies (Gilbert et al. ). Even if point-forecasts of social processes are not possible or even desirable, ABMS can be used to develop insights and improve qualitative reasoning (Chappin et al. ). With the advantage that ceteris paribus starting conditions can be ensured, model experiments can generate "real-world" counterfactuals and the process of policy modelling can enable "learning-bydoing", and can lead to a higher understanding for all stakeholders involved (Gilbert et al. ). In principle, this enables researchers to build a virtual test environment for energy and climate policy impact assessment (Chappin ). .
In AMIRIS, for instance, dynamic distortion e ects of di erent policy instruments for variable RES over time can be investigated, instead of just analysing the system optimal static dispatch like in optimisation models. In order to support renewable investment and feed-in, the variable market premium is calculated as the di erence between the market value of the renewable energy technology (usually averaged over an accounting period of e.g. one month) and a predefined reference (refinancing) tari level. When bidding at the wholesale power market this premium distorts the marginal bid of the plant operator in such a way that, from a micro-economic perspective, it might still be e icient to bid below its own actual marginal costs. Therefore, the link between the micro-and the macro-level of the systems needs to be addressed by the modelling approach. .
The e ect of this distortion can be seen in Figure   . E M calculates the system optimal dispatch from a macro-economic perspective, while AMIRIS considers the individual curtailment incentives induced by the variable market premium from the actors' perspective (Deissenroth et al. ). Due to the above-mentioned incentive, the market driven curtailed power is lower compared to a system driven optimal curtailment. Thus, as the amount of renewable energy in the market changes, so does the amount of energy fed into the grid for RES and all other market participants. This in turn leads to different revenue streams for thermal or storage plant operators, resulting in important insights for future policy or market design requirements. If these real-world policy incentives and their implications on the power system and its actors are not mapped in the models for policy making, severe misinterpretations on the decisions makers side might result. .
Other examples of explicit policy modelling have been performed with EMLab-Generation. These include the interaction between the power market and CO markets (Richstein et al. ), capacity markets (Bhagwat et al. ), and RES support mechanisms (Iychettira et al. ). Interestingly, these cases depict policies on their design element level, i.e. they dissect policy instruments into independent features that may span a policy design space, and which can be evaluated individually and in combination. This allows exploring which elements of intervention are most vital to achieve certain goals (Iychettira et al. ). In equilibrium or optimisation models only a policy goal like certain RES shares or the CO cap can be set as restriction to the optimisation problem, but the policy mechanisms themselves or the way on how to achieve these goals cannot be analysed itself.

Systems in disequilibrium .
Model-based energy systems analysis relies heavily on equilibrium and optimisation models (Fernández-Blanco Carramolino et al. ). These models implicitly assume that the energy market is in long-term equilibrium.
However, the reasons for non-existent or unstable economic equilibria can be manifold (Axtell ). Many energy markets are in (deliberate) imbalance as a transition may actually be politically desired. Actors trade on liberalized energy markets with imperfect information, limited foresight and bounded rationality (Tesfatsion ). In liberalized electricity markets, these imperfections prohibit that market actors invest in adequate generation capacities. Moreover, the system is complex with ever new interactions, legacy power plants did exist before market liberalization, and heterogeneous market actors with diverse motivations emerge with the increasing decentralization of the energy market (Wassermann et al. ). .
A further advantage of using ABMS in energy systems analysis is that rationality and equilibrium conditions do not have to be formulated necessarily in the model (Arthur ). The agents can optimize in a rational way, but do not have to. Since the modelled system properties are created from scratch at runtime, the dynamics of the system can be observed âĂŞ whether and how a balance is formed lies in the nature of the problem and its description and does not necessarily have to be constructed as a model assumption. .
In PowerACE, for example, there is no guarantee that su icient secured capacity is always available in the system, or that a mechanism exists that su icient power plant capacities are installed. In this way, potential conflicts of objectives can be identified and possible mechanisms that can lead to system failure can be identified. .
In a recent study, Keles et al. ( ) evaluate several design options of capacity markets that could potentially ensure that su icient investments in power plants take place. The generation adequacy constraint states that in equilibrium or optimisation models the restriction holds that e. g. the load has to be covered at all times in order to fulfill mathematical requirements for being able to solve the optimisation problem. Only with a model which has no generation adequacy constraint, it is possible to answer questions of how such markets could look like, and how well they would perform. .
Another study using PowerACE focuses on market power in energy markets (Genoese ). While these evaluations might also be tackled with game-theoretical approaches, the particular advantage here is that assertions could be made in a "realistic" scenario in a system context.

.
Yet another study performed by Sensfußet al. ( ) was responsible for bringing greater attention to the socalled 'merit-order e ect', a decline of wholesale market prices with increasing shares of renewable energy sources in the system. This particular case can also serve as an example on how AMBS can uncover emergent phenomena in energy systems.

Evolutionary approach and explorative energy scenarios .
The last example demonstrates a further advantage of ABMS in the context of energy systems analysis. Energy scenario studies distinguish between explorative scenarios, i.e. "what would be possible", and normative or target scenarios in which desirable target states are defined (Dieckho et al. ). Identifying the normative basis of the latter is o en a great challenge (Grunwald ). Many model-based energy scenarios based on optimisation or equilibrium models implicitly represent normative scenarios: In the model formulation, a goal must be specified, for example, via a target function for minimizing the overall system costs, for balancing trade flows, for minimizing emissions, etc. The goal must be set in the model formulation. Moreover, the solution of such an optimisation problem usually covers only a part of the future (e.g. a target year). Therefore, such scenarios are o en not consistent when questions about system dynamics, evolutionary developments, or the consequences for the involved actors are to be answered, i.e. via the path of goal achievement.

.
In contrast, there are explorative scenarios whose great benefit lies in the exploration of future possibility space (Weimer-Jehle et al. ). ABMS reveals the evolutionary perspective when studying the fitness of populations for environmental changes. Since the modelled system "arises" from the bottom up and no external target function restrictions have to be specified, ABMS is a suitable tool for creating explorative energy scenarios apart from stochastic methods and trend extrapolation. In addition, the ability to easily explore the possibility space assists in decision making under uncertainty.

.
EMLab, for example, allows changes in the regulatory framework to be imported as parameters for exploratory scenarios. Policy interventions can be pursued and explored dynamically, step by step, in the model. The impact of policy instruments can be assessed both at the micro level of individual actors and at the macro level of the system, taking into account the complex interdependencies of all interactions (Reeg ). For example, Richstein et al. ( ) evaluates several CO market design options and observes how CO prices and in turn the power generation portfolio would develop over time. The case is also an excellent example of how uncertainties in input assumptions like fuel prices are handled and presented.

Discussion
. The previous sections discussed the advantages of using ABMS. This section is concerned with its challenges, since ABMS face the same trade-o s like other modelling approaches. Therefore, several challenges have to be met. We will concentrate on three of them -validation, correct specification of the agents and setting as well as interpretation of emergent behaviour. .
A first problem is the empirical validation (Fagiolo et al. ). Validation of energy system models is o en seen as a minor issue, mostly because most optimisation models construct (theoretical) system-optimal futures that are due to their mathematical formulation inherently self-consistent as mentioned in the introduction. However, retrospective validation resulted in mixed results, indicating that long-term equilibria might not manifest themselves in reality (see Section . -. ). If ABMS tries to improve understanding energy system transitions, it thus has to be based on empirical findings and models which need to be checked and tested thoroughly. .
It is a well-known problem that sometimes even small parameter changes have large e ects on results. In order to avoid this problem, input data has to be carefully validated so that the basis of observed changes is sound. A complimentary next step is the check for robustness of parameter changes with sensitivity analyses. This problem is not helped by the lack of standard techniques and so ware, making comparisons hard. However, one approach is the systematic design of experiments (DOE) developed by Lorscheid et al. ( ) to increase the transparency of simulation model behaviour and the e ectiveness of reporting simulation results. .
In AMIRIS, for example, the modelled electricity sub-markets can be calibrated and validated as sub-models according to Carley ( ) -with a fundamental approach for the merit-order model and a multiple regression approach for the balancing energy market -using empirical data in the classical sense. The day-ahead spot market model was validated for three very di erent years ( , , and ) in order to be able to make robust statements about electricity price developments (Reeg ).
. Second, it is di icult to determine how to reduce complex problems correctly. This means that the correct specification of the realistic behaviour of agents is very hard (Helbing ). Thus, the two advantages of ABMS -the ability to consider individual decision-making and to combine multiple models within one frameworkare generally hard to implement, because the choice of parameters and validation is di icult. This is a typical trade-o faced by developers of ABMS.

.
Among others, there are three key e orts that try to overcome these problems. First, there is the platform OpenABM (https://www.comses.net/codebases/) which is about dissemination of ABMS, the opening of code and good practice. Unfortunately, two of the three models of the case studies in this paper -namely AMIRIS and PowerACE -as well as the contrasted optimisation model E M are proprietary models so far. For the AMIRIS model an open source strategy is followed since last year, resulting in an open-source framework that is planned to be made public this year.
. Then, there is the ODD-protocol (Grimm et al. , ) providing a standard protocol for making ABMS methodology more rigorous by making model descriptions more complete, thus enhancing reproducibility. Third, new calibration and validation techniques for (agent-based) simulation models have been developed like the 'indirect-calibration' technique by Dosi et al. ( ) and Dosi et al. ( ) and the 'Werker-Brenner' concept -incl. a Bayesian approach if no empirical data is at hand -(Werker & Brenner ) and the 'history-friendly' procedure by Malerba et al. ( ) and Malerba & Orsenigo ( ).
. A third challenge is the analytical interpretation of the results of energy system ABMS. This is best illustrated as follows: While in optimisation models there are usually exactly two possible results (either a solution to the mathematical problem is found or not), ABMS does not provide a "solution" in the classical sense. Similarly, there are no "archetypical" ABMS, because the structure of the decision rules (i.e. the internal models) varies depending on the question. Therefore, the creation of these models cannot be generalized -it must always be approached individually for each specific case. In other words, there is exactly one way for the entities in the system to behave rationally, but there are many ways to behave boundedly rational or even irrationally.
. For example, all the above-mentioned case studies are intended to analyse emergent system behaviour and explorative (policy) scenarios and transition pathways. However, the interaction of agent decisions causes system behaviour in the first place while the system behaviour in return influences the agentsâĂŹ decision making (feed-back-loops) and there is naturally limited information as to whether such decisions are actually made in reality. In fact, this also touches on the validation problem, since it is di icult to validate the results at macro level in absolute numbers.  . The large number of interactions between policy instruments and actors coupled with the increasing decentralisation make the energy system as a whole increasingly complex. Does this mean that energy system models also have to become increasingly complex? The same principle as in scientific modelling in general applies: The model should be kept as simple as possible but as complex as necessary to adequately address the research question. However, since emergent system properties are identified in the best-case scenario, complex ABMS can contribute to a better understanding of the system and reduce complexity at a higher system level. .
Possible fields for further sophistication for energy-system-analytical ABMS are numerous. Flexibility options like storage and demand response are mostly covered in the presented case studies, but grid constraints or cross-border e ects between multiple countries are considered rather crudely or not at all. Furthermore, while the presented models are already quite broad in scope, they only consider the electricity system, disregarding coupling e ects with both heating and transport sectors. Nevertheless, both of which are highly dependent on social processes and individual behaviour (e.g. driving cycles, di usion of heating technology, see Grunwald et al. ). ABMS might hence be a natural way to move forward in integrating these research areas into wider energy system models.

Conclusion
. The purpose of this paper was to give an overview on how the particular advantages of agent-based modelling can be used in the research field of energy systems analysis and in energy scenario studies. We identified that an in-depth analysis of the transformation of the energy system requires the actors involved and their interdependencies to be studied in an adequate degree of resolution. Using three case study electricity market models as example (AMIRIS, EMLab-Generation and PowerACE), we assert that this actor perspective makes it possible to draw attention to otherwise under-studied possible system conflicts -like market disequilibria or system failure -, to integrate decisions and other behavioural aspects into models and to explore unknown adaptations and emergent phenomena. ABMS are one tool to achieve that.
. We showed that ABMS allows di erent model paradigms -such as optimisation, simulation or stochastic models -to be integrated side by side. Equilibrium conditions do not necessarily have to be met. The object orientation and the framework of abstraction enable interdisciplinary research. The evolutionary approach of ABMS can increase the consistency of exploratory energy scenarios. The method allows interventions to be read directly into the actors' scope of action as scenario parameters. When developing agent-based models are used in energy systems analysis, the following points should be considered: . Improve validation: • Include stakeholder feedback early on, especially if business or policy recommendations shall be derived.
• In the beginning of the model development, focus on easy-to-validate macro-observables like day ahead electricity market prices or the development of capacities of technologies over time.
• Start with validation on a sub-model level. This step will ensure that validation e orts are modular and manageable, and provide more confidence in possible emergent system behavior that may arise from sub-model interactions.
. One example is the development of AMIRIS: to parameterise the agents' attributes, various expert interviews have been conducted to exactly calibrate their real-world perception and behaviour. ABMS can ensure that energy systems analysis focuses more on empirical research and thorough validation.
. Improve model consistency: • Avoid ad-hoc modelling decisions. While there is a strong KISS ("keep it simple, stupid") push to keep models parsimonious and manageable in the ABMS community, unsubstantiated decisions will be detrimental in later development phases.
• Object oriented programming (OOP) is the natural way to express agent-based models, since OOP formulates objects as a set of variables and methods. This is analogous to agents, which are defined by a set of attributes and behaviors.
• In the beginning, if appropriate, use existing ABMS frameworks (e.g. Repast, North et al. ) that take care of otherwise tedious programming tasks like time scheduling.
Again, AMIRIS is a case in point: it has been developed in Java, strictly following OOP-principles, using Repast.

.
. Improve transparency: • Use networks (OpenABM), common codes (ODD protocol) and systematic design of experiments (DOE) to ensure transparency and reproducibility.
• Transparency in energy systems analysis and energy scenario studies is important, and even more so for ABMS, since all modelling steps and behaviors have to be well-founded. As a guideline for model transparency, consider e.g. the checklist provided by Cao et al. ( ).
• For the same transparency reasons, consider using open data and open sourcing of your code (Pfenninger et al. ; Morrison ), and/or using already published open sourced models like EMLab-Generation.
One example is the development of EMLab which demonstrates how transparency can be enhanced by making it open source.
. ABMS could become increasingly widespread in the energy sciences in the future and can make valuable contributions to technology and policy assessment. We expect that a higher data availability, improved data management tools, and easy-to-use machine learning algorithms will make this endeavour even more fruitful in the future. Models have been getting more complex, with more specialisation taking place. Therefore, ABMS might serve as the framework for many future energy system models that integrate many di erent algorithms and modelling approaches.