Brian Heath, Raymond Hill and Frank Ciarallo (2009)

A Survey of Agent-Based Modeling Practices (January 1998 to July 2008)

Journal of Artificial Societies and Social Simulation 12 (4) 9
<https://www.jasss.org/12/4/9.html>

For information about citing this article, click here

Received: 26-Nov-2008    Accepted: 22-Aug-2009    Published: 31-Oct-2009

Abstract

In the 1990s, Agent-Based Modeling (ABM) began gaining popularity and represents a departure from the more classical simulation approaches. This departure, its recent development and its increasing application by non-traditional simulation disciplines indicates the need to continuously assess the current state of ABM and identify opportunities for improvement. To begin to satisfy this need, we surveyed and collected data from 279 articles from 92 unique publication outlets in which the authors had constructed and analyzed an agent-based model. From this large data set we establish the current practice of ABM in terms of year of publication, field of study, simulation software used, purpose of the simulation, acceptable validation criteria, validation techniques and complete description of the simulation. Based on the current practice we discuss six improvements needed to advance ABM as an analysis tool. These improvements include the development of ABM specific tools that are independent of software, the development of ABM as an independent discipline with a common language that extends across domains, the establishment of expectations for ABM that match their intended purposes, the requirement of complete descriptions of the simulation so others can independently replicate the results, the requirement that all models be completely validated and the development and application of statistical and non-statistical validation techniques specifically for ABM.

Keywords:

Agent-Based Modeling, Survey, Current Practices, Simulation Validation, Simulation Purpose

Introduction

1.1

Emerging from the fields of Complexity, Chaos, Cybernetics, Cellular Automata and Computers, the Agent-Based Modeling (ABM) simulation paradigm began gaining popularity in the 1990s and represents a departure from the more classical simulation approaches such as the discrete-event simulation paradigm (Heath and Hill 2009). A primary reason for the popularity of ABM and its departure from other simulation paradigms is that ABM can simulate and help examine organized complex systems (OCS). This means the ABM paradigm can represent large systems consisting of many subsystem interactions. These systems are typically characterized as being unpredictable, decentralized and nearly decomposable. Although computer simulation as an analytical tool has been around since the advent of computers, the ability of the ABM paradigm to simulate complex systems has moved into a breadth of fields ranging from engineering to mathematics to social science and economics where sometimes for the first time analysts can utilize simulation to explore these complex systems at a level of detail that was difficult or impossible to previously obtain.

What Seems to be Holding Back ABM?

1.2

Due to its characteristics and abilities, some claim that ABM represents a revolution in modeling and simulation. However, this statement is based primarily on the potential of ABM rather than the current results (Bankes 2002). One reason for the lack of meaningful results sometimes emanating from ABM studies, in general, is due to the type of complex systems that ABM is used to simulate and explore. Traditionally these types of systems are difficult to analyze given their non-linear behavior and size (Casti 1995). Nevertheless, there is no reason why analyzing these complex systems using ABM should not eventually always produce meaningful, model-based results. Systems that are large and difficult can be understood. History gives many examples of problems seemed nearly impossible to solve, but when given the proper tools scientists found solutions. For example, at one point we did not understand why an apple fell to the ground from a tree. Newton and others were able to develop theories and tools that helped them not only explain but also predict the behavior of the falling apple. By extension meaningful results regarding these complex systems will be consistently gained when the proper tools and models are in place, and ABM is, at least for the moment, the most suitable tool for analyzing these types of the complex systems.

1.3

ABM as a modeling technique and paradigm is really still in development. This statement is generally supported by the relatively recent development and popularity of the paradigm, its departure from traditional simulation paradigms and the "new to simulation" fields that are using ABM to study OCS. Whenever a new tool or technique emerges time is needed to flush out the details of its application, capability and limitations. For ABM, researchers must determine what simulation techniques/philosophies are appropriate and what new techniques/philosophies are needed specifically for ABM. Since ABM is being used in fields that traditionally have not used simulation, it will take some time for these researchers to hone their simulation skills and to effectively develop appropriate analytic models for their domain.

1.4

Two key things are needed to mature the ABM paradigm. First, techniques, philosophies and methods need to be developed specifically for ABM and distinguished from other simulation techniques, philosophies and methods. A fair amount of research in this area has already been done (for a few examples see Axtell, Axelrod, Epstein and Cohen 1996; Bonabeau 2002; Epstein and Axtell 1996; Epstein 1999; Macal and North 2006; Miller and Page 2007; North and Macal 2007). Then, teaching of ABM techniques, philosophies and methods must improve so those using ABM can build effective models. These key things are independent of the specific scientific domain of interest.

What is the Current State of ABM?

1.5

Specifically what do ABM researchers need to focus on? What specific problems exist in the ABM paradigm domain that are keeping ABM from reaching its full potential? To help answer this question, we present a comprehensive review of the state of ABM to determine research directions, needs and opportunities. We surveyed 279 published articles in which agent-based models were built and used for analysis. The survey helps to describe the last 10 years of the field's development as well as its current state of the art.

1.6

The remainder of the article is divided into four sections. Section 2 discusses the general survey methodology and provides justification for the categorization strategy employed. Section 3 discusses the results from the survey. Section 4 discusses the implications the survey results have on identifying the research opportunities in the ABM paradigm. Finally, Section 5 summarizes and concludes the article.

Methodology

2.1

Throughout the survey process every attempt was made to obtain ABM articles in an unbiased manner. However, the ABM literature is vast and covers many scientific domains of interest. Thus, it is quite likely, despite our efforts, that this survey will miss some domains using ABM. However, the issues and challenges associated with ABM are quite domain independent. Thus, our survey provides at least a starting point in determining the state of the art and the common research challenges.

2.2

It should also be noted that our research is primarily concerned with the philosophy of ABM and simulation and how these simulation models are developed and validated. As a result, the framework and methodology we developed is not concerned with the type and/or classification of the agent. In our framework an agent is an abstract representation of a distributed autonomous entity and the behaviors and attributes of the agent is the modeler's decision. While we do not consider an agent's characteristics in this survey, we encourage those interested in extending our survey to evaluate the characteristics of the agents in these models.

Collection of the Sample

2.3

The survey methodology involved obtaining a large sample of published works where the authors built some agent-based models and reported their analytical findings. There are several advantages to this approach. The first is that it more accurately reflects what simulationists are concerned with, the techniques they are using and what the publication outlets and reviewers deem acceptable practice. This type of information directly represents the main thoughts, feelings and techniques used by those constructing 'acceptable' agent-based models. This approach can also help capture trends by tracking when the works were published. Finally, this approach is less subjective to author opinion and biases. A good representative sample of works was collected and a well defined categorization scheme implemented to objectively capture the techniques used by the simulationists. We did not focus on articles discussing specific techniques or methods as this focus would yield limited information on ABM trends, issues and challenges.

2.4

The works included in this survey discuss development of an agent-based model, produced results, were published by a peer-reviewed outlet and were published within an approximate 10 year time frame (January 1, 1998 to July 20, 2008). Furthermore, we primarily focused on ABM where all decisions and actions were defined prior to execution of the computer simulation. Using this criteria, 279 works were obtain from a variety of outlets. The primary source used to collect the samples was OhioLINK's Electronic Journal Center. OhioLINK is a consortium of 89 Ohio colleges and universities as well as the State of Ohio Library. Specifically, the Electronic Journal Center (EJC) is one service of OhioLINK that was established in 1998 and is an online full-text collection of over 7,750 journals from many different disciplines (The ohio library and information network 2008). Using the EJC, the search "agent-based" provided the links to the works obtained.

2.5

In addition to the EJC, other sources were used to obtain samples from fields that are not as well represented within the EJC. One such source is the Journal of Artificial Societies and Social Simulation (JASSS). JASSS is one of the few journals dedicated to society and social computer simulations. All JASSS articles that met the search criteria were also included in the survey sample. One field that was noticeably missing in the original EJC sample was military applications of ABM. To incorporate some of the military work involving ABM, Master's Theses from the Naval Postgraduate School in Monterey, CA and the Air Force Institute of Technology in Dayton, OH were also included into the survey. Although not published journal articles, they are publicly available, reviewed and deemed to be acceptable enough to award students with a Master Degree. These works not only met the survey criteria but often provide much more detail about their models since they are not restricted by page limits. Appropriate articles from the Winter Simulation Conference (WSC) were included to capture ongoing work since WSC is one of the primary simulation conferences in the world and these papers particularly represent how more traditional simulationists utilized ABM. Note WSC articles are also reviewed before being published in the proceedings. Finally, duplicate works were excluded. Duplicate works would include papers using a common model but for differing purposes. Removing duplicates helped avoid skewing the survey results.

2.6

Altogether, a total of 279 samples were collected from 92 unique publication outlets from the 10 year sampling period. The distribution of the number of articles per year is shown in Figure 1 to provide a measure that is not influenced by variations in journal publication and topic schedules. In general, this distribution appears appropriate; it reflects what is intuitively expected. Since ABM has become more popular over time, there should be an increasing trend in the number of articles per year. Clearly the sample reflects this. Thus, this sample appears to be a relatively decent representation of the population. Note 2008 data only includes articles available before July 20, 2008. Projections of final 2008 number are not made since the survey focus is not on projecting ABM growth but on capturing ABM trends and research challenges.

Figure 1. Number of Articles per Year in the Sample

2.7

The breakdown of the number of articles per publication outlet is shown in Figure 2. Figure 2 indicates that the majority of the samples come from publication outlets with four or less articles in the sample. This means that many different outlets are accepting ABM articles, a nice trend for the field. Figure 2 also shows that the sample represents a wide variety of topics including military applications, biology, economic, social science, business, complexity theory and simulation. This topic diversity in the range of outlets further supports our claim that this sample is a meaningful representation of the ABM field. A complete list of the 279 works included in this sample is found in the Appendix.

Categorization and Data Collection Strategy

2.8

With a reasonable sample of literature, the next step was determining an appropriate categorization and data collection strategy that would give insight into the progression and current state of ABM. Some data was standard. For example, the author(s), publication outlet, general topic and year of publication were easily recorded from each sample. These data do not provide the insight needed into many of the techniques, methods and philosophies of the field. Therefore, other data were employed.

Software

2.9

Software data included whether general software packages or native languages were used to realize the agent-based model. If authors mentioned a software package, for example the ABM was built using Java or C++, the software package name used was recorded. If the authors said they programmed their model directly, for example by using Java or C++, then the programming language was recorded. This type of information gives insight into the popularity of particular software packages and helps to determine how modelers are creating their agent-based models.

Figure 2. Articles per Publication Outlet in the Sample

Field of Study

2.10

Accurate information regarding the author's domain or field of study helps infer whether different fields of study have different ABM practices. Each article was deemed from a field of study such as economics, social science, military, biology and public policy. The field applied was judged to best describe the topic of the model. Naturally, there were instances where a model could exist in multiple fields of study; only the best describing field was used. This categorization strategy gives insight into the differences and similarities between and within domains that are using ABM.

Reference to the Complete Model

2.11

Science and engineering is the process of developing models/theories of real systems for particular purposes. ABM is just a technique that aids science and engineering in gaining insight into the real world and how the real world behaves. As with any science or engineering model, ABM results must be independently replicated for the results to be considered scientifically valuable. Each article was reviewed to see if they provided some reference the complete model, or at least some way to obtain a complete description of the model such that it can be independently replicated.

Validation Technique(s)

2.12

To gain insight into a real system by using a model, that model must be an accurate representation of that real system. Since all models are incorrect representations of reality (Ashby 1970; Stanislaw 1986), the emphasis of simulation validation is ensuring the model is an appropriate representation of the real system of interest for a given set of objectives (Balci 1998; Banks, Carson, Nelson and Nicol 2001; Law 2007; Sargent 2005; Zeigler, Praehofer and Kim 2000).

2.13

There are two aspects to consider when considering the validation of ABM, or any simulation model. The first aspect is the piece of the simulation model being validated. There are many pieces of a simulation. For simplicity this survey examined validation of the two most basic pieces: the conceptual model and the simulation output. There are many different representations of how to build a good simulation model (Balci 1998; Banks et al. 2001; Law 2007; Sargent 2005; Zeigler et al. 2000). Figure 3 shows a simplified simulation development process. Notice there are two rounds of validation, each validating different parts of the simulation. The first round validates the conceptual model. The conceptual model is the abstracted model of the real system. It relies upon known system theories, drives model development and dictates the variety of assumptions required in any model abstraction process (Banks et al. 2001; Morris 1967; Robinson 2008;Robinson 2006; Sargent 2005). The conceptual model forms the foundation of an ABM model; an invalid conceptual model indicates the model may not be an appropriate representation of reality. The second round validates results of the simulation against results from the real system. For a model to be completely valid, it must be validated both conceptually and operationally. For the survey, each article in the sample was examined to check whether conceptual and operational validation of the model occurred.

Figure 3. A Simplified Simulation Development Process

2.14

The second aspect is the techniques used to validate each piece of the simulation model. Within the simulation domain are many different validation techniques (for several examples see Balci 1998). This survey partitioned these techniques into statistical and non-statistical techniques. Statistical techniques use formal statistical hypothesis tests to check the validity of some piece of the model. Non-statistical techniques do not use formal statistical hypothesis tests, but rely instead on more qualitative assessments such as expert opinion. For the survey, each piece (conceptual and operational) of a model was examined to determine if a statistical technique, a non-statistical technique, some mixture, or no validation technique was performed on that piece of the model.

2.15

All validation techniques involve the evaluator subjectivity in determining whether the simulation is a valid representation of reality. Some say that validation, which implies truth, should really be called sanctioning (Winsberg 1999), which implies more of a process in which evaluators agree that a model is close enough for useful purpose. For the survey, an article was reviewed and data recorded when a validation technique was performed within the framework established. No measure was assigned pertaining to the quality of the validation process as such a measure would be inherently biased based on the authors like or dislike of the technique.

Purpose of the Simulation

2.16

Defining the purpose of the model can be subjective and ambiguous. However, knowing a model's purpose allows conjectures regarding how different ABM techniques and model philosophies support differing ABM purposes. To reduce subjectivity and ambiguity another framework describing the different simulation purposes was established. This framework is based upon the level of understanding associated with the system of interest and more recent research concerning the role that simulation and modeling plays in modern science today (Kuppers, Lenhard and Shinn 2006).

Figure 4. Purpose of the Simulation

2.17

Figure 4 relates our three defined roles or purposes of the simulation (Generator, Mediator and Predictor) with the level of understanding known about the real system. When the system is well understood the simulation is called a Predictor; it is used like a calculator to provide clear and concise predictions about the system. An example of this could be a simple queuing system or a very well understood assembly line activity. As less is understood about the real system, the simulation moves toward a Mediator role. In this role the simulation provides insight into the system, but is not a complete representation of how that system actually behaves. When using a simulation as a Mediator, theories can be put forth and tested and the simulation can be subsequently improved. For more about simulations and models as Mediators see Morrison and Morgan (1999). When little is known about the real system of interest, the simulation takes on the role of a Generator; the simulation acts as a generator of hypotheses and theories about how the real system behaves. As a Generator, a simulation serves the same purpose as other mediums where theories and hypotheses are proposed (Ashby 1970).

2.18

These three roles are not mutually exclusive. Figure 4 shows that these roles exist on a continuum meaning simulations can exist between two different roles. For this survey, the model was recorded into the dominant role. For example, if a model was deemed 40% mediator and 60% generator, the model was classified as a Generator. For the survey, the following definitions were used:

• A Generator is a simulation where little is known about the system of interest and it is used primarily to determine if a given conceptual model/theory is capable of generating observed behavior of the system.
• A Mediator is a simulation where the system is moderately understood and it is used primarily to establish the capability of the conceptual model to represent the system and to then gain some insight into the system's characteristics and behaviors.
• A Predictor is a simulation where the system is well understood and it is used primarily to estimate or predict a system's behavior with little time spent on ensuring that the conceptual model is correct because this aspect of the simulation has already been established.

Results

3.1

This section provides the main results compiled from the survey. A further analysis of each topic and the implication of the results is discussed in the next section.

Software

3.2

Figure 5 displays a summary of the software packages or programming languages used. Overall, a total 68 unique software packages or programming languages were referenced with many of them (22.6%) being referenced less than three times overall. It is clear that both ABM specific software packages and generic programming languages are being used and that the most popular software packages are ones that are public domain. In fact, only AnyLogic and Matlab are commercial packages listed in Figure 5. A striking result is that 104 articles (37.3%) did not provide any details on what package or programming language was used to construct and execute the simulation. Also, there is no clear evidence in the data suggesting that software packages or programming languages are more popular.

Figure 5. Histogram of Top Used Software

Figure 6. Breakdown of Articles by Field

Fields of Study

3.3

The breakdown of the articles by domain is displayed in Figure 6. In the sample the three most popular fields of study using ABM are economics, social science and biology. In general, the fields of study in the survey show ABM being used by fields whose systems involve many interacting autonomous entities. This supports the fundamental belief that ABM is good at modeling and analyzing interacting entities. Although the majority of the fields of study in the survey are not traditional scientific disciplines, there are still a significant number of traditional disciplines using ABM. This supports the wide appeal of ABM as a methodology.

Purpose of the Simulation

3.4

In terms of model purpose, 111 (39.8%) of the models surveyed were Generators, 168 (60.2%) were Mediators and 0 (0.0%) were Predictors. This confirms the belief that agent-based models are used primarily to gain insight into the system of interest. It is interesting to note an almost equal number of generators and mediators. Simulationists are not only using agent-based models to generate theories about a system's behaviors but as a mediating instrument to capture certain behaviors of the system and to characterize how the system may behave under certain scenarios. This general characteristic of how agent-based models are used is relatively constant over the last 10 years, as shown in Figure 7.

Figure 7. Simulation Purpose by Year

3.5

There does appear to be differing model purposes by domain of interest. As shown in Figure 8, the only domains where the majority of the models were generators are social science (66.2%) and economics (65.8%). The domains with the lowest number of generator models are business (0.0%), public policy (4.3%) and the military (5.6%). These differences are reasonable. Social science and economics are still relatively new and in the process of developing theories about how their systems of interest operate. Thus, using agent-based models as generators allows them to explore hypotheses and ideas that are not easily manipulated using other theory generating techniques. Conversely, it makes sense that business, public policy and the military are more interested in mediating models that can be used to gain insight into the system to in order exploit some aspect of the system's characteristics.

Reference to the Complete Model

3.6

Only 44 (15.8%) of the articles surveyed gave a reference for the reader to access or replicate the model. This indicates that the majority of the authors, publication outlets and reviewers did not deem it necessary to allow independent access to the models. This trend appears consistently over the last 10 years as shown in Figure 9.

Figure 8. Simulation Purpose by Field

Figure 9. Reference to Complete Model by Year

3.7

Figure 10 depicts model references by domain. The domains with the most references to the complete model are social science (26.5%) and economics (19.0%), while those with the least are the military (2.8%) and business (0.0%). These results are again reasonable. Social science and economics are scientific fields interested in theory development, so they are more likely to provide their model to others. The military and business fields are more secretive (e.g., security, competitive advantage) so less they are less willing to share their complete model.

Figure 10. Reference to Complete Model by Field

Figure 11. Reference to the Complete Model by Purpose

3.8

The defined purpose of the simulation generally has little impact on the whether the complete model is referenced. Figure 11 indicates that only 21.6% of generator models and only 11.9% of mediator models gave references to the complete model. It may seem that this is a significant difference, but the correlation between purpose and domain better explains the difference depicted in Figure 11.

Validation (Not Considering Technique)

3.9

We next focus on whether a model was conceptually validated, operationally validated, conceptually and operationally validated or not validated at all. Figure 12 indicates that 29% of the models were not validated, 17% only had their conceptual model validated, 19% only operationally validated their model, and 35% validated their model both conceptually and operationally. A reasonable position is that a model is only validated, or sanctioned, when it is both conceptually and operationally validated. In this case, at least 65% of the models in the survey were incompletely validated. This is alarming since most outlets for scientific publication insist on some level of model validation.

Figure 12. Validation of the Simulations (Not Considering Technique)

3.10

Emphasis on model validation does seem to be changing. As seen in Figure 12, the percentage of models not completely validated is declining. The difference between the beginning and the end of the 10 year period is distinct and shows that the field is improving in terms of completely validating their models. However, between 2005 and 2008 the number of articles that both conceptually and operationally validate their model remains relatively constant and averages to just under 43%.

Figure 13. Validation by Year

Figure 14. Validation by Field of Study

3.11

Breaking down model validation by domain reveals that some fields are more concerned with validation than others. As shown in Figure 14, the fields with the highest percentage of completely validated models are ecology (77.8%) and biology (70.0%) and the fields with the lowest percentage of validated models are military (16.7%), economics (20.3%) and social science (27.9%). A reasonable conjecture regarding the differences is their scientific tradition. However, while economics and social science are relatively new fields and not as well connected to the classical scientific tradition, the military has a long history of using computer simulation and their issues with simulation validation are well documented (Davis and Blumenthal 1991). Thus, this aspect of validation for military agent-based models is somewhat surprising.

3.12

There does appear to be a relationship between the purpose of the simulation and validation efforts. In Figure 15, 11.7% of generator models were completely validated while 51.2% of mediator models were completely validated. Since generator models are based on systems that are less understood, these models are harder to validate because there is less information available about the system. Conversely, more 'validation activities' should occur for mediator models because more information is known about the system being modeled.

Figure 15. Validation by Purpose

Validation Techniques

3.13

Of the models validated in some way, 0.5% used only statistical validation techniques, 95.0% used only non-statistical validation techniques and 4.5% used a combination of statistical and non-statistical validation techniques. Thus, it appears that in ABM the primary validation techniques employed are expert opinion and qualitative comparisons of behaviors. The statistical validation techniques often taught in basic simulation courses are not as popular. This result may be due in part to difficulties in capturing statistics from the ABM simulation and the system being challenging to analyze due to nonlinear output. When examining validation techniques by year, as shown in Figure 16, a trend shows that a decreasing number of models not using any validation technique. For the most part the use of non-statistical validation techniques are being employed.

Figure 16. Validation Technique by Year

3.14

Figure 17 breaks out the validation technique used by field and again the most commonly used are non-statistical validation techniques, but with no strong relationship between validation technique and the field of study. Figure 18 displays validation techniques by model purpose. These results show thatÊnon-statistical techniques areÊthe most popular validation techniques. For mediator models there is a slightly higher use of statistical validation techniques; this is expected since more is understood about the real system.

Figure 17. Validation Technique by Field

Figure 18. Validation Technique by Purpose

Discussion

4.1

These survey results provide information about the development and current state of ABM. From this data research directions, needs and opportunities are identified. While there are many different implications these results may have depending upon a researcher's interest, in this section we discuss just some of the most important implications these results have on developing and maturing the field of ABM.

Software and Verification

4.2

With 68 unique software packages or programming languages used to build and execute the surveyed simulations it is clear that there are many ways that a model can be represented in a computer simulation. This variety can most likely be attributed to the background of the simulationist, programmer or non-programmer. Thus, no software package or programming language will likely ever become the standard in building agent-based models. This means that tools developed to aid in constructing and documenting agent-based models as well as teaching techniques, should not be specifically geared towards a particular software package or programming language. Instead, development and documentation tools and teaching techniques should be independent of software and programming languages and should focus on the general issues involved in constructing and executing an agent-based model while emphasizing the fundamental methods and issues of building a simulation. This idea is also reflected in many computer science higher education standards, such as ABET in the US.

4.3

There are also implications for reviewers and evaluators of agent-based models where there is a lack of common software package. ABM evaluators must understand basic simulation programming techniques. Because agent-based models can address a wide range of problems it is essential that researchers provide sufficient discussion of their application for the evaluator to assess the realization of the system abstraction into the simulation. Publication outlets, and their reviewers, do not seem to be requiring such detail.

Addressing the Many Fields of Study or Creating a New One

4.4

ABM is connecting diverse fields. The fields of biology, business, ecology, economics, the military, public policy, social science and traffic, among others, all use ABM. These diverse fields are trying to understand complex systems and are using ABM as a one common tool. If it holds that complex systems generally have similar properties, then these diverse fields should be actively sharing insights about their complex systems. Naturally, ABM publications promote sharing. However after reviewing the surveyed articles it is clear that each field has developed their own ABM terminology to describe techniques, applications and results, have their own ABM standards and their own ABM philosophies.

4.5

Observing the growth of multiple ABM theories points to a fundamental need for ABM to be studied as an independent discipline, a subset of simulation, so that standard ABM techniques, practices, philosophies and methodologies can be established. A common ABM theory means all disciplines could speak the same ABM language and develop techniques and models based on proven and accepted approaches. Similar measures of standardization supporting these same ideas are also found in the food, drug, manufacturing and collegiate industries. To gauge the depth of this division one only needs to realize that even the definition of an agent is not clear, depends upon who is writing and can vary widely. Bringing together the field of ABM will result in a better analysis tool for every field of study and considering that some believe that ABM and simulation is becoming the epistemological engine of our time (Kuppers et al. 2006) it is important that such standards be established.

Redefining the Meaning of Results by Purpose

4.6

Those considering ABM, as a simulationist or evaluator, must re-consider how they define results of the model. ABM naysayers argue the models do not produce results while this survey found otherwise. This contradiction is likely the result of different definitions of the term 'result' and the different expectations associated with simulation. There is a general belief that simulations should produce clear predictions and estimations of system behaviors to be considered successful. This expectation fits well with the long standing ability of systems simulated in the discrete-event simulation paradigm, but it does not necessarily fit well with the kind of systems that an ABM simulates.

4.7

It could be conjectured that the majority of simulations developed throughout history are of fairly well understood systems and that their general purpose was to provide some estimation or prediction about the behaviors of a particular system. In other words, the majority of past simulations are held up to predictor expectations. But from the survey it is clear that ABMs are being used as mediators (60.2%) and generators (39.8%). This survey finds that ABM is living up to its potential as a revolution in modeling and simulation by extending the applicability of simulation to new fields of studies and complex system abstractions. As the use of ABM expands and complex systems become more understood, we conjecture that eventually the ability of an agent-based model to provide predictions will improve as more is understood about the complex systems they are simulating.

Providing a Reference to the Complete Model

4.8

A low value of 15.8% of the surveyed articles provided a reference to the complete model. If the reader or evaluator does not have access to a complete model, how can they verify the results produced? In other sciences, such shortfalls would give the article little or no chance of publication. This prompts the question of why such limited model descriptions are allowed?

4.9

There are probably several main reasons why references to the complete model are not considered an important part of many ABM articles. The first is that simulationists may not be willing or able, due to propriety issues, to provide their complete model to the public. This is not likely to change. However, a potential remedy to this problem is to require authors to provide enough of a description of the model such that independent evaluators can reconstruct the model. Such detail does allow others to quickly review the logic and execution of the model and reproduce it in their choice of software package or programming language. For this to occur some model describing tools or diagrams from the fields of systems engineering or computer science may help by providing rich and complete descriptions of these models sufficient for independent evaluation and replication.

4.10

An ABM developmental tool offers other benefits to the ABM community. First, methods could help enforce good simulation programming practices by emphasizing particular aspects of the model that must be described. This information aids those building the model and provides evaluators a way to evaluate and validate every model. The tool could also be used as a teaching aid to help researchers build more effective models. This could mean more effective ABM employment resulting in improved understanding of modern complex systems.

Complete Validation is Required for Every Model

4.11

It could be argued that validation is one of the most important aspects of model building because it is the only means that provides some evidence that a model can be used for a particular purpose. Without validation a model cannot be said to be representative of anything real. However, 65% of the surveyed articles were not completely validated. This is a practice that is not acceptable in other sciences and should no longer be acceptable in ABM practice and in publications associated with ABM. One of the other potential reasons why models are not being completely validated is that the authors may consider that just conceptually or operationally validating their model is good enough. This survey found that overall 36% (the majority) of the articles only validated one aspect of the model. Our position is that both conceptual and operational validity are required for complete validity.

4.12

If a model is only conceptually validated, then it unknown if that model will produce correct output results. For example, consider a scientific experiment. In this experiment a hypothesis about some macro-level behavior is made based on some conceptual model that appears valid based on what is known about the system. However, when the experiment is performed the hypothesis is rejected because it did not properly predict the macro-level behavior. The operational-level hypothesis based on the conceptual model is invalid even though the conceptual model of the hypothesis appears valid prior to the experiment.

4.13

Conversely, if a model is only operationally validated, then it is unknown whether that model is based on any appropriate representation of reality. For example, consider a simulation of a standard single server queuing model where the objective is to achieve the theoretical performance (Jensen and Bard 2003). Typical performance measures are the average time in the queue or system throughput. The standard approach to build this simulation is to observe the real system, measure arrival rates, measure server processing times and then build a realistic representation of the system using some discrete event simulation packages. It is expected that the simulation will behave like the real system and therefore the simulation would be both conceptually and operationally validated. Now consider using an ABM simulation based on reproducing bugs to model the queue. In this simulation, the bugs move about their environment looking for food and reproduce with other bugs, much like those of the Sugarscape ABM (Epstein and Axtell 1996). Key measures about the bugs, such as lifespan and birthrate, are mapped to the goal performance measures of the single server queuing model. Parameters concerning the bugs and their environment are adjusted using some algorithm until the simulation's performance measures match the expected queuing performance measures. The bug model is then deemed useful for queuing analysis, even though it is unlikely that anyone would accept this conceptual construct as a queuing system construct. Although this is an extreme example, without complete validation the effectiveness and ability of the model to represent a system is unknown.

4.14

The importance of validation in science and simulation cannot be overstated. Not enough scientists using ABM as an analysis tool are properly validating and documenting their model. It is absolutely essential that all models be completely validated and that the articles associated with them clearly document the validation techniques used and the validation results. Likewise, publication outlets and reviewers should be stringent in their validation requirements in order to produce better models and to advance not only their field of interest but also the field of ABM.

Statistical vs Non-Statistical Validation Techniques

4.15

It may be surprising that so few of the articles surveyed used statistical validation techniques given the widespread use of statistical validation techniques in other simulation paradigms. Two conjectured reasons for this are that ABM is used to simulate systems whose output are not conducive to statistical analysis and that those building and evaluating these agent-based models have validation criteria that differs from validation criteria used in other simulation paradigms. The surveyed models are primarily being used in non-traditional simulation fields that may not be as influenced by the statistical validation techniques of other simulation paradigms. Further, the surveyed models generally reflect their use for generator and mediator purposes, as opposed to predictor purposes that are more focused on matching system outputs and therefore more conducive to statistical analysis.

4.16

The popularity of non-statistical validation techniques in ABM highlights potential research opportunities. First, the effectiveness of statistical validation techniques for ABM needs to be further explored and evaluated. Second, there is a need for new statistical validation and data collection techniques specifically for ABM. Researchers must then convey the usefulness of statistical validation techniques for these agent-based models. Unlike non-statistical techniques, which requires evaluator knowledge of the domain modeled, statistical techniques do not require complete domain knowledge about the system or field for the evaluator to judge the validity of the model. Finally, the field must develop more standardized and comprehensive non-statistical validation techniques specifically for ABM. Fundamentally, by developing and discussing the use of both statistical and non-statistical validation techniques for ABM, the resulting models will be validated to a higher standard, yielding more robust models that can advance the knowledge of the system being modeled and the field of ABM.

Conclusion

5.1

Based on a survey of 279 published articles this article portrayed the state-of-the-art in ABM and identified key research directions. It has been conjectured that ABM is an immature method and that standard practices promoting effective ABM modeling are neither clearly established nor accepted. This survey seems to support that conjecture. The lack of maturity and standard practices in the ABM field is reflected by the lack of models that were completely validated, the lack of references to the complete model and what is accepted as publishable results. A remedy is that techniques, philosophies and methods need to be adopted from other simulation paradigms, or developed specifically for ABM, and these techniques, philosophies and methods need to be taught to those using ABM such that they can build more effective models.

5.2

Six specific research directions, needs and opportunities for ABM were identified in the survey. The first is that development and documentation tools for ABM need to be independent of software and that published articles should detail the software package or programming language used to build and execute the simulation. The second identified research direction and need is that since ABM is a departure from other simulation paradigms, it needs to be studied as an independent discipline and as a subset of the simulation discipline. From this standard techniques, practices, philosophies and methodologies are needed to extend ABM as a functional analysis tool. Third, since ABM is used for different purposes, simulationists should have different expectations for ABM. The fourth research need is that articles need sufficient information about the model so other researchers can independently develop and evaluate the effectiveness of these models. The fifth, and most significant, conclusion reached from the survey is that reviewers and publication outlets must require that the model discussed be completely validated and be documented in the article. Finally, both statistical and non-statistical validation techniques specifically for ABM need to be developed and conveyed effectively to those building these models.

5.3

These six research directions, needs and opportunities represent some initial things needed to mature and help establish standard practices for ABM. If ABM is to reach its full potential as a modeling and simulation paradigm, these fundamental opportunities must be addressed. This is especially true as simulation takes on new roles and begins to extend our limited ability to comprehend and mentally analyze modern complex systems. By establishing clear research goals and standards, the field of ABM will continue to mature and progress and every field exploring complex systems will be better equipped to understand, evaluate and predict these systems through the exploitation of more appropriate and effective agent-based models.

Appendix: Surveyed Articles

Agar, M. H. (2001). Another complex step: A model of heroin experimentation. Field Methods, 13, 353-369.

Agar, M. H. (2005). Agents in living color: Towards emic agent-based models. Journal of Artificial Societies and Social Simulation, 8(1) 4, https://www.jasss.org/8/1/4.html

Akaishi, J. and Arita, T. (2006). Agent-based modeling for investigating adaptivity of misperception. Systems and Computers in Japan, 37(12), 482-491.

Aktipis, C. A. (2004). Know when to walk away: Contingent movement and the evolution of cooperation. Journal of Theoretical Biology, 231, 249-260.

Alam, S. I., Meyer, R., Ziervogel, G. and Moss, S. (2007). The impact of HIV/AIDS in the conext of socioeconomic stressors: An evidence-driven approach. Journal of Artificial Societies and Social Simulation,10(4)7, https://www.jasss.org/10/4/7.html

Alam, S. J. and Werth, B. (2008). Study emergence of clusters in a bus passengers seating preference model. Transportation Research Part C, 16, 593-614.

Albino, V., Carbonara, N. and Giannoccaro, I. (2006). Innovation in industrial districts: An agent-based simulation model. International Journal of Production Economics, 104, 30-45.

Alfonseca, M. and Lara, J. d. (2002). Two-level evolution of foraging agent communities. BioSystems, 66, 21-30.

An, G. and Lee, I. A. (2001). Computer simulation to study inflammatory response. Simulation and Gaming, 32(3), 344-361.

An, L., Linderman, M., Qi, J., Shortridge, A. and Liu, J. (2005). Exploring complexity in a human-environment system: An agent-based spatial MOdel for multidisciplinary and multiscale integration. Annals of the Association of American Geographers, 85(1), 54-79.

Anderson, J., Chaturvedi, A. and Cibulskis, M. (2007). Simulation tools for developing policies for complex systems: Modeling the health and safety of refugee communities. Health Care Management Science, 10, 331-339.

Andersson, M. R. and Sanholm, T. W. (2001). Leveled commitment contracts with myopic and strategic agents. Journal of Economic Dynamics and Control, 25, 615-640.

Andras, P., Lazarus, J., Roberts, G. and Lynden, S. J. (2006). Uncertainty and cooperation: Analytical results and a simulated agent society. Journal of Artificial Societies and Social Simulation, 9(1)7, https://www.jasss.org/9/1/7.html

Ang, H. H. (2005). The effects of military tactics, techniques, and procedures on peace support election operations in representative Iraqi towns. Naval Postgraduate School).

Athale, C., Mansury, Y. and Deisboeck, T. S. (2005). Simulating the impact of a molecular 'decision-process' on cellular phenotype and multicellular patterns in brain tumors. Journal of Theoretical Biology,233, 469-481.

Athanasiadis, I. N., Mentes, A. K. and Mylopoulos,ericles A.Mitkas and Yiannis A. (2005). A hybrid agent-based model for estimating residential water demand. Simulation, 81(3), 175-187.

Atkins, K., Marathe, A. and Barret, C. (2007). A computational approach to modeling commodity markets. Computational Economics, 30, 125-142.

Ausloos, M. and Pekalski, A. (2007). Model of wealth and goods dynamics in a closed market. Physica A, 373, 560-568.

Aydin, M. (2004). An exploratory analysis of village search operations. Naval Postgraduate School).

Babilot, M. (2005). Comparison of a distributed operations force to a traditional force in urban combat. Naval Postgraduate School).

Baek, J. and Kim, C. O. (2007). Learning single-issue negotiation strategies using hierarchical clustering method. Expert Systems with Applications, 32, 606-615.

Bagni, R., Berchi, R. and Cariello, P. (2002). A comparison of simulation models applied to epidemics. Journal of Artificial Societies and Social Simulation, 5(3)5, https://www.jasss.org/5/3/5.html

Bah, A., Toure, I., Page, C. L., Ickowicz, A. and Diop, A. T. (2006). An agent-based model to understand the multiple uses of land resources around drillings in sahel. Mathematical and Computer Modelling, 44, 513-534.

Bailey, A. M., Thorne, B. C. and Peirce, S. M. (2004). Multi-cell agent-based simulation of the microvasculature to study the dynamics of circulating inflammatory cell trafficking. Annals of Biomedical Engineering, 35(6), 916-936.

Bain, M. D. (2005). Supporting a marine corps distribution operations platoon: A quantitative analysis. Naval Postgraduate School).

Bantang, J. and Saloma, C. (2006). Co-existence of poisson and non-poisson processes in ordered parallel multilane pedestrian traffic. Complexity, 11(5), 35-42.

Barreteau, O., Garin, P., Dumontier, A. and Abrami, G. (2003). Agent-based facilitation of water allocation: Case study in the drome river valley. Group Decision and Negotiation, 12, 441-461.

Bars, M. L., Attonaty, J. M., Pinson, S. and Ferrand, N. (2005). An agent-based simulation testing the impact of water allocation on farmers' collective behaviors. Simulation, 81(3), 223-235.

Baydar, C. (2003). Agent-based modeling and simulation of store performance for personalized pricing. Proceedings of the 2003 Winter Simulation Conference, 1759-1764.

Bazzan, A. L. C. and Klugl, F. (2005). Case studies on the braess paradox: Simulating route recommendation and learning in abstract and microscopic models. Transportation Research Part C, 13, 299-319.

Beauchemin, C., Samuel, J. and Tuszynski, J. (2005). A simple cellular automaton model for influenza A viral infections. Journal of Theoretical Biology, 232, 223-234.

Beck, J., Kempener, R., Cohen, B. and Petrie, J. (2008). A complex systems approach to planning, optimization and decision making for energy networks. Energy Policy, 36, 2803-2813.

Beltran, F. S., Salas, L. and Quera, V. (2006). Spatail behavior in groups: An agent-based approach. Journal of Artificial Societies and Social Simulation, 9(3)5, https://www.jasss.org/9/3/5.html

Bentley, K., Gerhardt, H. and Bates, P. A. (2008). Agent-based simulation of notch-mediating tip cell selection in agiogenic sprout initialisation. Journal of Theoretical Biology, 250, 25-36.

Bentley, R. A., Lake, M. W. and Shennan, S. J. (2005). Specialisation and wealth inequality in a model of a clustered economic network. Journal of Archaeological Science, 32, 1346-1356.

Berg, J., Marsili, M., Rustichini, A. and Zecchina, R. (2003). Are financial markets efficient? phase transition in the aggregation of information. Complexity, 8(2), 20-23.

Bergman, M. (2004). Examining risk attitudes. Complexity, 9(5), 25-30.

Bernard, R. N. (1999). Using adaptive agent-based simulation models to assist planners in policy development: The case of rent controlSante Fe Institute.

Bharwani, S., Bithell, M., Downing, T. E., New, M., Washington, R. and Ziervogal, G. (2005). Multi-agent modeling of climate outlooks and food security on the community garden scheme in limpopo, south africa. Philosohpical Transactions of the Royal Society B, 360, 2183-2194.

Bhavnani, R. (2003). Adaptive agents, political institutions and civic traditions in modern italy. Journal of Artificial Societies and Social Simulation, 6(4)1, https://www.jasss.org/9/3/5.htmlhttps://www.jasss.org/6/4/1.html

Black, J. A., Oliver, R. L., Howell, J. P. and King, J. P. (2006). A dynamic system simulation of leader and group effects on contect for learning. The Leadership Quarterly, 17, 39-56.

Blaum, N. and Wichmann, M. C. (2007). Short-term transformation of matrix into hospitable habitat facilitates gene flow and mitigates fragmentation. Journal of Animal Ecology, 76, 1116-1127.

Boero, R., Castellani, M. and Squazzoni, F. (2004). Micro behavioural attitudes and macro technological adaptation in industrial districts: An agent-based prototype. Journal of Artificial Societies and Social Simulation, 7(2)1,https://www.jasss.org/9/3/5.html https://www.jasss.org/7/2/1.html

Boisot, M., MacMillan, I. C. and Han, K. S. (2007). Property rights and information flows: A simulation approach. Journal of Evolutionary Economics, 17, 63-93.

Borrelli, F., Ponsiglione, C., Iandoli, L. and Zollo, G. (2005). Inter-organizational learning and collective memory in small firms clusters: An agent-based approach. Journal of Artificial Societies and Social Simulation, 8(3)4 https://www.jasss.org/8/3/4.html

Bower, J. and Bunn, D. (2001). Experimental analysis of the efficiency of uniform-price versus discriminatory auctions in the england and wales electricity market. Journal of Economic Dynamics and Control, 25, 561-592.

Bradburd, R., Sheppard, S., Bergeron, J., Engler, E. and Gee, E. (2005). The distributional impact of housing discrimination in a non-walrasian setting. Journal of Housing Economics, 14, 61-91.

Brede, M., Foschetti, F. and McDonald, D. (2008). Strategies for resource exploitation. Ecological Complexity, 5, 22-29.

Brown, D. G., Page, S. E., Riolo, R. and Rand, W. (2004). Agent-based and analytical modeling to evaluate the effectiveness of greenbelts. Environmental Modelling and Software, 19, 1097-1109.

Brown, L. P. (2000). Agent based simulation as an exploratory tool in the study of the human dimension of combat. Naval Postgraduate School).

Buchta, C., Meyer, D., Pfister, A., Mild, A. and Taudes, A. (2003). Technological efficiency and organizational inertia: A model of the emergence of disruption. Computational and Mathematical Organization Theory, 9, 127-146.

Bullock, R. K., McIntyre, G. A. and Hill, R. R. (2000). Using agent-based modeling to capture airpower strategic effects. Proceedsing of the 2000 Winter Simulation Conference, 1739-1746.

Bunn, D. W. and Oliveira, F. S. (2003). Evaluating individual market power in electricity markets via agent-based simulation. Annals of Operations Research, 121, 57-77.

Cabaniss, S. E., Madey, G., Leff, L., Maurice, P. A. and Wetzel, R. (2005). A stochastic model for the synthesis and degradation of natural organic matter. part I. data structures and reaction kinetics.Biogeochemistry, 76, 319-347.

Canessa, E. and Riolo, R. L. (2003). The effect of organizational communication media on organizational culture and performance: An agent-based simulation model. Computational and Mathematical Organization Theory, 9, 147-176.

Carrillo-Hermosilla, J. (2006). A policy approach to the environmental impacts of technological lock-in. Ecological Economics, 58, 717-742.

Casal, A., Sumen, C., Reddy, T. E. and Lee,Mark S.Alber and Peter P. (2005). Agent-based modeling of the context dependency in T cell recognition. Journal of Theoretical Biology, 236, 376-391.

Chakrabarti, R. (2000). Just another day in the inter-bank foreign exchange market. Journal of Financial Economics, 56, 29-64.

Challet, D. (2008). Inter-pattern speculation: Beyond minority, majority and money-games. Journal of Economics Dynamics and Control, 32, 85-100.

Champagne, L. E. and Hill, R. R. (2007). Agent-model validation based on historical data. Proceedings of the 2007 Winter Simulation Conference, 1223-1231.

Chang, K. M. (2005). The peformance of edge organizations in a collaborative task. Naval Postgraduate School).

Chen, S. H. and Yeh, C. H. (1999). Modeling the expectations of inflation in the OLG model with genetic programming. Soft Computing, 3, 53-62.

Chen, S. and Chie, B. (2008). Lottery markets design, micro-structure and macro-behavior: An ACE approach. Journal of Economic Behavior and Organization, 67, 463-480.

Chen, S. and Huang, Y. (2007). Relative risk aversion and wealth dynamics. Information Sciences, 177, 1222-1229.

Chen, S. and Liao, C. (2005). Agent-based computational modeling of the stock price-volume relation. Information Sciences, 170, 75-100.

Chen, X., Meaker, J. W. and Zhan, F. B. (2006). Agent-based modeling and analysis of hurricane evacuation procedures for the florida keys. Natural Hazards, 38, 321-338.

Ching, W. S. (2002). An exploratory analysis on the effects of human factors on combat outcomes. Naval Postgraduate School).

Cope, D. R. (2005). Individuality in modeling: A simplifying assumption too far? Nonlinear Analysis: Real World Applications, 6, 691-704.

Damaceanu, R. (2008). An agent-based computational study of wealth distribution in function of resource growth interval using NetLogo. Applied Mathematics and Computation, 201, 371-377.

D'Aquino, P., Page, C. L., Bousquet, F. and Bah, A. (2003). Using self-designed role-playing games and a multi-agent system to empower a local decision-making process for land use management: The SelfCormas experiment in senegal. Journal of Artificial Societies and Social Simulation, 6(3)5 https://www.jasss.org/6/3/5.html

Dawid, H. and Dermietzel, J. (2006). How robust is the equal split norm? responsive strategies, selection mechanisms and the need for economic interpretation of simulation parameters.Computational Economics, 28, 371-397.

Dawid, H. and Reimann, M. (2004). Evaluating market attractiveness: Individual incentives versus industry profitability. Computational Economics, 24, 321-355.

Delre, S. A., Jager, W., Bijmolt, T. H. A. and Janssen, M. A. (2007). Targeting and timing promotional activities: An agent-based model for the takeoff of new products. Journal of Business Research, 60, 826-835.

Donangelo, R., Hansen, A., Sneppen, K. and Souza, S. R. (2005). Need, greed and noise: Competing strategies in a trading model. Physica A, 348, 496-504.

Dosi, G., Fagiolo, G. and Roventini, A. (2006). An evolutionary model of endogenous business cycles. Computational Economics, 27, 3-34.

Duffy, J. (2001). Learning to speculate: Experiments with artificial and real agents. Journal of Economic Dyanmics and Control, 25, 295-319.

Duffy, J. and Unver, M. U. (2008). Internet auctions with artificial adaptive agents: A study on market design. Journal of Economic Behavior and Organization, 67, 394-417.

Duffy, J. and Unver, M. U. (2006). Asset price bubbles and crashes near-zero-intelligence traders. Economic Theory, 27, 537-563.

Ebben, M. J. R., Boer, L. d. and Sitar, C. E. P. (2002). Multi-agent simulation of purchasing activities in organizations. Proceedings of the 2002 Winter Simulation Conference,

Edmonds, B. (2001). Towards a descriptive model of agent strategy search. Computational Economics, 18, 113-135.

Efimba, M. E. (2003). An exploratory analysis of littoral combat ships' ability to protect expenditionary strike groups. Naval Postgraduate School).

Ehlen, M. A., Scholand, A. J. and Stamber, K. L. (2007). The effects of residential real-time pricing contracts on transco laods, pricing and profitability: Simulations using the N-ABLE agent-based model.Energy Economics, 29, 211-227.

Eidelson, B. M. and Lustick, I. (2004). VIR-POX: An agent-based analysis of smallpox preparedness and response policy. Journal of Artificial Societies and Social Simulation, 7(3)6 https://www.jasss.org/7/3/6.html

Elliston, L. and Cao, L. (2006). An agent-based bioeconomic model of fishery with input controls. Mathematical and Computer Modelling, 44, 565-575.

Epstein, J. M. (2001). Learning to be thoughtless: Social norms and individual computation. Computational Economics, 18, 9-24.

Erlenbruch, T. (2002). Agent-based simulation of german peacekeeping operations for units up to platoon level. Naval Postgraduate School).

Etienne, M., Page, C. L. and Cohen, M. (2003). A step-by-step approach to building land management scenarios based on multiple viewpoints on multi-agent system simulations. Journal of Artificial Societies and Social Simulations, 6(2)2, https://www.jasss.org/6/2/2.html

Evans, T. P. and Kelley, H. (2004). Multi-scale analysis of a household level agent-based model of landcover change. Journal of Environmental Management, 72, 57-72.

Fehler, M., Kleinhenz, M., Klugl, F. and Tautz,Frank Puppe and Jurgen. (2007). Caps and gaps: A computer model for studies on brood incubation strategies in honeybees (apis mellifera carnica).Naturwissenchaften, 94, 675-680.

Fioretti, G. and Lomi, A. (2008). The garbage can model of organizational choice: An agent-based reconstruction. Simulation Modelling Practice and Theory, 16, 192-217.

Flache, A. and Mas, M. (2008). Why do faultlines matter? A computational model of how strong demographic faultlines undermine team cohesion. Simulation Modelling Practice and Theory, 16, 175-191.

Fleischmann, A. (2005). A model for a simple luhmann economy. Journal of Artificial Societies and Social Simulation, 8(2)4 https://www.jasss.org/8/2/4.html

Flentge, F., Polani, D. and Uthmann, T. (2001). Modelling the emergence of possession norms using memes. Journal of Artificial Societies and Social Simulation, 4(4)3, https://www.jasss.org/4/4/3.html

Fort, H. (2003). Exploring the cooperative regimes in an agent-based model: Indirect reciprocity vs. selfish incentives. Physica A, 326, 286-298.

Fowler, C. S. (2007). Taking geographical economics out of equilibrium: Implications for theory and policy. Journal of Economic Geography, 7, 265-284.

Garmire, L. X., Garmire, D. G. and Hunt, C. A. (2007). An in silico transwell device for the study of drug transport and drug-drug interactions. Pharmaceutical Research, 24(12), 2171-2186.

Gatti, D. D., Guilmi, C. D., Gaffeo, E., Giulioni, G., Gallegati, M. and Palestrini, A. (2005). A new approach to business fluctuations: Heterogeneous interacting agents, scaling laws and financial fragility.Journal of Economic Behavior and Organization, 56, 489-512.

Gaupp, M. P. and Hill, R. R. (1999). Using adaptive agents in java to simulate U.S. air force pilot rentention. Proceedings of the 1999 Winter Simulation Conference, 1152-1159.

Giardina, I. and Bouchaud, J. (2003). Volatility clustering in agent based market models. Physica A, 324, 6-16.

Gigliotta, O., Miglino, O. and Parisi, D. (2007). Groups of agents with a leader. Journal of Artificial Societies and Social Simulation, 10(4)1, https://www.jasss.org/10/4/1.html

Gill, A. W., Egudo, R. R., Dortmans, P. J. and Grieger, D. (2002). Using agent based distillations in support of the army capability development process - A case studyDSTO Systems Sciences Laboratory.

Gonzalex-Avella, J. C., Cosenza, M. G., Equiluz,Konstantin Klemm and Victor M. and Miguel, M. S. (2007). Information feedback and mass media effects in cultural dynamics. Journal of Artificial Societies and Social Simulation, 10(3)9, https://www.jasss.org/10/3/9.html

Gorobets, A. and Nooteboom, B. (2006). Adaptive build-up and breakdown of trust: An agent based computational approach. Journal of Management Governance, 10, 277-306.

Grieger, D. and Gill, A. (2001). Exploring the effect of the fog of war on the value of competitive edges in land warfare. Computational Techniques and Applications Conference,

Groff, E. R. (2007). Simulation for theory testing and experimentation: An example using routine activity theory and street robbery. Journal of Quantitative Criminology, 23, 75-103.

Gross, J. E., McAllister, R. R. J., Abel, N. and Maru,D.M.Stafford Smith and Y. (2006). Australian rangelands as complex adaptive systems: A conceptual model and preliminary results. Environmental Modelling and Software, 21, 1264-1272.

Gun, S. K. (2005). Evaluating sunni participation in an election in a representative iraqi town. Naval Postgraduate School).

Hakola, M. B. (2004). An exploratory analysis of convey protection using agent-based simulation. Naval Postgraduate School).

Happe, K., Balmann, A., Kellermann, K. and Sahrbacher, C. (2008). Does structure matter? the impact of switching the agricultural policy regime on farm structures. Journal of Economic Behavior and Organization, 67, 431-444.

Hazy, J. K. and Tivnan, B. F. (2004). On building an organizationally realistic agent-based model of local interaction and emergent network structure. Proceedings of the 2004 Winter Simulation Conference,

Hellweger, F. L. (2008). The role of inter-generation memory in diel phytoplankton division patterns. Ecological Modelling, 212, 382-396.

Hemelrijk, C. K., Wantia, J. and Gygax, L. (2005). The construction of dominance order: Comparing performance of five methods using an individual-based model. Behaviour, 142, 1037-1058.

Henrickson, L. (2002). Old wine in a new wineskin: College choice, college access using agent-based modeling. Social Science Computer Review, 20(4), 400-419.

Ho, S. T. (2006). Investigating ground swarm robotics using agent based simulation. Naval Postgraduate School).

Hodgson, G. M. and Knudsen, T. (2004). The complex evolution of a simple traffic convention: The functions and implications of habit. Journal of Economic Behavior and Organization, 54, 19-47.

Hoffmann, A. O. I., Iager, W. and Eije, I. H. V. (2007). Social SImulation of stock markets: Taking it to the next level. Journal of Artificial Societies and Social Simulation, 10(2)7, https://www.jasss.org/10/2/7.html

Holme, P., Karlin, J. and Forrest, S. (2008). An integrated model of traffic, geography and economy in the internet. ACM SIGCOMM Computer Communication Review, 38(3), 7-15.

Hu, X. and Sun, Y. (2007). Agent-based modeling and simulation of wildland fire suppression. Proceedings of the 2007 Winter Simulation Conference,

Huigen, M. G. A. (2004). First principles of the MameLuke multi-actor modelling framework for land use change, illustrated with a phillipine case study. Journal of Environmental Management, 72, 5-21.

Hummon, N. P. and Doreian, P. (2003). Some dynamics of social balance processes: Bringing heider back into balance theory. Social Networks, 25, 17-49.

Iizuka, H., Yamamoto, M., Suzuki, K. and Ohuchi, A. (2002). Bottom-up consensus formation in voting games. Nonlinear Dynamics, Psychology and Life Sciences, 6(2), 185-195.

Ikeda, Y., Aoyama, H., Iyetomi, H., Souma,Yoshi Fujiwara and Wataru and Kaisoji, T. (2007). Response of firm agent network to exogenous shock. Physica A, 382, 138-148.

Ipekci, A. I. (2002). How agent ased models can be utilized to explore and exploit non-linearity and intangibles inherent in guerrilla warfare. Naval Postgraduate School).

Jackson, J. (2007). Are US utility standby rates inhibiting diffusion of customer-owned generating systems? Energy Policy, 35, 1896-1908.

Jacob, C. and Burleigh, I. (2004). Biomolecular swarms - an agent-based model of the lactose operon. Natural Computing, 3, 361-376.

Joshi, S., Parker, J. and Bedau, M. A. (2002). Financial markets can be at sub-optimal equilibria. Computational Economics, 19, 5-23.

Katare, S. and Venkatasuramanian, V. (2001). An agent-based learning framework for modeling microbial growth. Engineering Applications of Artificial Intelligence, 14, 715-726.

Kendrick, G. A., Marba, N. and Duarte, C. M. (2005). Modelling formation of complex topography by the seagrass posidonia oceanica. Estuarine, Coastal and Shelf Science, 65, 717-725.

Kewley, R. H. (2004). Agent-based model of auftragstaktik: Self organization in command and control of future combat forces. Proceedings of the 2004 Winter Simulation Conference,

Khalak, A. (2003). Agent-based model for economic impact of free software. Complexity, 8(3), 45-55.

Khouja, M., Hadzikadic, M. and Zaffar, M. A. (2008). An agent-based modeling approach for determining optimal price-rebate schemes. Simulation Modelling Practice and Theory, 16, 111-126.

Kim, Y. S. (2007). Maximizing sellers' welfare in online auction by simulating bidders' proxy bidding agents. Expert Systems with Applications, 32, 289-298.

Kiskowski, M. A., Alber, M. S., Thomas, G. L., Glazier, J. A., Bronstein, N. B., Pu, J., et al. (2004). Interplay between activator-inhibitor coupling and cell-matrix adhesion in a cellular automaton model for chondrogenic patterning. Developmental Biology, 271, 372-387.

Klos, T. B. and Nooteboom, B. (2001). Agent-based computational transaction cos economics. Journal of Economic Dynamics and Control, 25, 503-526.

Kumar, S. and Mitra, S. (2006). Self-organizing traffic at a malfunctioning intersection. Journal of Artificial Societies and Social Simulation, 9(4)3, https://www.jasss.org/9/4/3.html

Kuscsik, Z., Horvath, D. and Gmitra, M. (2007). The critical properties of the agent-based model with environmental-economic interactions. Physica A, 379, 199-206.

Lalis, V. (2007). Exploring naval tactics with UAVs in an island complex using agent-based simulation. Naval Postgraduate School).

Lam, R. B. (2007). Agent-based simulations of service policy decisions. Proceedings of the 2007 Winter Simulation Conference,

Lao, B. J. and Kamei, D. T. (2008). Investigation of cellular movement in the prostate epithelium using an agent-based model. Journal of Theoretical Biology, 250, 642-654.

Lara, J. d. and Alfonseca, M. (2000). Some strategies for the simulation of vocabulary agreement in multi-agent communities. Journal of Artificial Societies and Social Simulation, 3(4)2, https://www.jasss.org/3/4/2.html

Lara, J. d. and Alfonseca, M. (2002). The role of oblivion, memory size and spatial separation in dynamic language games. Journal of Artificial Societies and Social Simulation, 5(2)1, https://www.jasss.org/5/2/1.html

Lauren, M. K. (2002). Firepower concentration in cellular automaton combat models-an alternative to lanchester. Journal of Operational Research Society, 53, 672-679.

Laver, M. and Schilperoord, M. (2007). Spatial models of political competition with endogenous political parties. Philosohpical Transactions of the Royal Society B, 362, 1711-1721.

LeBaron, B. and Yamamoto, R. (2007). Long-memory in an order-driven market. Physica A, 383, 85-89.

Lee, S. M., Ravinder, U. and Johnson, J. C. (2005). Developing an agent model of human performance in air traffic control operations using apex cognitive architecture. Proceedings of the 2005 Winter Simulation Conference,

Liang, L. A. H. (2005). The use of agent based simualtion for cooperative sensing of the battlefield. Naval Postgraduate School).

LiCalzi, M. and Pellizzari, P. (2006). Breeds of risk-adjusted foundamentalist strategies in an order-driven market. Physica A, 359, 619-633.

Lindquist, J. M. (2004). An analysis of degraded communications in the army's future force. Naval Postgraduate School).

Liu, X., Liang, X. and Tang, B. (2004). Minority game and anomalies in financial markets. Physica A, 333, 343-352.

Liu, Y. and Hunt, C. A. (2006). Mechanicistic study of the cellular interplay of transport and metabolism using the synthetic modeling method. Pharmaceutical Research, 23(3), 493-505.

Loper, M. L. and Presnell, B. (2005). Modeling an emergency operations center with agents. Proceedings of the 2005 Winter Simulation Conference,

Lopez-Paredes, A., Sauri, D. and Galan, J. M. (2005). Urban water management with artificial societies of agents: The FIRMABAR simulator. Simulation, 81(3), 189-199.

Lus, H., Aydin, C. O., Keten, S. and Atilgan,Hakan Ismail Unsal and Ali Rana. (2005). El farol revisisted. Physica A, 346, 651-656.

Lustick, I. S. (2000). Agent-based modelling of collective identity: Testing constructivist theory. Journal of Artificial Societies and Social Simulation, 3(1)1, https://www.jasss.org/3/1/1.html

Ma, T. and Nakamori, Y. (2005). Agent-based modeling on technological innovation as an evolutionary process. European Journal of Operational Research, 166, 741-755.

Makowsky, M. (2006). An agent-based model of mortality shocks, intergeneral effects and urban crime. Journal of Artificial Societies and Social Simulation, 9(2), https://www.jasss.org/9/2/7.html

Manson, S. M. (2005). Agent-based modeling and genetic programming for modeling land change in the southern yucatan peninsular region of mexico. Agriculture, Ecosystems and Environment, 111, 47-62.

Mansury, Y. and Deisboeck, T. S. (2003). The impact of search precision in an agent-based tumor model. Journal of Theoretical Biology, 224, 325-337.

Mansury, Y. and Gulyas, L. (2007). The emergence of zipf's law in a system of cities: An agent-based simulation approach. Journal of Economic Dynamics and Control, 31, 2438-2460.

Martin, C. W. and Plumper, T. (2005). Number of parties, endogenous part preferences and electoral turnout rates. Social Science Computer Review, 23(3), 347-359.

Mathevet, R., Bousquet, F., Page, C. L. and Antona, M. (2003). Agent-based simulations of interactions between duck population, farming decisions and leasing of hunting rights in the camargue (southern france). Ecological Modelling, 165, 107-126.

Mathevet, R., Page, C. L., Etienne, M., Lefebvre, G., Poulin, B., Gigot, G., et al. (2007). BUTORSTAR: A role-playing game for collective awareness of wise reedbed use. Simulation and Gaming, 38(2), 233-262.

McDonald, A. D., Little, L. R., Gray, R., Fulton, E., Sainsbury, K. J. and Lyne, V. D. (2008). An agent-based modelling approach to evaluation of multiple-use management strategies for coastal marine ecosystems. Mathematics and Computers in Simulation, 78, 401-411.

McDonald, M. L. and Upton, S. C. (2005). Investigating the dynamics of competition: Coevolving red and blue simulation parameters. Proceedings of the 2005 Winter Simulation Conference, 1008-1012.

McMindes, K. L. (2005). Unmanned aerial vehicle survivability: The impact of speed, detectability, altitude and enemy capabilities. Naval Postgraduate School).

Mehta, K. and Bhattacharyya, S. (2006). Design, development and validation of an agent-based model of electronic auctions. Information Technology and Management, 7, 191-212.

Mendes, R. V. (2008). The fractional volatility model: An agent-based interpretation. Physica A, 387, 3987-3994.

Micola, A. R., Banal-Estanol, A. and Bunn, D. W. (2008). Incentives and coordination in vertically related energy markets. Journal of Economic Behavior and Organization, 67, 381-393.

Micola, A. R. and Bunn, D. W. (2008). Crossholdings, concentration and information in capacity-constrained sealed bid-offer auctions. Journal of Economic Behavior and Organization, 66, 748-766.

Milton, R. M. (2004). Using agent-based modeling to examine the logistical chain of the seabase. Naval Postgraduate School).

Minett, J. W. and Wang, W. S. (2008). Modelling endangered languages: The effects of bilingualism and social structure. Lingua, 118, 19-45.

Miodownik, D. (2006). Cultural differences and economic incentives: An agent-based study of their impact on the emergence of regional autonomy movements. Journal of Artificial Societies and Social Simulation, 9(4)2, https://www.jasss.org/9/4/2.html

Mitrovic, I. and Dautenhahn, K. (2003). Social attitudes: Investigating with agent simulations using webots. Journal of Artificial Societies and Social Simulation, 6(4)4, https://www.jasss.org/6/4/4.html

Mizuta, H. and Nakamura, F. (2005). Agent-based simulation of enterprise communication network. Proceedings of the 2005 Winter Simulation Conference,

Mizuta, H., Steiglitz, K. and Lirov, E. (2003). Effects of price signal choices on market stability. Journal of Economic Behavior and Organization, 52, 235-251.

Mizuta, H. and Yamagata, Y. (2001). Agent-based simulation and greenhouse gass emissions trading. Proceedings of the 2001 Winter Simulation Conference,

Morone, P. and Taylor, R. (2004). Small world dynamics and the process of knowledge diffusion: The case of the metropolitan area of greater santiago de chile. Journal of Artificial Societies and Social Simulation, 7(2)5, https://www.jasss.org/7/2/5.html

Moulet, S. and Rouchier, J. (2008). The influence of seller learning and time constraints on sequential bargaining in an artificial perishable goods market. Journal of Economic Dynamics and Control, 32, 2322-2348.

Muller, G., Grebaut, P. and Gouteux, J. (2004). An agent-based model of sleeping sickness: Simulation trials of a fore focus in southern cameroon. Comptes Rendus Biologies, 327, 1-11.

Muller, M., Sensfuss, G. and Wietschel, M. (2007). Simulation of current pricing-tendencies in the german electricity market for private consumption. Energy Policy, 35, 4283-4294.

Nawa, N. E., Shimohara, K. and Katai, O. (2002). On fairness and learning agents in a bargaining model with uncertainty. Cognitive Systems Research, 3, 555-578.

Neuberg, L. and Bertels, K. (2003). Heterogeneous trading agents. Complexity, 8(5), 28-35.

Neugart, M. (2008). Labor market policy evaluation with ACE. Journal of Economic Behavior and Organization, 67, 418-430.

Nickel, D., Barthel, R. and Braun, J. (2005). Large-scale water resources management within the framework of GLOWA-danube - the water supply model. Physics and Chemistry of the Earth, 30, 383-388.

Niedringhaus, W. P. (2004). The jet:Wise model of national airspace system evolution. Simulation, 80(1), 45-58.

Ormerod, P. (2002). The US business cycle: Power law scaling for interacting units with complex internal structure. Physica A, 314, 774-785.

Ormerod, P. (2007). Extracting deep information from limited observations on an evovled social network. Physica A, 378, 48-52.

Ormerod, P. and Colbaugh, R. (2006). Cascades of failure and extinction in evolving complex systems. Journal of Artificial Societies and Social Simulation, 9(4)9, https://www.jasss.org/9/4/9.html

Parks, S. C., Farifullin, M. and Dronzek, R. (2005). Argus invasive species spread model constructed using agent-based modeling approach and cellular automata. Proceedings of the 2005 Winter Simulation Conference, 1038-1042.

Parra, C. M. and Yano, M. (2005). Evolutionary dynamics of knowledge. Complexity, 11(5), 12-19.

Pathak, S. D., Dilts, D. M. and Biswas, G. (2004). Simulating growth dynamics in complex adaptive supply networks. Proceedings of the 2004 Winter Simulation Conference, 774-782.

Pavon, J., Arroyo, M., Hassan, S. and Sansores, C. (2008). Agent-based modelling and simulation for the analysis of social patterns. Pattern Recognition Letters, 29, 1039-1048.

Peeta, S., Zhang, P. and Zhou, W. (2005). Behavior-based analysis of freeway car-truck interactions and related mitigation strategies. Transportation Research Part B, 39, 417-451.

Penzar, D. and Srbljinovic, A. (2004). Dynamic modeling of ethnic conflicts. International Transactions in Operational Research, 11, 63-76.

Pertoldi, C. and Topping, C. (2004). The use of agent-based modelling of genetics in conservation genetics studies. Journal for Nature Conservation, 12, 111-120.

Pfeiffer, V. (2006). Communication aspects in urban terrain. Naval Postgraduate School).

Pie, M. R., Rosengaus, R. B. and Traniello, J. F. A. (2004). Nest architectures, activity pattern, worker density and the dynamics of disease transmission in social insects. Journal of Theoretical Biology,226, 45-51.

Pogson, M., Smallwood, R., Qwarnstrom, E. and Holcombe, M. (2006). Formal agent-based modelling of intracellular chemical interactions. BioSystems, 85, 37-45.

Pujol, J. M., Flache, A., Delgado, J. and Sanguesa, R. (2005). How can social networks ever become complex? modelling the emergence of complex networks from local social exchanges. Journal of Artificial Societies and Social Simulation, 8(4)12, https://www.jasss.org/8/4/12.html

Purucker, S. T., Welsh, C. J. E., Stewart, R. N. and Starzec, P. (2007). Use of habitat-contamination spatial correlation to determine when to perform a spatially explicit ecological risk assessment.Ecological Modelling, 204, 180-192.

Raberto, M. and Cincotti, S. (2005). Modeling and simulation of a double auction artificial financial market. Physica A, 355, 34-45.

Ramos-Fernandez, G., Boyer, D. and Gomez, V. P. (2006). A complex social structure with fission-fusion properties can emerge froma simple foraging model. Behavioral Ecology and Sociobiology, 60, 536-549.

Reaney, S. M. (2008). The use of agent based modelling techniques in hydrology: Determining the spatial and temporal origin of channel flow in semi-arid catchments. Earth Surface Processes and Landforms, 33, 317-327.

Reuter, H., Jopp, F., Holker, F., Eschenbach, C. and Breckling, U. M. a. B. (2008). The ecological effect of phenotypic plasticity - analyzing complex interaction networks (COIN) with agent-based models.Ecological Informatics, 3, 35-45.

Riggs, T., Walts, A., Perry, N., Lynch,Laura Bickle and Jennifer N., Myers, A., Flynn, J., et al. (2008). A comparison of random vs. chemotaxis-driven contacts of T cells with dendtritic cells during repertoire scanning. Journal of Theoretical Biology, 250, 732-751.

Robbins, M. M. and Robbins, A. M. (2004). Simulation of the population dynamics and social structure of the virunga mountain gorillas. American Journal of Primatology, 63, 201-223.

Roberts, C. A., Stallman, D. and Bieri, J. A. (2002). Modeling complex human-environment interactions: The grand canyon river trip simulator. Ecological Modelling, 153, 181-196.

Roberts, M. E. and Goldstone, R. L. (2006). EPICURE: Spatial and knowledge limitations in group foraging. Adaptive Behavior, 14(4), 291-313.

Rocha, L. M. (2001). Evolution with material symbol systems. BioSystems, 60, 95-121.

Roginski, J. W. (2006). Emergency first response to a crisis event: A multi-agent simulation approach. Naval Postgraduate School).

Rothenstein, R. and Pawelzik, K. (2003). Evolution and anti-evolution in a minimal stock market model. Physica A, 326, 534-543.

Rouchier, J., Bousquet, F. and Antona,Melanie Requier-Desjardins and Martine. (2001). A multi-agent model for describing transhumance in north cameroon: Comparison of different rationality to develop a routine. Journal of Economic Dynamics and Control, 25, 527-559.

Rouchier, J. and Robin, S. (2006). Information perception and price dynamics in a continuous double auction. Simulation and Gaming, 37(2), 195-208.

Russo, A., Catalano, M., Gaffeo, E. and Napoletano,Mauro Gallegati and Mauro. (2007). Industrial dynamics, fiscal policy and R-and-D: Evidence from a computational experiment. Journal of Economic Behavior and Organization, 64, 426-447.

Sadedin, S., Dybiec, B. and Briscoe, G. (2003). A toy model of faith-based systems evolution. Physica A, 323, 715-725.

Sallans, B., Pfister, A., Karatzoglou, A. and Dorffner, G. (2003). Simulation and validation of an integrated markets model. Journal of Artificial Societies and Social Simulation, 6(4)2, https://www.jasss.org/6/4/2.html

Sanders, T. M. (2005). Exploring the effectiveness of the marine expeditionary rifle squad. Naval Postgraduate School).

Sapienza, M. (2003). Do real options perform better than net present value? testing in an artificial financial market. Journal of Artificial Societies and Social Simulation, 6(3)4, https://www.jasss.org/6/3/4.html

Satake, A., Leslie, H. M., Iwasa, Y. and Levin, S. A. (2007). Coupled ecological-social dynamics in a forested landscape: Spatial interactions and information flow. Journal of Theoretical Biology, 246, 695-707.

Sato, A. (2007). Frequency analysis of tick quotes on the foreign exchange market and agent-based modeling: A spectral distance approach. Physica A, 382, 258-270.

Sawhney, A., Bashford, H., Walsh, K. and Mulky, A. R. (2003). Agent-based modeling and simulation in construction. Proceedings of the 2003 Winter Simulation Conference,

Scheffran, J. and Hannon, B. (2007). From complex conflicts to stable cooperation. Complexity, 13(2), 78-91.

Schenk, T. A., Loffler, G. and Rauh, J. (2007). Agent-based simulation of consumer behavior in grocery shopping on a regional level. Journal of Business Research, 60, 894-903.

Schreiber, C. and Carley, K. (2004). Going beyond the data: Empirical validation leading to grounded theory. Computational and Mathematical Organization Theory, 10, 155-164.

Schreinemachers, P., Berger, T. and Aune, J. B. (2007). Simulating soil fertility and poverty dynamics in uganda: A bio-economic multi-agent systems approach. Ecological Economics, 64, 387-401.

Schwoon, M. (2006). Simulating the adoption of fuel cell vehicles. Journal of Evolutionary Economics, 16, 435-472.

Segovia-Juarez, J. L., Ganguli, S. and Kirschner, D. (2004). Identifying control mechanisms of granuloma formation during M. tuberculosis infection using an agent-based model. Journal of Theoretical Biology, 231, 357-376.

Sellers, W. I., Hill, R. A. and Logan, B. S. (2007). An agent-based model of group decision making in baboons. Philosohpical Transactions of the Royal Society B, 362, 1699-1710.

Shapiro, M., Duca, K. A., Lee, K., Delgado-Eckert, E., Hawkins, J., Jarrah, A. S., et al. (2008). A virtual look at epstein-barr virus infection: Simulation mechanism. Journal of Theoretical Biology, 252, 633-648.

Shendarkar, A., Vaudevan, K., Lee, S. and Son, Y. (2006). Crown simulation for emergency response using BDI agent based on virtual reality. Proceedings of the 2006 Winter Simulation Conference,

Shimokawa, T., Suzuki, K. and Misawa, T. (2007). An agent-based approach to financial stylized facts. Physica A, 379, 207-225.

Sickinger, L. R. (2006). Effectiveness of non-lethal capabilities in a maritime environment. Naval Postgraduate School).

Siebers, P., Aickelin, U., Celia, H. and Clegg, C. W. (2007). Using intelligent agents to understand management practice and retail productivity. Proceedings of the 2007 Winter Simulation Conference,2212-2220.

Sim, K. M. (2004). Negotiation agents that make prudent compromises and are slightly flexible in reaching consensus. Computational Intelligence, 20, 643-662.

Simao, J. and Todd, P. M. (2002). Modeling mate choice in monogamous mating systems with courtship. Adaptive Behavior, 10(2), 113-136.

Smith, N., Capiluppi, A. and Fernandez-Ramil, J. (2006). Agent-based simulation of open source evolution. Software Process Improvement and Practice, 11, 423-434.

Srbljinovic, A., Penzar, D., Rodik, P. and Kardov, K. (2003). An agent-based model of ethnic mobilisation. Journal of Artificial Societies and Social Simulation, 6(1)1, https://www.jasss.org/6/1/1.html

Steele, M. J. (2004). Agent-based simulation of unmanned surface vehicles: A force in the fleet. Naval Postgraduate School).

Stroud, P., Valle, S. D., Sydoriak, S. and Mniszewski,Jane Riese and Susan. (2007). Spatial dynamics of pandemic influenza in a massive artificial society. Journal of Artificial Societies and Social Simulation, 10(4)9, https://www.jasss.org/10/4/9.html

Sueyoshi, T. and Tadiparthi, G. R. (2008). An agent-based decision support system for wholesale electricity market. Decision Support Systems, 44, 425-446.

Sznajd-Weron, K. and Weron, R. (2003). How effective is advertising in duopoly markets? Physica A, 324, 437-444.

Sznajd-Weron, K. and Sznajd, J. (2005). Who is left, who is right? Physica A, 351, 593-604.

Takadama, K., Kawai, T. and Koyama, Y. (2008). Micro- and macro-level validation in agent-based simulation: Reproduction of human-like behaviors and thinking in a sequential bargaining game.Journal of Artificial Societies and Social Simulation, 11(2)9, https://www.jasss.org/11/2/9.html

Takahashi, H. and Terano, T. (2003). Agent-based approach to investory behavior and asset price fluctuation in financial markets. Journal of Artificial Societies and Social Simulation, 6(3)3, https://www.jasss.org/6/3/3.html

Takama, T. and Preston, J. (2008). Forecasting the effects of road user charge by stochastic agent-based modelling. Transportation Research Part A, 42, 738-749.

Tan, P. S. (2004). Using agent-based modeling to assess the impact of martial law on a representation iraqi town. Naval Postgraduate School).

Teitelbaum, D. and Dowlatabadi, H. (2000). A computational model of technological innovation at the firm level. Computational and Mathematical Organization Theory, 6(3), 227-247.

Tesfatsion, L. (2001). Structure, behavior and market power in an evolutionary labor market with adaptive search. Journal of Economic Dynamics and Control, 25, 419-457.

Tiburcio, F. M. (2005). Maritime protection of critical infrastructure assets in the campeche sound. Naval Postgraduate School).

Tillman, D., Larsen, T. A., Pahl-Wostl, C. and Gujer, W. (1999). Modeling the actors in water supply systems. Water Science and Technology, 39(4), 203-211.

Torrens, P. M. and Nara, A. (2007). Modeling gentrification dynamics: A hybrid approach. Computers, Environment and Urban Systems, 31, 337-361.

Vag, A. (2007). Simulating changing consumer preferences: A dynamic conjoint model. Journal of Business Research, 60, 904-911.

Valkering, P., Rotmans, J., Kywkow, J. and Veen, A. v. d. (2005). Simulating stakeholder support in a policy process: An application to river management. Simulation, 81(10), 701-718.

Vaughan, D. P. (2006). Exploration of force transitions in stability operations using multi-agent simulation. Naval Postgraduate School).

Walker, D. C., Southgate, J., Hill, G., Holcombe, M., Hose, D. R., Wood, S. M., et al. (2004). The epitheliome: Agent-based modelling of the social behaviour of cells. BioSystems, 76, 89-100.

Wang, Z., Birch, C. M. and Deisboeck, T. S. (2008). Cross-scale sensitivity analysis of a non-small cell lung cancer model: Linking molecular signaling properties to cellular behavior. BioSystems, 92, 249-258.

Wheeler, S. (2005). It pays to be popular: A study of civilian assistance and guerilla warfare. Journal of Artificial Societies and Social Simulation, 8(4)9, https://www.jasss.org/8/4/9.html

White, D. J. and Smith, V. A. (2007). Testing measures of animal social association by computer simulation. Behaviour, 144, 1447-1468.

Wilhite, A. (2001). Bilaterial trade and 'small-world' networks. Computational Economics, 18, 49-64.

Wilhite, A. (2006). Protection and social order-driven. Journal of Economic Behavior and Organization, 61, 691-709.

Wilson, R. (2007). Simulating the effect of social influence on decision-making in small, task-oriented, groups. Journal of Artificial Societies and Social Simulation, 10(4)4, https://www.jasss.org/10/4/4.html

Wittwer, L. N. (2006). An exploration of equipping a future force warrior small combat unit with non-lethal weapons. Naval Postgraduate School).

Wohlmuth, J. and Anderson, J. V. (2006). Model financial markets with agents competing on different time scales and with different amount of information. Physica A, 363, 459-468.

Wolf, E. (2003). Using agent-based distillations to explore logistics support to urban, humanitarian Assistance/Disaster relief operations. Naval Postgraduate School).

Worden, L. and Levin, S. A. (2007). Evolutionary escape from the prisoner's dilemma. Journal of Theoretical Biology, 245, 411-422.

Wu, J. and Hu, B. (2007). Modeling and simulation of group behavior in E-government implementation. Proceedings of the 2007 Winter Simulation Conference, 1284-1291.

Xie, Y., Batty, M. and Zhao, K. (2007). Simulating emergent urban form using agent-based modeling: Desakota in the suzhou-wuxian region in china. Annals of the Association of American Geographers,97(3), 477-495.

Xie, Y., Shortle, J. and Donohue, G. (2004). Airport terminal-approach safety and capacity analysis using an agent-based model. Proceedings of the 2004 Winter Simulation Conference, 1349-1357.

Yamamoto, H., Ishida, K. and Ohta, T. (2004). Modeling reputation management system on online C2C market. Computational and Mathematical Organization Theory, 10, 165-178.

Yan, L., Ropella, G. E. P., Park, S. and Hunt,Michael S.Roberts and C.Anthony. (2008). Modeling and simulation of hepatic drug disposition using a physiologically based, multi-agent in silico liver.Pharmaceutical Research, 25(5), 1023-1036.

Yniguez, A. T., McManus, J. W. and DeAngelis, D. L. (2008). Allowing macroalgae growth forms to emerge: Use of an agent-based model to understand the growth and spread of macroalgae in florida coral reefs, with emphasis on halimeda tuna. Ecological Modelling, 216, 60-74.

Yuhara, N. and Tajima, J. (2006). Multi-driver agent-based traffic simulation systems for evaluating the effects of advanced driver assistance systems on road traffic accidents. Cognition, Technology and Work, 8, 283-300.

Zarboutis, N. and Marmaras, N. (2004). Searching efficient plans for emergency rescue through simulation: The case of a metro fire. Cognition, Technology and Work, 6, 117-126.

Zhang, J. (2003). Growing silicon valley on a landscape: An agent-based approach to high-test industrial clusters. Journal of Evolutionary Economics, 13, 529-548.

Zhang, J. (2004). Residential segregation in an all-intergrationist world. Journal of Economic Behavior and Organization, 54, 533-550.

Zhang, T. and Zhang, D. (2007). Agent-based simulation of consumer purchase decision-making and the decoy effect. Journal of Business Research, 60, 912-922.

Zhao, J., Szidarovskzy, F. and Szilagyi, M. N. (2007). Finite neighborhood binary games: A structural study. Journal of Artificial Societies and Social Simulation, 10(3)3, https://www.jasss.org/10/3/3.html

Zhao, J., Szilagyi, M. N. and Szidarovszky, F. (2008). n-person battle of sexes games - a simulation study. Physica A, 387, 3678-3688.

Ziervogel, G., Bithell, M., Washington, R. and Downing, T. (2005). Agent-based social simulation: A method for assessing the impact of seasonal climate fore applications among smallholder farmers. Agricultural Systems, 83, 1-25.

References

ASHBY, W. R. (1970). Analysis of the system to be modeled. The process of model-building in the behavioral sciences (pp. 94-114) Ohio State University Press.

AXTELL, R., Axelrod, R., Epstein, J. M. and Cohen, M. D. (1996). Aligning simulation models: A case study and results. Computational and Mathematical Organization Theory, (1), 123-141.

BALCI, O. (1998). Verification, validation and accreditation. Proceedings of the 1998 Winter Simulation Conference, 41-48.

BANKES, S. C. (2002). Agent-based modeling: A revolution? Proceedings of the National Academy of Sciences of the United States of America, , 99(10) 7199-7200.

BANKS, J., Carson, J. S., Nelson, B. L. and Nicol, D. M. (2001). Discrete-event system simualtion (3rd ed.). Upper Saddle River, NJ: Prentice-Hall.

BONABEAU, E. (2002). Agent-based modeling: Methods and techniques for simulating human systems. Proceedings of the National Academy of Sciences of the United States of America, , 99(10) 7280-7287.

CASTI, J. L. (1995). Complexification: Explaining a paradoxical world through the science of surprise (1st ed.) HarperPerennial.

DAVIS, P. K. and Blumenthal, D. (1991). The base of sand problem: A white paper on the state of military combat modeling. Santa Monica, CA: RAND.

EPSTEIN, J. M. (1999). Agent-based computation models and generative social science. Complexity, 4(5), 41-60.

EPSTEIN, J. M. and Axtell, R. (1996). Growing artificial societies: Social science from the bottom up. Washington, D.C.: Brookings Institution Press.

HEATH, B.L. and Hill, R.R. (2009). The emergence of agent-based modeling. Journal of Simulation. forthcoming.

JENSEN, P. A. and Bard, J. F. (2003). Operations research models and methods. John Wiley.

KUPPERS, G., Lenhard, J. and Shinn, T. (2006). Computer simulation: Practice, epistemology, and social dynamics. In G. K. Johannes Lenhard and Terry Shinn (Eds.), Simulation: Pragmatic construction of reality; sociology of the sciences yearbook (pp. 3-22). Dordrecht, The Netherlands: Springer.

LAW, A. M. (2007). Simulation, modeling and analysis (4th ed.). New York, NY: McGraw-Hill.

MACAL, C. M. and North, M. J. (2006). Tutorial on agent-based modeling and simulation part 2: How to model with agents. Proceedings of the 2006 Winter Simulation Conference, 73-83.

MILLER, J. H. and Page, S. E. (2007). Complex adaptive systems: An introduction to computational models of social life. Princeton, NJ: Princeton University Press.

MORRIS, W. T. (1967). On the art of modeling. Management Science, 13(12), 707-717.

MORRISON, M. and Morgan, M. S. (1999). Models as mediating instruments. In M. S. Morgan and M. Morrison (Eds.), Models as mediators (pp. 10-37). Cambridge: Cambridge University Press.

NORTH, M. J. and Macal, C. M. (2007). Managing business complexity: Discovering strategic solutions with agent-based modeling and simulation. New York, NY: Oxford University Press.

THE OHIO LIBRARY AND INFORMATION NETWORK (2008). www.ohiolink.org

ROBINSON, S. (2008). Conceptual modelling for simulation part I: Definition and requirements. Journal of Operational Research Society, 59, 278-290.

ROBINSON, S. (2006). Conceptual modeling for simulation: Issues and research requirements. Proceedings of the 2006 Winter Simulation Conference, 792-800.

SARGENT, R. G. (2005). Verification and validation of simulation models. The Proceedings of the 2005 Winter Simulation Conference, 130-143.

STANISLAW, H. (1986). Tests of computer simulation validation: What do they measure? Simulation and Games, 17(2), 173-191.

WINSBERG, E. (1999). Sanctioning models: The epistemology of simulation. Science in Context, 12(2), 275-292.

ZEIGLER, B. P., Praehofer, H. and Kim, T. G. (2000). Theory of modeling and simulation (2nd ed.) Academic Press.

Return to Contents of this issue

© Copyright Journal of Artificial Societies and Social Simulation, [2009]