1.1

In recent years there has been an explosion of published literature utilising Multi-Agent-Based Simulation (MABS) to study social, biological and artificial systems. This kind of work is evidenced within JASSS but is increasingly becoming part of mainstream practice across many disciplines.

1.2

However, despite this plethora of interesting models, they are rarely compared, built-on or transferred between researchers. It would seem there is a dearth of "model-to-model" analysis. Rather researchers tend to work in isolation, designing all their models from scratch and reporting their results without anyone else reproducing what they found. Although the opposite extreme, where all that seems to happen is the next twist on an existing model, is not to be wished for, there are considerable dangers if everybody only works on their own model. Part of the reason for this is that models tend to be very seductive - especially to the person who has built the model. What is needed is a third person to check the results. However it is not always clear how people who are not the modeller can interpret or utilise such results, because it is very difficult to replicate simulation models from what is reported in papers.

1.3

It was for these reasons that we called on the MABS community to submit papers for a model-to-model (M2M) workshop. The aim of the workshop was to gather researchers in MABS who were interested in understanding and furthering the transferability of knowledge between models. We received fourteen submissions from which (after a process of peer review) eight were presented at the workshop. Of the six articles that comprise this special issue, five were presented at the workshop.

1.4

It is now clear (Gilbert and Troitzsch 1999) that MABS has more in common, methodologically, with the natural sciences and engineering disciplines than deductive logics or mathematics - it is closer to an experimental science than a formal one. With this in mind, it is important that simulations be replicated before they are accepted as correct. That is results from simulations cannot be proved but only inductively analysed. This indicates that the same kinds of methods used within other inductive sciences will be applicable. In its simplest form a result that is reproduced many times by different modellers, re-implemented on several platforms in different places, should be more reliable. Although never attaining the status of a proof we can become more confident over time as to the veracity of the results - i.e. progress can be made.

1.5

Another methodological technique that we can steal from the natural sciences is the comparison of results from seemingly diverse phenomena. Such a comparison may allow some kind of unifying phenomena or underlying theory to be discovered. In our case this would involve taking MABS applied to different phenomena and aligning them with each other (or a third generic model) in order to show where their behaviour intersects. This is an exciting possibility because it offers the potential for importing whole sets of results from one area or discipline into another. In this way MABS could be used as a kind of common language or translation mechanism for partial theories in a complex systems domain.

1.6

There are quite a number of ways that models can be compared, including:

• Rewriting models that others have described in papers so as to understand them more deeply and reproduce the stated results (Axelrod 1997)
• Composing models where different scales are inter-related in a larger model - the results of one model being used in the other
• Comparing different models that announce the same type of results and trying to see if they actually produce similar (or the same) results - sometimes termed "aligning" of models (Axtell et al. 1996)
• Comparing different models based on their fitness with respect to a set of data;
• Using one model as a post-hoc summary or abstraction of another model's results; thus constraining the scope of an existing model to enable more powerful techniques to be applied in a different computational framework
• Using models with different structures and assumptions to confirm each other's results

Most of the articles published here utilise some mixture of these techniques. We briefly summarise each of these below.

1.7

Takadama et al. compare different learning mechanisms within the domain of a classic "bargaining game". Specifically they vary a single element (learning) within the agents and then examine whether the agents acquire "rational" behaviours (in the neo-classical sense). Moreover they begin to sketch out what they propose as a more general methodology that they term "Cross-Element Validation". They argue that such a "within model" approach potentially offers a better insight into the complex dynamics that MABS often produce.

1.8

Juliette Rouchier re-implements a model of Duffy and Ochs (1999) which is an agent-based version of a model proposed by Kiyotaki and Wright (1989). Despite having a helpful email exchange with Duffy and discovering a lot about the model, she does not replicate his results. Her experience leads her to make some insightful suggestions concerning reporting simulation work, including: that it would be useful to give more detailed lists of individual behaviours (not just averaged data) so as to be able to compare processes; that when the main hypothesis is about learning, it would be useful to have adequate data about the knowledge of the agents and its evolution in time, so as to be able to judge the degree of misrepresentation and its importance; and that it is essential to give a genuine description of the dynamics of the model, with different indicators (and not just the one that is most central to the issue) so as to help the aligning of future models and aid the comprehension of the logical processes in the system.

1.9

Jürgen Klüver and Christina Stoica model group dynamics based on positive and negative associations between individuals represented in "Socio- or Moreno matrices". They compare results from Cellular Automata, Neural Networks (Kohonen Feature Map and Interactive Networks) and Genetic Algorithm implementations. Each of these adaptive algorithms has been used as model of agent learning and adaptation by others but they are rarely compared over the same domain. Not only are the different models shown to produce similar results but these common results are also compared to empirical observations of human behaviour within constrained domains. This process can be seen as a method by which the artefacts that might be produced by any single learning algorithm can be removed (Takadama et al do a similar thing in their paper in this issue) before comparison with empirical observation.

1.10

Margaret Edwards, Sylvie Huet, François Goreaud, Guillaume Deffuant compare two versions of a model of innovation diffusion (Young 1999). One of these is an individual-based model, representing each of the actors separately and one of these is an equation-based model formed as a result of applying a mean field approximation. They found that these two models sometimes aligned in terms of the results and sometimes did not. The equation-based model provided an explanation in terms of the local maxima and hence attractor basins of the equation-based model (as well as being a more computationally efficient way of computing outcomes when they did align). Where there was more than one attractor there was a chance that the equation-based model (being deterministic) would get trapped in the minority basin, whilst the individual-based model would eventually escape from this to the principal attractor due to is stochastic nature.

1.11

Claudio Cioffi-Revilla and Nick Gotts relate two seemingly unrelated models from distinct domains: GeoSim, a model of military conflict and FEARLUS, a model of land use and ownership change. They follow a process of relating both models to an abstracted model class they term "TRAP²". By looking for the common features and specify this as an abstracted class they initiate an approach that would seem to indicate a kind of extensible typology of MABS (based on identifying and abstracting classes and relating them to existing models in the literature). They claim that this process could facilitate the transfer of knowledge between distinct domains.

1.12

Bruce Edmonds and David Hales re-implement a model concerning the evolution of cooperation using "tags", originally published by Riolo, Cohen and Axelrod (Riolo et al. 2002). As part of the re-implementation they both re-implemented the model on different platforms and aligned (or docked) their models before comparing their results with the previously published results. By performing this initial "double" implementation they can be more confident in the accuracy of their implementation than they could be with a single re-implementation. Finally they docked their model with the published results. However, the process of duel implementation helped to uncover inaccuracies in the original interpretation placed on the model by Riolo et al. Indeed they claim to have invalidated the central claim the model was published to support.

1.13

The workshop was a tightly focused event that brought together researchers in agent-based modelling from across the globe. Those presenting work were given ample time to explain complex ideas and plenty of time for answering questions. This contributed to a stimulating and supportive atmosphere in which all participants had time to ask questions and discuss points of issue.

1.14

Overall it was agreed by all that the level of discussion was of a high quality and we hope that at least some of that debate is reflected in the updated articles that form part of this special issue.

1.15

During those discussions a number of "open issues" were identified. These included:

• Stimulated by Claudio Cioffi-Revilla and Nick Gotts' paper, a discussion began as to whether a "typology" of MABS could be produced? Can further "abstract classes" of models be identified and related to existing models? Should such classes be formally specified? Will different researchers agree on abstracted classes and who's going to do all the work? It is certainly clear that there is currently nothing close to a common framework in MABS, so any progress is to be welcomed.
• One interesting point raised by Marco Janssen was that MABS can be used to better understand existing (more traditional and more empirically validated) models by implementing agents following such models but relaxing previous constraints (such as agent homogeneity).
• Almost all the participants believed that it would be useful to integrate or align top-down with bottom-up models (i.e. equation based models with individual based models) as is demonstrated in Margaret Edwards's (and colleagues) paper. However, it was also noted that currently it is uncertain whether such techniques could scale up to more complex cases.
• Jürgen Klüver and Christina Stoica demonstrated the conditions under which several popular adaptation algorithms converged in their results - which is of interest in itself. However, it was agreed that also of interest (perhaps even greater interest) are the conditions under which models don't agree. It's currently unclear as to how one would go about achieving this.
• Andreas Flache raised the general issue of formulating heuristics or methodologies for model alignment. Currently it's a bit of a "black art" but surely a more structured approach can be outlined?
• As argued for by Bruce Edmonds, David Hales and Juliette Rouchier, there was general agreement that replication of MABS models was a worthwhile exercise for increasing confidence and understanding.

1.16

The M2M workshop was held in Marseille over two bright sunny days earlier this year (from the 31st of March to the 1st of April 2003). We believe it was a worthwhile and significant event, as well as being highly enjoyable. For more information about the workshop and the papers there see http://cfpm.org/m2m.

1.17

We hope that the workshop and this special issue will begin a serious strand of work aimed at the productive comparison and re-use of agent-based simulation models and their results. The ultimate aim of such work is to approach a level of rigor and reproducibility that has become the norm in the natural sciences. We hope that a second M2M workshop (M2M2) will be organised sometime in 2005.

1.18

All those involved in M2M wish to express their gratitude for the kind support offered by Grequam/CNRS [http://durandal.cnrs-mrs.fr/GREQAM/index.htm] and the professional manner in which the local organisation was executed. It really was a pleasure to attend.

References

AXELROD R., 1997, Advancing the Art of Simulation in the Social Sciences, R. Conte and R. Hegselmann and P. Terna (eds) Simulating Social Phenomena, Springer-Verlag. Selected Papers TBA, Berlin, pp 21-40.

AXTELL, R., Axelrod R., J.M. Epstein and M.D. Cohen (1996), "Aligning Simulation Models: A Case Study and Results", Computational and Mathematical Organization Theory 1(2), pp. 123-141.

DUFFY, J. and Ochs J. (1999) Fiat money as a medium of exchange: Experimental evidence, working paper, University of Pittsburgh. http://www.pitt.edu/~jduffy/papers.html#IER

GILBERT, N. and Troitzsch, K. G. (1999) Simulation for the Social Scientist. Open University Press.

KIYOTAKI , N. and Randall Wright, 1989, On Money as a Medium of Exchange, Journal of Political Economy, vol 97, pp 927-954.

RIOLO, R. L., Cohen, M. D. and Axelrod, R (2001), Evolution of cooperation without reciprocity. Nature, 411:441-443.

YOUNG, P. (1999) "Diffusion in Social Networks", Center on Social and Economic Dynamics, Working Paper No 2., Brookings Institution.

Return to Contents of this issue

© Copyright Journal of Artificial Societies and Social Simulation, [2003]