Learning opinions by observing actions Simulation of opinion dynamics using an action-opinion inference model

: Opinion dynamics models are based on the implicit assumption that people can observe the opinions of others directly, and update their own opinions based on the observation. This assumption significantly reduces the complexity of the process of learning opinions, but seems to be rather unrealistic. Instead, we argue that the opinion itself is unobservable, and that people attempt to infer the opinions of others by observing and interpreting their actions. Building on the notion of Bayesian learning, we introduce an action-opinion inference model (AOI model); this model describes and predicts opinion dynamics where actions are governed by underlying opinions, and each agent changes her opinion according to her inference of others’ opinions from their actions. We study different action-opinion relations in the framework of the AOI model, and show how opinion dynamics are determined by the relations between opinions and actions. We also show that the well-known voter model can be formulated as being a special case of the AOI model when adopting a bijective action-opinion relation. Furthermore, we show that a so-called inclusive opinion, which is congruent with more than one action (in contrast with an exclusive opinion which is only congruent with one action), plays a special role in the dynamic process of opinion spreading. Specifically, the system containing an inclusive opinion always ends up with a full consensus of an exclusive opinion that is incompatible with the inclusive opinion, or with a mixed state of other opinions, including the inclusive opinion itself. A mathematical solution is given for some simple action-opinion relations to help better understand and interpret the simulation results. Finally, the AOI model is compared with the constrained voter model and the language competition model; several avenues for further research are discussed at the end of the paper.


Introduction .
In social psychology, the relation between opinions and actions has been extensively studied with a di erent term called attitude-behaviour consistency/ inconsistency, where attitudes resemble opinions, and behaviors resemble actions in this paper. Intuitively, early studies assumed that "attitudes predict overt behaviors" (Zanna et al. ), but this notion has been challenged by psychologists who found that the attitude-behavior relationship is considerably weak (Wicker ), and may be influenced by other factors (Liska ). It is called the problem of attitude-behavior inconsistency. One of the most influential and successful models that aims to explain the problem should be the Fishbein-Ajzen model (Fishbein & Ajzen ). The model proposes that behavior is directly driven by behavioral intention, and intention depends on both attitudes and subjective norms (Liska ), providing a conceptualized framework to analyze multiplicity in attitude-behavior relations, which might shed light on opinion-action relations that will be used in our paper. .
In this paper, we propose a simple alternative model of opinion-learning to simulate opinion dynamics in an artificial society. The central assumption embedded in this so-called action-opinion inference (AOI) model is the notion of "learning opinions by observing actions", which means that the agent attempts to learn the opinions of others by observing and interpreting their actions. Crucially, our model assumes that actions are noisy signals of underlying opinions, which follows from the above-discussed multiplicity. Agents try to infer opinions from actions in a Bayesian way, acknowledging the uncertainty inherent to the opinion-action relation. More specifically, acknowledging the multiplicity of action-opinion relations, our model is able to describe the situation where an action is permitted by more than one opinion, and one opinion could result in various actions. Therefore observing an action will in most cases only allow the observer to partially update her beliefs about the other agent's underlying (latent) opinions. The model postulates that agents update their opinions and actions by a three-stage mechanism: the agent first uses Bayes' rule to update her beliefs regarding her neighbors' opinions, based on their actions which she observes. The inference of opinions from observed actions is determined by an action-opinion matrix, which defines, for each action and each opinion, if the action is either prohibited, permitted, or obliged by that particular opinion. In the second stage, the agent chooses her new opinion for the next time step according to the relative probability of each opinion in the neighborhood, calculated from the inferences of di erent opinions. In stage three she updates her action according to the new opinion she selected just now. Having performed an extensive literature review, we only found one model whose conceptualization of latent opinions and observable actions relates to our work in physics. This so-called CODA model (Martins ) and its relation to our work will be described in Section . .

.
We compare the AOI model systematically with the voter model, which we will prove can be formulated as a reduced form of the AOI model. More specifically, the AOI model is equivalent to the voter model when each action is only obliged by one opinion, and each opinion only obliges one action (when there is no uncertainty as to which opinion causes which action). The model setup o ers an opportunity for us to explore di erent actionopinion relations, which have rarely been studied before. In the paper we employ cellular automata (CA) to simulate the model with di erent action-opinion relations and analyze the simulation results both numerically and spatially, focusing on the density of each opinion in the final stable state, as well as the clustering features of the dynamic process. The key question we investigate in this paper is as follows: "what action-opinion relations induce consensus or diversity?" In other words, we would envisage the role of the action-opinion relations in the formation of macroscopic features of the society. This question is closely related to the well-known Axelrod's question, which asks why consensus is not always reached given that agents learn opinions from others (Axelrod ). In this sense, the AOI model provides an alternative approach to answer Axelrod's question besides the conventional models mentioned before. .
The rest of the paper is organized as follows: Section Public & Private Characteristics: A Brief Review, as the name indicated, briefly reviews the relevant works on public and private characteristics, a similar concept to our notion of learn opinions by observing actions. Section Model Setup presents the Action-opinion inference model in detail. Section Two-Action Situation and Section Three-Action Situation illustrate the simulation results of the model with two and three actions respectively. Additionally, a brief mathematical analysis is given in Section for the system with two opinions and two actions only. The mathematical analysis provides the first step towards validation of the simulation results, and helps us better understand the dynamic process. Section Discussion and Conclusion summarizes the major findings and discusses some critical issues concerning the AOI model and other complex system models. Furthermore, we discuss several avenues for further research.

Public & Private Characteristics: A Brief Review
. Although the notion of learning opinions by observing actions was not frequently acknowledged in previous literature, a similar pair of concepts -private and public characteristics -has been employed in previous opinion dynamics studies to capture the discrepancy in the learning process. Here, the term "characteristics" may refer to opinions, attitudes, actions, or any property of an agent that is open to the influences from others. "Public characteristic" represents the observable characteristic publicly expressed by an agent. Conversely, a private characteristic is defined as an agent's privately held characteristic. Therefore, a public characteristic can be observed directly but is not necessarily the same as the agent's private characteristic. An early example comes from information cascade. Information cascade, which is defined as the situation when agents simply follow the actions of the others sequentially without considering their own private information (Bikhchandani et al. ), is a powerful tool to explain localized conformity and its systematical fragility (Bikhchandani et al. ; Wu & Huberman ). Both information cascade and the AOI model have roots in the same idea that the public characteristic obtained from a neighbor may not be the same as the neighbor's private characteristic, and the agent takes actions by making inference from that possibly inaccurate public characteristic. In our notation, an action serves as the public characteristic, and the opinions beneath is the true but private characteristic. The discrepancy between public and private characteristic in information cascade originates from the fact that agents simply ignore their own private characteristic (i.e. opinions) when taking actions, but in the AOI model, it is because of the unobservable relations between actions (public characteristic) and opinions (private characteristic). Therefore the two notions have di erent underlying principles. .
A more recent representative of public and private characteristics is the persuasion model (Mäs et al. ; Mäs & Flache ; Mäs & Bischofberger ) based on psychological theories (e.g. Fishbein ; Petty et al. ). Di erent from traditional opinion dynamics models, the persuasion model assumes that opinions are formed based on arguments, and agents only exchange arguments, so opinions are not directly influenced by others (Mäs & Bischofberger ). One may realize that the underpinning of our AOI model is inherently close to this assumption in the persuasion model. In both models, opinions (i.e. the private characteristic) play no role in the communication (not necessarily verbal) directly, while some other public characteristics, which refer to actions in the AOI model and arguments in the persuasion model, serve as the messenger between agents. In the persuasion model agents learn arguments from others, and form new opinions based on the arguments; Meanwhile, in the AOI model agents observe actions of others, and update their opinions according to the interpretation of the observations. The primal di erence of the two models lies in the relations between private characteristic (i.e. opinions) and public characteristic (i.e. action or argument): in the persuasion model, opinion is a function of some relevant arguments, and thus arguments can a ect opinions, but not vice versa. In other words, an agent's opinions are only a ected by another agent's arguments. On the contrary, the opinion in the AOI model, together with the action-opinion matrix, determines the action; and only another agent's actions, via the inference process, can a ect the agent's opinion. That is, actions are a function of opinions. This structural disparity distinguishes the two models at a microscopic level, and thus will lead to distinct outcomes at a macroscopic level (see further below). It is noteworthy that in practice, persuasion models usually adopt opinion homophily, that is, each agent selects an agent she wants to interact with based on the similarities of their opinions (Mäs & Flache ), so opinions are still observable in such models, playing the role of labels in the partner selection phase. Opinion homophily should be partly responsible for the persuasion model's ability to explain opinion polarization. .
Another famous phenomenon describing disparities between public and private characteristics is pluralistic ignorance, in which most members of a society privately disapprove of, or are undecided about, an opinion but incorrectly believe that most other members accept it (Miller & McFarland ; Huang & Wen ). Considering themselves as the only dissident, they would express their approval of the opinion that they do not actually support. Pluralistic ignorance results in a global consensus although most members disagree with it, and hence the consensus is so fragile that it could be broken by the so-called minority influence (Huang & Wen ). As Seeme and Green explained, the term "opinion" in studies of pluralistic ignorance, rather than in the AOI model, refers to "the expression or behavior of a person towards a topic", instead of one's "true internal opinion" (Seeme & Green ). The "true internal opinion" is called "attitude" by Seeme & Green ( ) to avoid confusion. According to this claim, we find that the AOI model and the pluralistic ignorance studies both describe the discrepancy of the observable "public characteristic", which is actions in the AOI model and publicly expressed opinions in pluralistic ignorance, and the "private characteristic", which refers to the (underlying) opinions in the AOI model and (private) attitudes in pluralistic ignorance. The critical di erence between these studies is obvious: the dynamics of public characteristic (actions) in the AOI model is driven by the dynamics of private characteristic (opinions). As stated in Section . , public characteristics (actions) are not directly influenced. However, in pluralistic ignorance models, public characteristics are directly a ected by the "the pressure to conform" (Seeme & Green ) or "normative social influence" (Huang & Wen ), and private characteristics (attitudes) are later updated according to either "self-perception theory", "cognitive dissonance" (Seeme & Green ), or other psychological theories.
The table shows the similarities as well as di erences between some notable earlier work and the AOI model. In all, although previous researches have noticed the existence of public and private characteristics and described them in various models, these models do not capture the inference process that enables agents to learn the private characteristics of others by observing public characteristics. The absence of an inference process leads to the omission of uncertainty: in these three models, the relations between public and private characteristics are either deterministic (persuasion model & pluralistic ignorance) or unspecified (information cascade). In contrast, the AOI model creates a smokescreen between public characteristics (actions) and private characteristics (opinions), which represents the multiplicity of the action-opinion relations. This type of uncertainties, although rarely acknowledged in opinion dynamics papers, could lead to misunderstanding or obfuscation, and the role of the uncertainties in opinion dynamics will be one of the central problems we investigate in the rest of the paper.

Model Setup
. We consider a population of N agents on an L × L regular lattice with periodic boundary conditions as well as a Von Neumann neighborhood. Each cell of the lattice is occupied by an agent, and we set L 2 = N to avoid empty cells. Agent i (i = 1, 2, . . . , N ) chooses one action a (i) from the action set A = {a 1 , . . . , a g , . . . , a G } based on her opinion, described by a rule r (i) selected from the rule set R = {r 1 , . . . , r k , . . . , r K }. Note that, in terms of terminology, we choose to use the term "opinion" in colloquial discussions, and we use the term "rule" in the context of the mathematical model and simulations. The evaluation of action a g by rule r k is denoted by s kg . In case that r k is an exclusive rule, s kg ∈ {+, −} a g is either obliged (+) or prohibited (−) by r k ; however, if r k is an inclusive rule, then s kg ∈ {0, −}, where 0 implies that the action is permitted but not obliged by the rule. An exclusive rule can only oblige one action, but an inclusive rule always permits more than one action. All s kg (g = 1, . . . G; k = 1, . . . , K) constitute a K × G matrix S, called the action-opinion matrix, summarizing the action-opinion relations in the system. .
The behaviors of the agents are described as follows: if an agent follows an exclusive rule r k , then she will certainly take the action obliged by the rule, that is, P (a g |r k ) = 1 if s kg = +. Otherwise P (a g |r k ) = 0. If she believes in an inclusive rule r k , then P (a g |r k ) = 1/W if s kg = 0, where W is the number of actions permitted by r k ; P (a g |r k ) = 0 if a g is prohibited (i.e. s kg = −). Besides, the agent can observe the actions of her Von Neumann neighbors but cannot observe their opinions (in the form of rules) directly. In addition, each agent has full knowledge of A, R, S, and she assumes that other agents choose actions and update rules in the same way as she does so herself. .
Proceeding on the preliminary setup, the action-opinion inference process takes the following steps. Initially, each agent (say agent i) is randomly assigned a rule r (i) ∈ R, then chooses the action based on the assigned rule. At each time unit τ , an agent (say agent i) is randomly chosen to update her probabilistic inference about which rule is adopted by her neighbor j (j ∈ M i , M i is the von Neumann neighbors of agent i) based on the observation of j's action a (j) (τ ). Specifically, agent i's inference that neighbor j adopts r k a er observing a (j) (τ ) takes the form: where P (a (j) (τ )|r k ), which has been defined in 3.2, is the probability that an agent acts as a (j) given the rule r k . It should be noted that all the agents have the same inference strategy, thus . Equation is derived from the Bayes' rule by setting equal prior probabilistic beliefs P (r k ) for all k, that is, P (r k ) = 1/K, ∀k. The intuition behind this is that the agent a priori assumes that each rule is equally likely to be taken by her neighbor j. This assumption is reasonable in light of the fact that in each time unit only one agent is selected to observe the neighborhood and then update rules and actions, so it is highly likely that she did not observe her neighbor's action in a recent time unit.
. By updating her probabilistic inference of all neighbors' rules, agent i learns the local opinion distributions, based on which she will update her own opinion (i.e. rule). To do so, agent i evaluates the accumulative probability of each rule across the neighborhood: ..,K helps agent i to estimate the occurrence of each rule. Agent i will adopt r k as her rule for the next time unit with the probabilitŷ .
A er the rule-updating process, the world moves to the next time unit τ + 1. Because in each time unit only one agent updates, it is ine icient to study the dynamics between time units. Instead, we denote N consecutive time units as one time step t so that each agent has been selected once during one time step on average. This is a common practice adopted by many opinion dynamics models (Suchecki et al. ). We will present the dynamics of key variables in the scale of time step in the rest of the paper.
. The voter model is chosen as the benchmark for the AOI model not only because of its long-standing popularity in the discipline of opinion dynamics for explaining the emergence of consensus (Sood & Redner ; Barrat et al. ; Krapivsky et al. ), but also due to the fact that the AOI model is built upon the framework of the voter model: despite the AOI model's learning process, the basic dynamics of the two models are the same, thus using the voter model as a benchmark helps derive implications regarding the e ect of the inference. Moreover, variations of the voter model have been applied to a wide range of social phenomena besides opinion dynamics, which provides examples for the AOI model to be modified for other disciplines. The reason why we use the voter model as the basis of the AOI model is also related to its simplicity. As argued by many, the voter model is the simplest and minimal model for the study of opinion dynamics, so its basic framework helps maintain the simplicity and comprehensibility of the new model. For example, if the social influence model were chosen as the basis, we might have trouble deciding which type of social influence we would like to use -positive, negative, assimilative, or similarity biased. Combining the action-opinion inference with various social influence models is a promising line of further research, but for the first work on the AOI model, it is better to avoid unnecessary subtleties arising from model setup and focus on the role of action-opinion inference process in governing dynamics. .
The voter model is an example of discrete opinion models, where opinion is represented by a discrete variable. Therefore in the AOI model, rules and actions are also discrete. Besides discrete opinion models (e.g. voter model, majority rule model (Galam ), Sznajd model (Sznajd-Weron )), many sociological models describe variables of interest in a discrete way, including Latané's social impact theory (Latané ) and Axelrod's model of cultural dissemination (Axelrod ). Continuous opinion models have taken an alternative approach, where opinions can vary between extreme values smoothly (Castellano et al. ). De uant model (De uant et al. ), Hegselmann-Krause model (Hegselmann & Krause ) and social influence models (Flache et al. ) are famous examples of continuous opinion models. A model, mentioned in the introduction, which shares with our model its distinction between latent opinions and observable actions, is the so-called CODA model (Martins ). That model di ers from ours in the following fundamental ways: the CODA model postulates that opinions refer to a ground (or: universal) truth, which agents attempt to uncover by learning from each others' actions in a Bayesian fashion. In contrast, the AOI model is not concerned with learning truths, but inferring the latent beliefs of neighbours, induced by a wish to conform with those neighbours in terms of their latent beliefs. This is why the AOI model postulates that opinions and actions are discrete and multinomial, as opposed to the CODA model whose actions are discrete and binary, and whose opinions are statements in the form of a continuous probability function about something being true or not. For this same reason, the AOI model distinguishes between an inference step in which agents try to infer (learn) each others' opinions, and an update step in which agents update (adjust) their own opinion as a probabilistic function of neighbours' opinions. This distinction is absent in the CODA model, which is logical given that it is concerned only with learning about truths. Conceptually, the CODA model is related to literature about truth-seeking agents (Prelec ), whereas the AOI model is focused on opinion-conformity among agents with diverse latent opinions. An important distinction between the models in terms of what macro-level phenomena they tend to predict, is that the CODA model tends to generate extremism, whereas the AOI model, depending on the structure imposed on action-opinion relations may generate either consensus, coexistence of various opinions, or extremism. This di erence in predicted outcomes is rooted in the fact that whereas the CODA model features one relation between binary actions and an underlying opinion, whereas the AOI model features a broader set of relations between various actions and various opinions which may either be exclusive or inclusive (see Section . ).

Two-Action Situation
. For simplicity, we first focus on the two-action AOI model (i.e. G = 2). Given a two-action set {a 1 , a 2 }, there are in total possible action-opinion matrices S 1 to S 4 for K ≤ 3, if one does not allow for duplication. .
Note that S 2 and S 3 are identical in nature, we will study S 1 , S 2 and S 4 only, which cover all action-opinion relations in a two-action situation when there are more than one rule. AOI model with S 1 and the voter model .
When taking S 1 as the action-opinion matrix for the system, the AOI model reduces to a two-state voter model, a naive spin model where agents observe and learn opinions directly. This serves as a simple but representative example of studies in opinion dynamics and consensus formation (Krapivsky et al. ). In the voter model, a randomly chosen agent adopts the opinion of a neighbor who is also chosen at random (Dornic et al. ). The voter model can therefore be interpreted as a special case of the AOI model, the two models being equivalent when in the AOI model, each action is only obliged by one rule, and each rule only obliges one action (i.e. S 1 ). Unsurprising, the AOI model with S 1 shows all the features of a two-state voter model (Figure ). Starting with a random initial configuration, the population eventually converges to an absorbing state of consensus where everyone takes the same opinion (and action). Also, the emergence of opinion clustering and coarsening shown in Figure  An inclusive rule (r 2 in S 2 and r 3 in S 4 ) that permits both actions is introduced in S 2 and S 4 . S 4 constitutes a typical case in politics where both supporters of a party (believe in r 1 or r 2 ) and indi erent voters (believe in r 3 ) exist. See Section . and Section . for a discussion of how these indi erent voters can be seen as centrists. A striking observation from the simulation of the system with S 4 (Figure ) is that the two key features of the voter model (i.e. the AOI model with S 1 ), clustering and consensus, are no longer valid when an inclusive rule is introduced (i.e. using S 4 ).   Holley & Liggett ( ) have proved that coexistence of opinions is impossible in a two-dimensional voter model, and many e orts have been made to modify the model in order to reach a mixed state where more than one opinion survives. Such modifications include the threshold voter models (Liggett ) where the agent adopts the opposite opinion only when the number of neighbors with opposite opinions is large enough, and the threestate constrained voter model (Vazquez et al. ), where the le ists and rights only interact with the centrists. Additionally, if the voter model is run on a small-world network (Watts & Strogatz ), the system will be temporarily trapped in a metastable state where di erent opinions coexist, although it will escape from the metastable state and reaches consensus eventually (Castellano et al. , ). In Figure  old and constrained voter models) or modifying the network structure (as in the small-world network).
. Figure , which concerns S 4 , is straightforward. For example, if an observer (the focal agent) sees a neighbor acting as a 1 , she considers the neighbor believes in r 1 with probability P (r 1 |a 1 ) = 1 1+0.5 = 2 3 , and r 3 with a smaller probability P (r 3 |a 1 ) = 0.5 1+0.5 = 1 3 according to S 4 . So a neighbor acting as a 1 will increase the observer's probability of adopting r 1 as well as (albeit less so) r 3 . Similarly, an action a 2 of a neighbor will not only increase the observer's probability of adopting r 2 , but also increase her probability of adopting r 3 . The underlying opinion dynamics imply that r 3 will never die out. Likewise, an agent employing r 3 will take a 1 and a 2 with equal probabilities, and therefore a reciprocal loop of opinions is constituted (Figure (c)). The loop shows that each action or rule can reach any other action or rule through a finite number of arrows, which implies that all rules and actions are "beneficial" to all the others. This explains the coexistence of di erent opinions (and actions) in the AOI model with S 4 . Contrarily, Figure (a) shows that in S 1 , r 1 and r 2 are disconnected, so the ultimate consensus is always reached. . In Figure , we imposed an initial configuration with equal densities of the three rules, but this is not the reason why eventually the three rules have almost equal densities. Figure gives four cases of extreme initial configurations, which implies that regardless of the initial densities of the rules, the system will always reach a state where all rules have approximately the same density. On the contrary, the opinion (rule) in the final state of consensus in the voter model is completely determined by the initial configuration. Precisely, the consensus of r 1 occurs with probability P 1 = p 0 and the consensus of r 2 occurs with P 2 = 1 − p 0 , given that the system was initially composed of a fraction p 0 of agents believes in r 1 and a fraction 1 − p 0 of agents believes in r 2 (Krapivsky et al.

The intuition behind the coexistence of opinions in
). To summarize, the initial configuration determines the result of the voter model, but has no e ect on the result of the AOI model with S 4 . The di erent roles of the initial configurations, obviously, result from the emergence/ absence of the inclusive rule r 3 . Comparing Figure (a) with (c), we can see that r 3 plays the role of a bridge connecting the two opposite pairs (r 1 , a 1 ) and (r 2 , a 2 ), and the bridge helps to balance the densities of rules dynamically. Figure illustrates the dynamics in a simple way using the cartoon of a set of communicating vessels. (d) r 1 p = r 2 p = 0, r 3 p = 900 Figure : Simulation results of AOI model with S = S 4 on an L = 30 lattice with a random initial configuration. A = {a 1 , a 2 }, R = {r 1 , r 2 , r 3 }. The system has an initial population of r 1 p believers in r 1 , r 2 p believers in r 2 , and r 3 p believers in r 3 . Figure : Illustration of the AOI model with S 4 in the form of communicating vessels. Containers represent rules, and the liquid level in each container represents the population of the corresponding rule. The le part shows the initial liquid distribution, and the right part shows the stable state of liquid. .
In Figure we present the simulation result of the AOI model with S 2 . S 2 constitutes another interesting case where a 1 is obliged by r 1 and permitted by r 2 , while a 2 is only permitted by a 2 . Clearly, a 1 holds a major advantage over a 2 , but as we can see in Figure (b), a 2 still survives and holds a small but stable fraction of population with the help of r 2 . Again the composition of the stable state has no dependence on the initial configurations ( Figure ). The flowchart corresponding to S 2 can be found in Figure  (b) r 1 p = 900, r 2 p = 0 Figure : Simulation results of AOI model with S = S 2 on an L = 30 lattice with a random initial configuration. A = {a 1 , a 2 }, R = {r 1 , r 2 }. The system has an initial population of r 1 p believers in r 1 , and r 2 p believers in r 2 . .
To summarize, the inclusive rule has three non-trivial e ects on opinion dynamics. First, the inclusive rule prohibits the clustering process; second, consensus is never reached if an inclusive rule is present; finally, the composition of the final population is not determined by the initial densities of rules when inclusive rules are present. In other words, the three key features of the voter model, namely clustering, consensus, and the dependence on the initial configurations, disappear due to the introduction of the inclusive rule. Instead, the inclusive rule leads to a non-clustering dynamics of opinions and a diverse final state of opinions that is not related to the initial configuration at all.

Mathematical derivation .
Given the various forms of action-opinion relations, obtaining a general analytical solution for the AOI model is very di icult, and therefore we have so far used the cellular automata approach to investigate the evolution of the opinions in the previous subsections. However, it is beneficial to provide analytical results for some simple action-opinion matrices, which would help us better understand the evolution dynamics, especially the feature of (in)dependence of initial configurations in a precise manner. In this subsection, we provide the mathematical analysis for S 1 and S 2 , whose simulation results have been displayed in Figure and  .
In S 1 or S 2 there are only two rules in total, thus we can define the rules in a binary way: the rule adopted by the agent i, r (i) , can be either +1 (which means r (i) = r 1 ) or −1 (which means r (i) = r 2 ). We write r (i) (τ ) = r (i) to keep the notation simple. According to Equation and , the flip rate that the agent i changes her rule r (i) is: where s is the current configuration of the system, and the subscript i in w i (s) implies that only agent i changes her rule in an update (i.e. a time unit, see . for reference). The scalar r equals either +1 or −1. The flip rate shown in equation ( ) is analogous to the flip rate in the voter models (Krapivsky et al. ). .
The master equation is easy to derive, but di icult to solve. Instead, we focus on the average opinion (rule) for each agent, namely R(i) ≡< r (i) >, where < . > is the average notation, defined by < x >= x P (x = x )x . In a short enough time interval ∆τ , the rule of agent i changes according to: with probability w i (s)∆τ ( ) Following Krapivsky's path (Krapivsky et al. ), from Equation we notice that agent i's opinion changes by −2r (i) with an instant probability w i (s), then the evolution dynamics of the average opinion is: Substitute Equation into Equation and use (r (i) ) 2 = 1: http://jasss.soc.surrey.ac.uk/ / / .html Doi: . /jasss. and define < r P (r (j) = r|a (j) )r >≡ R (j), which is agent i's perceived average opinion of agent j, gives In the voter model, R(j) = R (j), so the equation reduces to dR(i) dτ = −R(i) + 1 4 j∈Mj R(j). Analogous to magnetization in the vote model, we define the mean magnetization of the system as: m ≡ i R(i)/N , which measures the average opinion of the whole system, and m = +1 means the system reaches the consensus of r 1 , while m = −1 means the consensus of r 2 . Summing Equation over all agents: If we take a close look at R (j), since r can be either +1 or −1, we can rewrite R (j) as: ( ) where we've used P (r (j) = 1|a (j) ) + P (r (j) = −1|a (j) ) ≡ 1. Now by using the action-opinion matrix, we can solve for < P (r (j) = 1|a (j) ) >. .
[S 1 ] From S 1 , we know: On the other hand, one can rewrite

Rearrange and simplify Equation gives
: . For S 1 , it is clear that P (a (i) = a 1 ) = P (r (i) = 1) because believing in r 1 is equivalent to acting as a 1 , and vice versa. Therefore we have N dm dτ = 0, which means the magnetization m is conserved in the AOI model with S 1 (voter model). The conserved magnetization helps to understand the features of the voter model stated in Section . . Also, the result is identical to the result solved for the voter model. .
[S 2 ] From S 2 , it can be calculated that: with probability P (a (j) = a 1 ) 0 with probability P (a (j) = a 2 ) ( ) Therefore < P (r (j) = 1|a (j) ) >= 2 3 P (a (j) = a 1 ) ( ) Substitute Equations , and into Equation and simplify it: By conditional probability calculus, we can obtain that: P (a (i) = a 1 ) = P (r (i) = 1)P (a 1 |r (i) = 1) + P (r (i) = −1)P (a 1 |r (i) = −1) ( ) S 4 shows that P (a 1 |r (i) = 1) = 1 and P (a 1 |r (i) = −1) = 0.5, thus Equation becomes: where we have used P (r (i) = 1) + P (r (i) = −1) = 1. Substitute Equation into Equation and rearrange it: Thus the fixed point is i P (r (i) = 1) = N/2, that is, the probability of believing in r 1 , averaged over the population, is 1/2. Starting from any configuration that i P (r (i) = 1) > N/2, for example, the configuration where r (i) = 1, ∀i, since N dm dτ < 0, will always converge to a (dynamic) state where i P (r (i) = 1) = N/2, which is the stable state shown in Figure a. Similarly, the system starting with the configuration where i P (r (i) = 1) < N/2, because N dm dτ > 0, will still evolve to the same state where i P (r (i) = 1) = N/2. The analytical result helps us understand that in the model with S 2 , why the composition of the stable state is independent of the initial configuration Figure , and why the mixed state of rules is always the final stable state (Figure a).

Three-Action Situation
. Although most studies only deal with two-state voter model due to simplicity, it is promising to study the AOI model with three actions because a larger number, and more subtle, action-opinion relations are possible compared to the two-action situation. For reasons of space limitations, we will not go through all the possible situations, but focus on the cases shown by S 5 , S 6 and S 7 specifically: The AOI model using S 6 is nothing but a three-state voter model, which is widely used in the studies of language competition (Castelló et al. ; Hadzibeganovic et al. ). Unsurprisingly, all features of the twostate voter model (Figure ) are still valid in the three-state voter model: Figure illustrates that consensus is always reached, and each type of consensus (i.e. Figure a, b, and c) has the same probability to become the final absorbing state because the initial densities of all rules are set equal. Additionally, the clustering phenomenon of the three rules is shown in Figure .  . S 6 is constructed by introducing an inclusive rule r 4 (that permits a 1 and a 2 but prohibits a 3 ) to S 5 , so the difference between the simulations result of S 5 and S 6 implies the role of what we call a preferentially inclusive rule. Rule r 4 in S 6 is called a preferentially inclusive rule because it shows strict preference for a 1 and a 2 over a 3 , although it is indi erent between a 1 and a 2 . On the other hand, r 2 in S 2 , r 3 in S 4 , and r 4 in S 7 are called non-preferentially inclusive rules because they are completely indi erent to any action.  In case A, r 4 dies out rapidly, while r 1 , r 2 , and r 3 coexist. Specifically, one of r 1 and r 2 is in a momentary majority alternatively, but r 4 holds a relatively stable share over time (Figure a). The dynamics of rules lead to similar evolutionary paths for actions (Figure b), where a 3 go extinct and a 1 and a 2 coexist. In case B, consensus of r 3 is reached eventually, and r 1 , r 2 and r 4 die out gradually. One can not predict which case we will obtain from one realization of the simulation. There relation between case A and B mimics a trade-o between a smaller chance to become the sole superpower that dominates everyone (r 3 in case B), and a larger chance to dominate around 1/3 of the population (r 1 , r 2 and r 4 in case A). In independent trials, we find that % trials are in case A, and % trials are in case B. The distribution of case A and B suggests that the trade-o is in equilibrium: consider a finite system with equal density for each rule r 1 , r 2 , r 3 and r 4 . Ultimately, the system reaches the consensus of r 3 with probability P B , and reaches the mixed state of r 1 , r 2 , and r 4 with probability P A . So the expected number of agents believes in r 3 , < N 3 >, should be P B N , and the the expected number of believes in any other rule, < N k > (k = 1, 2, 4), is 1 3 P A N . In our trials, we observed that trials are in case A and in case B, so the estimated P A ,P A , is . , and the estimated P B , P B is . . Substitute the two estimated probabilities, we find that: Based on the trials, we conjecture that P A = 0.75, and P B = 0.25, which leads to: which implies that all rules have the same expected population of believers. Consequently, the trade-o is in equilibrium. To better understand the result, imagine a gamble where the player is asked to bet on the most popular rule in the AOI model described by S 6 . Equation tells her that she should be indi erent to any choice, as all betting strategies lead to the same expected payo .
. Figure , describing how the system reaches the consensus of r 3 from a mixed state of all rules. An interesting observation is that besides the single-rule clusters of r 3 , there are also some mixed-rule clusters composed of r 1 , r 2 and r 4 . Comparing Figure with Figure , we can see that the preferentially inclusive rule r 4 reduces the ability of r 1 or r 2 to form a single-rule cluster of its own. The results of the AOI model with S 6 shows that there are two categories for exclusive rules: r 1 and r 2 form a category that coexist with r 4 and cannot form single-rule clusters, and r 3 itself is another category, as r 3 and r 4 are completely incompatible. Here we define that two rules are compatible with each other if there exists at least one action that is allowed (i.e. obliged or permitted) by both of them, and otherwise we say that they are incompatible.

The evolution of case B is illustrated in
If we look at S 6 carefully, it is clear that r 1 and r 2 are compatible with r 4 respectively: both r 1 and r 4 allow a 1 , and both r 2 and r 4 allow a 2 . On the contrary, the two actions permitted by r 4 (i.e. a 1 and a 2 ) are prohibited by r 3 , and the only action obliged by r 3 (i.e. a 3 ) is forbidden by r 4 , so r 3 and r 4 are incompatible. The di erent relations between exclusive rules and inclusive rules explain the di erent clustering features of each rule. Because both r 1 and r 2 are compatible with r 3 , the three rules coexist and no single-rule clusters can be formed. Meanwhile, r 3 and r 4 are incompatible, so r 3 is unlikely to coexist with r 4 . Since r 1 and r 2 coexist with r 4 , r 3 cannot coexist with r 1 and r 2 either. Thus the single-rule cluster of r 3 emerges if it dominates the population (i.e. case B in Figure ). .
The only di erence between S 6 and S 7 lies in the inclusive rule. In S 6 , r 4 is a preferentially inclusive rule that permits a 1 and a 2 but prohibits a 3 . However, in S 7 we are having a non-preferentially inclusive rule r 4 , which permits all actions. Given S 7 , now all exclusive rules and the inclusive rule are compatible, so all rules can coexist ( Figure ). It should be noted that although all rules share almost the same fraction of population on average, the variation of the population believing in r 4 is significantly smaller than other rules. It is equivalent to saying that the share of the population which employs exclusive rules is more likely to be either very small or very large, while the share of the population which employs the inclusive rule is of intermediate magnitude and relatively stable through time.
. To summarize, the inclusive rule "forms an alliance" with all the exclusive rules that are compatible with it (e.g. r 4 and r 1 , r 2 in S 6 ) to compete with, if it exists, the exclusive rule with which it is incompatible (e.g. r 3 in S 6 ). In S 6 r 1 , r 2 and r 4 form an alliance against r 4 , while in S 7 all rules constitute a large alliance. In both cases, the expected population of the believers in each rule in the alliance should be the same ( Figure & ).

Discussion and Conclusion
Discussion: Constrained voter model, language competition, and the AOI model . The above simulations have proved that the action-opinion matrix is the most important factor determining the results obtained from the AOI model. A question that arises naturally is where the matrix originates from? Actually, this question concerns the relations between actions, something which we have not discussed in depth yet.
In S 1 , a 1 and a 2 are two excluding actions, in the sense that it is impossible to be indi erent between a 1 and a 2 , and agents must have a strict preference. On the contrary, in S 4 , r 3 o ers an option for centrists (i.e. the believers in inclusive rules): believers in r 3 are indi erent between a 1 and a 2 , and thus they choose actions randomly. This is related to the constrained voter model (Vazquez et al. ) and its modification (de La Lama et al. ) where agents can be extremists (including le ists and rightists) or centrists. In those models, the centrists, or the undecided agents, serve as an intermediate group that can be converted to one of the extremists, while the extremists do not interact with each other (Castellano et al. ). This is also a common method to implement a bilingual state in language competitions, where any change between the two monolingual states must go through an intermediate state called the bilingual state (Castelló et al. ; Colaiori et al. ).
.  (Castellano et al. ). Conversely, the AOI model with S 4 , as stated in Figure , provides another outcome where the mixed state of all states: r 1 (resembles one of the extremists or monolingual states), r 2 (the same resemblance as r 1 ), and r 3 (resembles the centrists, or the bilingual state), which is an impossible outcome for the other two models.

Brief conclusion and outlook
.
The most important contribution of this paper is to provide an alternative and -in our view-more realistic approach to model the spreading of opinions compared to existing models of opinion dynamics. The new approach, called the Action-Opinion Inference (AOI) Model, is based on the postulate that opinions themselves are unobservable, but may be learned by observing the actions that governed by the opinion; this learning process may be partial given that actions are noisy signals of underlying opinions due to the multiplicity of action-opinion relations. The AOI model captures the "learning opinions by observing actions" process, which is an intuitive assumption but has been ignored in the studies of opinion dynamics. In the AOI model, an agent first observes the actions of her neighbors, and then infers her neighbors' opinions (represented by rules) according to the observations. Then the agent updates her own rule based on the perceived probabilities of each rule among her neighbors.
. We show that the outcome of the AOI model strongly depends on action-opinion relations, described by the action-opinion matrix. When the mapping of the action set A to the rule set R (or vice versa) is a bijective function (e.g. S 1 and S 5 ), the AOI model reduces to a classic voter model. When introducing an inclusive rule that permits all actions to the bijective relation (e.g. S 4 ), the model resembles the constrained voter model. The variation of the action-opinion matrix enables us to investigate a broad range of opinion dynamics. A striking finding from the simulation results for the two-action AOI model is the role of inclusive rules, defined as the rules that permit more than one action, in a competition with other rules. An inclusive rule bridges the actions it permits, which means the rule also bridges the exclusive rules that oblige these actions. An exclusive rule is defined as a rule that obliges only one action. The connection between exclusive rules via the inclusive rule(s) leads to a final mixed state of all these rules, regardless of the initial density of each rule. This phenomenon has never been found in either constrained voter models or bilingual language competitions. The three-action AOI model is more complex, where the inclusive rule forms an alliance with all the exclusive rules that share at least one allowed action to compete with the exclusive rule(s) that shares no allowed actions with the inclusive rule (i.e. S 6 ). The competition between the alliance and the incompatible exclusive rule is a winner-take-all game, but if the alliance wins the whole population, the members share the population equally on average.
. Admittedly, the major limitation of the AOI model is the di iculty to incorporate complex reality into a simple action-opinion matrix. As mentioned before, the simulation outcomes, as well as the analytical solutions, are based on the relations between actions and opinions, which in this paper are represented by a series of simple action-opinion matrices. In fact, the reality is far more complicated than all the matrices we have shown in the paper. First, we are not sure how many underlying opinions can people infer from an action, and sometimes multiple opinions collectively lead to one action. Taking the example of cycling again, it is unfeasible to list all the possible opinions that lead to the action of cycling: besides being environmentally-friendly or cost-sensitive, the cyclist may simply love this sport, or actually he just randomly choose a travel mode and it happens to be cycling today. Moreover, we are not sure if every agent in the society is aware of all the possible opinions. With a slightly di erent matrix, the simulation result could be di erent. Given this limitation, we would recommend first applying the model to some simple and obvious situations. Second, further modifications on the design of action-opinion matrices can be a major challenge in future work to enhance the model's capability to describe reality. It should also be noted that changing the current assumption of discrete opinions, described by "+", "−" and "0", to a more realistic but complicated assumption of cardinal opinions might fit the reality better. Cardinal opinions o er more ways to describe evaluations, rather than simply referring to an action being completely forbidden and completely obliged (or permitted); a consequence would be that a matrix of finite size would no longer represent the full set of possible action-opinion relations. Given that the central concept of the uncertainty in the relations between actions and opinions, which is the key to explaining simulation outcome's independence of initial configurations (see Section and Section ), remains unchanged, we optimistically speculate that the main result of the discrete opinion version should be robust in the cardinal opinion version of the model.

.
In all, the AOI model establishes a new framework for researchers to cope with the latency of opinions and with a variety of presumed action-opinion relations. We believe that the AOI model does not only serve as another modification of the voter model, but also constitutes an attempt to study the spreading of both actions and opinions while opening the floor for further discussions in opinion dynamics. Despite the fact that there are still some possible action-opinion matrices that we have not tested yet in the three-action situation, several avenues for further research are promising. First, the AOI model can be extended or adapted by employing other methods that represent processes of "learning opinions by observing actions" to make the model more realistic. For instance, a similarity-based mechanism may assume that an agent is more likely to take the opinion that is similar to her previous opinion (Teşileanu & Meyer-Ortmanns ; Flache et al. ). In addition, it would be interesting to explore opinion dynamics in the situation where (some) agents are reluctant to signal their opinions through their actions. Such obfuscation behavior, which is characterized by an agent choosing an action that provides minimal information to a focal agent regarding her underlying opinion, has been formalized in recent work (Chorus ). Second, the model can be tested in various network structures. We only test the model in the von Neumann neighborhood in the paper, and it is promising to analyze the dynamics of the AOI model in di erent networks to investigate the role of randomness, degree distribution, and dimensionality. Furthermore, analytical solutions to the model with general action-opinion relations (a simple example has been given in Section ) would be helpful to understand the simulation results. Finally, an obvious and important direction for further research consists of empirically validating -at a micro and macro level -our behavioral model and the emergent properties it generates.