empirically-based agent-based models

Reading: Marco A. Janssen and Elinor Ostrom, Empirically-based Agent-based modles, 2006

Summary

The authors of this paper discuss that agent-based models, though, have been proposed to show how simple rules of interaction could explain some macro-level phenomena such as spatial patterns and levels of cooperation, most of them are not rigorously tested with empirical data. They argue that both qualitative and quantitative empirical information can be either used as input to explore the possible explanation for a particular situation, i.e. for context-focused studies, or used to test the model to search for some generalizable arguments, i.e. for generalizability-focused studies. In order to combine agent-based modeling and empirical observations, they present a framework to deal with four types of empirical approaches: stylized facts, laboratory experiments, role games and case studies. The first two focus on generalizability, while the other two focus on the fitting of some special cases. Stylized facts such as power law distribution in number of hyperlinks in websites, can be used to investigate under that conditions the conceptual models can derive similar statistics. Laboratory experimental results can be used to compare alternative models of not only individual but also collective decision making process. The data derived from role games has been used to contrast, in particular situation, how agents and human behave differently. Case studies often include multi-sourced incomplete information; therefore models regarding certain specific cases can be developed and analyzed from different aspects. With this framework, agent-based modeling methodology can be justified and increase the acceptance in the broader domain of social science.

Some thoughts on this paper

This paper suggests a high-level methodology for researcher to examine and justify their agent-based models. Similar attempt can be found in [1]. In this paper, Grimm et. al. propose a pattern-oriented modeling framework to balance the complexity and uncertainty of models, to contrast alternative theories, and to search the best-fitting parameter set of models. I think such empirically-based modeling methodology will be significant in studies involving agent-based modeling approach.

When combining empirical data with agent-based models, the most important issue arises: how do we compare empirical data and the data from generative models? Some statistical methods such as goodness of fit or maximum likelihood may not be suitable because those classical statistical methods explain data based on certain assumption, which means we look at the empirical data from a specific aspect. However, such aspect may not be aware or carefully taken into account in modeling or comparison with modeling output. Other methods like Bayesian analysis could be useful to measure the distance between empirical data and the modeling output.

Another concern is more on epistemology. When social scientists decide to use role games or case studies to explore the social process, they may care more about meaningful social action, not just the external or observable behavior of people. Here I refer to interpretive social scientist. For interpretive researchers, it makes little sense to try to deduce social life from abstract, logical theories that may not relate to the feelings and experiences of ordinary people. Individual motives, i.e. the reasons for their action, are crucial to consider even if they are irrational, carry deep emotions, and contain false facts and prejudices [2]. I’m wondering if context and generalizability-focused studies are just two methodologies occupying both sides of spectrum, or they are located on different planets where researchers have completely different belief.

[1] V. GRIMM, E. REVILLA, U. BERGER, F. JELTSCH, W. M. MOOIJ, S. F. RAILSBACK, H.-H. THULKE, J. WEINER, T. WIEGAND and D. L. DEANGELIS (2005). Pattern-Oriented Modeling of Agent-Based Complex Systems: Lessons from Ecology. Science 310(5750): 987 - 991.
[2] W. L. NEUMAN (1997). Social Research Methods: Qualitative and Quantitative Approaches, Allyn and Bacon.