Using data from an artificial society to evaluate empirical approaches to the measurement of capabilities
Chávez Juárez, Florian (1); Krishnakumar, Jaya (2) (2018). 'Using data from an artificial society to evaluate empirical approaches to the measurement of capabilities' Paper presented at the annual conference of the HDCA, Buenos Aires, Argentina 2018.
Measuring capabilities has been a key challenge in the operationalisation of the capability approach for a long time. Over the last two decades a variety of proposals on how to measure capabilities have been made. The proposals range from simple averages of functionings, over factor analysis approaches to structural equation modelling. The main challenge for the evaluation of such proposals is the latent nature of the concept of capabilities, making it impossible to compare estimated to true capabilities. This challenge of validation is not unique to the capability approach, but present in all models aiming at measuring latent concepts.
One way of validating such models is to use Monte Carlo type simulations where the researcher simulates data and then applies the estimator to evaluate its performance. We argue that the main problem with this approach is that each proposal of estimator is accompanied by its own simulation exercise, which is likely to be inspired by the estimator itself. This might produce overly optimistic evaluations of the proposed estimator and makes it hard to compare the quality of different estimators. We therefore propose a novel approach which consists in dissociating the data generating process from the model we aim at testing. Using agent-based modelling techniques allows us to build an artificial society from which we can obtain data that looks very similar to survey data we generally use in empirical analyses. The key characteristic of this approach is that the model of the artificial society is completely disconnected from the estimation model and that it should allow researchers to compare different empirical strategies based on the same data.
The artificial society we present in this article is inspired by an emerging economy such as Brazil or Mexico. We calibrate the model with actual survey data from Mexico and include three development dimensions: education, health and social networks. Upon initialisation of the model using real world data, families endogenously evolve (birth/death) and must allocate resources to consumption, investment in education and health care services. Health care services are only required in the case of health shocks, which are depending on the previous health stock. The education process is modelled through a human capital accumulation process and directly affects the social network of individuals through the interaction with school mates. Other features of the model are a marriage market reproducing the generally observed assortative mating behaviour and regionally differentiated provision of public services such as hospital care and school supply. The three development dimensions are linked to each other through the behaviour of agents. For instance, social networks emerge from the interaction of agents at school and the school performance is to some extent dependent on the health status of the individual.
From the model we obtain data at the family and individual level for each simulation period (equivalent to a year). Besides generally observable variables such as gender, age, income and functionings, we also export a series of true capabilities which are directly derived from the conceptual definitions. In this sense, we include capabilities capturing potential outcomes on the one hand, and capabilities based on the actual choices individuals had.
These data can then be used by researchers to test different estimators of capabilities. The highly customisable model also allows researchers to generate different sets of data and evaluate the performance under different scenarios. To illustrate how the model and its data can be used to evaluate estimators, we present a short exercise evaluating the quality of estimation when using structural equation models. The findings are promising in the sense that they are more informative about the quality of adjustment of the method than previous exercises using simple Monte Carlo simulations. For instance, we find that using full information models might be more appropriate when the links between capability dimensions are strong.
In addition to presenting the general idea of using artificial societies to test empirical approaches, this article has two main goals. First, we aim at presenting our model in detail allowing interested researchers to understand what elements are present and how they interact with each other. Second and most important, we understand this article as an invitation to the research community in the capability approach to use our model and data to evaluate the different proposals of empirical models under a variety of scenarios. To facilitate this use, we follow an open-source strategy, allowing interested researchers to easily use and modify our model or directly the data generated from it.