An alternative strategy to identify deprivations in multidimensional poverty: a partial least squares approach

Pasha, Atika; Yoon, Jisu (2018). 'An alternative strategy to identify deprivations in multidimensional poverty: A partial least squares approach' Paper presented at the annual conference of the HDCA, Buenos Aires, Argentina 2018.


Increasing amounts of scientific literature establishes the need for a multidimensional measure of poverty, going above and beyond defining poverty along a unidimensional measure like income, consumption or other such similar indicators. The seminal concept of the capability approach, famously discussed by Sen, has always been at the frontier of this research. Following this philosophy, several indices (such as the HDI or the GDI) or dashboard approaches (like the millennium development goals) were implemented, to account for an overall broad-based measurement of wellbeing and development. The Multidimensional Poverty Index (MPI), proposed by the Oxford Human Development Initiative (OPHI) and the UNDP, is one of the new indices that operationalizes the capability approach to measure poverty. The idea of this approach is to determine household multidimensional wellbeing along the three dimensions of health, education and standard of living, using ten different indicators. These indicators are aggregated typically by normatively determined weights, but they are not free from the arbitrary judgement of a researcher, and it could be challenging to adjust the weights to a local context. Typical data driven weighting schemes such as principal component analysis (PCA) emphasize the largest variations in indicators, which are not necessarily informative in practice. If the largest variance in the indicators is not relevant to poverty, using PCA will result in a poor quality MPI. Therefore, this paper uses partial least squares (PLS) to determine the weights in the MPI, by which the covariance between per capita household income and poverty is maximized. Consequently, the weights are based on the structure in the data that low per capita household income is related to more poverty.

Apart from generating a weighting structure, this paper also utilizes an innovative data based approach to derive the indicator cut-offs within the MPI. So far, the indicator cut-off has been determined based on the empirical understanding and practical considerations of researchers. In other words, normative judgement has been made by researchers, which not only require understanding in the magnitude of the deprivation, but also the overall context within which policy can make the best improvements. However, researchers may not have enough information to make such a judgement, which can lead to misleading indicator deprivation cut-offs and overall picture of poverty. To account for this, an adjustment is made to the PLS method, so that it determines the first indicator cut-offs of the MPI, in addition to the weights of each indicator. This approach provides a new insight on the deprivation level for each indicator of the MPI and does not rely on external data or other literature. An additional adjustment to the PLS algorithm enables us to reflect existing knowledge to the data driven weighting scheme. The new PLS algorithm respects the existing tradition in the multidimensional poverty literature to treat the three dimensions (health, education and living standards) as equally important. The afore-mentioned adjustments are tailored to the MPI. However, we expect that the adjustments are useful to other composite indices as well.

In addition to the adjustments to the PLS algorithm, we propose an innovative modeling approach to the MPI. This introduces interaction terms between the black population dummy and the indicators in the MPI. The interaction terms have an intuitive interpretation in our application, that is, black households associate with lower per capita household income than non-black households for a given indicator deprivation. The black dummy itself is not included, since race is not a part of the MPI.

This approach is intended as an example to deal with heterogeneous observations in a composite index. If the meaning of indicators varies among observations, the resulting index will have only limited validity. PLS with interaction terms can be used to obtain a simple solution to this problem. We restricted our attention to the black dummy, as the source of heterogeneity, to provide an easy and intuitive example. However, we expect more sophisticated modeling to be possible by considering additional interaction terms. For example, housing is much more important in a cold region than in a warm region, which can be modeled with a cold region dummy interaction terms. 

For this analysis, we use the National Income and Dynamics Study (NIDS) data from South Africa, using all four available waves from 2008 to 2014. The most recent wave is used to calculate the weights and the cut-offs, while others are used to establish trend. The NIDS is a rich panel dataset, which allows one to track not only the spatial differences but also the dynamic changes within multidimensional poverty. It contains information on all indicators within the current formulation of the multidimensional poverty index, with the exception of the flooring variable. Therefore, the index created using this dataset contains only nine of the altogether ten indicators comprising the MPI. Although previous work shows that there has been a large drop in multidimensional poverty in South Africa from the period after the apartheid, the presence of high inequality prevalent in South Africa provides a ripe background to study these new sets of indices. With the help of this data, we are able to explore the racial divide between the blacks compared to the other sections of the South African population, where the blacks are largely found to be on the lower end of the income and multidimensional spectrum.

The results show that the adjusted PLS does not identify the first cut-offs as those from the traditional approach of the OPHI. Notable differences are found in the first cut-offs of years of education, child school enrolment, toilet, cooking fuel and the number of small assets. In terms of the weighting structure, some indicators receive highly different weights compared to the traditional ones. For example, in the standard of living dimension, water source is much more important than cooking fuel. The MPI using interaction terms assigns different weights to the black population, which can be interpreted as additional deprivation of the blacks compared to others in terms of household per capita income.  

scroll to top