Errors resulting from the implementation of PMTs

Exclusion by design: An assessment of the effectiveness of the proxy means test poverty targeting mechanism 13 errors into the targeting process. Many of those households that may have been ‘correctly targeted ’ in the first year are likely to be ‘inclusion errors’ in future years, as a result of improved circumstances. However, anyone falling into poverty between surveys – perhaps due to a crisis such as unemployment, disability, ill-health, death of a breadwinner or even the birth of a child – is excluded from accessing social protection no matter how challenging their circumstances. In Pakistan’s Benazir Income Support Programme BISP, for example, only 22 per cent of beneficiaries were living in poverty across each of three survey years 2011, 2013 and 2014, 20 per cent were never under the poverty line while almost 60 per cent spent one or two of the three years in poverty, remaining on the programme when above the poverty line Cheema et al, 2015. On the other hand, the vast majority of the population spending some time living in poverty over the same period were unable to access BISP. Figure 10 illustrates the challenges posed by changes in household composition, income, consumption and assets. It uses a panel dataset in Uganda to show who – in red – would be accurately selected by a PMT targeting the poorest 20 per cent of households in 2011 in the left hand graph. The right hand graph indicates where those same households would be in 2013, if they were retargeted. Only 55 per cent of those accurately targeted in 2011 would still be accurately targeted in 2013: around 38 per cent would no longer be in the poorest 20 per cent in terms of expenditure, while 21 per cent would have PMT scores above the cut-off. Many other households – in blue – would be newly eligible for the programme by 2013, both in terms of consumption and their PMT score, but would not be included, unless there were a new registration process. Figure 10: Scattergraphs showing how households accurately targeted in 2011 would be targeted using the same PMT in 2013 Uganda Source: Analysis undertaken by the authors using UNPS 201112 and UNPS 201314 datasets. See: Kidd and Bailey- Athias 2016. However, despite the reality of significant changes in household incomes over short periods of time, it is very rare for PMT surveys to be repeated on a frequent basis. Governments often perceive PMT surveys to be expensive: for example, the 2009 PMT survey in Pakistan and the 2011 survey in Indonesia both cost around US60 million, while the 2015 survey in Indonesia cost US100 million. 10 Indeed, some PMT surveys are very expensive: for example, Kenya’s HSNP programme spent around US10 million to survey only 380,000 households while achieving – as indicated earlier – not much better than 10 While these costs may seem high, they are, in reality, low when compared to the costs of implementing a more effective and robust targeting mechanism in a developing country. 14 Exclusion by design: An assessment of the effectiveness of the proxy means test poverty targeting mechanism random selection Fitzgibbon, 2014. As a result of the perceived high costs of PMT surveys, governments are reluctant to repeat them. Pakistan, for example, has not undertaken a PMT survey since 2009; in Indonesia, there was a four-year gap between the surveys of 2011 and 2015; while, in some areas of Mexico, registration for the Oportunidades programme had not been repeated for more than 10 years Zoletto, 2011. Furthermore, both the infrequency of information and the static nature of the PMT means that programmes using a PMT cannot function as safety nets. Since PMTs measure assets and other ‘stable’ characteristics such as education levels, the PMT is a very static mechanism. Even if it allowed people to apply for a social protection programme whenever they experienced a crisis, they are unlikely to b e identified as ‘poor’ because their proxies may not have changed, despite a significant fall in income. As discussed earlier, they would have to first divest themselves of many of their assets before they could be recognized as eligible. Yet, their loss of the assets would mean that their ability to recover from the crisis would be compromised. So, although the World Bank refer to social assistance programmes using PMTs as ‘social safety nets,’ they are nothing of the sort see Kidd, 2012 . A safety net should be available as soon as people experience a crisis: in contrast, a social assistance programme using a PMT will continue to exclude people who have fallen into poverty until their assets have been depleted, which may take time. Social assistance using PMTs is, in effect, Poor Relief – in other words, a programme for those who are already living in poverty – and nothing more and is similar to the approach used by developed countries in the 19 th Century. Advocates of PMTs have recently begun to claim that they can introduce ‘dynamic targeting ’ into PMTs. 11 By this they mean introducing on-demand applications to PMT rather than using one-off censuses. Yet, this will not solve the problem. It does not address the in-built design errors nor the challenges of errors introduced during implementation; and, it will further increase the costs of implementing the PMT, with no guarantee that it will be more accurate. Furthermore, as discussed earlier, even if households have been hit by a crisis and apply for a programme using a PMT, their assets are unlikely to have changed and so they would continue to be excluded. And, it would mean selecting households using data collected at different times: for example, if mandatory recertification of all households is only undertaken every five years, a targeting list using supposedly ‘dynamic targeting’ would be a mix of data from households assessed at any time over a five-year period, meaning that like would not be compared with like.

5. Grievance mechanisms for PMTs

Given the very high errors associated with the PMT methodology, one might assume that an appeals process should be introduced to allow them to be corrected. Unfortunately, this is not the case. With the PMT, it is not possible, in practice, to put in place effective grievance mechanisms that allow those living in poverty to appeal their exclusion. If people could appeal their exclusion on the basis of their poverty, the high level of exclusion error would mean that over half of the intended beneficiaries would be eligible to appeal. In effect, the appeal process would turn into a further application process, since it would be so large. Consequently, programmes using PMTs tend to restrict the appeal mechanism so that the design errors within the PMT mechanism itself are not challenged. For example, in the Philippines’ Pantawid Pamilya programme, if people living in extreme poverty appeal because they were judged by the proxy means test to be ineligible, their only option is to be re-surveyed, which is likely to give the same result as before Kidd, 2014. 11 Cf. Leite n.d. at: http:pubdocs.worldbank.orgen897101464039057078SPLCC-2016-SNCC-D4S1-Leite-Social- Registry.pdf Exclusion by design: An assessment of the effectiveness of the proxy means test poverty targeting mechanism 15

6. PMTs and the weakening of community cohesion

There is good evidence that proxy means tests cause social conflict in communities, weakening their cohesion despite this being one of the main assets of communities which should be strengthened rather than debilitated. 12 This is largely due to the inaccuracy and relatively arbitrary nature of the PMT methodology. Community members cannot understand why some people living in poverty are selected while others are excluded. In Mexico and Nicaragua, non-recipients – many of whom are living in poverty – have remonstrated about their feelings of despair, frustration, envy, resentment and jealousy. In Mexico, Nicaragua and Indonesia, non-recipients have withdrawn their labour from voluntary community activities. 13 There is evidence of direct conflict: for example, in some communities in Mexico, when recipients of the Progresa programme were cleaning the streets, the non-recipients threw rubbish; in others, fences mended by recipients were subsequently knocked down by non-recipients Adato, 2000 . In Kenya’s CT-OVC programme, the absence of significant discord in communities following the selection of recipients using a proxy means test was due to programme administrators deceiving those excluded by telling them that they would be incorporated into the programme in the near future when, in reality, this was not going to happen Calder et al, 2011. Widjaja 2009 found significant challenges in Indonesia when the Bantuan Langsung Tunai BLT programme – which used a proxy means test – was rolled out see Figure 11. Protests about the selection process took place in around 30 per cent of villages. Indeed, Cameron and Shah 2011 found that crime increased by 5.8 per cent as a result of the PMT. In a community visited by Hannigan 2010, the initial distribution of the Indonesia PKH programme – again, using a proxy means test – provoked stone throwing and the burning down of a building. Similar problems have been found in Lebanon where the introduction of proxy means testing led to riots in some refugee camps Kidd and Wylde, 2011. In Lesotho, Kardan 2014 found that the PMT ‘created a great deal of tensions in the communities between beneficiaries and non-beneficiaries due t o people’s limited knowledge of the selection criteria, their own sense of entitlement and the perceived exclusion of many deserving households. ’ In fact, it has been reported that the houses of village chiefs were burnt down which has not happened with the country’s universal pension, since this is a very popular and non-divisive programme. 14 Often, when people living in extreme poverty are excluded by PMTs, they accuse programme staff of stealing their money, since they cannot understand the reasons for their exclusion. 12 See Adato 2000, Adato et al. 2000, Adato and Roopnaraine 2004, Widjaja 2009, Huber et al 2009, Hannigan 2010, Kidd and Wylde 2011, Cameron and Shah 2011 and Hossain 2012. 13 See Adato 2000, Adato et al. 2000, Adato and Roopnaraine 2004 and Hannigan 2010. 14 Sharlene Ramkissoon personal communication.