Tag: Randomised Trials

  • Evidence-based Education in Africa

    I wonder what our current debates in Australia would or could contribute to the need to measure impact?

    Sub-Sharan Africa [Wikipaedia]The recent Evidence-Based Education:Policy-Making and Reform in Africa Conference raises numerous challenges, for not only an education system [Ghana], but policy makers across all sectors of Health, Education and Treasury throughout Sub-Saharan Africa.

    The Innovations for Poverty Action (IPA), and the Abdul Latif Jameel Poverty Action Lab (J-PAL) in collaboration with the Ghana Educational Service (GES) and support from the United States Agency for International Development (USAID) staged the two day event to review a large-scale evaluation of the Ghana Government’s Teacher Community Assistant Initiative (TCAI), aimed at improving literacy and numeracy levels in basic schools.

    The context of the work is data that shows within Sub-Saharan Africa that:

    • Net enrolment in primary school was 75% in 2009
    • Primary completion rates were 67% in 2009
    • In 2004, only 60% of students were able to progress to secondary school
    • Net enrolment in secondary school was 27% in 2008
    • Gross enrolment in tertiary institutions was 6% in 2008

    Natalie Colatosti and Laurence Dessein have an article here

    Links:

    First, Find out what works

    – Evidence-based Policy ” Build on or Spray on” David Halpern – Does Evidence Matter? [PDF]

    Random Trials for Service Provision

  • Random Trials for Service Provision

    As always, some very useful and timely links from Andrew Leigh. I would be interested in pursuing randomised trials in our Education Sector.

    Randomising in the UK.

    In 2003, the UK government started a major randomised evaluation of the Employment Retention and Advancement project (ERA), to test the effectiveness of interventions to improve job retention and advancement prospects for low wage workers (background here). Like the US and Canada, British policymakers decided that randomised evaluation needed to be a key part of understanding what active labour market programs work best. They are now reaping the rewards of their foresight, as the results couldn’t be arriving at a more useful time. Most of it is still under wraps, but a report on costs is now out.

  • First, Find out What Works

    This is re-produced from Andrew Leigh’s WebSite.

    Among the most interesting debates in economics today is the dispute between Jeffrey Sachs and William Easterly over how best to help the world’s poor. The discussion is interesting not only because it concerns the most important question in all of economics, but also because Sachs and Easterly happen to be exceptionally good communicators. And like the longtime Mets-Yankees rivalry, both happen to be at New York universities: Sachs at Columbia University, and Easterly at New York University.

    At stake is the question of whether development economics requires a Big Push or piecemeal reform. In The End of Poverty: Economic Possibilities for Our Time, Sachs argues that the world’s least developed countries are caught in a poverty trap, and that success requires a doubling of foreign aid. In The White Man’s Burden: Why the West’s Efforts to Aid the Rest Have Done So Much Ill and So Little Good, Easterly contends that much foreign aid has been wasted, and that it would be better to focus on taking modest, deliverable steps to make poor people’s lives better.

    Whoever is right (my own view is that the truth lies somewhere between the two), the Sachs-Easterly debate has helped to galvanise a revolution in development economics. Integral to the development economics revolution has been the advent of rigorous randomised trials to find out what works. Long regarded as the ‘gold standard’ in medicine, a spate of randomised trials are taking place across the developing world.

    In Kenya, researchers have shown that deworming drugs dramatically reduce school absences, while worm-prevention education programs had no impact whatsoever. In the Philippines, savings incentives have been shown to boost the assets of the poor. And in India, a program that paid teacher bonuses raised [“acrobat”] student test scores by a significant margin.

    The reason randomised trials are so powerful is that they allow us to know the counterfactual: what would have happened if the program was not implemented? Because participants are assigned to the treatment and control groups by the toss of a coin, we can be sure that any systematic differences in outcomes are due to the program itself.

    Beyond program evaluation, randomised trials have also been used to help researchers learn more about issues of governance, which have long bedevilled development efforts. In a famous randomised trial in Benin, researcher Leonard Wantchekon persuaded presidential candidates in the 2001 election to randomly alter their campaign strategies: giving high-minded public policy speeches in some villages, and pork-barrelling clientelist speeches in other villages. He found that candidates won more votes when they delivered a clientelist speech, and that male voters tended to be more easily swayed by offers of local patronage than female voters.

    More controversially, Marianne Bertrand and co-authors recently followed a group of Indian applicants through the notoriously corrupt process of obtaining a driving licence. One group were offered a financial reward if they obtained their licence fast, while others were not. The researchers found that the payment did indeed lead applicants to get their licences faster, but their skills were no better. Given a surprise driving test afterwards, 69 percent failed. Evidence from the randomised trial helps understand how corruption in the licensing process operates, and may assist reformers in improving the system.

    As the results from these and dozens of other randomised trials begin to pour in, development economists will continue to hone their knowledge about what works best, and whether the kind of grand strategy of Sachs or the more piecemeal approach of Easterly is likely to offer more promise for raising the living standards of those in developing countries.

    Yet while randomised policy trials are transforming development economics, it is startling that our homegrown development challenge – raising the living standards of Indigenous Australians – has seen so little attention to rigorous evaluation. With Indigenous life expectancy at third world levels, too much of Indigenous policy seems to be driven by instinct, and too little by results.

    Instead of digging in behind their favoured solutions, wouldn’t it be better if both sides of politics admitted that most social policies aimed at improving Indigenous living standards have failed? Once we acknowledge how little we know, we might then begin by applying the same gold standard evaluation technique to Indigenous social policies that we apply to new drugs.

    A dozen randomised trials of Indigenous social programs would cost comparatively little, but provide valuable new evidence. In health and education, employment and family policies, Indigenous policymaking could do with a little less rhetoric and a little more of the modest searching that characterises the best of development economics today. Think your policy works? Let’s toss a coin and find out.

    [Source: Dr. Andrew Leigh]

    Dr Andrew Leigh is an economist in the Research School of Social Sciences at the Australian National University.