Proven interventions can improve population health only if they reach those who need them. The Impact Evaluations research team focuses on developing new and improved methods to evaluate the impact of programs or interventions based on empirical evidence and conducting impact evaluations to determine the effect a health program had on a target population.
This work will help expand the evidence base about which health programs and interventions work and which do not to determine how well societies are addressing health problems.
IHME has pledged to undertake one to two impact evaluations every year. While proposed evaluation ideas can come from external organizations or from within the Institute, the IHME Board is responsible for making the final decision about which evaluations IHME undertakes.
Key Activities
- Evaluate the impact of Janani Suraksha Yojana on health outcomes
As part of their planned work to assess the impact of health programs, IHME researchers conducted an impact evaluation of Janani Suraksha Yojana (JSY), a “safe motherhood scheme,” that was launched in India in 2005 with the goal of reducing maternal and neonatal mortality.
JSY, a program that incentivized women to deliver in a health facility, was fully funded by the Indian government with a budget allocation of $342 million USD in 2009 to 2010.
IHME researchers found that significant increases in births took place in health facilities during the time period of observation. A wide range in uptake of the program was seen across districts; uptake was highest in many districts that previously had a very low proportion of births occurring in health facilities.
Researchers found evidence of a likely reduction of perinatal and stillbirth/neonatal mortality, but they were unable to detect a significant effect of JSY on maternal mortality.
- Assess malaria control policy in Zambia
In order to evaluate the impact of malaria control policy in Zambia, we will quantify the impact of Zambia’s expanding malaria control activities on under-5 mortality from 2000 to 2010, considering progress achieved on all major child health interventions. The findings will serve as evidence-based policy instruments for Zambia’s health programs, as well as guides for other countries and health initiatives tackling malaria.
- Assess the cost effectiveness of HIV antiretroviral therapy programs
As of 2011, the University of Zambia (UNZA) and IHME are collaborating to collect and analyze the costs and constraints of Zambia’s health system. A team assembled by UNZA will undertake data collection in a representative sample of health facilities across the country.
This project also collects specific information on the costs of antiretroviral therapy (ART) programs to estimate the cost per completed patient-year of ART for a representative sample of providers and to identify the determinants of program success or failure.
This work is part of the Access, Bottlenecks, Costs, and Equity (ABCE) project. Zambia is the first of at least four countries in which information will be collected. The data collection tool for ART costs has been integrated into the facility cost survey instrument of the ABCE project. The intent is to provide complete information on the facilities’ total costs and to streamline data collection efforts.
The ART clinical record extraction gathers data from de-identified patient charts to provide data on the sociodemographic and health characteristics of patients. When combined with the facility data, the patient data will be used to improve the cost effectiveness of ART delivery in each country.
The results will enable us to outline policy implications and recommendations for ministries of health and funding organizations, improve efficiency and effectiveness in the delivery of health service – including ART in resource-limited settings – and optimize resource allocation to HIV/AIDS in low- and middle-income countries.
- Evaluate the population-level impact of HIV prevention interventions in India
The goal of this project was to evaluate the impact of Avahan, an initiative to prevent HIV infection and transmission in India. The Impact Evaluations research team measured the observed incidence and prevalence of HIV between 2003 and 2008 and compared these data to what would be expected in the absence of Avahan.
This study is the first to quantify the impact of a nationwide HIV-prevention program. This work by IHME researchers provides critical input to India’s HIV-control strategy, which emphasizes comprehensive prevention programs.
IHME researchers estimated that 100,178 HIV infections were prevented at the population level between 2003 and 2008 as a result of Avahan. Researchers noted that the Avahan prevention efforts were associated with a small but significant reduction in HIV prevalence at the population level in three of the six states where Avahan operates.
- Carry out the monitoring and evaluation component of the Medical Education Partnership Initiative
The Medical Education Partnership Initiative (MEPI) aims to reform the medical education system at the University of Nairobi by training faculty in innovative teaching techniques, increasing the capacity of faculty to apply for research grants and conduct research projects, and establishing better linkages between Nairobi and training facilities in remote areas.
IHME is designing the monitoring and evaluation component of this project to measure the impact of the five‐year grant. IHME researchers are designing tools for baseline measurement of faculty attitudes toward various teaching methodologies and their use, as well as faculty performance reviews from students and peers. The baseline data will be compared to assessments at the end of the project.
IHME researchers are also designing the tools for baseline assessment of research capacity at the University of Nairobi and collaborating sites. Finally, the team is designing tools to measure baseline physician practices and adherence to treatment guidelines.
Impact
The Impact Evaluations research team’s work strengthens the evidence base of successful health care delivery mechanisms and health system-strengthening policies and programs.
We aim to increase the quality of information available to decision-makers who want to understand the effect of a program or to design a new program with the greatest possible positive impact.
