Validation of the symptom pattern method for analyzing verbal autopsy data
Published in PLoS Medicine, November 2007
Research published in PLoS Medicine in November 2007 validated a novel method for analyzing verbal autopsy (VA) data (the symptom pattern method, developed at IHME) and found that this method outperformed another common VA analytical method (physician-coded verbal autopsy, or PCVA). The study, Validation of the symptom pattern method for analyzing verbal autopsy data, suggests that the symptom pattern method (SP) provides more accurate estimates of cause-specific mortality fractions than PCVA at the population level, and at the individual level it is better than PCVA at assigning causes of death to individual responses. The work was done in collaboration with scientists at the University of Queensland, the Harvard Initiative for Global Health, and the China Center for Disease Control and Prevention. The study was funded by the Grand Challenges in Global Health Initiative and the US National Institute on Aging.
The researchers found that SP outperformed PCVA, at the population level, and especially at the individual level. Using data from China, the researchers found that at the population level, SP estimates population cause-specific mortality fractions (CSMFs) with 6% average relative error and 0.7% average absolute error, while PCVA results in 27% average relative error and 1.1% average absolute error. At the individual level, SP assigns the correct cause of death in 83% of cases, while PCVA does so for 69% of the cases. Additionally, when comparing the results of SP and PCVA at the population level without medical record recall, the SP method estimates CSMFs with 14% average relative error and 0.6% average absolute error, while PCVA results in 70% average relative error and 3.2% average absolute error. For individual estimates without medical record recall, SP assigns the correct cause of death in 78% of cases, while PCVA does so for 38% of cases. These findings suggest that PCVA relies heavily on household recall of medical records and related information, limiting its applicability in low-resource settings. SP does not require this additional information to adequately estimate causes of death.
Researchers explored analytical methods proposed by other researchers for analyzing VA data. They combined the advantages of these other methods to create a novel method, the SP method. The SP method uses two sources of VA data. First, it requires a dataset for which the true cause of death is known (i.e., deaths that occur in a hospital). The SP method can then be applied to a second VA sample that is representative of the population of interest. From the hospital data, researchers computed the properties of each symptom given the true cause of death. These properties allowed the researchers to estimate the population CSMFs and then use CSMFs as an input to assign a cause of death to each individual VA response. Finally, the researchers used individual cause-of-death assignments to refine population level CSMF estimates. The researchers applied their method to data collected in China to investigate the performance of the SP method in comparison to PCVA at the population and individual death levels.
Cause of death data are a critical input into formulating good public health policy. In the absence of vital registration data, VA data are commonly used to study causes of death. VA data are traditionally analyzed by using the PCVA method, which is expensive and its comparability across regions is questionable. Additionally, the accuracy of PCVA is limited in communities where few deaths had any interaction with the health system, because it relies on household members’ recall of medical record. The researchers aimed to develop and validate a statistical strategy for analyzing VA data that overcomes the limitations of PCVA.
Recommendations for future work
- Studies should explore ways to more accurately attribute causes of death to signs and symptoms.
- Because VA items may behave differently as a result of cultural factors, local validation studies are needed.
- Validation studies should be redesigned so that samples are chosen to get approximately equal numbers for important causes of death rather than a random sample of the population.
- The SP method should be tested and validated in other populations outside of China.
- A multi-country validation exercise and series of cause-of-death surveys to derive symptom sensitivity data, possibly using WHO standardized tools, should be conducted.
Citation: Murray CJL, Lopez AD, Feehan DM, Peter ST, Yang G. Validation of the symptom pattern method for analyzing verbal autopsy data. PLoS Medicine. 2007 Nov 20; 4(11):e327.