Nutritional Surveillance with RapidSMS in Malawi

Posted by admin on Aug 03, 2009

This case study was published originally by UNICEF's Innovation Group/RapidSMS. All photo credits: UNICEF Innovation

UNICEF Malawi deployed RapidSMS to address serious constraints within the national Integrated Nutrition and Food Security Surveillance (INFSS) System, which was facing slow data transmission, incomplete and poor quality data sets, high operational costs and low levels of stakeholder ownership.

Health workers now enter a child’s data, and through an innovative feedback loop system, RapidSMS instantly alerts field monitors of their patients’ nutritional status.

Automated basic diagnostic tests are now identifying more children with moderate malnutrition who were previously falling through the cracks. This system also increased local ownership of the larger surveillance program through two-way information exchange. Operational costs for the RapidSMS system are significantly less than the current data collection system. The Government of Malawi is considering a national scale-up later this year.

Country Profile

Malawi is one of the world’s most densely-populated, yet least-developed countries. A small, landlocked country bordered by Mozambique, Zambia, and Tanzania, Malawi has remained below the United Nation’s average human development index score of sub-Saharan Africa countries due to a combination of slow economic development, poor infrastructure, the catastrophic public health effects of HIV/AIDS, and chronic child malnutrition.

The 2008 UNDP Human Development Report ranked the country 164th out of 177 countries in factors such as average life expectancy (46.3 years), adult literacy (64 percent), GDP per capita (US$667), and percentage of underweight children under five years (22 percent). The country suffered a series of severe droughts over the last decade that led to widespread famine during 2002 and 2006.

It is estimated that 13 percent of children in Malawi die before the age of five, and that at least one-third of these deaths are related to acute malnutrition. Over half of all children suffer from stunting (52.5 percent) and 18.4 percent of children under five years old are underweight for their age.

The Scenario

In the wake of Malawi’s severe 2002 famine, the Integrated Nutrition and Food Security Surveillance (INFSS) system was set up with technical assistance from Action Against Hunger (ACF) and the support of the Malawi government, UNICEF, the European Union, and other partners.

The nutrition portion of the surveillance system was structured to monitor trends in nutritional status of approximately 9100 children, spread out in five growth monitoring clinics (GMCs) in each of the 26 districts. Monitored children are randomly selected from the population of children visiting GMCs, thereby including a combination of healthy, malnourished, and sick children. These same children are tracked and measured monthly for a period of 12 months. The nutritional data is collected on paper forms and then sent through district health managers to a central office in Lilongwe, where they are manually entered into Excel-based datasets for nutrition analysis.

In 2008, ACF handed the program over to the Malawi government. Data reporting dropped significantly. Paper data forms continued to trickle up to the central government level, yet much of the data was never entered into computers or analyzed. This left the government, UNICEF, and their development partners without any systematic means of identifying acute changes in nutrition status throughout the country and with little information with which to make effective decisions about allocation of resources.

The INFSS system faced several challenges:

Delays in transmission of data: there is an average of two to three months delay between data collection at health clinics and delivery of the actual forms at the government level, since data is recorded on paper and sent via mail or alternative means of transport to a centralized location. At the national level, lack of human resources lead to long delays before data is analyzed.

Poor data quality: paper data collection forms are sometimes illegible due to poor handwriting, or are incomplete or contain significant outliers (eg. height of a child listedas seven meters). In 2007, 14.2 percent of all forms were discarded by ACF as unusable.

One way flow of information: field based Health Surveillance Assistants (HSAs) rarely received any feedback or are given access to the analysis done at the national level, leading to low levels of ownership.

High operational costs: the paper based system was particularly labor intensive, with full time staff needed at a national level just for data entry.

Since chronic and widespread child malnutrition remains a serious problem in Malawi, the shortcomings of the system are a serious threat to the country’s ability to anticipate and plan for current and future food security crises. As a result of these limitations, there are too few complete datasets to analyze on a monthly basis, and effectiveness of the early warning system is subsequently compromised. Policy makers and development practitioners are unable to receive timely information regarding trends in child malnutrition throughout the country and are subsequently unable to react appropriately with increased support to areas facing high levels of malnutrition.

RapidSMS Implementation and Results

The Malawi RapidSMS platform was designed to keep the data input format as close to the paper format as possible. In order to make RapidSMS compatible with all mobile phones, the platform uses a string of variables entered in a predefined order within a single SMS, with the same health indicators inputted in the same order as the original system.

The RapidSMS platform was also designed to provide immediate feedback to the HSAs. Feedback loops were incorporated to guide HSAs in their work and alert them of any data entry errors. For example, after submitting a child’s measurements via SMS, HSAs automatically received an SMS confirming the data submission. In the event that the data submitted indicated malnutrition, the HSA would also receive an SMS providing them with specific instructions for treating the child. In the event of a significant data entry error (for example, a height measurement outside the range of physical possibility), the HSA would automatically receive an SMS instructing them to re-send the corrected data.

In January 2009, the pilot study was launched at three GMC sites in central Malawi. Approximately 30 health surveillance assistants, typically holding a secondary school diploma and paid by the government for their services, were given a two-hour training in RapidSMS reporting. They registered two hundred and ten children and tracked them for a period of four months using RapidSMS. During this period, total of 535 unique data sets were generated.

Delays in transmission of data: RapidSMS eliminated both the data transfer and data entry time delays by implementing automated data-entry into a central database. Once data was collected by the HSA and sent by SMS, it was also immediately stored, analyzed, and accessible to stakeholders at all levels. Transmission times that previously took from two to three months were reduced to an average of two minutes. This is essentially 64,800 times faster than the paper-based system.

Poor data quality: there were 15 data entry errors, representing an error rate of 2.8 percent. These errors consisted of wrong measurements entered, wrong association of child ID number with measurements, or entry of data strings with one or more missing values. This was a significant improvement over the 14.2 percent error rate under the previous system in 2007. Moreover, all the errors occurred in the first reporting period. During the final three months, there was not a single unusable data set.

One way flow of information: Approximately 30 feedback loops were programmed to assist health care workers in accurately reporting patient data and offer specific information on patient current health status. One of the most significant automated functions is the weight for height calculations. Despite being the most reliable method of identifying malnutrition in children, most HSAs stated that they were not trained in these calculations and did not feel competent carrying them out. RapidSMS automatically performed this calculation for every patient and instantly alerted HSAs of their patient’s nutritional status. At one GMC, HSAs proudly noted that they had identified and treated ten mildly malnourished children who would have otherwise been missed.

High operational costs: The need for centrally-based full-time staff dedicated to manually inputting the data was eliminated, along with the high cost of transporting the forms. After the two-hour initial training, the only additional operational cost was for the texts themselves, which was dramatically reduced through an agreement with the mobile providers.

The Government of Malawi plans on scaling RapidSMS up nationally later in the year. They are also interested in expanding this to a country wide campaign to register child births, as well as deploying RapidSMS in other sectors, including education and HIV/AIDS.

Basic Information

Website URL:

http://www.rapidsms.org

Organization involved in the project?:

UNICEF

Project goals:

Health workers now enter a child’s data, and through an innovative feedback loop system, RapidSMS instantly alerts field monitors of their patients’ nutritional status. Automated basic diagnostic tests are now identifying more children with moderate malnutrition who were previously falling through the cracks. This system also increased local ownership of the larger surveillance program through two-way information exchange. Operational costs for the RapidSMS system are significantly less than the current data collection system.

Brief description of the project:

RapidSMS is an open-source framework for data collection, logistics coordination and communication allowing any mobile phone to interact with the web via SMS text messages. in Malawi, the tool was used to monitor child nutrition levels.

Target audience:

UNICEF field staff.

Detailed Information

Status:

Ongoing

What worked well? :

Proper Training in Data Measurements

While RapidSMS cannot directly address many broader constraints of the INFSS system, including the high prevalence of improper measurement techniques, it can help identify potential problems. Measurement inaccuracy is difficult to identify in a system of INFSS’s size, since it is impossible to observe whether child measurements are actually being taken correctly.

However, improper height measurements are easy to identify within datasets, as children do not generally lose height from one month to the next. Subsequently, height-loss errors were used as a proxy for improper measurement techniques. At one site, height-loss was reported for 25.8 percent of children during this period. Not surprisingly, this site had received little supervision or training while using the original paper-based system.

Monitoring and Oversight

By using RapidSMS, stakeholders at the national level will be able to track nutritional trends in each district. The data’s easy accessibility and legibility facilitates the identification of data-entry errors. However, there is presently no one trained at the national or district levels on how to monitor incoming data using the RapidSMS platform. While this platform does not provide any new data, it supplies conventional data sets in real time. This offers its own set of rewards and challenges, and necessitates a different approach to monitoring.

What did not work? What were the challenges?:

Coordinators at the district and regional levels can immediately identify sites that fail to report data and contact them directly through the RapidSMS web user interface. However, this entails regular monitoring of the data. During the pilot, one site failed to report data for two weeks. While this was quickly identified by UNICEF workers in New York, despite the availability of this data outside consultants monitoring the project in Malawi were unaware of the situation. This points to a need to further simplify the internet user-interface and provide a concrete tool-box to support capacity building in Malawi. RapidSMS should be seen as a tool that allows health care workers at all levels to do their job more efficiently.

Data Bias

The INFSS system was set up not as a predictive model to extrapolate statistics for the larger population, but to identify changing trends in health in target populations over time. Participants are self selecting, as they only include children measured at the GMC. This likely leads to an overrepresentation of very young or sick children in the surveillance program. Additionally, many of the most vulnerable children who live great distances from the GMCs are probably underrepresented.

Furthermore, by automating the weight for height calculations which has shown to effectively identify previously uncaught children with mild malnourishment, it can be assumed that children whose data is being reported via RapidSMS are receiving better care than those that are not. This further biases the sample group, albeit in a positive and hopefully life-saving way. However, it is critical to recognize the limitations of the larger INFSS system due to this data bias.