Del.icio.us
Digg
TechnoratiReported by Leigh Jaschke and Melissa Loudon.
This spring, UNICEF Malawi and the UNICEF Innovations Team deployed RapidSMS to monitor child growth and nutrition. We wrote about it previously here.
Now there is a detailed report (pdf), evaluating the effort. The report, released on June 16th, was issued by Columbia University's School of Public Policy and Affairs (SIPA), UNICEF Malawi, UNICEF's Innovations team, and Mobile Development Solutions (MDS).
The report details the findings of the deployment, and outlines recommendations for the future use of SMS in Malawi. Raymond Short of Mobile Development Solutions says that,
“while there have been innumerable ICT development applications introduced recently, there have not been many independent studies of their efficiency”.
As the government of Malawi has expressed interest in scaling up the pilot nationally later in the year, as well as expanding into other health care applications such as registration of child births, this attempt to evaluate the RapidSMS-based pilot is particularly relevant.
UNICEF Malawi deployed RapidSMS to address serious constraints within the national Integrated Nutrition and Food Security Surveillance (INFSS) System. The system was initiated in 2002 in the wake severe famine and set up with technical assistance from Action Against Hunger (ACF) and the support of the Malawi government, UNICEF, the European Union, and other partners. The system faced slow data transmission, incomplete and poor quality data sets, high operational costs and low levels of stakeholder ownership.
Data collection for the pilot was accomplished using SMS-based data collection on health workers' mobile phones. Health workers at three Health Monitoring Clinics (HMC's) were trained to submit child nutrition data in a fixed-format SMS -- a text message containing series of data points -- in a predefined order and separated by a delimiter character. Using the open source RapidSMS platform, SMS are received by a central server and automatically analyzed for indicators of child malnutrition.
A very important aspect of this implementation is that healthcare workers received instant feedback messages confirming that the information was sent. The feedback messages also provided additional information if malnutrition was indicated by the data received.
Additionally, a website was created in order to provide the Malawian Government and other stakeholders real-time access to this data and the accompanying analysis. The report identified a need to investigate how feedback loops and provision of data to secondary users can be expanded. Both were key both to the success of the system, creating user buy-in by providing information back to users.
From a technical point of view, two aspects of the implementation stand out. One is the carefully considered use of fixed-format SMS for data collection, which is sometimes rejected as error-prone because users have to carefully type in a series of data points and delimiters.
In the pilot system, data was first captured into fields on pre-printed paper forms, before being copied in to the phone as an SMS. The data itself is restricted to numerical data (height, weight) or yes/no answers. Further error checking was done on the server side, and SMSes that are incorrectly formatted, or which indicate and impossible height or weight value, prompted an immediate response message asking the user to resubmit.
The report also mentions that much of the final system customisation was done in-country. This is a very positive departure from the model where a system, specified, designed and developed in the west is 'parachuted' into a developing country - and frequently turns out to be inadequate because of requirements that were not properly understood. In a data collection system, which frequently spans multiple levels of an organisation, this is particulary relevant.
As an open source project with an active community, RapidSMS is customisation-friendly and well-supported. This was a contributing factor to the initial success of the system, and bodes well for its sustainability and future expansion possibilities within Malawi.
In summary, the report indicated a number of positive findings that may be applicable to other mobile data collection projects:
- Significant reduction in data transmission delay
- Increase in data quality reported by health workers
- Elimination of the need for time consuming data-entry
- Increased two-way flow of information between stakeholders at the national government level and health workers in the field
- Increased system and personnel monitoring capabilities
- Elimination of costs related to transporting paper forms and manual entry of data
Key lessons learned include the need for proper training in data measurements, and for training at the district and national level on how to monitor incoming data using the RapidSMS platform. Further investigation of the information design of the entire child monitoring system, which currently uses a randomised data collection design that may limit its usefulness for surveillance purposes, is also recommended.
The report rightly concludes that technology can only aid development within a limited scope. As the authors say, “the true value of this innovation will depend on several other factors”, including integration of the SMS data collection system into the larger context of health activities and policy making in Malawi, willingness of the government of Malawi to take ownership of the platform, ability to build and develop local technical capacity, and maintenance of training and monitoring at all levels of participation.
Post new comment