Digital technologies are reshaping the way organizations conduct monitoring, evaluation, research and learning (MERL) processes. Improvements in information technology, availability of software solutions, and network capacity and capability continually increase opportunities for innovation in how data is managed and transformed into strategic insights.
To support development organizations with modernizing data management processes currently in place in a cost-effective way, the Institute for Development Impact (I4DI) has developed services that build their capacity at different points in this process.
Making technology work in the field
I4DI has recently conducted a capacity building process in Port-au-Prince, Haiti, for monitoring and evaluation staff of World Vision Haiti to facilitate their transition from paper-based to mobile data collection. I4DI has been working with World Vision Haiti to build a system for data collection, management and analysis for the implementation of a program called Kore Fanmi, which means “family support” in Haitian Creole.
With the support provided by I4DI, World Vision Haiti staff learned how to design surveys using mobile technology with Android devices (tablets or smart phones) and how to train staff and partners who will be in the field doing mobile data collection. Combined with ongoing technical assistance provided by I4DI over the past few months, the capacity building process was developed to equip World Vision Haiti staff with functional research and evaluation skills, data analysis and reporting skills, and processes for making data-driven decisions for program effectiveness.
Switching from paper surveys to tablet-based data collection is expected to result in quicker transfer of data and fewer data entry errors. Mobile survey platforms for collecting data online or offline using Android devices allow data collectors to use text or numbers, capture GPS coordinates and photos, and include audio or video. Data can then be exported to Excel or other databases or statistical software.
World Vision Haiti’s tablet-based system will allow them to avoid the timely and error-prone data input process associated with paper surveys and analyze data more quickly. They will be able to combine data collection from multiple programs into one survey and evaluate the data separately or analyze data from combined surveys if desired.
We at I4DI believe there are three key areas to increasing data quality and use of results: survey optimization, mobile data collection and rapid analytics.
The benefits of survey optimization – Reducing the number of surveys carried out by aligning and rightsizing questions reduces the burden on both data collectors and respondents, and significantly cuts down the costs by eliminating unnecessary data collection exercises. In addition, aligned surveys open opportunities for meta-analysis and evaluations across technical programs and area programs and allow for multiple reports to be produced seamlessly and with less effort. This optimization process requires a small upfront investment of time and coordination across teams, and in turn, brings significant returns in financial and knowledge assets to the organization.
The benefits of mobile data collection – Paper-based data collection is challenging to manage. Enumerators must carry physical surveys, carefully maintain confidentiality and store them appropriately to avoid them getting wet or lost. It is difficult to monitor, validate or analyse data until the enumerators reach the home office and survey results are manually input into a database, increasing the chance for human error.
Mobile data collection allows enumerators to efficiently collect data using light and easy-to-use Android tablets. Interface design tools allow for logic to be built into the survey to ensure irrelevant questions are not asked, and data entered is free from human errors. Once the survey is collected and the tablet is connected to the internet, data can be sent directly to the central server for storage and can be exported in several different formats for analysis (including .csv files to be uploaded to statistical software and GIS analysis, GPS coordinates to map data using Google Earth, and Microsoft Word formats for easy review). In-device and in-database encryption is built in with additional security features such as login passwords, and backups that are all used to protect sensitive information. Since data is exported for analytics using software that is available on local computer machines, organizations have full control of the data and do not nee to rely on costly third party solutions for data storing and hosting.
The benefits of rapid analytics – Analysing data is a time consuming process. Delayed analysis and reporting reduces the lead-time for program course corrections and broader strategic decision-making. R Statistical Software (free and open source) allows users to use standardized code to analyse survey data in minutes, producing standardized accurate results that can be used immediately for course correction within a given program, or aggregated across APs or TPs for strategic decision-making. Rapid analytics using R mitigates human error and saves resources by reducing the time and effort spent on data analytics. When statistics are done on optimized surveys using powerful analytic tools such as R, robust and meaningful insights can be created at multiple levels of the project and program bringing to light transformative information about what works, what does not and why.
World Vision Haiti will soon begin implementing the Kore Fanmi Program, a pilot program of the Haitian Government in partnership with the World Bank and UNICEF, which seeks to improve basic services to vulnerable families through regular visits from community social workers.
Instituting the tablet-based data collection system from the beginning of the Kore Fanmi program will enable World Vision Haiti to more easily make data-driven decisions to increase the wellbeing of the communities they serve and will strengthen their monitoring evaluation capacity.
