How do we know if development projects have impacted people’s lives? We can collect data on how many people participated in a project or how much their income increased. We can also measure the effect on the number of people with access to a service or we can count the amount of land that has been reforested. But when we know that complex development challenges take a long time to change, how do we clarify our impact beyond these specific measures and the very short project life cycle, which is usually three to five years?
Natural tree regeneration not only helps protect the environment and enhance livelihoods, but cuts down on women’s time collecting household fuel wood. Photo: Brent McCusker
This question was at the heart of a challenge recently taken up by the Agency’s Productive Landscapes Team in the Land Tenure and Resource Management Office.
Real landscape-level change takes a long time to detect and often eludes our most finely tuned impact indicators. Because environmental and landscape change happens over decades, and because human actions are often the result of many causes, E3 developed a holistic assessment methodology called “Stocktaking” and tested it in several rural Malawian landscapes.
In trying to understand both the unintended and long term impacts of our interventions, the team drew upon findings in the Sahel that show significant re-greening of the land over the last thirty years. That finding was identified only after interviewing local people and asking them about the reasons for their successes—not passing judgment on their actions, but by identifying the root causes of successful land transformations and the ways in which land users overcame barriers.
Stocktaking differs from traditional impact assessments or monitoring and evaluation methods. These latter techniques judge success or failure against a benchmark (indicator) to determine whether or not a project has met its specific goals over a bounded period of time. Stocktaking takes a different path. The focus is on long term, multi-sectoral changes, and in discovering hidden and/or unintended impacts. For instance, a Stocktaking approach might examine how an agricultural intervention led to increased food production and forest regrowth and an increase in the amount of credit in a village. This variety of different outcomes might not be captured in a traditional assessment technique. Stocktaking can be used to identify unintended impacts long after a program or development investment has ended.
With Stocktaking in mind, the E3 team traveled to Malawi in June and again in August of this year to search for the root causes of landscape change. Malawi’s north is relatively land abundant and USAID’s interventions have built value-chains from the local environment. Practices such as beekeeping, fishing along Lake Malawi, and sustainable cash crop production are all livelihood enhancing activities that put money in the hands of farmers without damaging the natural resource base.
After using the Stocktaking methodology to interview several households and community groups, the team learned valuable lessons about the longer-term impact of USAID interventions, and many of the positive unintended consequences of natural resource management projects. For instance, respondents remarked that natural tree regeneration resulted in significant labor savings. Women were able to reduce the amount of time they spent collecting fuel wood and transfer that labor savings to other income generating activities. Natural tree regeneration also reduced the amount of conflict with park rangers of nearby conservation areas. Beekeeping in the Nyika-Vwasa Forest Reserve generated sufficient capital for project beneficiaries to start a range of businesses.
The follow-up trip in southern Malawi in August 2013 discovered similar unintended consequences. The Stocktaking methodology was conducted on water projects in an irrigation and watershed management scheme. A key finding was that village savings and loans, a type of micro-lending institution, were critical in financing activities such as buying seeds for more diverse crops that will help farmers adapt to climate change.
Like the re-greening of the Sahel, these unintended consequences of natural resource management interventions may have fallen “under the radar” in normal monitoring and evaluation since they were not expressed goals of any single project. Additionally, natural regeneration is difficult to quantify with traditional assessment and is easy to miss with standard geospatial imagery. Stocktaking team members are in the process of examining advanced geospatial methods to determine when forested plots were either naturally regenerated or planted. By locating interventions on the map and using such images, a longer term time series analysis can be compiled to determine exactly when the landscape changed, so that Stocktaking teams can then probe deeper with stakeholders to discover why that change occurred. A instructional guide on how to conduct a Stocktaking evaluation and a community discussion board are found at: http://www.frameweb.org/CommunityBrowser.aspx?id=7050&lang=en-US
The Stocktaking approach is one way the USAID Forward principles of evidence-based decision making and local stakeholder participation are supporting improved development outcomes in the Malawi and beyond.
