The Climate Adaptation Learning Activity (CALA), funded by USAID’s Bureau for Humanitarian Assistance (BHA), was designed to strengthen how the agency and its partners learn from and adapt climate programming across diverse humanitarian settings. I4DI played a central role in CALA by designing and delivering a customized AI-powered system to support knowledge management and evaluation at scale.
Our team was tasked with analyzing more than 4,000 documents across 279 BHA-funded activities, including award materials, program reports, assessments, and other learning outputs. To manage this volume and complexity, we built a tailored generative AI workflow using open-source large language models, including Mistral, implemented through LangChain and deployed via custom Gradio and Streamlit interfaces. The system used natural language processing to retrieve, vectorize, and semantically classify unstructured content from PDFs, guided by a predefined analytical framework developed in collaboration with BHA and implementing partners.
Rather than relying on open-ended topic modeling, we structured the AI to tag content against a set of predefined themes spanning core climate adaptation approaches and cross-cutting issues like gender, equity, and localization. Human evaluators reviewed and refined outputs through multiple validation cycles to ensure analytical consistency and contextual accuracy. This allowed us to scale the review process while maintaining control over interpretation and meaning.
Beyond the technical system, I4DI facilitated a series of regional learning exchanges that brought together climate adaptation practitioners, donors, and local actors to share experiences, surface challenges, and reflect on practical lessons across geographies. We also contributed to the development of BHA’s climate learning agenda, helping prioritize future research and evidence synthesis efforts.
CALA demonstrated how generative AI can be applied not just as a tool for automation, but as part of a broader knowledge management system that supports evaluative thinking, adaptive programming, and strategic learning. By combining open-source AI with evaluator-led design, we built an approach that handled complexity without losing analytical depth—one that can inform similar efforts across the climate and humanitarian space.
