Monitoring and Evaluation Introduction: Food Nutrition Last Updated August 03, 2018. Original Release: March 28, 2013 Author: Kevin Boyle, President, DevTreks Version: DevTreks 2.1.4 This reference uses a food nutrition analysis example to demonstrate how DevTreks can be used to monitor and evaluate projects, programs, and technologies. Section Page Monitoring and Evaluation (M&E) Introduction 2 M&E in DevTreks 10 M&E Standards 14 Introduction to Food Nutrition M&E Example 14 Food Nutrition M&E Inputs 15 Food Nutrition M&E Operations and Components 24 Food Nutrition M&E Outputs 33 Food Nutrition M&E Outcomes 40 Food Nutrition M&E Budgets 48 Food Nutrition M&E using Algorithms and TEXT datasets 58 M&E in Other Industries 59 Summary and Conclusions 65 Versions 2.0.4 and 2.0.6 made substantial changes to the M&E calculations and analyses shown in this reference’s images. Although many of these images are out of date, their substance remains accurate. The associated Monitoring and Evaluation Analysis references have been updated with current images. The sibling M&E Calculation reference discusses using the M&E tools together with the Resource Stock tools. Not every base element needs M&E calculations –the Resource Stock Indicators provide complementary uses. The Social Performance tutorial demonstrates the use of algorithms, Indicators, and TEXT datasets, for these purposes. Introduction DevTreks faced a dilemma while researching the best way to carry out food nutrition analyses (see the Malnutrition Analysis tutorial). Food nutrition professionals seldom use the technology assessment and economic evaluation framework (i.e. budgets) found in DevTreks. Practitioners are not adding food nutrient composition properties to economic Inputs, combining them into economic Operations, and then assessing whether or not Outcomes, or combinations of Outputs, pay off. That is, do the Food Operating Budgets demonstrate that the benefits of the food consumption technologies (meals) exceed the costs? In addition, DevTreks’ “best practice” data recommendations are not followed by food nutrition experts. No mention of Work Breakdown Structures and no economic performance measures (net profits, cost per unit output). Instead of forging ahead with our (quite purposefully) self-righteous economics, we decided to try to accommodate the existing frameworks used by food nutrition practitioners. A prominent framework used by those professionals stands out –they carry out monitoring and evaluation programs for food nutrition projects and programs. Experts from several other industries use the same framework. Closer scrutiny (1*) of those monitoring and evaluation frameworks disclosed enough similarities to DevTreks’ technology assessments and economic evaluations (and the U.S. GAO’s recommendations for capital programs) to offer a potential solution. A. Monitoring and Evaluation (M&E) Introduction The following image displays a framework for monitoring and evaluating an agricultural development and food nutrition project (USAID, 2012). The following three images demonstrate that international organizations use similar frameworks for general monitoring and evaluation analyses (UN 2009, IFRC 2011, IIEP 2009) These M&E frameworks consistently use a system of indicators, also known as a “results map” (UN, 2009), or “Logic Frame” (logframe), to measure M&E elements (Inputs, Activities or Actions, Outputs or Results, Outcomes, and Impacts or Goals) for the purpose of monitoring and evaluating project or program results. IFRC (2011) defines a logframe as “a table (matrix) summarizing a project/programme’s operational design, including: the logical sequence of objectives to achieve the project/programme’s intended results (activities, outputs, outcomes and goal), the indicators and means of verification to measure these objectives, and any key assumptions”. IFRC definitions for each of these terms, which derive from OECD definitions, can be found in the sections that follow. The Social Performance Analysis tutorial uses the term “impact pathways” rather than “results map” or “logframe”. The quantitative or qualitative indicators measure Benchmark, Target, and Actual (or Alternative), conditions. Quantitative indicators are measured using numbers, rates, ratios, proportions, percentages, and probabilities. Qualitative indicators are measured using terms such as “Compliance with, Quality of, Extent of, Level of” (UN, 2009). The following image demonstrates one suggested step-by-step approach for using indicators with M&E logframes (IIEP, 2009): The following image displays a typical M&E quantitative indicator report tracking system (IFRC, 2011): The following image displays a mix of both potential quantitative and qualitative indicators that might be needed in M&E food nutrition projects (World Bank, 2013): Several M&E references also document the importance of accounting for risk in M&E analyses (UN 2009, World Bank 2013, IFCR, 2011). The following image demonstrates one example (World Bank, 2013). The related Monitoring and Evaluation references document how Version 2.0.4 upgraded the M&E 2 Calculator to accommodate measuring the risk and uncertainty associated with indicator measurement and valuation. The Social Performance Analysis tutorial demonstrates how to use Monitoring and Evaluation to integrate Performance Monitoring with Impact Evaluation. B. Monitoring and Evaluation in DevTreks A monitoring and evaluation logframe’s elements (Inputs, Activities, Outputs, Outcomes, Impacts) are consistent with DevTreks standard base elements. Combinations of Inputs, or Operations (i.e. Activities), result in combinations of Outputs (i.e. Results), or Outcomes. The Operations, with their Inputs, are used to determine costs. The Outcomes, with their Outputs, are used to determine benefits. DevTreks budgets, which combine the Operations and Outcomes, produce economic performance (net profit, benefit-cost ratio) and productivity (cost per unit output, cost per dollar revenue, output per unit input) measures. Those performance and productivity measures can be interpreted as economic “Impacts”. DevTreks base elements (Input, Output, Operation, Component, Outcome, Time Period, Budget) can be extended, using calculators and analyzers, with any properties needed by an analysis. For example, food nutrition Inputs can be extended with nutrient composition properties or other “input indicators”. Operations can be extended with household meal consumption properties or other “activity indicators”, food nutrition Outputs can be extended with food nutrient goods delivered properties or other “output indicators”, and food nutrition Outcomes can be extended with population health improvement change properties or other “outcome indicators”. Budget performance measures, such as improvements in regional household hunger, can be extended with other “impact indicators”. IFRC (2011) defines indicators as “a unit of measurement that helps determine what progress is being made towards the achievement of an intended result (objective)”. DevTreks supports the measurement of quantitative M&E indicators by using flexible, abstract, sets of indicator properties, such as the following: Although simple, these properties appear to be capable of handling most of the quantitative indicators used in M&E analysis. All of the Conservation Technology Assessment (CTA) algorithms documented in the Technology Assessment tutorials can be used to set these calculations. They’ll evolve to handle additional types of M&E indicators and analyses (i.e. Scheduling Risk, Timeliness Penalty, and Critical Path) in future releases. The current version allows up to 15 quantitative indicators and a Score to be used per M&E element. M&E practitioners have to decide the best approach for indicator measurement. In DevTreks, each M&E element has to be entered as a standalone entity (base element), which might or might not be associated with an actual program or project (budget). The whole logframe isn’t completed, or necessarily known, until all the elements are copied into a budget. Although some Operations have to be measured for, and added to, specific budgets, other Operations may not need to be associated with an actual budget. “Generic” Operations can be defined that are associated with “generic” targeted populations. For example, the “typical” food nutrients consumed in a meal can be defined for a representative sample of school children who live in a particular city neighborhood. A group of Operations can be completed for a given household, or typical household, that measures the family’s dietary diversity, and nothing more. Stand-alone Outcomes can be used to measure Disability Adjusted Life Years (DALYs) for representative malnourished children, and nothing more. In most cases, Output and Outcome indicators are measured as the results of specific, rather than generic, Operations (Actions). They often need to be identified as belonging to a specific, known, budget. The full results map (UN, 2009), or logframe table (IFRC, 2011), can be produced using a DevTreks Budget analysis (i.e. the IFRC’s indicator reporting system displayed above). All report styles in DevTreks can be customized without much trouble (by linking a custom stylesheet to a calculator or analyzer). The full results map can also be generated using the techniques introduced in the Social Performance Analysis tutorial (i.e. Indicators and TEXT datasets). Even when a full analysis, or budget, is completed, DevTreks does not allow changes to Input, Output, Operation, Component, or Outcome, base element M&E indicator properties. We don’t allow changes in these base element indicators because we think time and expense are better spent adjusting the original base element (Input, Operation/Component, Output, Outcome), rather than Budget element, indicators. A Budget’s Time Period, Budget/Investment, and Budget/Investment Group, base element M&E indicators can be changed (refer to the sibling M&E Calculation reference). C. M&E Standards The related references (UN 2009, IFRC 2011, World Bank 2013) point out the importance of limiting M&E analysis to only the most important M&E elements and their associated indicators. That’s critical in DevTreks, because budgets can technically contain an unlimited number of M&E elements, along with their associated indicators. M&E analyses can quickly become an exercise in futility if abused by using too many elements and indicators. The same references also point out the importance of using industry-recognized indicators (and by default, industry-recognized M&E elements). Unfortunately, the references don’t recognize the implications for computer-automated M&E analysis. Automated M&E analysis requires industry-recognized standards. In the “public goods” context of DevTreks, the standards must be non-proprietary. DevTreks recommends that those standards start by using Work Breakdown Structures (WBS) to classify all M&E elements. Second, the WBS should include industry-recognized indicators for measuring all of the elements. That’s asking a lot, but automation is already here and somebody has to do this “public goods” work (a good place to start is to build the WBS using the World Bank (2013) and USDA (2010, 2012) references). D. Introduction to Food Nutrition M&E Example Our literature review did not dig up a comprehensive logframe example that could be used for this food nutrition M&E example (2*). The challenge faced by DevTreks in developing new calculators is to derive a coherent set of M&E indicators, or logframe, that work together for all of the base element elements found in a DevTreks’ budget. For this example, we chose to keep it simple. The goal of the example food nutrition program is to increase child food nutrition by educating households about better food nutrition practices and providing supplementary nutrients to children. The image below (World Bank, 2013) gives an overview of this type of food nutrition intervention. Note that without the use of a standard WBS, M&E elements and indicators become much harder to identify and use (and M&E analysis becomes much more adhoc). These M&E logframes, with their calculators, analyzers and WBSs, will evolve over the years. The Malnutrition Analysis 1 tutorial explains the food nutrition calculators and analyzers used to produce the food nutrition calculations discussed in this reference. Some of the URLs references in the following section contain calculations and analyses conducted using the M&E 1 set of tools. Appendix A in the M&E Calculation reference documents the status of those tools. E. Food Nutrition M&E Inputs The IFRC (2011) define inputs as “the financial, human and material resources needed to carry out activities”. Economists think of inputs as being the resources need to carry out economic production. The starting point emphasized by many food nutrition experts is to keep track of food nutrition quantitative intake. Logical Inputs to measure are food inputs. The following image introduces a logical data source for food inputs and their nutrients (USDA ARS 2012): This image demonstrates that useful food nutrition input indicators to track include the amount of food nutrients in food inputs consumed. Additional Input Indicators: Depending on the type of food nutrition analysis needed, additional Input indicators might include: any input associated with the Actions listed in Annex E-2 of the 2013 World Bank reference; quantity of food nutrients; quantity of food contaminants; source of food item x; storage properties; social acceptability; container garbage quantity; recycle material; substitute food items; complementary food items; change in quantity used per 1% change in substitute price x; private cost; public cost; environmental damage; low, moderate, and strenuous physical activity of food consumer, training workshop materials; Calculations DevTreks measures food nutrition Input indicators using Food Nutrition Input Calculators and Analyzers. These tools measure the food nutrient composition of food inputs based on their actual, rather than typical, serving sizes. The calculators can measure additional indicators specific to the Input. For example, indicators can be used to measure target levels of food nutrients. A typical calculation produces these of types of indicator measurements (USDA, 2010): A current DevTreks Food Nutrition Input Calculator generates the following actual results. Although food nutrient calculators have been bulk uploaded for over 7,000 food items in the USA, only enough calculations have been run to support these tutorials. Refer to the Malnutrition tutorial for further details on calculating or analyzing food nutrients. Also note that the Social Performance Analysis tutorial introduces techniques that store domain-specific data in TEXT datasets. https://www.devtreks.org/hometreks/preview/farmworkers/input/BARLEY,PEARLED,RAW/2147395842/none A current DevTreks M&E Input Calculator generates the results displayed in the following image. Note that this URL contains examples of 2 M&E calculations. The sibling Monitoring and Evaluation Calculation reference explains that the M&E 1 calculators and analyzers have been deprecated –only the M&E 2 tools are supported. Rather than maintain the M&E 1 tools, DevTreks preference for Version 2.0.4 was to upgrade the M&E 2 tools using newer techniques discovered from the Resource Stock calculators and analyzers. https://www.devtreks.org/hometreks/preview/farmworkers/input/2013 Nutrition Training Workshop/2147397489/none A current DevTreks M&E Input Statistical Analyzer generates the following actual results: https://www.devtreks.org/hometreks/preview/farmworkers/input/2013 Food Package/2147397521/none F. Food Nutrition M&E Operations The IFRC (2011) defines Activities (Operations) as “the collection of tasks to be carried out in order to achieve an output”. DevTreks defines Operations as groupings of Inputs. Economists think of Operations as distinct steps in the production process, or technology, used to produce output. Logical Operations need to keep track of the combined effects of the food inputs, or meals, on household members. The following images displays measurement of common food nutrition-related indicators (USDA 2010): These images demonstrate that useful food nutrition operation indicators to track include the benchmark and targeted levels of food nutrients in meals consumed and physical activity for each household member. For the purposes of M&E analysis, the actual levels must also be tracked over time. Additional Operation Indicators: Depending on the type of food nutrition analysis needed, additional Operation indicators, expressed as Actions, might include: provide quantity x of Child Health Days (or any of the other Actions listed in Annex E-2 of the 2013 World Bank reference); provide quantity x of food nutrient supplement x in food operation x to family member x; measure % of recommended food nutrient x in food operation x consumed by household member x; eliminate quantity x of food contaminant x consumed by family x; measure % of recommended food contaminant x in food operation x consumed by household x; determine tastiness of food operation x; measure the social acceptability of food operation x; measure the time spent on shopping and food preparation for food operation x; record substitute or complement activities possible for food operation x; measure height, weight and body mass index of household member x consuming food operation x; record hours of low, moderate, and strenuous physical activity for household x; develop food nutrient training materials for nutrient workshops; develop affordable food recipes for target population; organize and deliver food nutrient workshops; provide access to food nutrition intervention program to household x; reduce food cost for food operation x; reduce cost of garbage generated by food operation x; change in input quantity per 1% change in total cost; document the environmental impact associated with food operation x Example Calculations DevTreks measures food nutrition Operation indicators using Food Nutrition Operation Calculators and Analyzers. These tools combine an Operation’s Inputs and analyze the aggregated numbers. The result can be as simple as a report displaying summations of all of the food nutrients in an Operation. The analyzers also measure indicators specific to the Operation element. For example, indicators can be used to measure the exact quantity of the food Operation consumed by each household member. A current DevTreks’ Food Nutrition Operation Analyzer generates the following actual results: https://www.devtreks.org/hometreks/preview/smallholders/operation/Turkey Packaged Meat Sandwich/2091557249/none A current DevTreks M&E Operation Calculator generates the following actual results: https://www.devtreks.org/hometreks/preview/farmworkers/operation/2012 Packages Delivered/2091557267/none A current DevTreks M&E Operation Analyzer generates the following actual results: https://www.devtreks.org/hometreks/preview/farmworkers/operationgroup/M and E 2 Food Delivery/758/none G. Food Nutrition M&E Outputs The IFRC (2011) defines outputs as “tangible products, goods and services and other immediate results that lead to the achievement of outcomes”. Output indicators “measure whether project activities are occurring on schedule and budget” (i.e. number of meals provided in one week, number of food nutritionist visits, number of food nutrition training workshops held). They tend to need frequent measurement. Economists think of outputs as being the goods and services produced as a result of those activities. They think of Output indicators as supporting measurements of performance (i.e. net profits) and productivity (i.e. quantity of output produced per quantity of input). The following image (UN, 2009) demonstrates that Output indicators measure tangible project results (i.e. level of, percentage of, proportion of). In contrast, Outcome indicators measure changes in performance, behavior, or quality of life (i.e. attitudes, knowledge and practices). The following image (World Bank, 2013), doesn’t specify whether these are Output or Outcome indicators. DevTreks inexperience conducting M&E food nutrition analyses kicks in when trying to decide what makes a good Output vs. Outcome indicator. Our literature review suggests that Box 3, in the following image, lists Output indicators because they measure tangible project results. They would be measured using indicators such as “number of child health days held” and “number of child contacts made”. We’re less sure about Table 2. Although the World Bank appears to consider these to be Outcome indicators, under some circumstances they might be considered tangible project results, or Output Indicators, as well. Clearer Output indicators associated with Table 2 might include “number of food nutrient packages distributed” and “quantity of food nutrient stocks delivered to health centers”. Please refer to the Outcomes section, next, for further discussion about the differences between food nutrition Output and Outcome Indicators. This image demonstrates that useful food nutrition output indicators to track include tangible project results. Additional Output Indicators Depending on the type of food nutrition analysis needed, additional Output indicators might include: number of food nutrient packages distributed, number of food nutrient stocks delivered to health centers, number of growth monitoring days completed, proportion of children ages 6 to 25 who have received nutrient supplements (or any of the indicators listed in the World Bank reference that clearly measure project tangible results); amount of economic output and income x produced by member x; degree of malnourishment property x; rate of morbidity or mortality; degree of good food hygiene practices by household x; number of food nutrition workshops attended by targets; number of new food recipes used by household x per week; level of client satisfaction with new diets; height, weight and body mass index of household member x at time x; amount of obese weight loss; hours of physical activity per week; amount of wealth; number of incidents of diarrhea reduced; % change in quantity of good demanded (or supplied) from 1% change in price; change in quantity demanded from 1% change in income; change in quantity demanded per change in price of related output; dollar benefits for output x; amount of garbage reduced; amount of environmental benefit increased; Example Calculations DevTreks measures food nutrition Output indicators using Food Nutrition Output Calculators and Analyzers. These tools measure the tangible results of food nutrition interventions using the indicators identified. A current DevTreks’ Health Care Output Calculator (3*) generates the following actual results (the QALY shown here may be supplemented with a DALY for food nutrition): A current DevTreks M&E Output Calculator generates the following actual results: https://www.devtreks.org/hometreks/preview/farmworkers/output/2013 Number of children examined for malnutrition/2141223448/none A current DevTreks M&E Output Analyzer generates the following actual results: https://www.devtreks.org/hometreks/preview/farmworkers/outputgroup/M and E Nutrition Distribution, Stats and Changes/1936433722/none H. Food Nutrition M&E Outcomes The IFRC (2011) defines outcomes as “the primary results that lead to the achievement of the goal (most commonly in terms of the knowledge, attitudes or practices of the target group)”. The UN (2009) defines outcomes as “actual or intended changes in development conditions that interventions are seeking to support”. The IIEP (2009) defines an outcome as “a short-term or medium-term effect of an intervention’s outputs.” DevTreks defines Outcomes as groupings of Outputs. Economists tend to leave outcomes out of their equations, but economic outcomes can be thought of as being the changes in desired economic benefits (increased revenue, decreased costs) resulting from a production process. The following image shows that the UN (2009) uses Outcome indicators to measure changes in performance, behavior, or quality of life (i.e. what the IFRC (2011) calls knowledge, attitude, or practice): As mentioned in Section G. Output Calculators, the indicators in the following table (World Bank 2013) could be either Output or Outcome indicators. They might be more clearly used as Outcome indicators by using language like: “proportion of mothers who ensure children ages …” (change in practice), “proportion of adults who believe contraceptives improve household well-being” (change in attitude) and “increase in proportion of women who understand, and practice, breast-feeding” (change in attitude and practice). Make sure to work with M&E food nutrition experts to define these M&E elements and their indicators (better yet, work with them to develop WBS standards). These images demonstrate that useful food nutrition outcome indicators to track include changes in the practices, attitudes, knowledge, performance, and quality of life, of targeted populations and targeted health delivery clinics. Additional Outcome Indicators Depending on the type of food nutrition analysis needed, additional Outcome indicators might include: proportion of mothers who ensure children ages 6 to 25 receive food nutrient supplements, proportion of health centers that maintain an adequate stock of …(or any of the indicators listed in the World Bank reference that clearly measure knowledge, attitude, practice, performance, or quality of life); change in disability adjusted life year probability (DALY) for household member x; change in quality adjusted life year probability (QALY) for household member x; change in Dietary Diversity Index; change in Health Eating Index 1 by household x; change in Food Contaminant Index 1 consumed by household x; change in body property x (bmi) by household member x; change in food expenditures by household x; change in productive output and income generation; change in women discretionary income; dollar benefit for household x from food technology x; complementary outcome x; substitute outcome x; environmental benefit by household x; increase in household x income Example Calculations DevTreks measures food nutrition Outcome indicators using Food Nutrition Outcome Calculators and Analyzers. These tools combine an Outcome’s Outputs and analyze the aggregated numbers. The result can be as simple as a report displaying summations of all of the children receiving nutrient supplements in an Outcome. The analyzers also measure indicators specific to the Outcome element. For example, indicators can be used to measure Disability Adjusted Life Years (DALYs) for targeted household members. A current DevTreks’ Health Care Outcome Calculator generates the following actual results (the QALY shown here may be supplemented with a DALY for food nutrition): Further information about these deprecated Health Care tools can be found in the Outputs section of this reference. A current DevTreks M&E Outcome Calculator generates the following actual results: https://www.devtreks.org/hometreks/preview/farmworkers/outcome/2012 Families no malnourished children/5742/none A current DevTreks M&E Outcome Analyzer generates the following actual results: https://www.devtreks.org/hometreks/preview/farmworkers/outcomegroup/Nutrition Delivery/36/none I. Food Nutrition M&E Budgets (Impacts) The IFRC (2011) defines impacts as “the positive and negative, primary and secondary long-term effects produced by an intervention, directly or indirectly, intended or unintended”. The UN (2009) defines impacts as “actual or intended changes in human development as measured by people’s well-being”. Impacts generally capture changes in people’s lives”. DevTreks defines Budgets as groupings of Operations, along with their associated Inputs, and Outcomes, along with their associated Outputs. Economists think of budgets as defining the complete technology, encompassing all inputs and outputs, used in a production process. By adding prices to the inputs and outputs, economists can measure Impacts using indicators for economic performance (net profit, benefit-cost ratio) and productivity (cost per unit output, cost per dollar revenue). Our literature review suggests that appropriate Impact indicators for the type of food intervention project displayed in the following image (World Bank, 2013) would be tied to the Nutrition Objectives listed in the first column. Impact indicators might be: Percent of population with reduced micronutrient deficiencies, Percent of women who are no longer anemic, Percent of population who have adopted healthy eating practices. Food nutrition experts need to be on the M&E team that decides on the indicators to measure (or better yet, on the standards-setting team that builds the WBSs containing these M&E elements and indicators). The following image displays common food nutrition economic performance impacts: The following image displays an indicator, Dietary Diversity Index, used by food nutrition experts as a good proxy for household food nutrition status (FAO, 2011). These images demonstrate that useful food nutrition impact indicators to track include physiological indicators (Proportion change in Anemic Women), food nutritional status indicators (% Population with Target Dietary Diversity Index) and economic performance indicators (Cost per DALY Saved). Additional Budget Indicators Depending on the type of food nutrition analysis needed, additional Budget (or Impact) indicators might include: : percent of population with reduced micronutrient deficiencies; percent of women who are no longer anemic; percent of population who have adopted healthy eating practices; percent of women delivering normal weight babies; percent of children who are food secure (i.e. or any of the indicators listed in the World Bank reference that directly measure food intervention impacts); change in Dietary Diversity Index x for targeted population x (the World Bank reference lists additional food nutrition indexes); change in Health Eating Index x by targeted group x; change in Food Contaminant Index 1 consumed by household x using food technology x; change in body mass index x by household member x using food budget x; change in food expenditures by household x using food budget x; change in QALY or DALY for target population x; dollar benefit per dollar cost for targeted population x from food technology x; incremental benefit per incremental cost; benefit cost ratio probability Example Calculations DevTreks measures food nutrition Budget indicators using Food Nutrition Budget Calculators and Analyzers. These tools combine Operations and Outcomes and analyze the aggregated numbers. Besides dollar benefits and costs, an analysis might display summations of all of the food nutrients contained in all food Operations and proportions of children receiving supplements in all food nutrition outreach Outcomes. The analyzers also measure indicators specific to the Budget element. For example, indicators can be used to measure overall project impacts, such as reduction in poverty level, total increase in proportion of well-fed children in a targeted population, and cost per DALY saved. A current DevTreks’ Food Nutrition Budget Analyzer generates the following actual results: https://www.devtreks.org/hometreks/preview/smallholders/budget/Food Nutrition Subsistence Stocks SR01/273083905/none The calculations for all of the food nutrient calculators and analyzers can be found in the Malnutrition Analysis tutorial. A current DevTreks M&E Operating Budget Calculator generates the following actual results: https://www.devtreks.org/hometreks/preview/farmworkers/budgettimeperiod/2012 Malnutrition Progress/5747/none A current DevTreks M&E Operating Budget Analyzer generates the following actual results: https://www.devtreks.org/hometreks/preview/farmworkers/budget/M and E 2 Operating Budget/273083904/none J. Food Nutrition M&E using Algorithms and TEXT datasets The Social Performance Analysis tutorial demonstrates how to define M&E pathways using alternative data structures. Specifically, by using the Resource Stock and M&E calculators with custom algorithms and TEXT datasets to define “social impact pathways”. The pathways can be defined directly in the Indicators and TEXT datasets, rather than in all of the base elements. The Resource Stock Analysis reference discusses the development of consumer-oriented software that allows shoppers to assess the “social soundness” of companies prior to making purchases (i.e. consequential digital activism). The latter applications may be necessary when current institutions fail to safeguard society properly and consumers decide to take independent action. K. Monitoring and Evaluation in Other Industries DevTreks prefers using frameworks, and building related tools, that will support multiple industries. While vetting the monitoring and evaluation system for food nutrition, we investigated whether it could be used in other industries. The following examples demonstrate how the process can be adapted for use in other industries. Refer to the Resource Stock Calculation, Resource Stock Analysis 1, CTA, CTAP, and Social Performance, tutorials for concrete examples from some of these sectors. Health Care M&E The following image (IFCR, 2011) suggests that health care investment, as well as other civil engineering, projects are amenable to standard M&E analysis. Also review the Social Performance Analysis tutorial for further examples of using M&E in the health care sector. The following image (IHP and WHO, 2011) lists core indicators that government health ministries might find useful in M&E programs that track the efficiency of their health care sector: Natural Resources Conservation M&E The author has trained conservationists in the use a USDA, Natural Resources Conservation Service conservation planning process known as SWAPAH (Soil, Air, Water, Plants, Animals, and Humans). This planning process can be considered a comprehensive logframe that can be used for natural resources M&E programs. Conservation projects are amenable to M&E analysis as follows: Input Operation Output Residue Harvest N runoff Native seed Wildlife Habitat Land Prep Hectares planted Outcome Impact % Farmers adoption Change in Water Quality Index Habitat restoration Change in T&E species count Although the actual planning process has many more ingredients (conservation practice standards, checklists of resource concerns, conservation practice physical effects, conservation impact report), this covers the core capacity needed to monitor and evaluate natural resources conservation projects. Refer to the Resource Stock Calculation 1 tutorial for further examples. Natural Resource Stock Damage (Climate Change) Prevention M&E The CTA-Prevention (CTAP) reference in the Social Performance Analysis tutorial demonstrates how M&E Indicators can be used to support disaster risk prevention analysis in the context of climate change. The following images of “disaster impact pathways” come from the Social Performance Analysis 3 reference. Education and Human Capital M&E We haven’t thoroughly researched the tutorial yet, but the following image (IIEP, 2009) suggests that the industry is amenable to standard M&E analysis: The following image comes from the Social Performance Analysis 3 reference and further demonstrates using M&E for human capital improvements. Summary and Conclusions Clubs using DevTreks can monitor and evaluate projects and programs involving malnourishment elimination, healthy lifestyle improvement, natural resources conservation, flood damage assessments, health care efficiency, and related project and programs, with “real data”, “real evidence”, and “transparent data”. “Real data” means having real wetland preservation cost and benefit data for every wetland investment in a county, real time series food budgets for the rural sector, real databases of food nutrition monitoring and evaluation projects, and real coastal recovery flood damage assessment databases. “Real evidence” means having real statistical analysis of any city’s food nutrition improvement projects, real food nutritional statistics for all of a school district’s food budgets, and full cost-benefit statistics for all of a country’s health care treatments. “Transparent data” means that all of the data and all of the evidence can be accessed by people and machines using URIs. In summary, DevTreks provides technology that can improve peoples’ lives and livelihoods. Footnotes 1. The author started studying monitoring and evaluation frameworks while majoring in International Agriculture Development as a graduate student at UC, Davis, USA. The author made the connection between the elements in economic budgets and monitoring evaluation frameworks a long time ago, but didn’t have the time to automate it until recent years. The author is not a food nutrition expert, so don’t be surprised if the food nutrition example needs substantial improvement. As usual, we encourage readers to build their own references, produce their own video tutorials, and develop their own algorithms. The main point of the reference is to document DevTreks’ general monitoring and evaluation framework. 2. We tend to be suspicious when references, like those below, don’t include comprehensive, real, examples. None of the references include an A to Z example, with a complete data set, of a completed M&E analysis. Nevertheless, they have enough consensus, and single purpose examples, to prove their point. DevTreks “proofs” our calculators and analyzers using real data sets –so we’ll keep looking for examples that include comprehensive data sets. The root cause of not having readily available data sets is not having really good automated M&E analysis (with the required WBS standards). In the meantime, complete, but sample, M&E data sets can be found in DevTreks other Monitoring and Evaluation references. 3. This Health Care calculator was built several years ago to prove the point that existing institutions were not doing a very good job with documenting performance. In other words, if 1 developer with a zero budget can do all of this –what excuse do the existing institutions, with their hefty budgets, have? These types of domain-specific calculators are the responsibility of social networks –not software developers. These types of networks, even if they don’t currently exist, are a type of institutional reform. Who knows, they may even prove that “$28” donations are a key part of the needed reforms (i.e. rather than rent seeking). References Food and Agriculture Organization. Guidelines for Measuring Household and Individual Dietary Diversity. 2011 International Federation of Red Cross and Red Crescent Societies. Project/programme monitoring and evaluation (M&E) guide. 2011. (www.ifcr.org) International Health Partnership and World Health Organization. Monitoring, Evaluation, and Review of National Health Strategies. A country-led platform for information and accountability. 2011 International Institute for Educational Planning. Manual for Monitoring and Evaluating Education Partnerships. UNESCO 2009. (www.iiep.unesco.org) US Agency for International Development. Nutrition by Design, Design and Measuring for Nutrition Impacts. Presentation made December, 2012 for Agriculture and Nutrition Global Learning and Evidence Exchange (NGLEE). U.S. Centers for Medicare and Medicaid Services. Roadmap for Implementing Value Driven Healthcare in the Traditional Medicare Fee-for-Service Program. (no date, last accessed May, 2013) U.S. Department of Agriculture, ARS. Composition of Foods Raw, Processed, Prepared USDA National Nutrient Database for Standard Reference, Release 25, September, 2012. U.S. Department of Agriculture and U.S. Department of Health and Human Services. Dietary Guidelines for Americans, 2010. 7th Edition, December 2010 US Government Accountability Office. Applied Research and Methods. GAO Cost Estimating and Assessment Guide. Best Practices for Developing and Managing Capital Program Costs. March, 2009. United Nations Development Programme. Handbook on Planning, Monitoring and Evaluating for Development Results. 2009 World Bank. Improving Nutrition Through Multisectoral Approaches. 2013 References Note We try to use references that are open access or that do not charge fees. Improvements, Errors, and New Features Please notify DevTreks (devtrekkers@gmail.com) if you find errors or can recommend improvements. Video tutorials explaining this reference can be found at: https://www.devtreks.org/commontreks/preview/commons/resourcepack/Monitoring and Evaluation 1/476/none/ DevTreks –social budgeting that improves lives and livelihoods 1