DevPacks Analysis Last Updated: September 17, 2018; First Released: September 3, 2014 Author: Kevin Boyle, President, DevTreks Version: DevTreks 2.1.6 (5*) A. Introduction DevPacks are a regular data service that any club can deliver (1*). They are arbitrary hierarchies of data that contain structured xml. The xml must derive from base element data: Inputs, Outputs, Operations, Components, Outcomes, Operating Budgets, or Capital Budgets. DevPacks offer a flexible data service that permits science and technology data, such as randomized control trial (RCT) data, to be collected, analyzed, stored and explained in online knowledge banks (2*). Their data hierarchy consists of a Service element, DevPackGroup elements, one or more DevPack element layers, DevPackPart elements, and Resource Pack elements. The DevPack layers can contain up to 4 layers. An xml document is uploaded to the DevPackPart element. The DevPackPart and DevPack xml data can be analyzed using standard Linked View calculators and analyzers. The Resource Pack contains any multimedia needed to support the part, such as a style sheet or image. The following image displays randomized control trial budget data that has been structured using this hierarchy. The analyses conducted using this data can be found in the References. The typical raw, hierarchical, xml data appears as follows. The devpackpart xml element refers to the xml data that will be analyzed. Those xml files are stored statefully in file/blob storage. Version 2.0.6 tests conducted using this type of multi-hierarchical dataset initially did not work on localhost because the stateful files had been deleted so that “clean machine” tests could be conducted. An important point to remember when conducting similar tests on localhost. Section Page * Social Budgeting Economies of Scale, Scope, and Limitations 5 * Workflows 6 * NPV Calculation and Resource Stock Analysis 14 * NPV Calculation and Monitoring and Evaluation (M&E) Analysis 21 * NPV Calculation and Malnutrition Analysis 28 * NPV Calculation and NPV Analysis 31 * NPV Calculation and LCA Analysis 34 * NPV Calculation and Capital Input Analysis 36 * Conservation Technology Assessments 39 * Sustainable Supply Chain Analysis (Production and Consumption Analysis) 42 * Multimedia and Stories 45 * Knowledge Banks and Summary and Conclusions 46 B. Social Budgeting Economies of Scale, Scope, and Limitations DevPacks contribute to economies of scale and scope in the delivery of social budgeting data services by leveraging existing data, calculators, analyzers, and information technology infrastructure. The ability to conduct metadata analysis, in particular, can aid decision making (see the calculator pattern introduced in the Resource Stock Example). A formal definition for the term “economies of scale and scope” can be found in the Performance Analysis tutorial. Given their potential for decision support, customers should expect to pay for the resources needed to conduct large-scale analysis, including physical infrastructure, analyst time, and new algorithm development. As footnote 3 mentions, the current version has not undergone testing with full datasets yet. The manipulation of the xml/html data used in these services imposes potential limitations on the size of the datasets that can be analyzed (currently). C. WorkFlows The workflows associated with DevPacks work differently than regular base element workflows. The major differences include: Add New Element Workflow: The following image shows that this workflow is similar to any other hierarchical workflow. The major difference is that multiple layers of DevPacks can be added (i.e. Treatment 1, SubTreatment 1, SubSubTreatment 1). As with the Linked View data service, the final element, a DevPackPart, uses the “Select and Save ResourcePack” to add children Resource Packs that support the part, such as images used to enhance the display of the part. DevPackPart Workflow: The following image displays the “Make Base” command button which generates the base document that is uploaded. With the exception of Inputs and Outputs, these commands appear when calculators and analyzers are run for base elements. This document can be downloaded using the Pack workflow that is explained in the Calculators and Analyzers tutorial. The following image shows that the base document can be distinguished from sibling documents by its name –a base document’s file name starts with “Name” rather than “Addin”. This document is usually edited, using a text editor, prior to uploading. Alternatively, the results of NPV calculations can be downloaded and used as an initial DevPackPart. The advantage to using NPV documents is that the NPV calculations don’t need to be rerun for DevPackParts. The disadvantage is that any custom edits made to the NPV calculations, such as quantities and prices, need to be manually calculated for the NPV totals prior to being uploaded. The following image highlights the naming conventions used to save the files generated after saving NPV calculations –file names end with the name of the principal base element being analyzed. In this example, since Operating Budgets are being analyzed, files ending with “_budget.xml” or “budget_full.xml” must be uploaded. When files with both names exist, the “full” suffix means a full dataset, the other file is a summary dataset. Also note the size of the “full” document –it’s often the largest file size. Edit Linked Views Workflow: The following image explains why this workflow differs substantially from regular base elements. This image shows that the base element that is loaded into the Views panel (in this case, a DevPack named Treatment 02) displays its children elements and their linked views, rather than its own. When a selection is made in the top drop down selection list (i.e. Plot 02, Plot 03), the bottom list (NPV Budget Calculator) is filled with linked views associated with that selection, not the parent DevPack (Treatment 02). The following image shows that when a selection is made in the top drop down selection list and that list consists of DevPackParts (i.e. Plot 02, Plot 03), the document displayed below the calculator is the base DevPackPart that was uploaded (in this image, a budget that was built in a precursor to DevTreks). This view is convenient for checking that all edits have been made correctly. If the list contains DevPacks, nothing will be displayed. Calculations cannot be run using this step –a specific calculator or analyzer must be chosen from the second drop down list. When a Linked View is selected from the second drop down selection list (NPV Budget Calculator), the document displayed below the calculator is the calculated results (see the first image in this section). The advantage to this workflow is that a large number of calculations and analysis can be run without having to navigate to and open each one. The disadvantage is that care is needed when using the Edit Linked Views workflow (at the top of the form). In the image above, that workflow is associated with the parent DevPack (Treatment 02). In order to link calculators to each of the DevPackParts displayed in the top selection list (Plot 02, Plot 03), the Edit Linked Views workflow must be accessed from the Edit panel where those elements can be edited. This version does not support the automatic insertion or updating of children calculators and analyzers. The Edit Linked Views command button in the previous image opens the following list of Linked Views. This form is used to add and delete the Linked Views associated with DevPack and DevPackPart base elements. This list differs from non-DevPack lists because it does not allow calculators and analyzers to be opened directly from this form –it’s missing the 4th Views command button (DEL, UNDEL, VIEW, VIEWS). This version requires opening linked calculators and analyzers from the Preview panel. Select Existing DevPack Workflow: This workflow allows existing DevPacks and DevPackParts to be selected and then reused in new data sets. This workflow is not available in this version. It may become available if a need arises. On a related note, the reason that the forms used to edit DevPacks contain 2 sections, with the bottom section being an “Owners Properties”, is because when this feature was first built (over a dozen years ago), the top section allowed existing DevPacks to be renamed for alternative uses. Testing on localhost:5000: If testing on localhost, the devpackpart base document may need to be uploaded. In addition, NPV calculations need to be rerun if the base document is edited in any manner. The NPV calculator will store stateful files in file/blob storage for subsequent analyses. The current version lacks seasoning and may return unnecessary error messages that may need to be ignored. D. NPV Calculation and Resource Stock Analysis DevPacks support the Resource Stock calculators and analyzers documented in the Resource Stock Calculation and Analysis references. NPV calculators are run for each DevPackPart prior to running Stock analyzers for the DevPacks. The NPV calculators process edits made to the base documents uploaded into DevPackParts. New NPV calculators do not need to be linked or run for DevPackParts when the base xml documents already have good NPV calculations. The following image demonstrates that the Media View works exactly the same in DevPacks analysis. The following image displays the result of running a Resource Stock Operating Budget Totals Analysis from a DevPack base element. This example can be found in Appendix C of the Resource Stock Analysis reference. This example demonstrates how to use DevPacks for the Resource Stock Analysis of non-sibling Operating Budgets. Version 2.1.4 changed the datasets in this example to demonstrate the recommended calculator pattern when using Resource Stock and M&E calculators (i.e. the pattern employed by many types of Health Technology Assessments): Indicator metadata – TEXT datasets – custom algorithm – mathematical/statistical library The Resource Stock Calculations used with these Inputs and Outputs employed TEXT datasets holding the NO3 and CO2 Indicators. The 2nd Indicator used PRA techniques (i.e. subalgorithm1). The remaining Indicators were ignored (i.e. and led to the consequent mistakes in Indicator alignment in the Change Analyses). This data is owned by the Carbon Emission club in GreenTreks. http://localhost:5000/greentreks/linkedviews/carbon/devpackgroup/Carbon Budgeting DevPack Group/43/none The following image shows that when NPV calculated results are directly uploaded to each DevPackPart, individual NPV calculations do not need to be rerun for each DevPackPart. This analysis took less than 5 minutes to set up and run. A second dataset was built using this same data, but the 1st DevPack element contains 2 children DevPacks (i.e. Treatment 01, Treatment 02). The children DevPacks each contain the same 3 DevPackParts holding NPV-calculated budgets. No calculators were run for the DevPackParts because the files uploaded held NPV-calculated results. The image demonstrates that the Time Period elements in both budgets have been added to the same budget (2012). The data was structured in this manner because that was deemed the best way to analyze the original 504 budget dataset used to test DevPacks. The associated reference, in the References section, explains more about using Labels to aggregate data. As mentioned, this version has not been tested using large datasets. E. NPV Calculation and Monitoring and Evaluation (M&E) Analysis DevPacks support the M&E calculators and analyzers documented in the M&E Calculation and Analysis references. NPV calculators are run for each DevPackPart prior to running M&E analyzers for the DevPacks (i.e. unless the base documents already contain good NPV calculations). The following image highlights the public goods context behind DevTreks and the aspirations that customers should have when using the software. The following image displays the result of running an M&E 2 Operating Budget Totals Analysis from a DevPacks data service. The underlying datasets employ pre-2.0.4 calculations and are missing the Lower and Upper Estimate properties. This data is owned by the Food Nutrition club in HomeTreks. http://localhost:5000/hometreks/preview/smallholders/devpackgroup/M and E RCT Tests/78/none DevTreks recommends using the calculator pattern introduced with the Resource Stock example for advanced M&E analysis (i.e. store the Indicator data in TEXT datasets). The following image uses the CTA Output TEXT DevPack at the same URL to carry out an M&E Totals Analysis of Indicator data stored in background TEXT files. The base document was uploaded to the devpackpart after running an M&E Output Calculator for the base Output Series. F. NPV Calculation and Malnutrition Analysis DevPacks support the Malnutrition calculators and analyzers documented in the Malnutrition Calculation and Analysis references. NPV calculators are run for each DevPackPart prior to running Malnutrition analyzers for the DevPacks (i.e. unless the base documents already contain good NPV calculations).. The NPV calculators process edits made to the base documents uploaded into DevPackParts. The following image demonstrates typical decision supported by these types of analysis. The following image displays the result of running a Malnutrition Operating Budget Totals Analysis from a DevPacks data service. This data is owned by the Food Nutrition club in HomeTreks. http://localhost:5000/hometreks/preview/smallholders/devpackgroup/Food Nutrition RCT Tests/79/none G. Net Present Value (NPV) Calculation and NPV Analysis DevPacks support the NPV calculators and analyzers documented in the NPV Calculation and Analysis references. The images in Section A show the type of Treatments used to conduct this analysis. The following image demonstrates one potential way to tie scientific theory to concrete evidence The following image displays the result of running a NPV Totals Analysis for the parent DevPack (Treatment 12 new) holding the budgets displayed in the previous image. This data is owned by the Iowa Corn and Soybean club in AgTreks. Only Treatment 12 new in the following dataset (the machinery analysis reflects this deprecated budget structure) http://localhost:5000/agtreks/preview/cropsconservation/devpack/Iowa, Nashua Conservation Tillage 108 Corn and Soybean Budget data set/117/none H. NPV Calculation and Life Cycle Analysis (LCA) DevPacks support the LCA calculators and analyzers documented in the LCA Calculation and Analysis references. The following image emphasizes that the purpose for most CTA analysis is to increase performance and to conserve scarce resources. The following image displays the result of running a LCA Capital Budget Totals Analysis from a DevPacks data service. This data is owned by the Reconstruction Science club in BuildTreks. http://localhost:5000/buildtreks/preview/commercial/devpack/Treatment 01/309/none I. NPV Calculation and Capital Input Analysis DevPacks support the Capital Input calculators and analyzers documented in the Capital Input Calculation and Analysis references. The following image stresses the importance of using multimedia to communicate the results of complex calculations to target audiences. This data is owned by the Iowa Corn and Soybean Science club in AgTreks. Only Treatment 12 new in the following dataset (the machinery analysis reflects this deprecated budget structure) http://localhost:5000/agtreks/preview/cropsconservation/devpack/Iowa, Nashua Conservation Tillage 108 Corn and Soybean Budget data set/117/none The following image displays the result of running a Capital Input Operating Budget Totals Analysis from a DevPacks data service. J. Conservation Technology Assessment (CTA) The health care sector uses Health Technology Assessments, or HTAs, to determine the worth of health care technologies. These assessments often involve the meta-analysis of randomized control trial data. HTAs require large amounts of professional staff time to complete –largely because of the need to find and interpret the data. The Resource Stock Analysis tutorial explains that DevTreks uses the complementary term, Conservation Technology Assessment, or CTAs. This reference demonstrates one way to store CTA data and run basic analyses. For example, by using the standard calculator pattern: Indicator metadata – TEXT datasets – custom algorithm – mathematical library pattern (i.e. the pattern employed by many types of Health Technology Assessments). One reasonably trained professional club is capable of collecting and managing the data and running the CTA. Future releases will expand the types of analyses that can be run, eventually supporting full scale CTAs. The following image derived from DevPacks RCT data documented in the References and demonstrates a typical CTA along with full multimedia support for analytic content. The original dataset used to conduct this analysis can be found at: https://www.devtreks.org/agtreks/select/cropsconservation/devpack/Iowa, ARS-NRCS 2, Treatments 1 through 35, Full Set/80/none/ The following list demonstrate the steps taken to produce this data set include (further information about this data set can be found in the References (3*)). A current example can be found in the Resource Stock Analysis reference. 1. Use standard base elements and calculators to build a base data set. In this example, the base data set included 108 crop budgets that documented the basic technology employed in each plot over 14 years (12 plots x 9 base budgets = 108). Note that these crop budgets were built with a precursor to DevTreks –they are not consistent with the current structure of operating budgets and cannot be used for current analysis. 2. Copy an appropriate base document into a corresponding RCT treatment. Make copies for each observation in the treatment. Use standard file system folders for this purpose. 3. Edit each observation according to the actual data results (4*). The data for each observation often matched more than 95% of the data found in the base document data. The remaining 5% required changing a small number of input and output quantities, such as fertilizer amounts, corn yield, dates, or nitrate emissions. 4. Build a hierarchical DevPacks data set that corresponds to the RCT data structure (see the URL for examples). The sample data set required 36 DevPack Treatments and 504 DevPackPart budgets. 5. Upload an edited observation document into a corresponding DevPackPart. Add one or more ResourcePacks to each part containing any necessary multimedia support, such as an image that can enhance the display of previewed parts. 6. Link each DevPackPart to an NPV calculator needed to carry out new calculations. Run and save each calculated result. This example required linking NPV operating budget calculators to the 504 DevPackParts. 7. Link appropriate analyzers to Treatments, or DevPacks base elements. Run and save the analyses. In this example, NPV Statistical Analyzers were run for the parent treatment that held the 36 individual treatments. A total of 504 budgets were analyzed. 8. Use the Pack panel to download the analytic results. Upload them into statistical analysis packages and run any statistics needed for the RCT analysis. Produce graphics and tables summarizing the results. 9. Add the graphics and tables to a multimedia, or Resources, data service. Use the Linked Views data service to link the multimedia to a story. Link the story to the root element in the DevPacks data set. K. Sustainable Supply Chain Analysis (Production and Consumption Analysis) DevPacks support the sustainable supply chain analyses introduced in the Performance and Social Performance Analysis (SPA) tutorials. Modern decision support software must help producers to make sustainable production decisions and consumers to make sustainable consumption choices. Many consumers do not want their money going to private sector companies and their executives who undermine their value systems. Many producers recognize the business opportunities this opens up to sustainable production practitioners. The SPA techniques included Life Cycle Impact Analysis, Product Life Cycle Analysis, Organization Life Cycle Analysis, Social Life Cycle Analysis, Life Cycle Cost and Benefit Budgeting, Incremental Cost Effectiveness Analysis, and custom algorithms related to population impact states, Performance Monitoring, Impact Evaluation, and Machine Learning. The following example employs the dataset introduced in Example 6 of the Social Performance Analysis 3 reference. This example carries out a custom Disaster Risk Reduction Totals Analysis of 2 devpackparts, each of which represents 1 stage of a 2 part disaster risk reduction chain (i.e. interpreted anyway you choose). Each base document employed 7 Indicators with data stored in Indicator.URL and MathResult.URL. Supply Chain RCA Value DevPack in: https://www.devtreks.org/greentreks/select/carbon/devpackgroup/RCT Emissions and Env Performance/48/none http://localhost:5000/greentreks/preview/carbon/devpackgroup/Carbon Budgeting DevPack Group/43/none L. Multimedia and Stories DevPacks contain the most complex data and analyses. Explanatory stories explaining the data is particularly important. M. Knowledge Bank Standards Most DevPack data should be entered into online knowledge banks (i.e. production servers as contrasted to development servers) that can be used to analyze the data. That structured evidence must be passed down to future generations. These knowledge banks aggregate and analyze all of the data in a network. Future references will discuss how these knowledge banks will evolve (i.e. semantic data, forecasts) to support future decision making needs. The flexibility offered by DevTreks in managing DevPack data means that networks need to develop “rules” explaining the “standards” that should be followed by clubs in their network. The “standards” make it possible to build knowledge banks. Summary and Conclusions Clubs using DevTreks can start to carry out the basic analysis of Conservation Technology Assessment (CTA) data. Clubs can solicit help understanding this data better and share structured evidence explaining the best technologies identified by CTAs. Networks can build knowledge banks of CTA data and pass that knowledge down to future generations. The result may be Indian smallholders who use CTAs to adapt effectively to changing monsoons, Somalian health care administrators who routinely carry out CTAs in support of efficient health care delivery, NYC city managers who use CTAs to mitigate rising sea levels more efficiently, and people who can make decisions that improve their lives and livelihoods in sustainable ways. Footnotes 1. DevPacks Analyses are somewhat harder to carry out than standard analyses. Consider using them when resources are particularly scarce and money really does need to be saved. A casual glance at newspaper stories suggest this circumstance may not happen very often in some countries –in many cases, accountability for budget expenditures is mostly a talking point. 2. Arbitrary hierarchies of structured xml data support a wide assortment of content management services, such as the Randomized Control Trial data used in advanced Conservation Technology Assessments (CTAs). RCT data is especially feasible when the structured xml data holds metadata analysis of TEXT Data URL datasets, such as those demonstrated in the CTAs, CTAP, and M&E references. Arbitrary structures of hierarchical URLs is also powerful and supported through the Resource Stock calculators and analyzers. Precursors to the current DevPacks data services were first built and tested more than a dozen years ago, but were not fully rebuilt and documented until Version 1.7.0. 3. The accompanying reference, Using Social Budgeting Web Software …, used a 504 RCT budget set for its analysis. The current version used much smaller data sets for testing. Full scale testing using larger data sets will be carried out when a particular need arises (such as the receipt of funds to pay for the analysis). In addition, this version only tested DevPacks holding Operating Budgets and Capital Budgets –the remaining base elements –Inputs, Outputs, Operations, Components, and Outcomes- must still be tested. 4. The precursor to DevTreks allowed the edits to be completed using the software, but 1.7.0 requires that the edits be completed in some other text editor before being uploaded. References K. P. Boyle, P. Heilman, R. W. Malone, L. Ma, and R. S. Kanwar. Using Social Budgeting Web Software and Natural Resources Software Models to Improve Agricultural Economics Data Collection, Dissemination, and Analysis. DevTreks Working Paper 01. January, 2012. (the pdf file can be found in this tutorial) Improvements, Errors, and New Features Please notify DevTreks (devtrekkers@gmail.com) if you find errors or can recommend improvements. Video tutorials explaining this reference can be found at: https://www.devtreks.org/commontreks/preview/commons/resourcepack/DevPacks Analysis/1535/none/ DevTreks –social budgeting that improves lives and livelihoods 43