Calculators and Analyzers Last Updated: July 31, 2018; First Released: 2014 Author: Kevin Boyle, President, DevTreks Version: 2.1.4 (1*) A. Introduction This reference explains how calculators and analyzers work. All calculators and analyzers are software modules known as Linked Views. These Linked Views must be linked to base elements (Inputs, Outputs, Operations, Components, Outcomes, Operating Budgets, Capital Budgets, Multimedia Resources) before calculations can be run. Linked Views are necessary because base elements can be calculated, analyzed, and manipulated in thousands of different ways. Section Page Calculators 2 General Calculator Use 8 Linked Views 10 Relations 12 Analyzers 15 Calculator or Analyzer Results 19 Service Agreement Subscriptions 20 Calculator or Analyzer State Management 21 Calculator or Analyzer Quality and Performance 22 Packages 23 Custom Analysis 25 Summary and Conclusion 26 B. Calculators The following image shows that calculators are run from the Views panel in DevTreks: The following image displays a typical analyzer: These images demonstrate that calculators and analyzers include the following user interface elements: Edit Linked Views: Opens a list of Linked Views that can be used to add, edit, or delete the Linked Views that are linked to a base element. The Linked Views section, below, explains this further. Make base: Pulls fresh data out the database and puts into a base XML document that can be used to carry out fresh calculations. This element must be used any time the data being calculated or analyzed has been changed. Early versions of DevTreks did not have this step –instead they automatically rebuilt a new document every time a calculator was opened. That proved too slow for most work. In general, this command should always be the first action taken when running calculations. Not running this command means that a lot of data may be meaninglessly entered in calculators only to find that the data cannot be saved (because no base document exists for saving it in). Media View (2*): Displays HTML multimedia views of calculations such as images, graphs, maps, or links to pdf files. This is the default view that loads from the Preview panel. The multimedia derives from the Media URL property which can be set in all calculators and analyzers. As shown in the following image, the URL to the multimedia should be copied from the Download Resource buttons found on the Preview panel when previewing Media content, such as: https://devtreks1.blob.core.windows.net/resources/network_crops/resourcepack_462/resource_7910/iowa-usa-corn-soybean-crop.jpg. The media should communicate the results of analytic content to decision makers. Networks should provide guidance to their clubs about the best communication aids to use. Multiple URLs can be entered by using a semi-colon string delimiter (“;”) between the URLs, as follows: Although the combined URLs can be up to 500 characters long, keep in mind that mobile devices, in particular, will load content faster by using smaller images and media that must be downloaded to view (i.e. pdf and videos). The name displayed with the URL is the filename of the URL. Multimedia should be named using standard search engine recommendations (i.e. iowa-usa-corn-soybean-crop.png). The following image is a typical example. Mobile View: Displays an HTML view of calculations that can be read in mobile, and touch-screen, devices. Generally displays data vertically. Desktop View: Displays an HTML view of calculations that can be viewed in non-mobile, and non-touch screen, devices. Generally displays data horizontally. Dataset: Links to a raw XML dataset containing the results of the calculation or analysis. This link is an early effort at making all raw data accessible to machines. The Package section, below, explains that all of the data associated with this IRI can downloaded by humans in HTML, TEXT, and XML formats. Version 1.7.6 began using additional ways to store raw data (i.e. Data URLs). Get Story Page: Opens a page from a story from the top drop down list. Get Calculator or Analyzer: Opens a calculator or analyzer from the bottom drop down list. Menu: Each menu item displays a section of the calculator or analyzer. The Introduction introduces the calculator. The Steps are used to run the calculator. The Help item provides an explanation of how each step in the calculator works. Data URL: Version 1.7.6 introduced a Data URL property that some calculators use to store a TEXT file holding data used by selected calculators. For example, calculators that use indicators might contain thousands of observations for each indicator. Rather that require manual entry of each observation, the TEXT file holds the data in a specific structure defined by each calculator. The TEXT file is uploaded to a base Resource element. As with Media URLs, the URL to the text data file can be copied from the Download Resource buttons. This data management technique can address the need to use large datasets, while still displaying reasonably sized html views of that data. The html data displayed becomes both the meta-data describing the related TEXT data and the results of running calculations on the dataset. The Resource Stock tutorial gives an example of using this property. Multiple URLs can be entered by using a semi-colon string delimiter (“;”) between the URLs, C. General Calculator Use Each step in a calculator can affect subsequent steps. For example, selecting machinery constants in Step 1 will insert default values for speed and width into step 3. When running calculators for the first time, this behavior is desirable. But when updating older calculations this can have undesired affects. In many cases, you may not want to have all of the parameters in subsequent steps replaced, especially when those parameters were customized for specific conditions. Before updating older calculations make sure you have a backup of the original calculations and double check the new calculations before saving them. When numbers change dramatically during updates, the cause often involves unintended, default, parameter changes. A lot of time and expense can be saved by running calculators at a ‘parent’ level, and selecting Relationships that automatically update descendants. For example, in most instances, a parent input calculator can be run that automatically inserts or updates calculations in children input series. This can go askew in two main ways. First, when a ‘base document’ is not first built before updating children, the document being calculated does not automatically check for changes in descendants before running a new calculation. If new children were inserted, or new children calculations saved, the base document may be out of date. Symptoms may include children that don’t have calculations automatically inserted into them, or children that have multiple, unnecessary, calculators inserted. Always run the ‘base document’ before updating any calculator or analyzer. This action puts fresh database data, containing all of the latest children with their calculators, into the base document being calculated. The second trouble can come from changing Relationship properties. The Relationship properties tell the parent which children calculations to update. If the ‘Related Calculator Type’ is changed, the parent will look for calculators with the same Relationship property in the children. When a relationship is not found, a new calculator or analyzer will be inserted into the children. Always check children calculations after running parent calculations to make sure that inadvertent insertions are not taking place. Unless you fully understand Relationships, we recommend not changing the default Relationship properties. Finally, DevTreks may not generate new HTML children views when no changes are detected for any calculation. When testing DevTreks for the first time, this behavior will become apparent. Children HTML views can be generated by making a minor change to the description field of the calculator, followed immediately by running the calculation. In general, Analyzers can’t be automatically inserted into children base elements. In general, analyses are usually documented using 1 primary base element (i.e. Input, Output, Component Group, Budget Group). D. Linked Views Opens the following form that can be used to add, edit, or delete the Linked Views that are linked to a base element: These user interface elements work as follows: Select is used to add a new Linked View to this list. View URI is used to open the base calculator or analyzer (i.e. to find out more about it). Views is used to open the Views panel and put the list of available calculators in a drop down list from which that can be opened. This element also opens specific calculators. Add Default AddIn is used to add whichever default Linked View has been defined by this club. This can be very helpful when a large number of base elements are being linked to the same calculator. E. Relations The following image displays the typical Relations properties found in Step 2 or 3 of a typical calculator: The Relations properties include: Use in Descendants: True to insert or update this same calculator in children. Set this property to false when children calculators should not be changed. Set this property to true and set Overwrite Descendants to false to update children calculators without overwriting them with the current calculator. OverWrite Descendants: Overwrites the children linked views with the parent’s linked view. The combinations of these two properties act as follows: Use in Descendants = true and Overwrite Descendants = true: Always updates the parent linked view in the database. If the children don’t have the parent’s linked view, the linked view will be inserted into the children and the children database table. If the children have this type of linked view, the linked view will be updated in the children and in the children database table; Use in Descendants = false and Overwrite Descendants = false: Always updates the parent linked view in the database. Makes no changes to the children linked views. Some analyzers, such as the Resource Stock analyzers, rerun input and output calculations for every analysis. These analyzers will show changed calculations in their descendants, but the children database table will not be updated with the changes. Use in Descendants = true and Overwrite Descendants = false: Always updates the parent linked view in the database. Runs new calculations for the children, but makes no database changes for the children. Some analyzers, such as the Resource Stock analyzers, rerun input and output calculations for every analysis. These analyzers will show changed calculations in their descendants, but the children database table will not be updated with the changes. Use in Descendants = false and Overwrite Descendants = true: Reserved for potential future use. When a new set of calculators or analyzers are inserted for the first time into children elements, the “base document” used to run calculations and analyses must be updated and new calculations run and saved. It must be updated anytime a change occurs in the database. In this case, new linked views have been inserted into a children database table. After saving the new calculations, and adding the new linked views to the base document, these two properties will update, rather than insert, the children linked views. If this action is not taken, but both of these properties are set to true, unneeded new linked views will continue to be inserted into the children database table, rather than being updated. When updating children calculators make sure to make at least one edit in the parent calculator, such as a slight change to a description. Updates won’t occur if no changes are detected in the calculators. What If Tag Name: Do not use yet. Related Calculator Type: Name of a related calculator in descendants. Aggregates and analyzes the data found in those related calculators. We recommend accepting the default value found here. Alternative Type: Used in Change by Alternative Type Analyzers to identify distinct alternatives that will be compared with one another. Target Type: Used in Progress Analyzers to identify benchmark, partial target, full target, and actual, base element states. F. Analyzers With the exception of Inputs and Outputs, DevTreks usually requires running Net Present Value (NPV) base calculators before running analyses. The NPV calculators include the Make Base command button for pulling fresh database data together. They also provide basic benefit and cost data which is a primary objective in Social Budgeting. The following image displays Step 1 of a typical analyzer: Step 1’s properties work as follows: Base Calculations to Analyze (3*): This is the name of base elements being analyzed. These base elements come from the calculator specified by this property. This base calculator must be run prior to running an analyzer. DevTreks usually requires the Net Present Value base calculators to be run before running analyses. The totals found in the NPV calculators are usually needed by the analyzer. In order to reduce duplicate code and maintenance, one analyzer can be shared by multiple elements (i.e. Operations and Components, Operating Budgets and Capital Budgets). Analysis Type: These are the types of analysis that can be carried out by this analyzer. As mentioned in the previous paragraph, less maintenance is needed when one analyzer can carry out multiple roles. Although any analysis can be run from any analyzer, the Linked View holding the Analyzer will specify the type of analysis to run -a Totals Analyzer should be used to save summations, a Stats Analyzer should be used to save statistics, and so on. Matching the correct Analysis Type and Analyzer is especially important when children analyzers are being inserted or updated. The following image displays Step 2 of a typical analyzer. Depending on the complexity of analytic results, some analyzers may not contain all 3 of the aggregation options. The last image shows that base elements can be aggregated using several options: Compare Using: Both the None and Compare Only options carry out the same type of analysis, but the Compare Only option will display the results using logical comparisons, such as lining up budgets side by side. Comparisons are only appropriate when for “Change by” and Progress Analyzers. Aggregate Using: The Labels option will aggregate data together using Work Breakdown Structure (WBS) Labels. This usually results in the most detailed aggregation. All economic content in DevTreks is stored using a table named Types at the top level of the data hierarchy (i.e. Operation Type) and a table named Group at the next level of the hierarchy (Operation Group). The Type option aggregates data using the Type’s table Id property. This usually results in the least detailed aggregation. The Group option aggregates data using the Group’s table Id property. This usually results in data that is aggregated somewhere in between the Labels option and the Types option. Only the Labels option can be used to aggregate data coming from different clubs (that’s why WBSs are so important in networks). Display Full View: These options determine the HTML view of the data that will be saved for future display. Yes means display all elements in the analysis. No means do not display all of the elements in the analysis. The No option generally shows only the children of the current element being used to run the analysis. The advantage of the No option is fast loading and less data to interpret. The Yes option should be used when all of the data in an analysis must be displayed. G. Calculator or Analyzer Results The results of running a calculator or analyzer appears directly below it: These results are often automatically “compacted” by DevTreks so that they load faster and scale better (unless a Display Full View property can be set). That means immediately after saving the results of a calculator or analyzer, only the children, and sometimes grandchildren, elements in the analysis are saved and displayed. For example, a Budget Group analysis of 50 crop budgets will only display the Budget and Time Period elements of the analysis. The Outcomes, Operations, Outputs, and Inputs are not displayed because the resultant HTML is just too large for standard Internet use. You can view missing elements by drilling down into those descendants. If all elements need to be displayed, a Totals analysis, such as an NPV Totals analysis, displays all base elements. H. Service Agreement Subscriptions Calculators and analyzers are usually built by one club and added to that club’s service agreement in one particular network. Some of these calculators and analyzers can be difficult to find. DevTreks discourages adding the same set of calculators or analyzers to more than one club or network because it complicates code maintenance. Instead, each club’s service agreement supports subscriptions to the services offered by other clubs. The following image shows that each club’s service agreement includes a “Select Existing Service” feature that allows clubs to subscribe to other clubs’ services, including their calculators and analyzers. That enables a club to quickly find only the calculators and analyzers they need. DevTreks believes that paying clubs for these services will provide well-earned incentives for high quality services. I. Calculator and Analyzer State Management The results of calculations and analyses are stored in both raw XML and HTML formats in file/blob storage. The raw data can be downloaded using Packages (explained below) or using the hyperlink (Corn Soybean Rotation IRI) displayed at the bottom of the Views panel. Prior to saving any result, all older calculated files are deleted. If children linked views are also being saved, the children HTML files associated with the linked view are deleted. They need to be regenerated by opening the child and viewing both the Mobile and Desktop Views. J. Calculator and Analyzer Quality and Performance During the debug of Version 2.0.0’s Azure storage, files in the debug version of the storage were deleted. Basically to find out what happens. The NPV and Resource Stock stylesheet files that were deleted caused the calculators and analyzers tested from the previous day to stop working. During the hunt to find out what happened, an important point about the performance of calculators and analyzers was highlighted. Calculators and analyzers evolve over time. Discoveries are made that lead later stage calculators and analyzers to be built differently than earlier stage ones –often because of improvements in performance or programming. Even though a calculator or analyzer works, that doesn’t mean they have been built for optimal speed and performance or that their architecture is perfect. Leading to the point –serious IT needs serious teams of IT professionals. That may be why most software development seems to be carried out by companies with a serious commercial focus. The nonprofit, public goods, model of software development may be personally satisfying to a software developer or an IT team. It may also be satisfying to a particular group of customers. That doesn’t mean it results in the best software. Ngos need to experiment with business models that can support serious public goods software. An example is the subscription-based services introduced in the Containers reference found in the Source Code tutorial. Until those business models are discovered, expectations need to be tempered about the quality and performance of their software products –including this product. K. Packages All analytic results can be downloaded and used on local clients. Two steps are used to package data: Step 1. Click on the PackIt command button found at the top of the Views and Edit panels. See the previous images of these 2 Views. The following form opens. Choose options for packaging and downloading data. Click on the Make Package command option. Multiple selections can be included in the package. The TEXT data option generates comma-separated-value files that can be imported into statistical analysis programs, such as R. The row/column nature of csv files means that they need to be edited prior to using them in statistical packages. Although all calculator properties are included in the TEXT files, the vast majority of columns are not needed for most subsequent analysis. Although the csv data appears in row column format, many columns of data are blank because they document descendant base element properties that are not relevant to ancestors. It’s pretty apparent that the TEXT data conversion needs to evolve. Step 2. Download the package by clicking on the “Click to download package” link. The package will be downloaded into wherever you store downloads (i.e. the Downloads folder). L. Custom Analysis This reference explains how to calculate and analyze base element resource stock data. The structure of base element data will not support many types of analyses, such as Conservation Technology Assessment (CTAs) that use randomized control trial data. Three options for custom analysis can be used to overcome this shortfall: 1. DevPacks: The DevPacks tutorial introduces base elements that allow analysis of hierarchical structures of data, including randomized control data. 2. Data URLs: As explained above, Data URL properties of calculators and analyzers allow custom datasets to be linked directly to calculators and analyzers. The Resource Stock Analysis and Technology Assessment 1 and 2 tutorials explain how to use this property. 3. Packages: This reference and the Performance Analysis references explain that datasets can be packaged, downloaded as zip files, cleaned up, imported into statistical packages, and further analyzed. Summary and Conclusions The professional calculation, analysis, explanation, and storage of economic, technologic, and performance data can help people to improve their lives and livelihoods. Footnotes 1. Version 2.1.4 has been refactored using Microsoft’s ASP.NET CORE 2 technologies. Details about the refactor can be found in the Containers reference in the Source Code tutorial. In terms of this reference, the refactor changed how html files downloaded in packages work. Html files generated prior to Version 2.0.0 used different paths and naming conventions for css and js files. Older html files will not have any css styling, and will appear as black and white html. In the case of some analytic content, the black and white content appears better than the css styles, mostly because the server processes css differently than the downloaded files. That means that css styles need to be tested as both server-based html and stand-alone html. 2. Videos should not be included in Medial URLs because some browsers begin downloading videos when the links are displayed. If a URL to a video is included in the Media URL property, an error message will be generated in the Media View panel. 3. As of version 1.6.0, the authors of Calculators and Analyzers must set a LinkedView.FileExtensionType property when adding a new Linked View for the first time. Analyzers can find base calculations using the base calculator’s FileExtensionType property (as well as the Related Calculator type property). On a related note, the Story Telling tutorial demonstrates how to maintain Linked Views, which, besides calculators and analyzers, includes stories. DevTreks recognizes the need to provide more thorough explanation of how software developers can build new Calculators and Analyzers. It’s on the To-Do list. References No references have been used with this tutorial. Improvements, Errors, and New Features Please notify DevTreks (devtrekkers@gmail.com) if you find errors or can recommend improvements. A video tutorial explaining this reference can be found at: https://www.devtreks.org/commontreks/preview/commons/resourcepack/Calculators and Analyzers/515/none/ DevTreks –social budgeting that improves lives and livelihoods 1