Social Performance Analysis 2
Last Updated: March 26, 2020; First Released: May 12, 2017
Author: Kevin Boyle, President, DevTreks (1*)	
Version: DevTreks 2.2.0
Introduction

This Appendix uses online datasets to explain how to carry out Social Performance, or Quality of Life, Assessments. The examples also demonstrate how to use communication aids, such as graphs and tables, to explain the results of analyses. The examples and their datasets are used to illustrate social performance analysis and should not be interpreted differently. [Because, to date, complete empirical datasets that can support the RCA Framework, or the 4 underlying frameworks, have not been found.]

Although many of these examples focus on the agricultural sector (i.e. because that’s the author’s area of expertise), these algorithms can be used in any industry. The Hyati (2017) reference found in Example 1 provides historical context for measuring agricultural sustainability.

Example
Page
* 1. Coffee Company Social Performance Trends Score (RCA2)
2
* 1A. Coffee Company Trends SAFA Score (RCA2)
55
* 2. Coffee Company PRA Social Performance Score (RCA1)
68
* 3. Product Life Cycle Impact Assessments (P-LCIA) for Representative Small Scale Coffee Farms (RCA3)
82
* 3A. Organization Life Cycle Impact Assessments (O-LCIA) for Representative Small Scale Coffee Farms 
125
* 3B. Social Life Cycle Assessment (S-LCA) for Representative Coffee Production Stakeholders 
151
* 4. Life Cycle Costs, or Benefits, (LCC or LCB) for Representative Small Scale Coffee Farms (RCA4)
172
* 4A. Coffee Farm Compliance Cost Effectiveness Analysis (CEA) (RCA5)
195
* 4B. Generalized Cost Effectiveness Analysis (GCEA) with Quality Adjusted Stock Years (QASYs) (RCA5)
224
* 4C. GCEA and LCIA (RCA5)
275

All of the algorithms in this reference were tested using the upgraded Version 2.1.6 calculator patterns.
A video tutorial explaining this reference can be found at:	
The Performance Analysis tutorial on the DevTreks home page.


Example 1. Coffee Company Social Performance Trends Score (RCA2)

URLs: 

https://www.devtreks.org/greentreks/preview/carbon/resourcepack/RCA Images/1549/none
https://www.devtreks.org/greentreks/preview/carbon/resourcepack/Coffee Firm RCA Example 1/1550/none
http://localhost:5000/greentreks/preview/carbon/resourcepack/Coffee Firm RCA Example 1/538/none
Resource Stock Assessment
https://www.devtreks.org/greentreks/preview/carbon/output/Coffee Firm RCA2 Stock/2141223476/none
http://localhost:5000/greentreks/preview/carbon/output/Coffee Firm RCA2 Stock/2141223482/none
Monitoring and Evaluation Assessment
https://www.devtreks.org/greentreks/preview/carbon/output/Coffee Firm RCA2 MandE/2141223477/none
http://localhost:5000/greentreks/preview/carbon/output/Coffee Firm RCA2 MandE/2141223485/none

A. Introduction

Several U.S. states allow corporations to register for public benefit purposes. Although these corporations are still judged on their profitability, customers and investors are also interested in proof of the social benefits they generate. This example illustrates how public benefit corporations, such as socially sound farms, can supply this proof.  This proof is usually a requirement for companies to obtain certification of compliance with third party standards, such as organic farming standards. The example also illustrates how private companies can assess the business continuity risks associated with probable future scenarios, such as a 1.5 degree temperature increase. The example uses the RCA2 algorithm, Resource Conservation Value Accounting Trends, together with data that has been abstracted from the Natural Capital Protocol coffee farm example (NCC, 2016). No actual on-farm, raw, data was found (10*). 

Although focused on coffee production, where more than 25% of production is certified, these types of “proofs of social soundness” are becoming increasingly used throughout the agricultural sector. Lernoud et al. (2017) describe this trend as follows:

“… sustainable commodities, as defined by products that are demonstrably (e.g. third-party verified) compliant with internationally recognized standards, are growing rapidly, and at a pace that far outstrips markets for conventional commodities. Highlights of the current market context are continued exceptional growth, expanding coverage of agricultural land, and dominance in some sectors of single-sector standards”

The following image (Antonopoulos et al, 2016) introduces an “instruction manual” that can serve as a training manual for this example. The authors demonstrate using best environmental management practices (BEMPs) and indicators, benchmark levels of excellence, strategic farm management plans, accreditation schemes, algorithms (i.e. that calculate N, P, and K cycles), scoring systems, and ecosystem services metrics, to assess “measurable resource and environmental efficiency”, or to conduct Social Performance Analysis, for farms and ranches. 

In the context of this algorithm, each BEMP (i.e. Strategic farm management plan) shown in the image’s table, corresponds to either a Categorical or Locational Index. The “Key environmental performance indicators” correspond to the Indicators used to assess the BEMP. Example 4’s crop and household budgets and Example 5’s Stakeholder Impact Assessments can be used to supply the demographic, geographical, and economic, variables needed to more fully understand social performance. FAO SAFA’s (2013) instructions for distinguishing performance, practice, and target indicators are particularly relevant with these indicators.



The following image of a small-scale agricultural WBS (Sustainable Food Lab, 2016) demonstrates that, for developing countries, the COSA (2014) and Sustainable Food Lab references, and those organizations’ related publications, can serve as primary instruction manuals. The reason that some capitals, notably natural capital, receive short shrift in this WBS is that the primary goal of many agricultural development schemes in developing countries is to lift producers out of poverty and to achieve national goals for food security. In addition, data scarcity forces the use of proxy, or SAFA’s practice, indicators. Asten et al (2015) provide a good example of a BEMP for coffee production in developing countries that addresses both climate change and improved farm income.


The following image of a general purpose sustainable agricultural WBS (FAO SAFA, 2013)) demonstrates the use of 4, rather than 7, capitals to conduct social performance assessment. This publication, and similar agricultural sustainability WBSs (see Hayati, 2017), can also serve as primary instruction manuals for applying this tutorial. The FAO and Hyati references use the terms “dimensions”, “themes” and “subthemes” for its hierarchical Indicators in a similar manner to this algorithm’s Total Risk, Locational, and Categorical, Indexes. In the context of the RCA Framework, Institutional Capital = Good Governance, Natural Resources and Physical Capital = Environmental Integrity, Economic Capital = Economic Integrity, and Human, Social, and Cultural Capital = Social Well Being. 


Keep in mind that most of the existing financial reporting systems introduced in SPA1 are grounded in “impact pathways”, “causal chains", or “results chains”, based on the principles of social performance assessment. Even if the 7 capital stocks are not overtly elements of a Social Performance Assessment’s Indicator system, they must still be implicitly addressed in the alternative Indicator systems, such as SAFA’s, COSA’s, or EU’s. Mbowa et al (2014) present an example for coffee production that uses the Sustainable Livelihoods Framework (SLF), which “emphasizes access to or ownership of livelihood assets – (i.e. human capital; social capital; [natural capital]; and physical capital) that are key in influencing livelihood strategies.” It’s worth mentioning that the SLF can be modeled using this algorithm: Indicator 1. Vulnerability Factors, Indicator 2. Livelihood Assets, Indicator 3. Livelihood Strategies, and Indicator 4. Livelihood Outcomes (i.e. the SPA3 reference further demonstrates the use of SLF).

UNFSS (2016 in SPA1) points out that more than 400 sustainability standards systems currently exist. Although it’s likely that they use similar Indicators in each standard, their independent systems lead to problems collecting, aggregating, understanding, advancing the science, communicating, and achieving results with social performance data. UNFSS (2016) states that this lack of mainstreamed standards can undermine the fundamental credibility of standards systems. Appendix A’s concluding advice that firms and public entities work in tandem and use mainstream, generic, open source, IT platforms and standards, applies. Several examples in this tutorial will revisit this issue.

B. Indicator Thresholds 

An objective, science-based, social network (i.e. Coffee Resource Conservation Value Accounting Network) has developed an Indicator Threshold system that their clubs can follow when developing resource conservation accounting and financial reports. The network started by prioritizing risks faced by their industry and communities. They then built Indicator Threshold Systems by using Indicators from established systems, such as the SDG, Sendai DRR, FAO, IPBES, COSA, EMAS, ISO, and IPCC, systems. Their goal is to be able to use the same Indicator WBS to support the full industrial sector of their countries, with supplements appropriate for specific industries, such as agriculture. The industrial sector is defined as all product supply chain participants, from producer to consumer, in multiple economic sectors. Several of the standards systems introduced in SPA1, such as SDG, EMAS, SASB, and GSSB, are examples.

The following image (FAO SAFA Guidelines, 2013) demonstrates the use of an agricultural Indicator Threshold system applicable to agricultural supply chains. FAO defines the upper and lower thresholds (FAO SAFA Indicators, 2013) while leaving the 3 remaining thresholds to be defined by the social network. The latter publication provides guidance for each separate Indicator, including descriptions, relevance, units of measurement, collection methods, limitations, and further sources of information.


The network’s clubs are engaged in coffee growing, processing, and advising. Resource conservationists, working with this case study firm’s managers, have used the tables displayed throughout this Example to customize the network’s Thresholds for the needs of the company and its associated stakeholders (i.e. SAFA contextualization). The first four factors are based on the NCC (2016) coffee firm example. Three of these target natural capital stocks and the fourth targets physical capital stocks. They included the physical capital stock risk, Flood Control, for their business continuity plan. 

The fifth factor, Employee Management, has been added to reemphasize the importance of accounting for all 7 capitals. The following image (Global Coffee Platform, 2016) demonstrates how a similar socioeconomic indicator, with thresholds, is currently being used in the coffee industry. The Global Coffee Platform includes a WBS with Indicator Thresholds for 8 economic principles and performance indicators, 9 social principles and performance indicators, and 10 environmental principles and performance indicators. Their stated objective “is that, over time, all coffee producers around the world, and therefore all coffee production, will achieve a baseline level of social, environmental and economic sustainability.” Although focused on the coffee industry, most of their principles are broadly applicable to the agricultural sector. [The NCC 2016 coffee example was believed to have more complete decision support data and therefore used in this case study.]



The following image (Fairtrade International, 2011) demonstrates a similar socioeconomic standard developed by the Fairtrade Labelling Organization. These types of certification organizations require producers, producer organizations, and supply chain participants, to comply with these types of standards in order to label their products as “socially sound”. The image also demonstrates the use of performance targets that can be modeled using this algorithm’s Trends.

The following image (IITA and COSA, 2016) illustrates how other international Voluntary Sustainability Standards (VSS) groups use both Indicators and Indexes to measure social impacts in the small-scale agricultural sector. The authors based this Indicator system on the “Theory of Change” system developed by the Fair Trade Labelling Organization (2015). The M&E tutorials, and Gertler et al (2016), explain their Theory of Change, or “results chain”, of Inputs->Activities->Outputs->Outcomes->Impacts. In the context of the RCA Framework’s 4 or 5 level hierarchy of indicators, these indexes coincide, loosely, with either Locational or Categorical Indexes. 



The following images shows that Indicator Thresholds have been defined separately for Actions, Conditions, Services, and Impacts. These Indicators, which are added to separate Indicator.URL datasets, define the social impact pathway used in this example. Other social networks may prefer using other systems of Indicators such as “results chains”, “exposure pathways”, “causal chains”, or “disaster impact pathways”. Care, combined with a thorough understanding of the background science, must be used when developing these Thresholds. The Global Coffee Exchange image demonstrates using 3 Indicator Threshold categories. SAFA demonstrates using 5 Threshold categories. Future releases will continue exploring how to automatically link these separate Indicators (i.e. using AI techniques). 




The following statement (Loconto et al, 2014) hints at why Indicators from all 7 capitals must, at the very least, be considered when conducting social performance assessments associated with standards systems. Several capitals, especially institutional, social, and cultural, may not have received their due in past Social Performance assessments. As an example for coffee production, Mbowa et al (2014) discuss the importance of Uganda’s National Coffee Policy (NCP) and land tenure (i.e. both are institutional capital indicators) in smallholder poverty alleviation.

“The institutional contexts within which smallholders operate are important. Recent research has begun to pay attention to institutional contexts in order to understand how standards interact with pre-existing norms of production and trade. A necessary but insufficient condition for increasing smallholder participation in markets is national institutions to support compliance by farmers with standards that reflect a market demand.”

C. Quality of Life Scenarios

The company chose to use the following two scenarios for the purpose of building a business continuity plan (BCP). The plan helps them understand the primary risks that may impact the company’s performance and employees in the near future. They use the BCP and their Social Performance Score to identify needed investments in mitigation and adaptation Actions. They use Part D’s Social Performance Score to monitor and evaluate how well the Actions are actually reducing risks.

Scenario 1. Current Quality of Life or Current Social Performance
Scenario 2. Threatened Quality of Life or Threatened Social Performance
Stressors: High GHG result in 1.5 degree temperature increase with higher incidence of droughts, severe heat waves, crop and livestock production risks, air pollution, floods, and social discord. 
Targeted Stakeholder Groups: Small Scale Coffee farmers; Coffee company employees; Consumers concerned about coffee produced using socially sound principles; Managers concerned about consumers who want companies to independently comply with socially sound business practices
Mitigation and Adaptation Actions: Improvement 1 consists of a) …, b)…, and c)…. (see Antonopoulos et al, 2016)

D. Social Performance Score (1*)

The following image (SAFA, 2013) demonstrates the use of a sustainable company scoring, or rating, system that is used with this algorithm. Landert et al (2017) describe this scoring system as follows: 

“Using Multi-Criteria Analysis, the degree of goal achievement for each subtheme, using the following equation, was calculated. The degree of goal achievement [DGA] accordingly expresses, on a scale from 0 to 100%, the extent to which a subtheme’s sustainability goal was reached. This type of metric was selected to yield meaningful and comprehensible assessment results and to reduce the complexity when comparing between different assessments.

DGAi = ?Nn=1 (IMni × ISn) / ?Nn=1 (IMni) 

where N is the number of indicators per subtheme, i is the index of the subtheme, IMni is the subtheme-speci?c weight of an indicator [1–3], and ISn is the rating of an indicator (0–100%).”

RAND (2016, in SPA1) describes this same scoring system in the following statement. Subalgorithms that employ normalization, including 13 and 14, carry this out (i.e. carry out Multi-Criteria Analysis) by using the sum of a Locational Index’s Indicators, along with a normalization type = weights.

“The third column in this section is the product of the weight and score. The values of this product may then be normalized in the fourth column. This normalization involves dividing the product of the weight and score by the total sum of the weights. These normalized values may then be summed to produce an unmitigated normalized value for the risk component.”

The following image and tables show that Indicator TEXT datasets store this firm’s initial Scoring system. The Thresholds are used to assign quantitative ratings for each time trend in these tables. Resource conservationists have been trained to understand how to use this social impact pathway (or, if applicable, causal chain or results chain). A key part of that training is to understand the linkages, interactions, and cause-effect relationships, among all of these Indicators and Indexes. The author is certain that he does not fully understand these relationships, but he’s also certain that machines are getting smarter and it’s likely they’ll help out (i.e. via more advanced algorithms). Additionally, UNFSS (2016, in SPA1) points to current international efforts to gather better evidence proving the actual impacts of “sustainability reporting”, or Voluntary Sustainability Standards, towards public and private sector performance objectives. A concrete example for coffee production can be found in IITA and COSA (2016). The latter publication demonstrates using public surveys to provide scientific evidence of social performance.

This example demonstrates a 1:1 relationship among all of the Indicators and Indexes within each of the 4 base Indicators. With this technique, the same Indicator is described and scored differently depending on whether it’s being used as an Action, Condition, Service, or Impact (i.e. labels AF1A, CF1A, SF1A, and IF1A). This technique supports a sophisticated understanding of how to use social impact pathways, causal chains, and results chains, to reduce risks, but it fails to capture the reality that multiple factors interact in multiple ways. Alternatively, 1:1 relationships can be described only for the Indexes, with the Indicators themselves only being used to “explain” each Index (i.e. labels NCA, NCB, and ECA). The same Indicator can appear in more than one Index. Decision making and reporting mostly focus on the Index results (refer to the IITA and COSA, 2016, image above for an example). 

Example 7 in the Social Performance Analysis 3 (SPA3) reference begins to demonstrate using machine learning algorithms to account for multiple factors that interact in multiple ways, or SPA3’s “complex intersecting patterns”.

This example also illustrates using separate benchmark, target, and actual, Indicators to monitor and evaluate performance. In this algorithm, the targets append a 2 character suffix such as “_A” and the actuals append a 3 character suffix such as “_AA”. That’s believed to be the most transparent way to score, but there’s no requirement to use separate benchmarks, targets, and actuals. The benchmarks in this example can be replaced with actuals, and the existing targets and actuals eliminated. That scoring system is how most existing reporting systems work. In that type of reporting, the Action Indicators include company activities that serve as mitigation and adaptation improvements and the Trend scores can serve as future targets.

The trend periods in this scoring system correspond to the firm’s 20 year planning horizon, similar to the replacement life of a coffee orchard. Although useful for financial accounting purposes, that horizon may not be as appropriate for resource conservation accounting –many capital stocks take longer to improve. In general, private sector planning horizons should be aligned, or at least in accord, with public sector horizons. For example, a company’s current GHG emission levels may not have serious impacts on the firm in the next 20 years, but they will have serious consequences on society for the foreseeable future (i.e. 100+ years). Version 2.1.2 upgraded this algorithm by requiring trend periods for each separate Categorical Impact –trend dates for long term climate change data have different scales than trends for current particulate matter air pollution.

The trend periods can be defined flexibly. The Eurostat 2014 reference demonstrates that, when in doubt, linear trends may serve as appropriate “guesstimates”.  When used for short term project accounting, their simplest use may be to ignore benchmarks, and just record targets and actuals over 7 periods, with periods such as quarterly, semiannual, annual, or biannual. If more periods are needed, add sibling base elements or add additional calculators to existing base elements. 

The normalized, weighted, time trends are summed and averaged in the final Indicator.QTM, Indicator.QTL, and Indicator.QTU, properties. For that reason, the “Actuals” should use 0’s for time trend periods that haven’t occurred. Further documentations about these calculations can be found in Appendix B.

SAFA’s Accuracy Score can be replicated, reasonably, with the certainty1 and certainty2 columns in the TEXT datasets.

Version 2.1.0 upgraded algorithms 13 and 14 by using average, rather that total, scores, in the Total Risk (TR) Indexes for QTMost,  QTLow, QTUp, certainty1, and certainty2 (i.e. see the Landert 2017 images used to communicate these types of scores). Version 2.1.2 supported separate trend dates for each Categorical Index, rather than uniform trend dates for all data. Some images may still reflect earlier versions. 

The first image displays the Output Stock Calculator and the second image displays the equivalent Monitoring and Evaluation Output Calculator. The Input Stock Calculator, and the Input, Operation, Component, Outcome, Operating and Capital Budget, M&E calculators can also be used to run the algorithms. Their respective tutorials, Resource Stock Calculation and M&E Calculation, discuss the appropriate uses of the various calculators.

 

Indicator 1. Actions Meta (all scores are filled in automatically)
When the Indicator.DistType property has been set to none, each Indicator in a dataset calculates an Indicator.QTMost, Indicator.QTLow and Indicator.QTUp, based on the average, lowest, and highest respective ratings in the trend periods. When this property is set to normal or triangular, those values are generated from a Probability Distribution derived from combining the 7 trend period ratings. Additional PRA distributions are not yet supported.
 
  
 
 Indicator1.URL TEXT dataset
Benchmarks
The following image confirms that Version 2.1.2 upgraded this algorithm by moving the trend dates from the title row to each Categorical Index row. Although, for convenience, all dates are the same in this example, each Category might require substantial differences. For example, Climate Change trend dates may require 100 year projections while Air Quality trend dates may be needed annually. 

Targets

Actuals

Indicator1.MathExpression
I1.Q1.factor1 (used as a placeholder)
Indicator1.MathResult (stored using a Resource URL in the MathResult)
Benchmarks
The following image confirms that that Categorical Index rows now append their numeric results to the starting performance measurement dates. In this example, the Indicator.QTMost measures the average, normalized and weighted, rating for the 7 trend periods. Indicator.QTLow and Indicator.QTUp measure low and high estimates derived as either the lowest trend period rating, or the result of PRA calculations. Indicators that are rated 0, as shown in the 3rd row, are no longer included in the calculations for average Indicator.QTM, QTL, and QTU.
The source code should be examined to verify that normalization and weighting are carried out by vectors of Locational Index Indicators, not Categorical Indexes (i.e. because many of the categories will only have 1 or 2 children Indicators).



Targets

Actuals


Indicator 2. Conditions Meta
 
 
Indicator2.URL TEXT dataset (note what happens with the row that has a weight = 0)
Benchmarks


Targets


Actuals

Indicator2.MathExpression
I2.Q1.factor1 (used as a placeholder)
Indicator2.MathResult
Same format for results as Indicator 1

Indicator 3. Services Meta
  

Indicator3.URL TEXT dataset 
Benchmarks (spot the major mistake in this dataset)

Targets

Actuals


Indicator3.MathExpression
I3.Q1.factor1 (used as a placeholder)
Indicator3.MathResult
Same format for results as Indicator 1

Indicator4. Impacts Meta
  

Indicator4.URL TEXT dataset (note that this normalization type was not used in the final data)
Benchmarks

Targets

Actuals


Indicator4.MathExpression
I4.Q1.factor1 (used as a placeholder)
Indicator4.MathResult
Benchmarks

Targets

Actuals


Optional Indicator5. Impacts Meta

As of Version 2.1.8, all of the Social Performance Analysis algorithms can be run from all of the 15 Indicators and the Score.


Social Performance Score


Version 2.1.2 investigated the use of separate algorithms to shore up Scores. For example, by assuming that enough Social Performance Assessments have been completed to train a neural network and then using a time series regression algo to predict cause and effect – will the conservation practices cause sustainability to change, by how much, and by when? Version 2.1.4 introduces new algorithms, including machine learning, that demonstrate using the Scores to conduct Impact Evaluations.

Scores can be defined several ways: 1) actual score / target score, 2) actual score / benchmark score, 3) Indicator 4’s direct Impact scores, or 4) by custom mathematical algorithms that use Indicators 1 to 4, or even separate datasets. The reporting standard developed by this social network provides guidance to their clubs.

Example 4B demonstrates how to also define a Social Performance Score as a Quality of Life Score that can be used to assess the cost effectiveness of sustainable technologies.

This firm uses their final Performance Score to monitor and evaluate how well they are accomplishing targeted goals. The following image confirms that they chose to use Indicator 4’s Impacts directly in the Scores as well.


E. Communication 

The following image (Landert, 2017) demonstrates applying the SAFA guidelines to communicate the results of social performance assessments. This assessment compares the performance of 3 separate coffee production alternatives, Fairtrade Organic, Fairtrade, and Conventional, using assessments completed for 180 Ugandan coffee farms. 


For this example, the company uses the following types of multimedia to communicate the results of their Performance Scores to stakeholders (TR = benchmark Impact Scores, TR_A = target Impact Scores, TR_AA = actual Impact Scores).


Why are Trends in the previous image lower in the future than now? Because empirical datasets (i.e. URIs) are not yet available for these types of assessments.

F. Decisions

The goal of this company’s conservation efforts, and their overall Social Performance Assessment, is to increase the quality of life for their stakeholders. In order to do so, they must understand the interactions, linkages, and cause-effect relationships, between this ecosystem and their 5 impact and dependency pathways. Their resultant knowledge of the tradeoffs that must be made between services, mitigation actions and impacts, and stakeholder values, assists them to achieve their business performance goals.

That understanding is enhanced by the monitoring and evaluation that takes place using Part D’s scoring system. Each mitigation and adaptation improvement is scored in terms of benchmarks, targets, and actuals. They use their resultant knowledge of the efficiency and performance of their investments in Alternatives (i.e. _A to _Z), along with Adaptive Management, to reduce the risks to their stakeholders’ quality of life.

The following URL provides a graphical explanation of economic Tradeoffs.

https://www.devtreks.org/agtreks/preview/crops/resourcepack/Tradeoffs/67/none



The most straightforward example of evaluating stakeholder tradeoffs is covered in the UNEP/SETAC S-LCA (Social Life Cycle Assessment) references (2009 and 2011) found in Example 3B.

G. Impact Evaluation

To understand the relation between Social Performance Assessments and formal Agricultural Project Impact Evaluations, Winters et al (2010) use the following statement to describe using “list of indicators [that are correlated within] a vertical logic” to evaluate the impact of agricultural development projects. That vertical logic corresponds to the “social impact pathways”, “causal chains”, or “results chains” used by the RCA Framework. 

“To successfully identify the important set of final and intermediate indicators requires considering the logic of the intervention model; that is, how is the project going to improve the well-being of farmers? This should be reflected in lists of indicators used such as those found in a project’s Results Matrix often used by development organization such as the Inter-American Development Bank. The indicators should have a vertical logic that show how project investments alter farmer behavior and, in due course, lead to an impact on farmer well-being. Ultimately, these sets of indicators create a series of hypothesis regarding how the project will be successful and if an evaluation is carefully designed, these hypotheses can be tested. Box 1 provides an example of the indicators used to assess the logic of a project in Ecuador, the Plataformas de concertación, that links poor potato farmers to high-value potato markets.”

Gertler et al (2016) provide the following definition for Impact Evaluation and explain the importance of understanding the causal relation, or attribution, between action and result.

“[A]n impact evaluation assesses the changes in the well-being of individuals that can be attributed to a particular project, program, or policy. This focus on attribution is the hallmark of impact evaluations. Correspondingly, the central challenge in carrying out effective impact evaluations is to identify the causal relationship between the program or policy and the outcomes of interest.”

Several references (COSA, Sustainable Food Lab, ISEAL, Gertler et al) demonstrate that Impact Evaluation takes a more comprehensive, long term approach, often using public surveys, than the “social impact pathway” performance analysis demonstrated in this example. The Sustainable Food Lab (2013, 2016) discusses the relation between performance analysis and impact evaluation as follows:

“Performance Measurement is an approach that assesses current status and tracks change over time. The goal is cost effective ways to measure performance that can complement more intensive and expensive in-depth assessment. Performance measurement approaches are not designed to measure attribution between specific interventions and specific outcomes the way an impact assessment might.”

COSA (2014) further distinguishes Performance Monitoring’s short term goal of tactical decision making, with Impact Evaluation’s long term goal of strategic decision making. The Sustainable Food Lab (2016) uses the following images to describe how results chains employ both short term Performance Monitoring and long term Impact Evaluations. ISEAL (2014) defines guidelines that explain how these assessments relate to Monitoring and Evaluation systems. ISEAL also provides a glossary for the principle terms used in these types of assessments.



The following image (Bamberger, 2010) is used to provide historical context explaining the relation between quantitative Impact Evaluation, such as Randomized Control Trials (RCTs), and qualitative mixed methods Impact Evaluations, such as this image’s M&E-based system. Example 4B demonstrates how the health care sector is addressing these authors’ recommendations about “[the need to] provide guidelines on minimum levels of acceptable methodological rigor when drawing on diverse data sources that are often collected under tight budget and time constraints or where much of the data is collected under difficult circumstances”.



A major goal for any social network is to aggregate the individual Social Performance Assessments completed by their clubs. The Sustainable Food Lab (2016) discusses the complementary relation between Performance Monitoring and Impact Evaluation -the aggregated Performance data serves as input to Impact Evaluation, which in turn feeds back to improve the [QOL] Assessments. Formal Monitoring and Evaluation (M&E) systems, with adaptive learning mechanisms, provide the formal linkage. They use the term, Performance Monitoring Assessment, for that type of performance reporting. The following image (ISEAL, 2014) shows the M&E, Performance Monitoring, and Impact Evaluation, requirements that ISEAL employs for “scheme owners”, or organizations who administer certification standards.




Farm record keeping associations commonly aggregate their members’ data to support advanced decision support, such as the identification of “best farm management practices” and the establishment of targets for financial, or in this case, social, performance. IITA and COSA (2016) demonstrate using aggregated farm production data to identify the most and least efficient production practices used by groups of farmers. Several examples in this tutorial, such as Example 4B and Examples 5 through 8, demonstrate how to integrate Impact Evaluation, M&E, and Social Performance Assessment.

Gertler et al (2016) use the following image to indirectly address the integration of Impact Evaluation, M&E, and Social Performance Assessment. The simple “benchmark -> target -> actual->” M&E Indicator approach introduced in this example can completely fail to capture real cause and effect attribution. That approach can easily lead to this image’s conclusion that rice yields increased 100 kg (B to A) due to the project, compared to “counterfactual benchmark” real yield differences (C to A and D to A). Full Impact Evaluations are addressed in the Social Performance 3 reference.
 

Grenz and Sereke (2017) demonstrate software that also uses a survey approach for assessing agricultural sustainability. This RCA algorithm assumes that a separate survey or interview, containing questions for assessing each Indicator, as demonstrated in the FAO (2013), EPA (2016, in Example 4B), and COSA references, has been completed prior to completing the TEXT data files. [DevTreks is reluctant to supply automated surveys (i.e. by adding a new schema to the Story-Telling application) because numerous automated surveys are already available and because DevPacks remain on the horizon.]

Footnotes
1. [Although Version 2.1.8 began relaxing this requirement, it’s still considered sound advice. The source code can be changed, easily, to support as many trend periods as needed.] DevTreks limits TEXT.csv input datasets to 14 columns (3 descriptive, 1 y, 10 x) for 2 reasons. First, Occam may be right –it’s the lowest, acceptable, common denominator for professional data. Second, almost every single reference used in this reference, along with the CTAP tutorial, uses different data formats making it impossible to compare, aggregate, analyze, or use data in any serious way (i.e. modern IT way). That’s part of the reason, along with conventional academics, that almost no useful data could be found for the examples in this reference. Standardized, raw, TEXT datasets, mean that terms like “futile” will no longer be needed by authors completing these assessments (10*). Make sure to check TEXT.csv files for incompatible characters, such as commas, prior to saving them.


Case Study References
Antonopoulos, Ioannis-Sofoklis, Paolo Canfora, Marco Dri, Pierre Gaudillat, David Styles, Julie Williamson, Alice Jewer, Neal Haddaway, Martin Price. Best environmental management practice for the agriculture sector - crop and animal production. European Commission. Final Draft, 2016.
Piet van Asten, Dennis Ochola, Lydia Wairegi, Anaclet Nibasumba, Laurence Jassogne, David Mukasa. Coffee-Banana Intercropping: Implementation guidance for policymakers and investors. Practice Brief Climate-smart agriculture. Global Alliance for Climate Smart Farming. 2015
Michael Bamberger, Vijayendra Rao, Michael Woolcock. Using Mixed Methods in Monitoring and Evaluation Experiences from International Development. Policy Research Working Paper 5245.The World Bank Development Research Group Poverty and Inequality Team. 2010
Committee on Sustainability Assessment (COSA). Sustainability Performance Monitoring. Outline of Steps. 2014
Evelien M. de Olde, Frank W. Oudshoorn, Claus A.G. Sørensen, Eddie A.M. Bokkers, Imke J.M. de Boerc. Sustainability at farm-level: Lessons learned from a comparison of tools in practice. Ecological Indicators 66 (2016) 391-404
Fairtrade International. Fairtrade Standard for Small Producer Organizations Current version: 01.05.2011_v1.4
Fairtrade International. Fairtrade Theory of Change. Version 2.0. 2015
Food and Agriculture Organization of the United Nations (FAO). SAFA (Sustainability Assessment of Food and Agriculture systems) Guidelines version 3.0. 2013
Food and Agriculture Organization of the United Nations (FAO). SAFA Indicators. 2013
Gertler, Paul J., Sebastian Martinez, Patrick Premand, Laura B. Rawlings, and Christel M. J. Vermeersch. 2016. Impact Evaluation in Practice, second edition. Washington, DC: Inter-American Development Bank and World Bank. doi:10.1596/978-1-4648-0779-4. License: Creative Commons Attribution CC BY 3.0 IGO
Global Coffee Platform. BASELINE COMMON CODE GCP_Doc_01_Baseline Common Code_v2.1_en; April. 2016
Hayati, D. A literature review on frameworks and methods for measuring and monitoring sustainable agriculture. Technical Report n.22. Global Strategy Technical Report: Global Strategy to improve agricultural and rural statistics (GSARS), Rome. 2017
ISEAL Alliance. Assessing the Impacts of Social and Environmental Standards Systems ISEAL Code of Good Practice. Version 2.0 – December 2014
Jan Grenz and Firesenai Sereke. Response-Inducing Sustainability Evaluation (RISE). Bern University of Applied Sciences (last accessed June, 2017: http://rise.hafl.bfh.ch)
Paul Hoebink, Ruerd, Ruben, Willem Elbers, Bart van Rijsbergen. The Impact of Coffee Certification on Smallholder Farmers in Kenya, Uganda and Ethiopia. Center for International Development Issues. Radboud University, Nijmegen, the Netherlands. 2014
International Institute for Tropical Agriculture (IITA) and Committee on Sustainable Agriculture (COSA). Impacts of Certification on Organized Small Coffee Farmers in Kenya. Baseline, 2016. [Young software developers should note the use of “advanced algorithms” in this reference.]
Jan Landert. SMART (Sustainability Monitoring and Assessment Routine) and GI (Geographic Indications) sustainability assessment (slide presentation). Research Institute of Organic Agriculture. (last accessed June, 2017: www.fibl.org). 
Julia Lernoud, Jason Potts, Gregory Sampson, Salvador Garibay, Matthew Lynch, Vivek Voora, Helga Willer and Joseph Wozniak, The State of Sustainable Markets – Statistics and Emerging Trends 2017. ITC, Geneva.
Loconto, Allison and Dankers, Cora. (2014). Impact of International Voluntary Standards on Smallholder Market Participation in Developing Countries: A Review of Literature. Food and Agriculture Organization of the United Nations. Rome, 2014. (last accessed June, 2017: http://www.fao.org/3/a-i3682e.pdf)
Swaibu Mbowa, Tonny Odokonyero, and Ezra Munyambonera. The potential of coffee to uplift people out of poverty in Northern Uganda. Economic Policy Research Center, Uganda. Research Report No. 11, 2014
Natural Capital Coalition (NCC). 2016. Natural Capital Protocol. (Online) Available at:www.naturalcapitalcoalition.org/protocol (last accessed March, 2017)
Sustainable Food Lab. Performance Measurement in Smallholder Supply Chains: A practitioner’s guide to developing a performance measurement approach. 2014
Sustainable Food Lab. Towards a Shared Approach for Smallholder Performance Measurement: Common indicators and metrics. 2016
Winters, Paul., Salazar, L., Maffioli, A. 2010. Designing Impact Evaluations for Agricultural Projects. Impact Evaluation Guidelines. Technical Notes No. IDB-TN-198. (last accessed June, 2017: https://publications.iadb.org/bitstream/handle/11319/1956/ Designing%20Impact%20Evaluations%20for%20Agricultural%20Projects. pdf?sequence=1)


Example 1A. Coffee Company Trends SAFA Score (RCA2)
URLs: 
Data
https://www.devtreks.org/greentreks/preview/carbon/resourcepack/Coffee Firm RCA Example 1A/1554/none
http://localhost:5000/greentreks/preview/carbon/resourcepack/Coffee Firm RCA Example 1A/542/none

Resource Stock Assessment
http://localhost:5000/greentreks/preview/carbon/output/Coffee Firm RCA1A Stock/2141223491/none

Monitoring and Evaluation Assessment
https://www.devtreks.org/greentreks/preview/carbon/output/Coffee Firm RCA1A M and E/2141223481/none

Stories
https://www.devtreks.org/greentreks/preview/carbon/linkedviewpack/Social Performance Stories/183/none

http://localhost:5000/greentreks/preview/watershed/linkedviewpack/Social Performance Stories/183/none

A. Introduction

This example has been added to address the de Olde et al (2016) conclusion that, in practice, some farmers are reluctant to fully apply recommendations derived from these types of Social Assessment, and sustainability assessment, tools. This example addresses that concern with the following, increasing likely, agricultural production scenario:
1. Farmer versus Resource Conservationist Reporting: Example 1’s pathways are completed by professional Resource Conservationists who have been trained how to use the 4 or 5 Indicator-based impact pathways. Farmers focus mostly on the Impact Indicators that directly impact their financial and social performance. They have been trained to complete and use the 1 Impact Indicator TEXT dataset needed to carry out the final part of the pathway. Alternatively, Example 4 demonstrates how to use the hierarchical TEXT files to define impact pathways directly. Although not demonstrated in this example, full results chains can be documented using 1 Indicator’s TEXT dataset.
2. Certification-Required Indicators: The only SAFA Indicators used in this assessment correspond to the Global Coffee Platform’s (GCP) 8 economic indicators, 9 social indicators, and 10 environmental indicators. The coffee company can’t receive certification unless those 27 Indicators meet the GCP certification requirements. Lernoud et al (2017) document the increasing trend by commodity producers to comply with these types of standards (i.e. because more supply chain buyers and end product consumers are demanding evidence of the social soundness of the products and services they buy).
3. M&E Decision Making: The certification organization bases their production and marketing assistance, and certification verification, on accurate M&E reporting. This coffee company will comply if their net returns have increased more than the increased costs associated with the reporting and certification requirements. Farms that have received organic certification are typical examples (refer to Loconto et al, 2014).

B. Indicator Thresholds

The following image displays the 27 elements of the Global Coffee Platform as “harmonized” with the SAFA Indicator Thresholds and WBS. The complete SAFA list can be found in the URLs. The FAO Indicators reference (2013) defines the upper and lower thresholds for each Indicator, but the coffee certification organization defines the middle 3 thresholds. An example can be found in Example 1. 


Version 2.1.0 upgraded several subalgorithms, including 13 and 14, to accommodate the single letter labels used by the SAFA WBS. The new convention is that any “Total Risk” index must be prefaced with the characters “TR”. The former “TR” labels meet this criteria. In the SAFA WBS, those labels are now TRG, TRE, TRS, and TRC.
The image confirms that although most Coffee Indicators have straightforward relations to SAFA Indicators, some relationships are strained. In addition, whole categories of SAFA Indicators had to be eliminated. One take home message from this exercise is that a WBS that can support both on-farm decision making and international reporting norms is probably somewhere in the middle of these 2 systems. In addition, these WBSs and Thresholds systems need active support and evolution. SAFA hasn’t changed in 4 years (i.e. recent FAO publications suggest they are moving towards Example 1’s COSA Indicator system), while the Coffee Platform was updated last year. Feedback in the form of the de Olde et al (2016) on-farm testing, provides valuable insight into how to improve these systems.

C. Quality of Life Scenarios

Same as Example 1.
D. Social Performance Score
The SAFA scoring system, that normalizes Indicator quantities by dividing them by the sum of the Indicator weights for each Locational Index, and then multiplying the Indicator’s normalized value by its individual weight, is used with this example.
Version 2.1.0 upgraded algorithms 13 and 14 by using average, rather that total, scores, in the Total Risk (TR) Indexes for QTMost,  QTLow, QTUp, certainty1, and certainty2 (i.e. see the Landert 2017 images in Example 1 used to communicate these types of scores). Version 2.1.2 supported separate trend dates for each Categorical Index, rather than uniform trend dates for all data. 

Indicator 1. Impacts Meta (all scores are filled in automatically)
 

The following table displays the calculations for the Indicator1 QT Most 1 property, 15.94, from the “actuals” dataset explained next. As mentioned throughout DevTreks, these types of “data standards” are the responsibility of full social networks.

Indicator1.URL TEXT dataset Benchmarks (same rating for each Indicator because real datasets are not available)

Actuals (no separate Targets were used; Actuals set arbitrarily at 50% of Benchmarks, partial data displayed)


Indicator1.MathExpression
I1.Q1.factor1 + I1.Q2.factor2 + I1.Q3.factor3 + I1.Q4.factor4 + I1.Q5.factor5 + I1.Q6.factor6 + I1.Q7.factor7 + I1.Q8.certainty1 + I1.Q9.certainty2 + I1.Q10.norm + I1.Q11.weight

Indicator1.MathResult (partial results displayed)
The certainty values, or SAFA Accuracy Scores, are all equal because the original dataset set them equal for each separate Indicator. It’s easier to confirm the results –dividing the aggregated totals by the number of Indicators in the aggregation must return the original Indicator values.

E. Communication

The certification standards organization requires complete documentation for every Indicator used in an assessment. Besides the quantitative scores, an explanation for each rating for each Indicator must also be completed. The standard tool in DevTreks for explanatory content is the paragraph editor explained in the Story Telling tutorial. The following image displays the story completed as part of the Social Assessment. Each paragraph in the story corresponds directly to an Indicator scored in the Social Assessment. The first image demonstrates a metadata cover page, while the second image demonstrates the first page of an explanatory story. 

The Calculator tutorial points out that the Calculator.MediaURL property can also be used to store a link to a pdf file holding this data.

 

 
 

F. Decisions
This example left out the Targets demonstrated in Example 1 that many full M&E systems employ. Instead, in keeping with this example’s simplification theme, only Benchmarks and Actuals have been completed. The Actuals are associated with the coffee firm’s adoption of new conservation practices needed to comply with the certification requirements. The certification organization bases their decision on progress demonstrated by the Actuals.

Case Study Footnotes
1. Although it took less than 10 minutes to upgrade subalgorithms 13 and 14 to accommodate SAFA, it still took several hours to run this algorithm successfully because, even with 1 TEXT dataset, several mistakes still took place. Labels mistakenly had trailing spaces that had to be found and fixed with new code, some rows of data were put in the wrong place, and the normalization and weighting used by the algorithm had been forgotten (even with Example 1’s clear documentation). Footnote 11 was added to SPA1 for good reason.
Case Study References
Same as Example 1.


Example 2. Coffee Company Probabilistic Risk (PRA) Social Performance Score (RCA1)
URLs: 
https://www.devtreks.org/greentreks/preview/carbon/resourcepack/Coffee Firm RCA Example 2/1551/none
http://localhost:5000/greentreks/preview/carbon/resourcepack/Coffee Firm RCA Example 2/539/none
Resource Stock Assessment
https://www.devtreks.org/greentreks/preview/carbon/output/Coffee Firm RCA1 Stock/2141223475/none
http://localhost:5000/greentreks/preview/carbon/output/Coffee Firm RCA1 Stock/2141223483/none
Monitoring and Evaluation Assessment
https://www.devtreks.org/greentreks/preview/carbon/output/Coffee Firm RCA1 MandE/2141223474/none
http://localhost:5000/greentreks/preview/carbon/output/Coffee Firm RCA1 MandE/2141223484/none

G. Introduction

This is the same coffee company as in Example 1, except this example uses the RCA1 algorithm. RCA1 substitutes RCA2’a Time Trends with probability distributions to calculate scores. Instead of RCA2’s emphasis on calculating relative, qualitative, Social Performance Trends, this algorithm provides an absolute, quantitative, snapshot in time of the uncertainty of the firm’s social performance. The firm can, of course, account for trends by completing this Measure multiple times, or by deriving the probability distributions from time trend data.

Example 4B demonstrates how to use this algorithm for the meta-analysis of social performance data. For example, networks that aggregate their individual club Performance Monitoring Assessments can use this algorithm to conduct an Impact Evaluation of the aggregated data.

Because this algorithm follows the same 4 to 6 Indicator pattern as already demonstrated with Example 1, only a few Indicator images are displayed. 

H. Indicator Thresholds
Same as Example 1.
I. Quality of Life Scenarios
Same as Example 1.
J. Social Performance Score

Version 2.1.0 upgraded algorithms 13 and 14 by using average, rather that total, scores, in the Total Risk (TR) Indexes for QTMost,  QTLow, QTUp, certainty1, and certainty2 (i.e. see the Landert 2017 images in Example 1 used to communicate these types of scores). Some images may still reflect earlier versions. 

Indicator 1. Actions Meta (the scores are filled in automatically)
  
  
Indicator1.URL TEXT dataset
Benchmarks

Targets

Actuals


Indicator1.MathExpression
I1.Q1.factor1 (a placeholder)
Indicator1.MathResult
Benchmarks

Targets

Actuals

Indicator 2. Conditions Meta
Not displayed
 Indicator 3. Services Meta
Not displayed
 
Indicator 4. Impacts Meta
 
 
 Indicator4.URL TEXT dataset
Benchmarks

Targets
Not displayed
Actuals
Not displayed
Indicator4.MathExpression
I4.Q1.factor1
Indicator4.MathResult
Benchmarks
Not displayed
Targets
Not displayed
Actuals


Optional Indicator5. Impacts Meta

Version 2.1.0 upgraded this algorithm so that it can be run for Indicators 1 to 5. That accommodates the full “results chain”, of Inputs->Activities->Outputs->Outcomes->Impacts, explained in the M&E tutorials. For testing purposes, the same data used in Indicator 4 produced the following result for Indicator 5.

Version 2.14 upgraded algorithms 13 and 14 by allowing up to 6 separate calculator Indicators to be used to define impact pathways (i.e. SPA3, Example 6’s disaster impact pathway of Drivers -> Hazards -> Exposure –> Vulnerability -> Capacity -> Impacts). Version 2.1.8+ started supporting this tutorials’ algorithms for all 15 Indicators.


Social Performance Score

Version 2.1.2 investigated the use of separate algorithms to shore up Scores. For example, by assuming that enough Social Performance Assessments have been completed to train a neural network and then using a time series regression algo to predict cause and effect – will the conservation practices cause sustainability to change, by how much, and by when? Version 2.1.4 introduced algorithms that demonstrate using the Score to conduct both formal and informal Impact Evaluations.

This firm uses their final Performance Score to monitor and evaluate how well they are accomplishing targeted goals. They use the formula, (Impact actual score / Impact target score) * 100) to display their overall progress in the image below, but use tables and charts to communicate the full results to company leaders. The reason that the most likely, low, and high, scores are equal is that all of the data used in Indicator 2, 3, 4, was built by multiplying Indicator 1’s data by 1.1, 1.2, and 1.3, respectively.
The Score’s Iterations, Confidence Interval, and Random Seed, are used with each Indicator’s Monte Carlo calculations.

K. Communication

The company uses the following types of media to communicate the final scores to company stakeholders. 



L. Decisions

Same as Example 1.

Case Study References

Same as Example 1.



Example 3. Product Life Cycle Impact Assessments (P-LCIA) for Representative Small Scale Coffee Farms (RCA3)
Version 2.2.0. This release includes an SDG Plan reference which provides a fuller context and additional examples for LCIA. Example 12 in that reference explains that this subalgorithm has been upgraded to be compatible with subalgorithm20. The example demonstrates running Example 4’s subalgorithm16 jointly with this algorithm by putting subalgo16’s data URL in the second position.
URLs
https://www.devtreks.org/greentreks/preview/carbon/resourcepack/Coffee Firm RCA Example 3/1552/none
http://localhost:5000/greentreks/preview/carbon/resourcepack/Coffee Firm RCA Example 3/540/none

The cloud datasets used Inputs rather than Outputs in order to mix things up a bit.
Resource Stock Assessment
https://www.devtreks.org/greentreks/preview/carbon/input/Coffee Firm RCA3 Stock/2147397559/none
http://localhost:5000/greentreks/preview/carbon/output/Coffee Firm RCA3 Stock/2141223486/none
Monitoring and Evaluation Assessment
https://www.devtreks.org/greentreks/preview/carbon/input/Coffee Firm RCA3 M and E/2147397561/none
http://localhost:5000/greentreks/preview/carbon/output/Coffee Firm RCA3 M and E/2141223488/none (has both Stock and MandE calculators)

The reason that the Stock Indicators are in index position 1, 7, and 8, while the M&E Indicators are in position 1, 2, and 3, is that the M&E calculators don’t allow Indicators to be skipped. And the only reason for skipping Indicators was to test that up to 8 life cycle stages are supported. Version 2.1.8+ started supporting this tutorials’ algorithms for all 15 Indicators.

A. Introduction or Goal and Scope

This example continues with Example 1 and 2’s coffee farm to demonstrate how to use life cycle assessments to complete social assessments for products. UNSETACc defines the purpose of life cycle assessments as follows:

“LCIA is about the quantification of potential environmental impacts caused by the supply chain of products and services [Product LCA], as well as by the activities of organizations including the upstream and downstream suppliers [Organization LCA].”

Large commercial firms commonly complete standard LCAs as further proof of their “emissions management”, but small scale growers do not (refer to the UNSETACd and UNSETACf references). This example demonstrates that Product LCAs may still be appropriate for small scale farms, but not necessarily for each individual farm. Instead, representative LCAs are completed by the network’s resource conservationists for typical, small scale farm-made products within some geographic region (i.e. watershed), ecosystem (i.e. humid, upland, tropical savannah), or industrial sector (i.e. the European Commission’s, 2016, Product Environmental Footprint Category Rules). The latter use can help avoid confidentiality issues dealing with divulging company technology secrets.
	
This example, along with Examples 3A, 3B, 4C and 5, demonstrate combining several of the approaches recommended by UNSETAC: Social Life Cycle Assessments (S-LCA: UNSETACa and UNSETACb), Organization Life Cycle Assessments (O-LCA: UNSETACd and UNSETACf), Life Cycle Impact Assessments (LCIA: UNSETACc) and Life Cycle Costs. 

To set the context for this algorithm, UNSETACc describes the reasons for developing strong standards to conduct LCIA as follows: 

“With the globalization of economies there has also been a steadily growing need to create a worldwide consensus set of environmental impact category indicators embedded in a consistent, methodological framework. Such a set of indicators is expected to be used in environmental product information schemes, benchmarking in industry sectors, corporate reporting by companies, intergovernmental and national environmental policies, and common LCA work commissioned by governments and companies.”

This algorithm can be used several different ways:

Standalone Life Cycle Assessments. This algorithm can be used with standalone Indicators that are not related to the RCA Framework. Instead, the approaches recommending in the ILCD Handbook (LCA), UN-SECTAC S-LCA, UN-SECTAC LCIA, and UN-SECTAC O-LCA, references are followed. This example demonstrates combining several of those approaches. The current release supports LCAs for Indicators 1 to 8 for supply chain analysis. 

RCA Framework. The RCA Framework’s “social impact pathway” of Actions->Conditions->Services->Impacts (or “results chain” of Activities->Outputs->Outcomes->Impacts), might be used in a similar manner to how those pathways are used in LCA. UN-SETAC (2016) uses the term “impact pathway” to define the cause and effect relationship contained in their recommended 4 level LCIA hierarchy. They encourage further experimentation and expansion of their LCIA techniques to include such factors as cultural heritage, ecosystem services, and “identification and further modeling of a general environmental mechanism, applicable to all resources”. Suggesting that the RCA Framework’s 4 or 5 Indicator impact pathways may also be an appropriate approach.

Substitutes for Indicators that use subalgorithms 13 or 14. The algorithm can be used to replace specific Indicators in subalgorithms 13 and 14. For example, some assessments may find the ILCD LCA techniques to be a suitable replacement for Actions (i.e. quantified damages serve as stressors or risks). Others may find the UN-SETAC LCIA techniques, or Example 4’s Life Cycle Cost or Benefits techniques, to be a suitable replacement for Impacts. They can, of course, also be used as a supplemental set of calculations for each of the 4 or 5 Indicators (i.e. by linking 2 calculators to each base element).

Whole Operating or Capital Budget. The Ag Production tutorial demonstrates that DevTreks supports complete farm budgeting, where all of the inputs and outputs used by a company are quantified and categorized. Each of those inputs and outputs can use this algorithm to quantify their associated social impacts, including emissions. Alternatively, Components, Operations, Outcomes, Time Periods, and/or Budget, base elements can be used. In addition, rather than the Ag Production tutorial’s focus on commodity cost and return estimating, these budgets are also appropriate for supply chain budgeting. The full use of the hierarchical base elements can be used to address the UN-SETACc recommendations for spatial, regional, and temporal scales. For an example, navigate around the national crop rotation database URI found in the Ag Production tutorial and in Section C (regional = state Budget Groups, spatial = specific crop rotation Budgets, temporal = enterprise Time Periods).

Organization LCA (O-LCA): Example 3A explains the differences between Organization LCIA and Product LCIA. 

The following uses support the fuller sustainability assessments identified by several references (EC, 2012, UNSETACe) as being logical extensions of Product and Organization LCIA.

Social Life Cycle Analysis (S-LCA): Example 3B demonstrates conducting S-LCA to assess the socioeconomic impacts of company production on stakeholders. 

LCIA and Cost Effectiveness Analysis: Example 4C demonstrates how to use this algorithm to conduct Cost Effectiveness Analysis (CEA). It demonstrates how to ensure that money spent mitigating and adapting to environmental damages generates cost effective value. It also demonstrates the use of complementary algorithms that use harmonized data (i.e. the 22 properties of this algorithm also use the 22 properties of other algorithms, such as Example 4’s economic and temporal properties).

LCA and Stakeholder Impact Analysis (SIA): Example 5 in the Social Performance Analysis 3 reference demonstrates integrating population algorithms with this algorithm to conduct SIAs.

Social Sustainability Accounting Platforms: Example 12 in the Social Performance Analysis 4 reference demonstrates using social sustainability media platforms to conduct more thorough LCIs and LCIAs.

B. Indicator Thresholds, or System Boundary and Resource Inventory

The RCA Coffee social network has developed the following typical LCA resource inventory using a “system boundary” defined for their members’ typical coffee farms (the image comes from Coltro et al, 2006). Resource conservationists in these networks work with individual firm managers to customize the final system boundary, elements, and Indicators, to fit individual company, or representative farm, requirements. The Frischknecht et al reference (2016) demonstrate inventories for a product’s multiple life cycle stages. The fact that finding a complete, rather than this image’s summary, LCA raw dataset for even a single coffee farm “proved futile” (2* and 10*) suggests that social networks and clubs need to get busy.


The following image and tables show that Indicator TEXT datasets store this firm’s initial Scoring system. This stylized system uses the same Indicators and Categories as used with Indicator 4 in Examples 1 and 2, because this social network uses the same WBS for all of its resource accounting. It then adapts, or “harmonizes”, those categories to the UN-SETAC techniques to carry out LCIA in a stylized fashion as demonstrated in the following table. The UNSETAC references focus on frameworks, standards, and detailed calculations, not harmonized data that fits nicely into tabular datasets, so no actual data was available for this example (2*). 



Example 4C will demonstrate how to harmonize this type of data with actual on-farm budget data. To summarize that approach, each farm operation, such as Nutrient Management, can have multiple Damage Categories, or Categorical Indexes (CIs), such as Human Health and Ecosystem Quality. This algorithm sums, normalizes, and weights the Damage Categories to produce one Locational Index. The Locational Indexes (LIs) can then be independently normalized, weighted, and summed in the Total Risk Index. The LIs, if carefully defined, can support the “Areas of Protection” identified in the UNSECTACc reference.

To conduct cost effectiveness analysis, the LIs from this algorithm must then be allocated to each farm operation. In effect, this algorithm’s LIs must correspond to Example 4’s CIs (or careful use must be made of Example 4’s CategoricalIndex.factor6, allocation factor). The result will be cost per unit normalized damage index, or cost per unit damage per citizen.

UN-SETACc makes the following recommendation, along with caveats, for using their LCIA techniques (1*). Their “Hotspots Analysis” will be demonstrated with this example’s Score.

“The environmental impact category indicators recommended in this guidance are primarily suited for hot spot analyses in product and organizational LCA. Some of them are also suited for identifying Hotspots in consumption-based assessments of the environmental impacts of nations (Frischknecht et al. 2015; Tukker et al. 2014) and intergovernmental organizations such as the European Union (JRC 2012). The indicators try to model complex cause-effect chains in general and disregard specific local aspects. Therefore, they are not (yet) fully suited for the identification of environmentally optimal agricultural management practices for a particular farm, a particular [agriculture, or forest land] with respect to terrestrial biodiversity protection. They are also not fully ready for the measurement of actual human health impacts of particulate matter emissions in a particular city district, nor in the prediction of human health effects of a severe drought period in a given year in Central Africa.”

C. Quality of Life Scenarios

The company’s overall Quality of Life Scenarios are the same as explained in Example 1, but they want to use LCAs as further, quantitative proof, of the claims they report using Example 1 and 2’s techniques. Small scale farms use the representative LCAs to further define mitigation and adaptation actions that help them achieve their resource conservation accounting goals.

The ILCD Handbook defines scenarios very technically:

“Scenario for the analysed process or system that varies data and method assumptions with the purpose of evaluating the robustness of the study results and conclusions. If more than one alternative system or option are compared, each of them would have its own assumption scenarios.”

In effect, they are saying that quantifying the inputs and outputs associated with any product, production process, technology, or supply chain, requires making a lot of assumptions. More so, when the LCA is being done for representative farms or farm products –what are typical soil conditions, slopes, rainfall amounts, plant varieties, fertilizer applications, pesticide practices, yields, and bean grades? Making those assumptions for representative farm budgets alone is difficult. This issue can be addressed by developing several scenarios that define sideboards for the uncertain production characteristics. 

Example 3A will use examples of comparative LCIA, such as between organic and conventional production, to address this issue. Example 4C also demonstrates using Scenario Analysis with LCIA and CEA.

D. Social Performance Indicators and Life Cycle Impact Assessment

For LCIA calculations, the following image (UNSETACc) demonstrates a representative LCIA impact pathway used to assess air pollution’s impact on human health. 



These calculations support the general LCIA impact pathway approach of (see the Resource Calculation reference for an introductory example)
1) Pathway Part 1: starting with a field measurement, such as mass emitted to air from an input or output
2) Pathway Part 2: using multipliers, or factors, to allocate the first measurement to account for the parent Indicator contribution to emissions (i.e. when Co-Inputs or Co-Outputs also contribute to the first measurement) –in the previous image, this factor is part of the Intake fraction calculation
3) Pathway Part 3: using a multiplier, or factor, to convert the second calculation to an emission unit –in the previous image, this factor is similar to the Intake fraction
4) Pathway Part 4: using a multiplier, or characterization factor, to convert the third calculation to an Environmental Impact Category –in the previous image, this factor is similar to the Effect factor
5) Pathway Part 5: optionally using a multiplier, or characterization factor, to convert the Environmental Impact Category to a Damage Category –in the image above, this factor is the final DALY per kgPM25

The UNSETAC references demonstrate that the simplified multipliers, or factors, in these pathways can derive from complex calculations (i.e. land use conversions, YLL and YLD factors used to calculate DALYs, waste recycling, and energy consumption). The UNSETAC recommendations to document these related calculations can be handled using the paragraph editor introduced in Example 1A, or the Calculator.MediaURL property to link a pdf file that shows the calculations. Example 12 explained how to use either Version 2.2.0’s new Math Expressions, or automated Guidance Documents, to conduct the calculations.
 
The following 24 properties (12 factors for Indicators and 12 factors for Categorical Indexes) define the full LCIA impact pathway from initial Input or Output quantity to final LCIA damages. Unlike the “vertical impact pathways” introduced in Example 1, which are defined using multiple Indicators, these horizontal impact pathways are defined by the 24 properties. Although a case can be made that additional properties may be needed to conduct more comprehensive LCAs, examples will be added that demonstrate using complementary subalgorithms for that purpose (i.e. using Example 4’s economic and temporal properties; using Example 5’s socioeconomic properties). Some of the references use the term “full sustainability assessment” for LCAs that address the latter purposes.

Although the algorithm doesn’t care about property names, DevTreks recommends using the generic property names. Although these multipliers demonstrate typical uses, they can be used for whatever purpose fits best with the impact pathway (i.e. product footprints). 
* factor1: Most likely quantity of Indicator (i.e. input or output quantity). This will be used to calculate Indicator.QTM. Version 2.1.2 upgraded this algorithm to also support Example 2’s PRA techniques. If using this to conduct PRA, this factor is equivalent to QT.
* factor2: Low quantity of Indicator. This will be used to calculate Indicator.QTL. Version 2.1.2 upgraded this algorithm to also support Example 2’s PRA techniques. If using this to conduct PRA, this factor is equivalent to QTD1 (i.e. shape). 
* factor3. High quantity of Indicator. This will be used to calculate Indicator.QTU. Version 2.1.2 upgraded this algorithm to also support Example 2’s PRA techniques. If using this to conduct PRA, this factor is equivalent to QTD2 (i.e. scale). 
* factor4. Unit of measurement for factor1. 
* factor5: probability distribution type (see Example 2: none, normal, lognormal, triangle …) when factor1, factor2, and factor3 are used to conduct PRA. 
* factor6: General multiplier applied to the quantities. The references show that the simplest use of this factor is as percent Q to be allocated to this Indicator (because of co-inputs or co-outputs). 
* factor7: Unit of measurement for the factor6 conversion (i.e. when factor6 is not a simple allocation factor). 
* factor8: Multiplier used to convert the unit of measurement of Q from the input or output quantity (kg N / ha) to the emission unit (kg NO2 / ha).
* factor9: Unit of measurement for the factor8 conversion.
* factor10: Characterization factor used to convert the calculated emission quantity to the Categorical Index, or Impact Category, Unit (CO2 equivs).
* factor11: Unit of measurement for the factor10 conversion (i.e. CO2 Equivalents). 
Version 2.2.0 broke Occam’s rule for these datasets to make them consistent with Example 12, sibalgorithm20, in the SDG Plan reference.
* factor12: math expression used to calculate the 4 LCIA midpoint measurements; the expression takes the form of Q1*Q2*Q3*Q4 or, as explained in the Resource Stock Calculator reference, Q1*(Q2/3)*Q3*Q4*2; the term “none” signifies that no calculations should be run

The following formula is used with these columns.

Indicator.QTM = Version 2.2.0 uses the math expression, Q1*Q2*Q3*Q4, in factor12
Indicator.QTL = uses factor2 in the math expression
Indicator.QTU = used factor3 in the math expression

The Categorical Indexes used in this example were upgraded in Version 2.1.2 to allow the results of Environmental Impact categories to be converted to Damage categories as displayed in the following image (UNSETACTc). That reference also explains that the damage category can be substituted for the Impact category, with no additional conversion (i.e. not all impact pathways use Midpoint Impact Categories). 



The UNSETAC references explain that each stage in an LCA pathway has uncertainty. The properties in the following list accounts for the uncertainty of the Damage Category characterization factor. For example, the following image (UNSETACc) shows that factor1, factor2, and factor3, can be documented using characterization factors that derive from the marginal, or average, slope of an exposure-response curve at most likely (shown), low (not shown), and high (not shown), emission concentrations (also refer to Figure 4.2 and Figure 5.5).



To support these calculations, the following list defines the final 11 columns of data for Categorical Indexes. Although the normalization and weights used in this example demonstrate typical uses, they can be adjusted for whatever purpose fits best with the production technology.
* factor1: Most likely quantity of Damage Category characterization factor. If using this to conduct PRA, this factor is equivalent to QT. The results of this calculation will be multiplied by the Indicator.QTM (most likely Impact Category quantity).
* factor2: Low quantity of Damage Category characterization factor. If using this to conduct PRA, this factor is equivalent to QTD1 (i.e. shape). The results of this calculation will be multiplied by the Indicator.QTL (low Impact Category quantity). 
* factor3. High quantity of Damage Category characterization factor. If using this to conduct PRA, this factor is equivalent to QTD2 (i.e. scale). The results of this calculation will be multiplied by the Indicator.QTU (high Impact Category quantity). 
* factor4. Unit of measurement for Damage Category characterization factor. Also used to aggregate multiple CIs into Elementary Flows for the Score’s Hotspots Analysis.
* factor5: probability distribution type (see Example 2: none, normal, lognormal, triangle …) when factor1, factor2, and factor3 are used to conduct PRA. 
* factor6: Label used to aggregate multiple CIs into Life Cycle Stages for the Score’s Hotspots Analysis (i.e. when each Indicator has multiple life cycle stages).
* factor7: Label used to aggregate multiple CIs into the Production Processes for the Score’s Hotspots Analysis. When possible, use the Labels from Example 4’s budgets. 
* factor8: certainty1 of Categorical Index
* factor9: certainty2 of Categorical Index 
* factor10: normalization (use “none” if the calculated Categorical Indexes, such as Human Health and Ecosystem Quality, are not being normalized)
* factor11: weight (use 1 if the calculated Categorical Indexes are not being weighted)
Version 2.2.0 broke Occam’s rule for these datasets to make them consistent with Example 12, subalgorithm20, in the SDG Plan reference.
* factor12: math expression used to calculate the final LCIA damage measurements; the expression takes the form of Q1*Q2 or (Q1*2)*(Q2/4.5), with Q1 derived from the Indicator calculations and Q2 derived from these Categorical Index properties; the term “none” signifies that no calculations should be run; using a prefix of “mcda:” in the expression (i.e. mcda:Q1*Q2) signifies that the Indicators should also be normalized (i.e. Example 4B’s MCDA calculations)

The following list defines how the final 11 columns of data for Locational Indexes are used.

* factor1: not applicable, use 0
* factor2: not applicable, use 0
* factor3. not applicable, use 0
* factor4. Unit of measurement for LocationalIndex.QTM calculation
* factor5: not applicable, use “none” 
* factor6: not applicable, use “none” 
* factor7: not applicable, use “none” 
* factor8: not applicable, use “none”
* factor9: not applicable, use “none”
* factor10: normalization (use “none” if the calculated Locational Indexes are not being normalized). This factor normalizes the Locational Indexes, which start with normalized Categorical Indexes. Some of the references use the term “Area of Protection” in a similar way to the Locational Indexes.
* factor11: weight (use 1 if the calculated Locational Indexes are not being weighted)
* factor12: none

The MathResult section, below, explains the calculations used with these properties to produce the final LCIA report. The results add 4 additional columns to hold the final normalized, weighted, calculations. The final column, percent, displays the percent contribution of the row to the parent aggregator. The final column supports the Hotspots Analysis explained for the Score.

Analysts should follow the recommendations for normalization and weighting found in the references. Those recommendations involve deriving the values from national or international standards, such as those displayed in the following image (European Commission, 2016). This algorithm upgraded the techniques demonstrated for Examples 1 and 2. Besides containing the standard options (weights, minmax, zscore, tanh, logit, logistic), the “norm” column can now contain doubles which will be used as a multiplier with each Indicator total (i.e. the population multipliers shown in the following image that normalizes damages to damage per EU person). Depending on the source of the normalization multiplier, the normalization factors may need to be entered based on the calculation, factor10 = (1 / normalization factor). Example 9 demonstrates how to use a new option, modzscore, to conduct modified z-scores.

This example uses multipliers for the Categorical Index normalization factors, and minmax for the Locational Index normalization factors.


The certainty factors (factors 8 and 9 of Categorical Indexes) can be used to address the “qualitative communication of uncertainty” discussed in Appendix A, including the following UNSETACc recommendations for handling uncertainty:

“It is strongly recommended to make all uncertainties explicit by reporting them at least in a qualitative way”

 “We recommend to include a vulnerability term for impacts on species richness and possibly ecosystems, to reflect that there are species and ecosystems that are more at risk due to specific interventions than others.”

Indicator 1. Life cycle stage 1. Production LCIA Meta. The benchmark, target, and actual metrics shown in the following image reflect the final normalized, weighted, environmental, or damage, impact for this life cycle stage, or Indicator. It’s important to understand that the algorithms demonstrated in this reference produce these types of final aggregated results, because these are the metrics that can be further analyzed using the Resource Stock and M&E Analyzers (i.e. Totals, Statistics, Change By, and Progress).

UNSETACd provides the following guidance for using these “single aggregated impacts”. The actual Indicator.MathResults are used to fully address the mentioned caveats. The Score will further explain these metrics. This statement verifies that the most useful analyses must be derived from the Score’s Categorical Index Hotspots data.

“Single-score impact category indicators (i.e., expressing the results of the environmental multi-impact assessment with only one aggregated indicator) have potential for O-LCA (see Report 9 on p.80). Based on value choices, they ease the interpretation of the results for non LCA experts, like managers. However, single-score indicators hide information trade-offs and have higher uncertainties as long as normalization and weighting factors are used. If the organization is aware of the limitations of inventory-level indicators and single-score impact category indicators, they may be used in the study.”
  
    

Indicator1.URL TEXT dataset
Although the Frischknecht et al (2016) reference documents the rice crop Resource Inventory used with the UNSETACc LCIA techniques, no complete LCA reference dataset, containing a complete TEXT dataset of the full LCIA impact pathways, including final Damage Category calculations and Indexes, is offered by any of the references (i.e. 2*).  Social networks must work with their information technologists to proof these LCA calculations before using them for professional work. For example, alternative normalization and weighting techniques can certainly be devised that might fit better with some LCIA impact pathways (i.e. and therefore require a separate subalgorithm). Just remember Appendix A’s concluding advice –use generic over custom whenever possible.

Benchmarks. UNSETACc refers to this as a Reference case; the functional unit is based on crop production or per kg product (2*). The CategoricalIndex.factor10s, or normalization factors, are straight multipliers. The LocationalIndex,factor10s use minmax for normalization. The primary purpose of this stylized data was to proof this algorithm, not as representative data. The Indicators, Categorical Indexeds, and Locational Indexes, all have equal properties, and, with allowances for the PRA randomization techniques, will return similar final calculations.

The following image confirms that Version 2.1.2 upgraded this algorithm to use the standard scientific notation format used in LCAs. Although the TEXT input data sets can still use digital formats, the MathResults will always return scientific notation. Version 2.2.0 standardized the labeling system to Example 9’s SDG labeling system and added Example 12’s MathExpression.



Targets
Not displayed -for testing same data as Benchmarks, but with _A suffix
Actuals
Not displayed -for testing same data as Benchmarks, but with _AA suffic

Indicator1.MathExpression
I1.Q1.factor1 

Indicator1.MathResult
Benchmarks. Although this image displays numbers in digital format, the raw csv data is in scientific notation. The calculations that are displayed below the image explain why these results don’t easily support simple “eyeball verification”.
 


For Indicators, the columns, QTMost and QTMost2, are derived by:
1) Running the PRA (i.e. normal distribution) on the first LCIA measurement (i.e. Q1 in the Indicator.MathExpression). 
2) Running the Indicator.MathExpression for the 4 LCIA measurements and adding the result to Indicator.QTM.
3) If the parent CategoricalIndex.MathExpression uses a “mcda:” prefix, the calculated children Indicators will be normalized and the column, QTMost will hold the normalized results (i.e. Example 4B’s MCDA technique). If the Indicators have not been normalized, the same column holds the nonnormalized calculated results. 
4) Adding the nonnormalized calculated results to the QTMost2 column.
For Indicators, the columns, Benchmark and Target Percents, are derived by:
1) For subalgorithm20, dividing the benchmark and target amounts by their respective Indicator.benchmark and target amounts (prior to normalizing and weighting the CIs). Subalgorithm15 doesn’t have benchmark or target amounts and doesn’t have these columns.
For Indicators, the column, Total Percent, is derived by:
1) Dividing column Indicator.QTMost by the parent CI.QTMost prior to normalization of the CIs. 

For Categorical Indexes, or CIs, the columns, QTMost and QTMost2 are derived by:
1) Running the PRA (i.e. normal distribution) on the fifth LCIA measurement (i.e. Q1). 
2) Summing the children Indicator.QTMs (i.e. Q2), running the CI.MathExpression (i.e. Q1 * Q2), storing the calculated results for display in QTMost2, and adding the calculated results to a vector of CI.QTMs.
3) Normalizing and weighting the vector of CI.QTMs and adding each normalized value to CI.QTM for display in the QTMost column.
For CIs, the columns, Benchmark and Target Percents, are derived by:
1. For subalgorithm20, dividing the benchmark and target amounts by their respective CI.benchmark and target amounts (prior to normalizing and weighting the CIs)..  Subalgorithm15 doesn’t have benchmark or target amounts (i.e. although DevTreks convention of conducting scenario analysis using “_A” and “_AA” suffixes can be used for that purpose).
For CIs, the column, Total Percent, is derived by:
1) Dividing each separate normalized CI by the sum of the normalized CIs. 

For Locational Indexes, the columns, QTMost and QTMost2 are derived by: 
1) Summing the children normalized and weighted Categorical Index.QTMs.
2) Normalizing and weighting a vector of LocationalIndex.QTMs, from step 1’s results, and adding each normalized member to the LocationalIndex.QTMost property.
3) Summing the children Categorical Index QTMost2 calculations (nonnormalized amounts) and adding the result to QTMost2.
The column, Percents, are derived by:
1) For subalgorithm20, summing the children benchmark and target percents and adding the average of the sum to the percent columns.
2) For TotalPercent, Dividing each separate normalized LI by the sum of the normalized LIs. 

For Total Risk Indexes (TR), the calculated results derive from simple summations or averages of their children Locational Indexes.

Targets
Not displayed –for testing, same at Actuals
Actuals
Not displayed–for testing, same at Actuals

Optional Indicators 2 to 15. Additional Life Cycle Stages (or Results Chains)

The following image (UN-SETACe) demonstrates typical life cycle stages in full LCA product evaluations. Version 2.1.8 upgraded this algorithm so that it can be run for Indicators 1 to 15. The CategoricalIndex.factor6 property, Life Cycle Stage Label, allows each Indicator to contain multiple life cycle stages. These 15 Indicators can be used for alternative uses, such as thoroughly analyzing primary product production processes or to support the RCA reporting demonstrated in the SPA3 reference. Example 1 demonstrates using separate Indicators to document the results chains and impact pathways needed for thorough RCA reports.

For testing purposes, the same data used in Indicator 1 produced the following result for Indicator 8.




Continuing with national and international data standards, the following image (USDA, 2015) demonstrates the data standards required when submitting LCAs to alternative databases. It’s the role of social networks and clubs to follow these types of standards (i.e. Example 1A demonstrates how to link a story page holding the Dublin Core Meta data standards to base elements). Appendix C in SPA4 discusses potential limitations with existing LCI databases.


E. Communication and Interpretation

Indicators
The following images (UNSETACc) demonstrate using the results of each Indicator to communicate the final Damage Impacts to network stakeholders. These images demonstrate conducting the water consumption and climate change categories of an LCIA conducted for representative rice farms in the 3 regions. 


Scores and Hotspots Analysis

Scores can be defined using standard mathematical expressions that use combinations of Indicators 1 to 15. They can also be calculated by using this algorithm, or subalgorithm20, to produce a Hotspots Analysis. UNSETACe defines Hotspots Analysis as follows:

“The rapid assimilation and analysis of a range of information sources, including life cycle based studies, market, and scientific research, expert opinion and stakeholder concerns. The outputs from this analysis can then be used to identify and prioritise potential actions around the most significant economic, environmental and social sustainability impacts or benefits associated with a specific country, city, industry sector, organization, product portfolio, product category or individual product or service. Hotspots analysis is often used as a pre-cursor to developing more detailed or granular sustainability information.”

They use the following image to define 2 options for identifying Hotspots:


The European Commission (2016) defines Hotspots Analysis as follows:

“In the context of PEF/OEF pilot phase we can define a hotspot as either:  
OPTION A: (1) life cycle stages, (2) processes and (3) elementary flows cumulatively contributing at least 50% to any impact category before normalisation and weighting (from the most contributing in descending order). 
OPTION B: At least the two most relevant life cycle stages, processes and at least two elementary flows (minimum 6).  Additional hotspots may be identified by the TS.”

The following image (European Commission, 2016) highlights the general process of using Indicator 1 to 15’s data to conduct Hotspots Analysis (UNSETACe), or Product Environmental Footprints (PEF) (European Commission, 2016) (3*). The Lippiatt reference (2007) provides a good overview of how software can be designed to conduct these analyses.

1. Elementary Flows: Identify the most important environmental impacts or elementary flows. The flows are measured using the CategoricalIndex.factor4 property to aggregate similar impacts and then calculating a % damage to determine importance. 
2. Life Cycle Stages: Identify the most relevant life cycle stages. The flows are measured using the CategoricalIndex.factor6 property to aggregate similar impacts and then calculating a % damage to determine importance. 
3. Production Processes: Identify the most impactful production processes or farm operations. The flows are measured using the CategoricalIndex.factor7 property to aggregate similar impacts and then calculating a % damage to determine importance. 


UN-SETACe describes the following advantages to using Hotspots Analysis.

“When applied to Life Cycle Assessment, the benefits of Hotspots Analysis include ensuring:
* Focus on priority issues (e.g., waste, water, materials of concern)
* Focus on the right life cycle stage (e.g., material acquisition, manufacturing, use, end of life)
* Focus on the right actors (e.g., producers, manufacturers, suppliers, retailers, customers, government officials) to evaluate, influence and implement solutions
* Implications of trade-offs are understood
* Resources (e.g., time, money) can be effectively allocated to actions.”

The following image (European Commission, 2016) displays the recommended data to include in a Hotspots Analysis, or Product Environmental Footprint (PEF) screening analysis, reports.
.
The following images of the Score show that the Score produces the basic data required by the PEF screening and UNSETACe recommendations. Use the Score.MediaURL property to link pdf files that address the full reporting recommendations. These aggregation techniques were first documented in the Resource Analysis reference for Version 1.8.2 and further developed for this algorithm.

Score Math Result

No separate TEXT datasets are added to the score to generate the following MathResult. Instead, the CategoricalIndex data from Indicators 1 to 15 are simply added to the MathResult with no additional calculations. In this analysis, only Indicator 1, 7, and 8 were completed. That data must then be manipulated in a spreadsheet to produce the analysis. To repeat, the purpose of this stylized data is to proof this algorithm, not to demonstrate representative LCA data. The wide gap between Most Likely, Low, and High, estimates, if based on real LCA data, would be extremely difficult to use for making decisions (i.e. should the Locational Indexes’ normalization use min-max or another option?).

This method was chosen because the Indexing used with these datasets support a wide range of aggregated reporting. For example, the EC (2016) reference recommends using Indicator summations that are not normalized or weighted when calculating the percent contribution to total Categorical Impacts. UNSETACe recommends using normalized and weighted data in the calculations. Neither publication fully discusses the Categorical Damage Impacts, PRAs, or the Locational Indexes, used in this algorithm. Rather than make tenuous assumptions about the best way to conduct Hotspots Analysis with this data, this release chose to just return raw Categorical Index data for further spreadsheet manipulation. Factors 4, 10, and 11, can be used to sort the data for Hotspots Analysis.




An alternative to using the Categorical Index data in this Score.MathResult is to copy the Indicator.MathResults into 1 dataset and manipulate the full dataset to conduct other types of LCA or Hotspots Analysis.

Score Properties
Prior to Version 2.2.0, this reference failed to mention that, with the exception of the Score.MathResult, this subalgorithm requires the manual completion of the remaining Score properties. The Score’s Hotspots Analysis data can be manipulated to produce multiple Scores for multiple purposes –select the one needed in subsequent Stock and M&E Analyses and fill in the properties.

The following Score properties were manually set by adding together each Indicator.QTM, QTL, and QTU. In effect, the sum of the normalized and weighted Indicator.LocationalIndexes. Example 3A will demonstrate how similar “ecopoints” are used with Organization LCIA. The Score’s Iterations, Confidence Interval, and Random Seed, properties are used with each Indicator’s probability distribution, or PRA, calculations. Score.MathExpressions are not run with this subalgorithm.

UNSETACe recommends not using “single score impact assessment indicators” when making Organization LCIA comparisons among companies. Care must be exercised if using these results to carry out the comparative analyses supported by the Resource Stock and M&E Analyzers. Example 3A discusses this further.

 

The following image shows why Score.MathResults should be stored in URLs, not in the Score itself.  Score.MathResults are also displayed with base element results. That technique is acceptable for small amounts of data, but it is not good practice with tabular datasets.



F. Decisions

The network uses the LCIA to advise their clubs about how alternative mitigation and adaptation actions impact public capital stock services, particularly ecosystem services. UNSETACe presents case studies demonstrating the central role played by stakeholders who must use LCA to make decisions. Their study of the tradeoffs that must be made community-wide between stakeholder values, ecosystem services, and mitigation and adaptation practices, ensure that “the severe conflict, inequity, and dissatisfied customers, which can arise from making decisions and taking actions that don’t account for the tradeoffs and synergies needed to balance the interests of diverse stakeholders“ is avoided. Example 3B and Example 5 through 8 in SPA3 addresses this further.

The following image (UNSETACe) demonstrates how some industries apply LCA very seriously. In this example, the appliance industry in the USA used a combination of LCA, Hotspots Analysis, and stakeholder engagement, to develop sustainability standards for appliances. Stores display those numbers with each appliance so that consumers can make informed decisions.



Notarnicola et al (2017) review the state of LCA art in food system analysis and highlight the numerous challenges remaining to be tackled in this industry. These challenges include 
* the need for dietary shifts to sustainable food systems 
* field level LCAs that don’t adequately address landscape level sustainability impacts on soil quality and fertility, land erosion, reduced ecosystem services, and biodiversity loss 
* integration of social, economic, and cultural factors into LCA studies 
* the reliance on average LCAs for predominant food production systems rather than the reality of extreme production variability 
* technical deficiencies dealing with product quality, geographical contexts, temporal variability, machinery, functional units, ecosystem services and biodiversity 
* consumer education that results in behavioral change 
* missing supply chain phases such as food waste 
* integration with “mixed methods explanatory approaches” used in food production studies 
* accounting for accidents and disasters

Notarnicola et al (2017) point out that failure to address these challenges could surpass planetary food system sustainability, and therefore food security, needs. In the context of this reference, it’s the job of social networks and clubs to address those challenges, including the development of open access, rather than commercial, sustainable food system databases that make all of their TEXT data available through URIs (2*). 

The Malnutrition Analysis tutorial explains that a complete database of the nutritional composition of most U.S. food products can be found in HomeTreks. (also refer to the ENVIFOOD 2013 reference) i.e.
https://www.devtreks.org/hometreks/select/farmworkers/servicebase/Food Inputs, USDA SR24 codes/2635/none

The Ag Production tutorial explains that a complete database of U.S. crop rotations can be found in AgTreks. i.e.
https://www.devtreks.org/agtreks/select/crops/servicebase/Profits, Texas AM and NRCS national crop rotations/1076/none/

The Health Care tutorial explains that health care professionals can find complete ICD-10s for several base elements in HealthTreks. i.e.
https://www.devtreks.org/healthtreks/select/urbandelivery/serviceaccount/HealthTreks West/459/none

The Construction Analysis tutorial explains that construction analysts can find good examples of the UNIFORMAT II in BuildTreks (also refer to the Lippiatt 2007 reference). i.e.
https://www.devtreks.org/buildtreks/select/commercial/servicebase/Inputs, WBS Examples/1220/none

Although challenging, the URLs demonstrate that it is still perfectly feasible for social networks to extend those types of datasets with “bulk uploaded” RCA calculators (2*). 

Footnotes
1. Example 4B discusses WHO’s recommendation not to complete analyses that are too localized –because the results may be more useful at country and sector scale rather than local scale. In a similar fashion, the European Commission (2016) provides standards for “Product Environmental Footprint Category Rules” that can be used throughout industries. In other words, the UN-SETAC caveats may be strengths rather than weaknesses. The take home message is that people need to be getting their hands dirty and gaining experience building these algorithms and applying their tools so that they know when and how to use them properly.
2. Lack of applied IT knowledge and skills is a recurring theme throughout this reference. For example, the use of URIs as the standard way to access data should not be novel to anyone anymore, but no LCA data could be found using that universal storage access mechanism (i.e. try for yourself). For reasons that are not entirely clear, research peers and government agencies, in particular, are culpable. The intent of these tutorials is to help the next generation stop making the same mistakes over and over again because too much is at stake.
3. The European Commission (2016) and UNSETAC references target full-time professional LCA researchers and practitioners. For example, Annex B in the EC 2016 reference summarizes the recommended reporting that the EC requires for LCA and PEF submissions. In contrast, this reference targets professional resource conservationists, such as sustainability officers working for firms and community service organizations, who must apply LCA as one important set of tools in a more comprehensive RCA toolkit. Social networks are encouraged to develop similar standards for their clubs to follow, recognizing that applied work may require compromise.

Case Study References

European Commission - Joint Research Centre - Institute for Environment and Sustainability: International Reference Life Cycle Data System (ILCD) Handbook - General guide for Life Cycle Assessment - Detailed guidance. First edition March 2010. EUR 24708 EN. Luxembourg. Publications Office of the European Union; 2010  
European Commission - Joint Research Centre - Institute for Environment and Sustainability: International Reference Life Cycle Data System (ILCD) Handbook. First edition March 2010. EUR 24708 EN. Luxembourg. Publications Office of the European Union; 2012  
European Commission, 2016, Environmental Footprint Pilot Guidance document, - Guidance for the implementation of the EU Product Environmental Footprint (PEF) during the Environmental Footprint (EF) pilot phase, version 5.2, February 2016. 
Coltro, Leda, Anna Lucia Mourad, Paula A.P.L.V. Oliveira, Jose Paulo O.A. Baddini, and Rojanne M. Kletcke, Environmental Profile of Brazilian Green Coffee. Int J LCA 11 (1) 16-21. 2006
Food SCP RT (2013), ENVIFOOD Protocol, Environmental Assessment of Food and Drink Protocol, European Food Sustainable Consumption and Production Round Table (SCP RT), Working Group 1, Brussels, Belgium. 2013
Frischknecht R, Fantke P, Tschümperlin L, Niero M, Antón A, Bare J, Boulay AM, Cherubini F, Hauschild MZ, Henderson A, Levasseur A, McKone TE, Michelsen O, Mila-i-Canals L, Pfister S, Ridoutt B, Rosenbaum RK, Verones F, Vigon B, Jolliet O. Global guidance on environmental life cycle impact assessment indicators: Progress and case study. Int J Life Cycle Assess. 21(3):429–442. 2016
Lippiatt, Barbara. BEES 4.0 Building for Environmental and Economic Sustainability Technical Manual and User Guide. US National Institute of Standards and Technology, US Department of Commerce. 2007
Notarnicola, Bruno, Serenella Sala, Assumpcion Anton, Sarah J, McLaren, Erwan Saouter, Ulf Sonesson. The role of life cycle assessment in supporting sustainable agri-food systems: A review of the challenges. Journal of Cleaner Production 140 (2017) 399-409
UNEP/SETACa. Guidelines for Social Life Cycle Assessment of Products. 2009
UNEP/SETACb. The Methodological Sheets for Subcategories in Social Life Cycle Assessment (S-LCA). Prepublication Version. 2013
UNEP/SETACc. Global Guidance for Life Cycle Impact Assessment Indicators Volume 1. 2016
UNEP/SETACd. Guidance on organizational life cycle assessment. 2015
UNEP/SETACe. Hotspots Analysis. An overarching methodological framework and guidance for product and sector level application. 2017
UNEP/SETACf. Road testing organizational life cycle assessment around the world. Applications, experiences and lessons learned, 2017
USDA-National Agricultural Library. LCA Commons Submission Guidelines. 2015
Zampori L., Saouter E., Castellani V., Schau E., Cristobal J., Sala S.; Guide for interpreting life cycle assessment result; EUR 28266 EN; doi:10.2788/171315. 2016



Example 3A. Organization Life Cycle Impact Assessments (O-LCIA) for Representative Small Scale Coffee Farms (1*)
URLs:
Conventional Inputs and Outputs (using stylized data)
https://www.devtreks.org/greentreks/preview/carbon/input/Coffee Firm RCA3A Conv/2147397563/none
http://localhost:5000/greentreks/preview/carbon/output/Coffee Firm RCA3 Stock, Conventional/2141223496/none
Organic Inputs and Outputs (using stylized data)
https://www.devtreks.org/greentreks/preview/carbon/input/Coffee Firm RCA3A Org/2147397564/none
http://localhost:5000/greentreks/preview/carbon/output/Coffee Firm RCA3 Stock, Organic/2141223486/none
Operation and Outcome Group Comparison
https://www.devtreks.org/greentreks/preview/carbon/operationgroup/RCA Crop Operations/765/none
http://localhost:5000/greentreks/preview/carbon/outcomegroup/RCA Conv vs Organic Coffee Crop Outcomes/50/none
A. Introduction or Goal and Scope

Example 3 demonstrated conducting a Product LCA for coffee production. This example demonstrates conducting an Organization LCA for a coffee production firm. This example explains that, while the substance and purpose of the 2 analyses differ, the actual subalgorithm calculations are the same. UNEP/SETACd defines an Organization LCA as follows:
“Compilation and evaluation of the inputs, outputs and potential environmental impacts of the activities associated with the organization as a whole or portion thereof adopting a life cycle perspective (ISO, 2014c).”
UNEP/SETACd use the following image to explain that one of the primary purposes of O-LCAs is to identify “opportunities for interventions”, such as mitigation and adaptation actions that companies can take to reduce environmental damages. 

Pelletier et al (EC, 2012) use the term Organization Environmental Footprint (OEM) for a similar purpose and define the term as follows:
“Based on a life cycle approach, the OEF is a method for modelling and quantifying the physical environmental impacts of the flows of material/energy and resulting emissions and waste streams associated with Organisational activities from a supply-chain perspective (from extraction of raw materials,through use, to final waste management).”
The following image (Appendix D in UNEP/SETACd) compares Product LCAs (the 2nd column) with Organization LCAs (the 3rd column). They further explain that O-LCAs do not necessarily have the same level of detail, and therefore cost, as P-LCAs.


The following definition of Life Cycle Impact Analysis, or LCIA, was introduced in Example 3. 
“The environmental impact category indicators recommended in this guidance are primarily suited for hot spot analyses in product and organizational LCA. …”
The definition explains why no changes had to be made in subalgorithm15, or subalgorithm20, to conduct either a Product LCIA or an Organization LCIA. Although the previous image shows the substance and purpose for these analyses differ, the LCIA calculations are the same. For that reason, and because Example 3’s data was originally based on coffee firm, not coffee product, analysis, Example 3’s Indicator calculations will not be repeated. Instead, this example focuses on prominent differences between the two LCIA approaches. 
B. Indicator Thresholds, or System Boundary and Resource Inventory

Several distinct differences arise in defining the system boundary and resource flows used to develop Resource Inventories between O-LCA and P-LCA. Organizations can have multiple ownership structures with varying degree of product ownership between business entities and facilities. They must further identify upstream and downstream resource boundaries where they still have product responsibilities. The following image (UNSETACd) demonstrates representative recommendations available for identifying system boundaries and defining “cradle-to-grave” product responsibility.
 

The following image (UNSETACd) summarizes the methodology organizations employ when developing Resource Inventories for O-LCA. UNSETACf provides several case studies showing how companies have carried this method out. Example 4 explains how budgets can be used as a sound initial step in identifying all supply chain Inputs and Outputs. Although the NPV tutorials point out that Operating Budgets must generally include all of an organization’s Operations and Inputs, and Outcomes and Outputs, UNSETACd explains that O-LCAs need not be as rigorous. For example, farm operations that don’t have major environmental impacts need not be included. UNSETACd advises “[f]ocusing resources based on significance can enable organizations to collect higher quality data for the priority activities in the value chain.” 

UNSETACd provides the following table to assist Small and Medium Enterprises (SMEs) apply O-LCIA according to their needs. Note that “activity data” is generally collected in DevTreks using the Operating Budgets and Capital Budgets explained throughout the tutorials. The TEXT datasets explained in this reference are introduced as more concise, flexible, integrated, complementary, alternatives that companies can use to “establish an enhanced environmental, [economic, and social,] collection system[s]”. Refer to the images in Section F.

Once an organization’s Inputs, Activities, Outputs, and Outcomes, have been identified and prioritized, they must then be converted to the physical resource flows needed in Resource Inventories. UNSETACd explains how to use the “emission and consumption factors” introduced in Example 3’s LCIA Impact Pathways, for that purpose. For example, in the case of multi-functional products, such as when an organization only purchases a portion of a supplier’s full production, Indicator.factor6 can be used to allocate the responsible portion in the LCIA.

C. Quality of Life Scenarios

The concerns raised in Example 3 about farm production heterogeneity complicating P-LCIA is exactly the same with O-LCIA. A latter section of this example will discuss the use of comparative LCIA to address this important issue.

D. Social Performance Indicators and Life Cycle Impact Assessment

UNSETACd confirms that Life Cycle Impact Assessment is conducted in the same way for products and organizations. They use the following image to explain that firms often include additional Indicators, such as Inventory Indicators (i.e. energy or water consumed), in the full LCIA. The additional Indicators assist them to manage their resources. The image explains why these types of Indicators must be reported separately from the “ISO-required” potential Damage Impact-related Indicators. The next section will make this point clearer.

The statement related to “single score Impact Indicators” is addressed in the Score section of Example 3 and in this example.



E. Communication and Interpretation

Indicators
The following image (UNSETACf) demonstrates how to communicate final O-LCIA results. It also demonstrates reporting ISO-required LCIA Damage categories separately from Resource Use and Inventory Level categories.

Scores and Hotspots Analysis
UNEP/SETACe describes how business organizations are using O-LCIA and Hotspots Analysis:
“Businesses are using hotspots analysis to focus their resources, drawing up action plans and practical programmes of work to eliminate, reduce or mitigate hotspots in their global value chains; and tackling major societal and commercial issues like food waste, food and resource security (future supply risk and resilience issues); and water use in agriculture.”
The following image (UNEP/SETACe) demonstrates that some O-LCAs employ normalized and weighted Indexes to generate final single score “ecopoints”.  Ecopoints correspond to the normalized and weighted Locational Indexes that this algorithm aggregates into the final Total Risk, or TR, Indexes. Example 3 demonstrated adding together each life cycle stage’s TR Indexes to produce the final Scores. This image demonstrates that the single score impacts support the 100% reference point from which this image’s Overall Ecopoints can be measured and communicated to stakeholders.



F. Decisions
This example addresses, in particular, the recommendations from the 12 road testers who proofed the O-LCA approach, related to “establishing an enhanced environmental [, economic, and social,] collection system[s]” and “setting reduction targets, tracking performance [and taking mitigation and adaptation actions]” as identified in the following image (UNEP/SETACf). 

This reference discourages companies from keeping O-LCIA results private. That is, by using the localhost or Intranet version of DevTreks, rather the cloud version. As mentioned throughout this tutorial, companies with a real concern about sustainability must also be concerned about transparency. Informed investors, supply chain participants, end-product consumers, and local communities, want evidence, not more marketing pabulum.
G. Resource Stock and M&E Analyzer Scenarios and Comparisons
The following image (UNSETACd) demonstrates how firms use O-LCIA to conduct Scenario Analysis.
 
The following image comes from the Resource Stock Analysis reference and demonstrates comparing the environmental performance of 3 different Scenarios, such as alternative mitigation and adaptation portfolios. Note carefully, that these comparisons, as with the previous image, are being made within an individual firm. 
Also note that numerous examples can be found throughout DevTreks (i.e. NPV 02, Construction Analysis, EVM, and CTA 01, references) of using Capital Budgets for the purpose of comparing Investment Scenarios (i.e. using the Change By Alternative Analyzers).

The following image also comes from the Resource Stock Analysis reference, but this time demonstrates making comparisons between representative organic and conventional farms (2*). 


The following image (UNEP/SETACf) explains a prominent difference in using Product and Organization LCIAs, and their related Hotspots Analyses: UNSETAC recommends that firms should not use O-LCIA and Hotspots Analysis to make comparisons between organizations. 


The image comparing organic and conventional production proves that this technology facilitates making organization comparisons –the question arises whether that approach is even sound. The tutorials throughout DevTreks argue, cautiously, that objective, science-based, 3rd party resource conservationists who use strong data standards applied using the RCA Framework can make consistent Organization LCA and Hotspots comparisons. Furthermore, the references used in this example suggest that companies might welcome having these high quality benchmark LCA standards and targets with which to compare their performance because their local communities, customers, supply chain buyers, and investors, demand this proof (2*). 
For example, the following image uses data from Example 3 and Example 3B to illustrate using O-LCIA to make comparisons. This example ran Output Stock Calculators for 2 separate base element Output Series which were added to 2 separate base element Outcomes. An Outcome Resource Stock Analyzer, explained in the Resource Stock Analysis reference, was linked to an Outcome Group, RCA Conv vs Organic Coffee Production, to make the comparison.  
 
The following data come the Score.MathResults from 2 Output Series, Conventional and Organic Coffee Production. The raw data readily supports comparisons based on Elementary Flows (i.e. factor4, bdp), Production Processes (factor6, not shown), or Life Cycle Stages (factor7, not shown).

H. Performance and Product Tracking
UNSETACd uses the following image to explain how companies use O-LCIA to also track their performance over time.

This reference originally planned to address “cradle-to-grave” product tracking with a separate example. Closer study concluded that it would mostly duplicate this example’s O-LCIA and Example 1 and 2’s techniques (i.e. refer to the previous section’s image comparing Conventional Orange production over 3 years). Dynamic product tracking is a separate issue that isn’t central, at this time, to this reference’s focus on “social performance analysis” (3*). 
Footnotes	
1. Although the UNSETACa and UNSETACc references in Example 3 present case studies for multi-national corporations, this example demonstrates that the same results are fully applicable to small-scale representative farms and SMEs. Although not demonstrated in this reference yet, the multi-national corporation examples can be replicated by a) using large Indicator data sets, or b) using all of the hierarchical base elements in DevTreks’ standard Operating and Capital Budgets (i.e. the Resource Stock Analysis reference demonstrates complementary techniques and examples that use the hierarchies).
2. Nemecek et al (2016) point out why multi-national company supply chain efficiency can’t be dismissed on ideological grounds but should be investigated on empirical grounds: “Alternative food systems such as community supported agriculture, food processing, and distribution directly from farms have gained popularity in the last decades and have been shown to have better resource efficiency in some cases (Markussen et al. 2014; Schramski et al. 2013) and were also proposed as solutions to reduce food waste (Caputo et al. 2014). However, compared to industrial large-scale production, such systems often lack efficiency, so that in many cases they have higher impacts on the environment than standard systems (Kulak et al. 2015).” The SPA3 reference points out that environmental efficiency must be balanced with socioeconomic efficiency (i.e. some cultures place high value on small, local, farms).
3. Previous statements still hold true about the importance of supporting “$28” contributions for open-source scientific software and criticizing public entities’ propensity for spending $multi-millions for proprietary software that serves the same purposes. The related issue of rent seeking carried out by special interest groups to get those $$ can begin to be addressed using the S-LCA techniques demonstrated in Example 3B and the consequential digital activism introduced in SPA3.

Case Study References

Example 3’s references, plus:

Nancy Auestad and Victor L Fulgoni. What Current Literature Tells Us about Sustainable Diets: Emerging Research Linking Dietary Patterns, Environmental Sustainability, and Economics. American Society for Nutrition. Adv. Nutr. 6: 19–36, 2015; doi:10.3945/an.114.005694.
Jan Landert, Christian Schader, Heidrun Moschitz and Matthias Stolze. A Holistic Sustainability Assessment Method for Urban Food System Governance. Sustainability 2017, 9, 490; doi:10.3390/su9040490
Christa Liedtke, Carolin Baedeker, Sandra Kolberg, Michael Lettenmeier. Resource intensity in global food chains: the Hot Spot Analysis. Wuppertal Institute, Germany. 2010
Thomas Nemecek & Niels Jungbluth & Llorenç Milà i Canals & Rita Schenck. Environmental impacts of food consumption and nutrition: where are we and what is next? Int J Life Cycle Assess (2016) 21:607–620 
Nathan Pelletier, Karen Allacker, Simone Manfredi, Kirana Chomkhamsri, Danielle Maia de Souza. Organisation Environmental Footprint (OEF) Guide. European Commission Joint Research Center. 2012 

Example 3B. Social Life Cycle Assessment (S-LCA) for Representative Coffee Production Stakeholders 
Version 2.1.8 and 2.2.0. These releases introduce an SDG Plan reference which provides a fuller context and additional examples for conducting S-LCA. Example 12 in that reference demonstrates how to use either subalgorithm15 or subalgorithm20 to conduct these analyses. The reference can be found in the Social Performance Analysis tutorial.
URLs:
Localhost:
Example 3A’s Conventional and Organic Outputs 
Cloud: 
https://www.devtreks.org/greentreks/preview/carbon/input/Coffee Firm RCA3 M and E/2147397561/none
A. Introduction or Goal and Scope
The following images (European Commission, 2012) summarize shortfalls in the Environmental Damage Impact emphasis of the LCIAs demonstrated in Examples 3 and 3A.
 
UNSETACa uses the following definition of S-LCA to address these shortfalls.

“A social and socio-economic Life Cycle Assessment (S-LCA) is a social impact (and potential impact) assessment technique that aims to assess the social and socio-economic aspects of products and their potential positive and negative impacts along their life cycle encompassing extraction and processing of raw materials; manufacturing; distribution; use; re-use; maintenance; recycling; and ?nal disposal. S-LCA complements [LCIA] with social and socio-economic aspects. It can either be applied on its own or in combination with [LCIA].”

The following image (UNSETACa) summarizes the major differences between S-LCA and Environmental LCA, or E-LCA (i.e. P-LCIA and O-LCIA). The functional unit in S-LCA relates primarily to stakeholders, not to product units or organization units. Impact categories are supplemented by stakeholder categories.



The following image (UNSETACa) summarizes how the Goal and Scope phase of S-LCA incorporates Stakeholder categories, Impact categories, and Subcategories, discussed in the previous image.


B. Indicator Thresholds, or System Boundary and Resource Inventory

The previous image and the following image (UNSETACa and UNSETACb) outline the process of defining a system boundary and Resource Inventory for S-LCAs. In relation to Example 3’s Damage Impact pathways, the “categorical pathway” displayed in these 2 images consist of “Stakeholder -> Impact categories -> Subcategories > Inventory Indicators”. 


This pathway is more of a classification scheme rather than the impact pathways introduced in Examples 1 to 3. UNSETACa describes the purpose of these classifications:

“The purpose of the classi?cation into impact categories is to support the identi?cation of stakeholders, to classify subcategory indicators within groups that have the same impacts, and to support further impact assessment and interpretation.”

Like O-LCIA, the Resource Inventory phase identifies and prioritizes an organization’s Activities, Outcomes, Inputs, and Outputs. For example, Inputs and Activities are needed to assess hours worked per week per employee. Subjective data about employee satisfaction, as collected with Output and Outcome data, is needed to assess employee welfare. The key requirement for conducting S-LCA is to relate company production processes with stakeholder impacts, as defined by S-LCA subcategories.
C. Quality of Life Scenarios

The concerns raised in Example 3 about farm production heterogeneity complicating P-LCIA are compounded in S-LCA. Social science tends to be more subjective and qualitative than physical science. For the current release, use the comparative Scenario approach discussed in Section F, Decisions, of this example. As explained in other examples, the S-LCA emphasis on measuring stakeholder impacts is particularly suited for tradeoff analysis. 
D. Social Performance Indicators and Life Cycle Impact Assessment

Examples 1 and 2 make use of a “nondiscrimination” Indicator in those datasets. Those socioeconomic indicators are analyzed in the context of “social impact pathways”, “results chains”, or “causal chains”, which use multiple Indicators in vertical pathways. In contrast, Example 3’s LCIA Damage Impact pathways use “horizontal impact pathways”. The overall goal of all algorithms in this reference is to use these pathways to integrate Social Performance Assessment, M&E, and Impact Evaluation. The quest is for cause and effect attribution (or, as explained by UNSETACa in Annex 2, consequence). 

The UNSETACa S-LCA calculations do not support the impact assessment techniques described in this reference for LCIA (i.e. impact pathways). UNSETACb explains the reason: 

“Because in S-LCA there is very little information regarding cause-effect chain models that would enable practitioners to aggregate results (characterization) in an accurate manner”. 

Recognizing this reality, this example’s S-LCA treats each LCIA Indicator as a Multi-Criteria Decision Analysis (MCDA) rating. Similar to the following MCDA scoring system introduced in Example 1, but adjusted for the LCIA properties:

DGAi = ?Nn=1 (IMni × ISn) / ?Nn=1 (IMni) 

where N is the number of Indicators per Categorical Index, 
i is the index of the Indicators, 
IMni is the Categorical Index-speci?c weight of an Indicator [1–3] = Indicator.factor10 and Indicator.factor11, 
ISn is the rating of an Indicator (0–100%) = Indicator.factor1 PRA calculation
Normalization type, or “weights” = Indicator.factor9
and factor5, factor6, factor7, and factor8 are not used (yet)

Version 2.2.0 will run this MCDA calculation when the CategoricalIndex.MathExpression for subalgorithms 15 or 20 begins with the prefix “mcda:”.

Advanced scientific networks may be capable of using Example 3’s horizontal impact pathways to conduct S-LCA. As explained throughout this reference, networks need to work with their information technologists to fine tune these approaches and to develop more advanced algorithms to address deficiencies with these algorithms. 

This example demonstrates how to convert the vertical impact pathways into the S-LCA MCDA scoring system. The following social Indicators repeat the vertical pathway introduced in Examples 1 and 2 (1*).

Vertical Pathway Part 1. Actions (or, in terms of results chains, Activities)
Indicator 1. 5.1.1 Degree to which legal frameworks are in place to promote, enforce and monitor equality and non?discrimination on the basis of sex (i.e. this comes from the SDG)
Indicator 2. Percent company supply sourced with SSIFs, small-scale food producers who are female and indigenous
Vertical Pathway Part 2. Conditions (or, in terms of results chains, Outputs)
Indicator 1. Percent employees filing sexual discrimination complaints per total number of employees
Indicator 2. 2.3.2 Percent SSIF coffee producers supplying company versus total SSIF coffee producers.
Vertical Pathway Part 3. Services (or, in terms of results chains, Outcomes)
Indicator 1. Percent employees satisfied with company discrimination policies
Indicator 2. Average income of small-scale food producers who are female and indigenous
Vertical Pathway Part 4. Impacts (the same for results chains)
Indicator 1. Percent employees leaving company per year due to discrimination enforcement
Indicator 2. Stability and quality of supply from SSIF sources

The following Indicators and properties illustrate a corresponding MCDA scoring system as introduced in Example 1. UNSETACs explains that qualitative and subjective Indicators derived from threshold systems, as introduced in Example 1, are appropriate metrics to use in these measurements. UNSETACb has a specific subcategory, Discrimination/Equal Opportunities, covering this topic. The SDG have several Indicators addressing both discrimination and sexual harassment. Although terms such as “percent workers” and “number of employees” can be used with these measurements, the metrics are usually based on MCDA-related qualitative rating scales and weights.

Indicator 1. Legal Frameworks: Degree to which sexual harassment legal frameworks are in place
factor1. Most likely quantity (and factor2 = low estimate or PRA, factor3 = high estimate or PRA)
factor4. Unit of measurement for the factor1 rating, DegreeSHL
factor9. normalization type: weights
factor11. Weight: 0.25
Indicator 2. Enforced Frameworks: Degree to which sexual harassment legal frameworks are enforced.
      factor1. Most likely quantity
      
factor4. Unit of measurement for the factor6 rating, DegreeSHLEnforced
factor9. normalization type: weights
factor11. Weight: 0.25
Indicator 3. Complaints: Percent employees reporting sexual harassment complaints per total number of employees	
      factor1. Most likely quantity

factor4. Unit of measurement for the factor8 rating, PercentEmployeeSH

factor9. normalization type: weights
factor11. Weight: 0.25
Indicator 4. Employee Satisfaction: Percent employees satisfied with company sexual harassment enforcement.
factor1. Most likely quantity
factor4. Unit of measurement for the factor10 weight, PercentEmployeeSHS
factor9. normalization type: weights
factor11. Weight: 0.25
Once the Indicators have been normalized, weighted and summed into a Categorical Index, the following Categorical and Locational Indexes use the exact same techniques demonstrated in Example 3. Footnote 3 discusses this example’s approach of combining normalized socioeconomic CIs with normalized environmental damage CIs. The CategoricalIndex.factor1, factor2, and factor3, are still treated as uncertain characterization factors that act as multipliers. This example uses the assumption that these characterization factors do not exist yet, and sets them equal to 1, but stills normalizes and weights the CI in the same manner as the sibling CIs (i.e. using a normalization multiplier).
Categorical Index 1 (LCIA Damage Category or S-LCA Subcategory). Workers at 100% capacity: Workers able to work at 100% capacity due to sexual harassment enforcement (2*). 
A more general S-LCA subcategory can be defined if the vertical impact pathway’s 2nd Indicator, related to small-scale, indigenous, female farmers, needs to be included in the same category (i.e. Effectiveness of Harassment/Discrimination Enforcement).
factor1. Most likely quantity
factor4. Unit of measurement for factor1, PercentWorkersatCapacity

Version 2.2.0 upgraded this algorithm by adding the prefix “mcda:” to Example 3’s CategoricalIndex.factor12, or MathExpression.
factor6: life cycle stage, production

[Version 2.2.0 deprecated this technique:
factor6: life cycle stage, productionslcia. S-LCA must add the characters “slcia” as a suffix to this property. That “tells” subalgorithm15 to use MCDA calculations, not LCIA calculations.]

Locational Index: In S-LCA, this corresponds to Stakeholder categories, such as “Workers”. UNSETACa defines this category as follows:
“A stakeholder category is a cluster of stakeholders that are expected to have shared interests due to their similar relationship to the investigated product systems.”
Very detailed S-LCA may want to use TR, or Total Risk Indexes, for this purpose as well. UNSETACa emphasizes the importance of relating Impact Assessment measurements to specific groups of stakeholders. In effect, that’s what distinguishes S-LCA from other types of assessments.
Indicator 7. Life cycle stage 1. Production LCIA Meta. The benchmark, target, and actual metrics shown in the following image reflect the final normalized, weighted, environmental damages and socioeconomic impacts, for this life cycle stage, or Indicator. Indicator7 was chosen strictly for testing purposes. 
  
     
  

Indicator7.URL TEXT dataset 
The following images verify that the data from Example 3 has been modified for S-LCA. When using a Threshold system, such as SAFA, to rate each Indicator, the data is actually scaled according to the threshold ratings (i.e. 0 to 100, 1 to 5).

An alternative is to put all of the damage indicators into 1 Indicator dataset and all of the socioeconomic indicators into a separate Indicator dataset (i.e. Indicator 2). 

Benchmarks. The following datasets confirms the S-LCA changes and the required property change: the row beginning with the label, ECA, is a Categorical Index, Workers at 100% capacity, with a factor6 property, or life cycle stage, of “production”. 

When Locational Indexes are used as S-LCA Stakeholder categories, as demonstrated by this dataset, tradeoff analysis focuses on differences, such as equity, among those stakeholder groups. Section F. Decisions, of this example will also demonstrate using the CIs alone to study stakeholder tradeoffs.



Targets
Not displayed -for testing same data as Benchmarks, but with _A suffix
Actuals
Not displayed -for testing same data as Benchmarks, but with _AA suffic

Indicator7.MathExpression
I7.Q1.factor1

Indicator7.MathResult
Benchmarks. The following results confirm that the socioeconomic Categorical Index, Working at 100% capacity, and its children Indicators are calculated using the MCDA formula, and all the environmental damage Categorical and Locational Indexes are calculated the same as regular LCIA. 



The Indicator.QTMost column for the row, IF5A. Legal Frameworks, is calculated as follows:

	Math Expression: “none” (don’t run mathematical calculations prior to normalization)

12. 5 = (50: Indicator.QTM * 2: weight) / 8: sum of Indicator weights

The CategoricalIndex.QTMost column for the row, ECA. Working at 100% Capacity, is calculated as follows:

	Math Expression: “mcda:Q1*Q2” (normalize and weight the children Indicators (mcda:) and then multiply the sum of the Indicators (Q1) by CategoricalIndex.QTM (Q2)

50 = 50: sum of Indicator.QTM for the 4 children Indicators * 1: CategoricalIndex.QTM * 0.25: normalization factor * 4: weight

E. Communication and Interpretation

Indicators
Although the references don’t provide good examples of communication aids that summarize the results of S-LCA, the following 2 images (The Sustainability Consortium, 2017) gives an example of the ingredients found in “toolkits” that use Indicators for P-LCIA, O-LCIA, and S-LCA for coffee suppliers. In terms of the images displayed in Examples 3 and 3A for this section, additional socioeconomic impact categories, or S-LCA subcategories, are added to the Indicator reports. The second image also demonstrates the use of quantitative measurements. Many “explanatory mixed methods” approaches combine quantitative and qualitative data –the dataset used in this example uses qualitative data for the S-LCA categories and quantitative data for the O-LCIA categories.



Scores and Hotspots Analysis
The following image (UNSETACe) confirms that LCA practitioners are going “beyond LCA” to more fully assess social, economic, and good governance, impact categories in LCA and Hotspots studies. In the context of the UN’s multi-sector SDG Sustainability Assessment Indicator system, the authors state: “Hotspots Analysis can be used to identify and prioritise actions for each of [the UNs Sustainability Development goals] at a product category / sector / city / nation or other level”.  Example 5 through 8 in SPA3 fully address the integration of SDG and sustainable accounting business data systems.




UNSETACe use the following statement to further confirm the role that S-LCA and Hotspots Analysis can play in fuller sustainability assessments. Annex 3 in that publication summarizes how more than 20 diverse organizations are applying those techniques (i.e. the National Cattlemen’s Beef Association Hotspots Analysis).

“The outputs from [S-LCA and Hotspots] analysis can then be used to identify and prioritise potential actions around the most significant economic, environmental and social sustainability impacts or benefits associated with a specific country, city, industry sector, organization, product portfolio, product category or individual product or service.”

The following image displays part of the Score.MathResult that will be used in the same manner as explained in Example 3 to conduct Hotspots Analysis. The S-LCA data was only added to the second Indicator, or Indicator 7.


F. Decisions
The following image derives from the O-LCIA comparison demonstrated in Example 3A, Section G.  The data illustrates how firms use both S-LCA and O-LCIA Hotspots Analysis data.to conduct Scenario Analysis for stakeholder tradeoff analysis. The Categorical Index, ECA8. Working at 100% Capacity, for Indicator 2 came from this example’s S-LCA dataset. The labels with “_A” and “_AA” now represent 2 additional stakeholder groups. The data supports the study of potential tradeoffs among 3 groups of stakeholders, 3 life cycle stages (Indicator 1, 2 and 3), and 2 production technologies (Organic and Conventional). Specifically, the analysis uses the S-LCA data to determine whether some of the technologies disproportionately impact some stakeholder groups more than others.


Example 5 in the Social Performance Analysis 3 reference, illustrates how to tie specific stakeholder populations, with socioeconomic attributes, to this subalgorithm’s stakeholder categories, or Locational Indexes. That example demonstrates how to use complementary subalgorithms to complete more thorough sustainability assessments.

Footnotes
1. Recent US news reports confirm that professionals in this field can develop better Indicators. For example, some of those professionals have mentioned the need to monitor the effectiveness of punishments metered out to transgressors.
2. In general, although many S-LCA Indicators are used for the purpose of changing “bad actor” behavior, they should be reported in positive terms because the goal is “good actor” behavior (and actual company use).
3. The legitimacy of combining normalized socioeconomic data with normalized environmental damage data is explained in the Lippiatt (2007) reference and by Landert (2017) and RAND (2016) in Example 1. MCDA allows normalization and weighting among “apples and oranges” LCA Indicators. The Liquete (2016) and Antioch et al (2017) references in Example 4B explain best practice techniques to employ when conducting professional MCDA. 

Case Study References

Same as Example 3, plus:

The Sustainability Consortium. Coffee Category Sustainability Profile. Version 03.10.001. Arizona State University and University of Arkansas. 2017



Example 4. Life Cycle Costs, or Benefits, (LCC or LCB) for Representative Small Scale Coffee Farms (RCA4)
Version 2.1.8 and 2.2.0. Version 2.1,8 introduced a new SDG Plan reference which provides a fuller context and additional examples for using LCC/LCB. Example 9 of that reference introduces new labeling requirements for these budgets. Version 2.2.0 introduced a new algorithm, subalgorithm21, for conducting LCC/LCB and refactored subalgorithm16 to make the 2 algorithms use consistent techniques. Example 12 changed this dataset by adding Adjusted Gross Living Wealth and Adjusted Sustainability scores. The SDG Plan reference can be found in the Social Performance Analysis tutorial.
URLs
https://www.devtreks.org/greentreks/preview/carbon/resourcepack/Coffee Firm RCA Example 4/1553/none
http://localhost:5000/greentreks/preview/carbon/resourcepack/Coffee Firm RCA Example 4/541/none

The cloud datasets used Inputs rather than Outputs in order to mix things up a bit.
Resource Stock Assessment
https://www.devtreks.org/greentreks/preview/carbon/input/Coffee Firm RCA4 Stock/2147397560/none
http://localhost:5000/greentreks/preview/carbon/output/Coffee Firm RCA4 Stock/2141223487/none
Monitoring and Evaluation Assessment
https://www.devtreks.org/greentreks/preview/carbon/input/Coffee Firm RCA4 M and E/2147397562/none
http://localhost:5000/greentreks/preview/carbon/output/Coffee Firm RCA4 M and E/2141223489/none 

Sample Coffee Farm Budget
https://www.devtreks.org/greentreks/preview/carbon/output/Coffee Crop Budget/2141223478/none
http://localhost:5000/greentreks/preview/carbon/output/Coffee Firm Crop Budget/2141223490/none

Sample Household Living Wage Budget
https://www.devtreks.org/greentreks/preview/carbon/output/Living Wage HH Budget/2141223482/none

A. Introduction

This example uses the RCA4 LCC/LCB algorithm to carry out a LCC and LCB of Example 3’s coffee farm. This algorithm can be used for stand-alone cost and benefit estimating or to add a cost and benefit dimension to the analyses demonstrated in other Examples used throughout this reference (i.e. as demonstrated in Example 4A, 4B, and 4C). 

In order to add a social performance assessment perspective to this algorithm, the following budgets have been completed, or at least demonstrated, for this example:
1. Coffee Farm Budget – agricultural cost of production study
2. Household Living Wage Budget – household budget study
3. Coffee Supply Chain Social Costs Budgets – social cost of supply chain example
4. Compliance Cost Effectiveness Analysis – see Example 4A
5. Country or Sector Cost Effectiveness Analysis – see Example 4B (WHO, 2016, explains how to use cost estimates and budgets at country scale).
6. LCIA Cost Effectiveness Analysis – see example 4C
 
B. Indicator Thresholds

Given the agricultural context of this example, many agriculturalists may recognize that the underlying 4 Level Indicator hierarchy readily supports the standard 4 level budgeting data structure, or results chain, used for Operating/Capital Budget analysis in DevTreks. Level 4 supports Inputs and Outputs, Level 3 supports Outcomes/Operations/Components, Level 2 supports Time Periods, and Level 1 supports Budgets. Again, that’s a perfectly fine data structure to use –provided that it’s part of an underlying WBS. 

C. Quality of Life Scenarios

These stakeholders derive the majority of their income from coffee farming. Their quality of life depends on the net income they derive from their company’s revenues and costs. Agricultural advisers are eager to assist their customers’ clubs desire to understand the impacts of their advice in terms of on-farm, and off-farm, costs, benefits, and performance.
Example 4A and 4B demonstrate typical uses of scenarios.
D. Social Performance Score

The following image and tables show that Indicator TEXT datasets store this firm’s initial Scoring system. To reinforce the LCC/LCB techniques introduced in the Life Cycle Calculation reference, this dataset uses that reference’s same “Examples”. We recommend sticking with a standard WBS for all of resource accounting. The goal of the WBS is uniform accounting, aggregation, and communication. This dataset holds the same 5 LCC examples as found in the Life Cycle Calculation reference.

The Indicator.URL TEXT datasets used with this algorithm have columns defined as follows. Although the algorithm doesn’t care about property names, DevTreks recommends using generic property names.

For Indicators, the final 11 columns of data are defined as follows:
* factor1. Most likely quantity of Indicator. 
* factor2. Low quantity of Indicator.
* factor3. High quantity of Indicator. 
* factor4. Price of Q
* factor5. Unit of measurement for final LCC cost or LCB benefit (P * Q). 
* factor6: Type of escalation or discounting as defined in the next paragraph
* factor7: Either an escalation rate used with the escalation type, or a multiplication factor representing a pre-discounted value
* factor8: number of years to use in discounting formulas
* factor9: recurrent times
* factor10: certainty1 of Indicator (same as Example 1’s certainty1 column)
* factor11: certainty2 of Indicator (same as Example 1’s certainty2 column)
For Categorical Indexes, the final 11 columns of data are defined as follows:
* factor1: see Example 4A and 4B (0 in this example). Setting factor1 and factor4 equal to zero tells this algorithm to calculate Categorical Indexes as summations of children Indicators. Otherwise, this algorithm will assume that the cost effectiveness analysis (CEA) demonstrated in Example 4A and 4B is being run. When CEA is run, performance indicators (QASYs) are discounted exactly the same as costs and benefits.
* factor2: see Example 4A and 4B (0 in this example).
* factor3: see Example 4A and 4B (0 in this example). 
* factor4. see Example 4A and 4B (0 in this example)
* factor5. see Example 4A and 4B (none in this example)
* factor6: general multiplier, or allocation factor, applied to sum of Indicators (equivalent to OC.Amount in Operation/Component/Outcome base elements)
* factor7: Planning Construction Years: Number of years in the planning and construction period. Also known as the preproduction period. 
* factor8: Service Life: The life span of the Input or Output. 
* factor9: Years from Base Date: The base date is the Input or Output’s date. The specific year within the Planning Construction Years when the Input is installed or Output’s revenue received. 
* factor10: Real discount rate (in digital format: 2.125)
* factor11: Nominal discount rate

Several references (i.e. WHO 2003 in Example 4A) demonstrate using general demographic variables (i.e. age, education, gender, and race), applied using population models, with aggregated budgets. Example 5 in the Social Performance Analysis 3 reference illustrates how to integrate population algorithms with budget and performance algorithms. The Social Performance Analysis 4 tutorial demonstrates this further.

Escalation Type Options. The following options, which are defined in the Life Cycle Calculation reference, can be used for this column. Examples 4A and 4B’s cost effectiveness analyses derive from techniques used in the health care industry. That industry is particularly concerned about the impacts of price escalation on health care affordability. 
* none = no escalation, 
* upvtable = use the factor1 property as a multiplier (the factor has been pre-calculated from a discounting formula)
* spv = single present value, 
* upv = uniform present value (based only on discounting years), 
* caprecovery = capital recovery or amortized, 
* caprecoveryspv = uses the Years from Base property to run a spv calculation and then uses the Service Life plus Planning Construction years to run the caprecovery calculation.
* uniform = uniform escalation (same techniques as NIST, including use of service years and planning construction years), 
* geometric = geometric escalation (same techniques as NIST, including use of service years and planning construction years), 
* linear = linear escalation (same techniques as NIST, including use of service years and planning construction years), 
* exponential = exponential escalation (based only on discounting years), 
* eaa = equivalent annual annuity


The formulas documented in the Life Cycle Calculation and NIST 135 references are the exact same functions used with this algorithm with 1 exception. These TEXT datasets do not have a separate property for Salvage Value. Instead, Salvage Value is treated as a separate discounted cost. The current release supports these calculations for all Indicators. 

1. Life Cycle Calculation Reference Budget

Indicator 1. LCC Meta (the scores are filled in automatically)
  
 
Indicator1.URL TEXT dataset

Benchmarks

Target A [for testing same data as Benchmarks]


Indicator1.MathExpression
I1.Q1.factor1 + I1.Q2.factor2 + I1.Q3.factor3 + I1.Q4.factor4 + I1.Q5.factor5 + I1.Q6.factor6 + I1.Q7.factor7 + I1.Q8.factor8 + I1.Q9.factor9 + I1.Q10.factor10 + I1.Q11.factor11

Indicator1.MathResult
Benchmarks
These mathematical results differ slightly from the Examples shown in the Life Cycle Calculation reference because of rounding (i.e. Example 2’s $544,983 compared to NIST’s $545,035) and because those calculators treat Salvage Value differently. In the context of DevTreks, the first 4 of these examples correspond to Capital Budgets while the 5th applies to Operating Budgets.


Target A


2. Coffee Farm Budget Math Result (1*)

This following farm budget comes from the Fleming et al (1998) reference. No WBS was available for classifying this data –social networks must provide uniform guidance to all of their clubs before these budgets can be used for professional work. The Fairtrade International (2011) reference provides further guidance for developing agricultural cost of production studies applicable to “sustainability reporting”. The latter organization requires adding a $0.30/pound premium to the revenue line items to account for their certification requirements (as of June, 2017). 

The objective of most agricultural cost of production studies is to derive a final Net Return per Hectare calculated by subtracting expenses from revenues. This algorithm can’t distinguish expenses from revenues –the algorithm considers them be generic cost or benefit indicators that get cumulatively summed up the hierarchy chain. The following example demonstrates using negative prices for the cost items, which means their costs will be correctly subtracted from revenues in the final Net Return calculation. A positive Net Return signals profits, a negative Net Return signals losses.

Indicator1.MathResult


3. Coffee Farm Household Living Wage Math Result (2*)

The Global Living Wage Coalition (ERC, 2017) defines living wages as follows:

“Remuneration received for a standard work week by a worker in a particular place sufficient to afford a decent standard of living for the worker and her or his family. Elements of a decent standard of living include food, water, housing, education, health care, transport, clothing, and other essential needs including provision for unexpected events.”

ERC (2017) calculates living wages using household budget data based on “a nutritious low-cost diet, basic acceptable housing, […] non-food non-housing costs, unforeseen events, and statutory payroll deductions and taxes.” They use the following images to summarize these components of a living wage. Anker (2011, 2013) developed this methodology because of shortfalls relying solely on secondary household survey data for household budgeting. [The Ankers’ 2017 reference charges fees, which has limited usefulness in the public goods context of DevTreks.] 


The following household budget TEXT dataset calculates monthly living wages for Vietnamese seafood industry workers. The general methodology is applicable for any household, including farm household living wages (4*). The Anker (2011, 2013) and ERC references (2017) explain the calculations in depth. This budget had to alter some of their methods, especially wage adjustments, to account for the cumulative sums accruing in these budgets. The Sustainable Food Lab (2016 in Example 1) and Shipman et al (2016) demonstrate techniques to apply when this type of household is also the farmer (4*).
Benchmarks


Actuals (partially displayed; testing purposes used benchmarks, but with only 2 digits of accuracy in the spreadsheet used to generate the TEXT dataset; Example 3 shows that although Version 2.1.2 now supports scientific notation, this algorithm was not upgraded because very few budgets use that data format)


Indicator.MathResults - Benchmarks


Indicator.MathResults – Actuals (2 digits of accuracy makes a difference when the magnitude of numbers is this high)


4. Coffee Supply Chain Social Costs Budget

The following image (True Price, 2017) summarizes household budgets for Ugandan coffee producers. 



The following images (IDH and True Price, 2017) summarize household and supply chain budgets for coffee production. The first image demonstrates how this organization calculates the social costs, such as water pollution and child labor, associated with agricultural products. The second image demonstrates how they determine social costs for full supply chains (3*). Loconto et al (2014, in Example 1) also discuss using supply chains to figure out which supply chain participants benefit from compliance with certification standards, and by extension, the credibility of the claims made by the standard bearers (i.e. producers capture most benefits).



These studies did not include the underlying raw data which explains why no budgets have been prepared. This reference tries to convey the message that raw, standardized, TEXT datasets, referenced with URIs, should always be included in Social Performance Assessments. That way, authors of these studies don’t have to keep using terms such as “futile” (10*).

Optional Indicators 2 to 8. Additional Life Cycle Stages

Example 3 explained that Version 2.1.0 was upgraded to permit subalgorithm15 to be used with Indicators 1 to 15 so that up to 8 life cycle stages could be evaluated for supply chains. Subalgorithm16 has also been upgraded for that purpose, or any other purpose. For testing purposes, the same data used in Indicator 1 produced the following result for Indicators 2 and 3. 



E. Communication

The firm uses the following types of communication aids to help managers interpret the raw data.



F. Decisions

The coffee firm mostly uses LCC/LCB to support on-farm financial and economic decision making, such as calculating the return on investment of new coffee plantations (i.e. Capital Budgets), or the projected profitability in the coming year (i.e. Operating Budgets). The first 4 cost estimates in the “NIST” example demonstrate the former. The coffee farm budget (Fleming et al, 1998) demonstrates the latter decision support.

Resource conservationists use the “true price”, or social cost of production, studies, to help farmers receive fair compensation for adoption of improved mitigation and adaptation conservation practices. If needed, they also use those budgets to make the consequences of bad actor behavior fully transparent to the impacted communities, investors, supply chain buyers, and end product consumers. They use the household living wage budgets to assist public entities carry out social protection actions that target rural households.

Footnotes
1. The author has completed several hundred farm budgets, using a variety of techniques, but prefers the LCC/LCB methodology, especially the basic, standardized, RCA TEXT datasets.
2. The author has completed several hundred household budgets, mostly for farmworkers, using much simpler techniques, but prefers the Anker methodology, especially applied with the basic, standardized, RCA TEXT datasets.
3. Social networks are assumed to have enough knowledge of economics, or access to someone who does, to understand the difficulties of fully monetizing social benefits and costs (i.e. especially using conventional approaches). Hence Examples 4A. 4B, and 4C.
4. Version 220 discovered the Voorend et al. Guatemalan coffee household budget. Please use that budget to understand smallholder farm budget.

Case Study References

Richard Anker. Estimating a living wage: A methodological review. Conditions of Work and Employment Series No. 29. International Labor Office. 2011 
Emily Shipman (Sustainable Food Lab), Gabriela Soto (COSA), Jessica Mullan (COSA), Marta Maireles González (ISEAL Alliance), and Stephanie Daniels (Sustainable Food Lab). Measuring guidance document of the Committee on Sustainability Assessment (COSA), the ISEAL Alliance and the Sustainable Food Lab. Version 1.0. October 2016

Fairtrade International. Guideline for Estimating Costs of Sustainable Production. 2011

IDH and True Price. Joint report by IDH (Sustainable Trade Initiative) and True Price. The True Price of Coffee from Vietnam. 2016 

Kent D. Fleming, H. C. “Skip” Bittenbender, and Virginia Easton Smith. The Economics of Producing Coffee in Kona. Cooperative Extension Service. University of Hawaii at Moanoa. 1998 (finding a more recent, complete, coffee crop budget proved “futile” 10*)
Marianna Lena Kambanou, Mattias Lindahl. A Literature Review of Life Cycle Costing in the Product-Service System Context. ScienceDirect, December, 2016
Research Center for Employment Relations (ERC). Living Wage Report Rural Vietnam Soc Trang to Thai Binh Context Provided in the Seafood Processing Industry. Series 1, Report 11. Prepared for: The Global Living Wage Coalition June 2017  
TruePrice. Assessing coffee farmer household income. Executive Summary. (last accessed July, 2017: www.trueprice.org)
UNEP/SETAC. Guidelines for Social Life Cycle Assessment of Products. 2009 (primarily covers SLCA but also includes a section on LCC –they published a reference on this subject in 2011 but charge fees which means the reference has limited usefulness in the public goods context of DevTreks)
U.S. Department of Commerce, National Institute for Standards and Technology. Handbook 135, Life-Cycle Costing Manual. 1996 Edition. 
U.S. Department of Energy. Life Cycle Cost Handbook. Guidance for Life Cycle Cost Estimation and Analysis. Office of Acquisition and Project Management U.S. Department of Energy Washington, DC, 2014
U.S. Government Accountability Office. Applied Research and Methods. GAO Cost Estimating and Assessment Guide. Best Practices for Developing and Managing Capital Program Costs. March, 2009.
U.S. General Services Administration. 1.8. Life Cycle Costing. https://www.gsa.gov/portal/content/101197 (last accessed May, 2017)
Koen Voorend, Richard Anker and Martha Anker. Living Wage Report. Rural Guatemala Central Departments: Context Provided in the Coffee Sector. Series 1, March 2018. First released 2016.

World Health Organization. Strategizing national health in the 21st century: a handbook. 2016. Chapters 7 and 8 cover cost estimating and budgeting at national scale.


Example 4A. Coffee Farm RCA4 Compliance Cost Effectiveness Analysis (CEA) (RCA5)
URLs
https://www.devtreks.org/greentreks/preview/carbon/resourcepack/Coffee Firm RCA Example 4A/1555/none
http://localhost:5000/greentreks/preview/carbon/resourcepack/Coffee Firm RCA Example 4/541/none

Resource Stock Assessment and Monitoring and Evaluation Assessment

https://www.devtreks.org/greentreks/preview/carbon/output/Coffee Firm RCA4A Both/2141223479/none

http://localhost:5000/greentreks/preview/carbon/output/Coffee Firm RCA4A Both/2141223493/none

A. Introduction

This example uses the RCA4 (LCC/LCB) algorithm to carry out a Cost Effectiveness Analysis (CEA) for a representative coffee farm. This example analyzes the cost effectiveness of complying with certification programs. It also illustrates how to “harmonize” different sustainability WBSs. In this example, typical crop budget WBSs will be harmonized with an agricultural sustainability WBS, GLOBALG.A.P., summarized as GG in this example, which certifies the social soundness of agricultural buyers and producers. 

The following techniques will be demonstrated simplistically in this example:
1. Harmonized Indicator Systems: The GG process allows alternative certification Indicators, with corresponding proof of compliance, to be written next to the GG “Indicators” on a spreadsheet checklist. This example uses a modification of Example 4’s crop budget as the alternative scheme. The point is not to endorse 1 standard system over another, but to understand real, alternative, ways to harmonize Indicator systems and apply different algorithms.
2. Crop Budgets and Cost Effectiveness Analysis (CEA): Unlike Example 4, where the point was to understand crop Net Returns, Return on Investment, and household Gross Adjusted Income, the point of this example is to understand basic cost effectiveness analysis. Example 4’s results are used as the numerator in the CEA formula, cost or benefit per unit of performance.
3. Compliance Scenarios and Incremental Cost Effectiveness Ratios (ICERs). Example 1, 2, or 3’s results are used as the denominator in the CEA formula, resulting in a total cost or total benefit per unit performance score. Subalgorithms 13 and 14 calculate TR Indexes as averages, while subalgorithm15 uses totals. Example 4’s final Net Returns are replaced with Net Return per Unit (Average or Total) Performance. Scenarios use ICERs to demonstrate the cost effectiveness of complying with different degrees of certification compliance, or social performance interventions.

The CEA references used with this Example come from the health care industry because they apply CEA very practically, seriously, and with full decision support context. In terms of agricultural sustainability, the Sustainable Food Lab (in Example 1, 2016) discusses CEA as follows: “Alongside the increased investment in smallholder sourcing is interest—and even urgency—in finding cost-effective approaches to better understanding farm-level conditions and sustainability”.

B. Indicator Thresholds

GG is an accreditation organization, based in Germany, which certifies the social soundness of food buyers and producers. They (2016) describe their farm production certification process as follows:

“GG is the internationally recognized standard for farm production. By complying with a single harmonized global standard for safe and sustainable food production, producers can show their commitment to Good Agricultural Practices [G.A.P.].”

The following image (GLOBALG.A.P., 2017) shows part of the certification assessment spreadsheet, or checklist, completed by farm companies and verified by trained assessors (i.e. Resource Conservationists). The assessments define Control Points and associated Compliance Criteria, which producers must comply with for certification. In practice, the buyers of agricultural products require these types of certifications from their farm producers. Many of the buyers also supervise overall certification compliance. Separate checklists are available for all farms, all crop production, fruit and orchard “subscope”, aquaculture subscope, tea subscope, and food processing firms. 

The SAFA system refers to these types of Indicators, or Criteria, as “Practice Indicators” because they don’t measure performance directly, such as quantity of GHG emissions. Instead, performance is assumed to be the result of the best management practices defined by the Indicators, such as proof that the manager is keeping records proving energy efficiency. In addition, the majority of the Indicators, or Compliance Criteria, focus on food safety and worker welfare.


GG’s checklists use 3 thresholds –Major Must that require 100% compliance, Minor Must whose Criteria must sum to 95% compliance, and Recommended which must be included in the assessment but not rated.  These thresholds have been replaced with the SAFA 5 threshold and scoring system demonstrated in Example 1 and 1A. The top SAFA threshold, best, is the only threshold that actually complies with GG. The GG recognizes that many producers already use other types of “certification schemas”, such as SAFA. Their spreadsheet checklist includes separate columns that allow the existing certification Indicators to be listed next to their respective GG Indicators. 

A full example of standards harmonization should employ the techniques demonstrated in Example 1A –showing the GG Criteria next to their related SAFA Indicators. That technique will not be demonstrated again because the goal of this example is to demonstrate cost effectiveness by using Example 4’s budgets with the performance data from subalgorithms 13, 14, or 15.

In fairness, the GG is not a true WBS nor is it used as a complete scoring system. WBSs typically have both names and descriptions for Indicators. GG uses descriptions but not names. Their descriptions have been truncated to 100 characters for the purpose of using a name in this example’s TEXT datasets. They use duplicate labels for both Control Points and Compliance Criteria, which were retained, but would be typically handled by making the Criteria children Indicators of the Control Point Categorical Indexes. Some of their labels were changed slightly to accommodate DevTreks character length conventions (2, 3, or >=4).
GG also supports the social, cultural, human, and institutional capitals with a separate checklist, the Risk-Assessment on Social Practice (GRASP, 2016). This example includes that checklist in the full WBS and treats it like another TR Index. The following image (GRASP, 2015) demonstrates how South Africa has interpreted that checklist for the needs of their country. Example 4B explains the pros and cons of including these type of “context-specific” Indicators in CEA –in many cases, the results of CEA can be so demonstrably efficient, that some countries are better off not making CEAs overly dependent on single locations, or single companies.


The following 2 images illustrates a GG – SAFA Indicator Thresholds system. The top image shows complete Compliance Points and Certifications. The second image demonstrates actual GG TEXT datasets that could be used directly in other Social Performance Assessments demonstrated in this reference. These TEXT datasets can be found in the URLs.




The following image demonstrates using the GG process directly to harmonize their Compliance Points and Certifications WBS with a WBS for crop budgets. 


This table points to at least two ways for relating RCA4 budgets to RCA1, RCA2, and RCA3, Social Performance Assessments. The first is to complete a partial budget, completing revenue and cost “line items” for every category in the Assessment. That’s feasible, especially if hourly labor costs are recorded for the compliance work, but probably overkill. 

The second is to reorganize the GG WBS into the typical capital categories demonstrated throughout this reference and then use a lump sum partial budget to assess the aggregated GG Indexes. The second approach may be more practical, because administrative expenses, such as record keeping and professional administrative services, are usually lumped together in farm budgets. The lump sum expenses can be allocated to different Indexes by using the Categorical Index.factor6 property. This example demonstrates the second approach. This approach ignores Loconto et al’s (2014, in Example 1) conclusion that, in practice, compliance costs are often paid by buyers, not producers.

C. Quality of Life Scenarios
Agricultural advisers are eager to assist their customers’ clubs to understand the incremental, or marginal cost and benefits, of achieving greater certification compliance levels. Specifically, will increased revenues or decreased costs, offset compliance costs? Will reduction in social costs result in higher price premiums because of supply chain buyer or end product consumer satisfaction? The sibling Performance Analysis 1 reference refers to this analysis technique as Incremental Cost Effectiveness Analysis or Incremental Cost Effectiveness Ratios (ICERs).
The following 4 Scenarios are evaluated for 2 separate Indicators. The first Indicator will focus only on changes in costs, the convention for most CEAs. The second Indicator adds changes in revenues along with the changes in costs. Although these scenarios suggest that it’s possible to precisely judge the effect of compliance on benefits and costs, in practice, precision is unlikely. The datasets used with these scenarios reflect the imprecision.
Scenario 1 or Alternative 1: Benchmark, or No Certification, Costs and Benefits: many current certification requirements have limited or unacceptable ratings. 
Scenario 2 or Alternative 2: 50% Compliance, Certification Costs and Benefits: all certification requirements are completed to at least the 50% Threshold Level, or moderate rating. 
Scenario 3 or Alternative 3:  70% Compliance, Certification Costs and Benefits: all certification requirements are completed to at least the 70% Threshold Level, or good rating. 
Scenario 4 or Alternative 4: 100% Compliance, Certification Costs and Benefits: all certification requirements are completed to at least the 100% Threshold Level, or best rating. 
The terms “Alternative” are also used because formal Project Analysis often study the cost effectiveness of project alternatives. The Alternatives are usually proposed as part of an overall decision support process explained in Appendix A, Part I, Decision Making Processes and Valuations. The PR&G references in SPA1 demonstrate that federal agencies often complete these types of assessments (i.e. the author has completed them). This example displays the results of a simple cost effectiveness for each scenario. 
The analyst has to then use that data for more advanced decision support. WHO (2003) describes the general CEA approach that will be used in Examples 4A and 4B, as follows (1*): 
“The approach of generalized CEA (GCEA) proposed in this Guide seeks to provide analysts with a method of assessing whether the current mix of interventions is efficient as well as whether a proposed new technology or intervention is appropriate. It also seeks to maximize the generalizability of results across settings.
… GCEA proposes the evaluation of interventions against the counterfactual of “doing nothing”, thereby providing decision-makers with information on what could be achieved if they could start again to build the health system, i.e. reallocate all health resources.”
D. Social Performance Score

Example 4 demonstrated that subalgorithm16 employs 22 generic factor properties stored in the Indicator and Categorical Index TEXT data rows. This example demonstrates how to use the first five properties of the Categorical Index to carry out Cost Effectiveness Analysis (CEA). 

The first 5 Categorical Index columns of data are defined as follows. 
* factor1: Most likely quantity of the corresponding CategoricalIndex.QTMost from either subalgorithm 13, 14, or 15. Setting factor1 and factor4 equal to zero tells this algorithm to calculate Categorical Indexes as summations of children Indicators as demonstrated in Example 4. Otherwise, this algorithm will assume that the cost effectiveness analysis (CEA) demonstrated in Example 4A is being run and will calculate Categorical Indexes by dividing the calculated LCC/LCB Categorical Index.QTM, by factor1. When CEA is run, performance indicators (QASYs) are discounted exactly the same as costs and benefits. 
* factor2:  Low quantity of the corresponding CategoricalIndex.QTLow from either subalgorithm 13, 14, or 15. 
* factor3:  High quantity of the corresponding CategoricalIndex.QTUp from either subalgorithm 13, 14, or 15.
* factor4:  Certainty of Categorical Index from the corresponding Categorical Index certainty score, or certainty calculation, from either subalgorithm 13, 14, or 15. 
* factor5:  Unit of measurement for the corresponding Categorical Index.QTM from either subalgorithm 13, 14, or 15. 

Indicator1.URL TEXT dataset (using partial budget)
The purpose of Indicator 1 is to demonstrate traditional cost effectiveness where costs, and costs alone, are divided by performance scores.
Benchmarks

Alternatives

Indicator2.URL TEXT dataset (using partial budget)
The purpose of Indicator 2 is to demonstrate nuanced cost effectiveness where revenues, costs, and net returns, are factored into the cost effectiveness analysis. With this Indicator, the second Categorical Index property, factor6, is a general multiplier = 0.5. Oftentimes, “lump sum” farm budget expenses, especially administrative expenses such as record keeping, must be allocated in a similar manner to specific enterprises, or in this case, Categorical Indexes within enterprises. It’s less typical to allocate revenues, as done here.
Benchmarks

Indicator 2 Alternatives (partial display, AA = scenario 2, AB = scenario 3, and AC = scenario 4)

Math Expressions
Same as Example 4.

Math Results
The MathResults display cost effectiveness ratios in the final 3 columns for each hierarchical Index. These ratios are calculated as follows:
1. Categorical Indexes:  The Categorical Index LCC or LCB monetary sum, demonstrated in Example 4, will be divided by the Categorical Index factor properties (factor1, factor2, and factor3) to derive a cost or benefit per quantity of factor. The resultant cost effectiveness calculations support decisions associated with “degree of compliance” or degree of social performance. Simplistic, but a starting point. Unlike the Locational and TR Indexes, the columns QTMost2, QTLow2, and QTHigh2, do not display the Total Costs used as the numerator in their CEA ratios because 1) the base data in those columns, such as allocation multiplier, is more important, and 2) eyeball, or calculator, Total Costs estimates can be easily made by summing the Indicator Total Cost data.
2. Locational Indexes: The sum of the Categorical LCC or LCB calculations will be divided by the sum of the Categorical Index factors, to derive a cost or benefit per quantity of factor. Subalgorithms 13, 14, and 15 normalize and weight all of the Indicators within a Locational Index, not for each separate Categorical Index, supporting “apples with apples” Indicator assessments.
3. Total Risk (TR) Indexes: Uses the same techniques as Locational Indexes, except uses all of the Categorical Indexes within a Total Risk Index. Given that each Locational Index uses separate normalized Indicator values, this row is suspect –the Indicators in one Locational Index have been normalized differently than the Indicators in all sibling Locational Indexes (i.e. “apples with oranges”). For convenience, this example ignores that possibility, but Example 4B will introduce an approach for handling this issue.
4. Final CEA Ratios: In the following images, the TR Index column (last) displays the sum of the Categorical Index performance scores in the factor1, factor2, and factor3 columns. The QTMost2, QTLow2, and QTHigh2 columns display the sum of the Indicator QTMost, QTLow, and QTUp, LCC/LCB calculations. The column QTMost is the CEA ratio calculated: Indicator 1: 1.42 (QTMost) = 520 (factor7) / 365 (factor1), and Indicator 2: 3.35 = 1423.3 / 425. The column, factor4, is the average certainty score for the Categorical Index performance scores. The columns, factor10 and factor11, are the average certainty scores for the Indicator LCC/LCB calculations. The combination of certainty scores are used to assess the uncertainty of the CEA ratios.

Indicator 1.MathResult
Benchmarks (better data, or better data assumptions, will result in better Most, Low, and High ratios)


Indicator 2.MathResult
Benchmarks

The following image displays Indicator 1. The Benchmark scores come directly from the previous image’s TR Index row. The Actual scores derive from the alternative with the lowest CEA ratio in the TRIndex.QTMost column. As will become clear in the next section, when Indicator 2’s Revenues are included in the analysis, the alternative with the lowest CEA ratio may not be the best one. Rather than lower CEA ratios reflecting lower cost per unit, higher CEA ratios reflecting higher net returns per unit are desired.

Indicator 1. CEA Meta (the scores are filled in automatically)
  

E. Communication 

HIQA, Ireland (2010) recommends: “the results for cost-utility analysis should be presented as incremental cost effectiveness ratios (ICERs). ICERs present the cost per unit of outcome, e.g.  the expected additional total cost to the expected additional QALYs (LYG) [, or Performance Indexes,] and are calculated as follows: 

ICER =   (cost A- cost B) (outcome of A-outcome of B) 

As the ICER becomes larger, the intervention is said to be less cost effective. Where more than two technologies are being compared, the results should be reported in tabular form, presented in the order of increasing costs. Technologies that may be excluded on the basis of simple dominance (they are more costly and less effective than the alternatives) are eliminated from further calculations. The initial ICER should then be calculated by comparing each programme with the one above it, excluding those programmes that are dominated. The final ICER is then calculated after eliminating technologies that are subject to extended dominance (other alternatives available that are more effective and more costly, but provide better value for money as identified by the initial ICER).”

The following image (HIQA, 2010) demonstrates an example.



They recommend the following types of graphics to communicate the results of Cost Effectiveness Analysis: 
 
* “cost-effectiveness plane to present the incremental costs and effects of two (or more) comparator technologies 
* tornado diagrams to display the results of subgroup effects and one-way sensitivity analysis  
* scatter plots to present ICER results from probabilistic sensitivity analysis of two comparator technologies on the cost-effectiveness plane  
* cost-effectiveness acceptability curve to present the probability that a technology is more cost-effective than its comparator. In a study comparing more that two technologies, it should present the probability that a technology is the most cost-effective as a function of the threshold willingness to pay for one additional unit of benefit.”

Ireland HIQA (2010) provides the following example of a cost effectiveness plane.


The following 2 analyses apply these techniques. Scenario 1 is considered the No Certification Compliance benchmark from which Alternatives AA, AB, and AC are compared. The Locational Index CEA results (PC, EC, and HC) have been included in the dataset but will not be used in this example. They are used in more advanced analysis to better understand the “drivers” of cost and performance (i.e. which capitals are most cost effective? which least? why?). The health care industry’s cost utility analysis approach, based on QALYs and DALYs, rather than this example’s Performance Impacts, will be fully addressed in Example 4B.

Indicator 1. Costs Only (2*)
The Indicator1.MathResults were rearranged in a spreadsheet to produce the following table. The objective is to select low ICERS with low costs per unit performance. 



The cost effectiveness plane reflects the least cost combination of alternatives. In the following graphic, the horizontal axis, with zero costs, is interpreted to be the most efficient frontier. At first glance, TR_AB appears dominated because its slope starts to increase, relative to TR_AA and TR_AC.



WHO (Hutubessy, 2003) discusses in detail why the No Certification alternative has to be included in these analyses: “it can identify current allocative inefficiencies as well as the efficiency of opportunities presented by new interventions”.

Indicator 2. Revenues Included
The Indicator2.MathResults were rearranged in a spreadsheet to produce the following table. The objective is to select high ICERS with high net returns per unit performance. Alternative AC has the highest Net Return ICER and may be the most cost effective alternative. The next section discusses using the low and high estimates and the certainty scores to base final decisions. 



The cost effectiveness plane reflects the point that both higher net returns and higher performance scores are desired. A line bisecting the graph from origin to top right corner is interpreted to be the ideal efficiency frontier (i.e. highest performance and highest net returns). In the following graph, TR_AC has the best slope in relation to the efficiency frontier and appears to dominate TR_AA and TR_AB. If the budget permits “2 alternatives”, TR_AB appears preferable over TR_AA because its slope increases more.



F. Decisions
Decision making focuses on the risk and uncertainty associated with the ICER ratios presented in the previous section. These analyses accommodate uncertainty with 3 metrics: 
1. high and low cost and performance estimates, 
2. average certainty (factor4) rating for the performance scores which derives from 2 uncertainty ratings in the base Social Performance Analysis, 
3. average certainty1 (factor10) and certainty2 (factor11) rating for the LCC/LCB calculations. 

For testing purposes, the datasets set the certainty factors equal for all Indicators and Indexes. The resultant average values can still support decisions based on their full values, but not their incremental differences. The following examples use fictitious certainty factors to illustrate incremental uncertainty. The author is not familiar with studies that apply the same ICER techniques to certainty values. Use their full values if that approach proves questionable.

Indicator 1. Cost Uncertainty ICERS
The following table applies the ICER techniques to the certainty scores. The low and high ICERS reflect summaries. The descriptions, or interpretation, of the certainty averages, derive from approaches introduced in Appendix A.



Discussion
The most cost effective alternative combination, AB-AC, is most likely to have an ICER = 0.497, with a high estimate of 0.854, and low estimate of -0.939. This alternative has total costs = 1,684 and total performance = 2,129. The risk and uncertainty associated with this alternative is substantially higher than the other possibilities, because the costs have “low confidence” and the performance is “inconclusive”. We recommend this alternative only for decisions where risk and uncertainty are not of primary concern. Otherwise, AA-AB should be chosen because the costs have “high confidence” and the performance is “established but incomplete”. When risk and uncertainty is the primary decision criteria, Alternative AA, alone, should be chosen because the costs are “likely” and the performance is “well established”.

Final Summary
In practice, a table similar to following image (Kim et al, 2016), is presented to decision makers. These authors use Mean and Standard Error statistics to communicate uncertainty to decision makers. For this example, a single uncertainty “Index” could be developed in a way that supports Appendix A’s “qualitative communication of performance”. This table will be revisited in Example 4B. In the meantime, the full article, and the related book, are good examples of why this [hard] work should be completed by social networks and clubs.



Indicator 2. Revenues Included

Alternative AA is a good example of the importance of low and high estimates. AA’s Low ICER=1.097 is almost double Alternative AC’s Low ICER = 0.667. These types of clear differences must be pointed out to decision makers. Example 4B will relate Net Returns to the more comprehensive Net Benefits (NMB) displayed in the previous image.

Example 4B discusses the importance of using CEA at appropriate scales and scopes. Specifically, if too location, or context, specific, the results may not be able to be used to allocate resources efficiently in regions and sectors. Example 4B addresses these concerns. 

Case Study Footnotes
1. Prior to Version 2.1.0, the Technology Assessment 2 reference actually explained WHO’s approach to cost effectiveness incompletely. The CEA calculations are accurate in that reference because they are extremely basic, but WHO recommends comparing Alternatives against a “Doing Nothing” Alternative, not a Current Practice Alternative. Refer to Footnote 11 in this reference and Footnotes 6 and 10 in the CTAP reference. DevTreks is a software development company, which happens to be run by an economist. Our role is to introduce basic tools and algorithms applied using concrete, modern, open source code. Your role is to figure out how to do it better.
2. These ICERS reflect calculations that were run prior to fully debugging this algorithm –the High Estimates are all wrong. This footnote is added because of the importance that the CEA references place on replicating CEA results (i.e. in the context of DevTreks, by clicking on, or touching, a Calculator.Run button).
References
GLOBALG.A.P. Risk-Assessment on Social Practice (GRASP). GRASP Module – Interpretation for South Africa. 2015
GLOBALG.A.P. Risk-Assessment on Social Practice (GRASP). GRASP Checklist – Version 1.3. Checklist Producer Group (Option 2). 2016
GLOBALG.A.P. The GLOBALG.A.P Database, Managing Complexity the Easy Way. 2016
GLOBALG.A.P. Integrated Farm Assurance. Crops Base (all sub-scopes). Benchmarking Cross-Reference Checklist. Version 5.0_July, 2017
QALY/DALY CEA References

Raymond Hutubessy, Dan Chisholm, Tessa Tan-Torres Edejer and WHO CHOICE. Generalized cost-effectiveness analysis for national-level priority-setting in the health sector. Cost Effectiveness and Resource Allocation 2003, 1:8
Health Information and Quality Authority (Ireland). Guidelines for the Economic Evaluation of Health Technologies in Ireland. 2010
David D. Kim, Anirban Basu, Sarah Q. Duffy, Gary A. Zarkin. Worked Example 1: The Cost-Effectiveness of Treatments for Individuals with Alcohol Use Disorders: A Reference Case Analysis. ResearchGate. November, 2016. (last accessed August, 2017: https://www.researchgate.net/publication/309591908) 
Peter J. Neumann, and Gillian D. Sanders. Cost-Effectiveness Analysis 2.0. The New England Journal of Medicine. 376;3 nejm.org January 19, 2017
Second Panel on Cost Effectiveness in Health and Medicine (2nd Panel on CEA). December, 2016. (last accessed August, 2017: https://www.youtube.com/watch?v=KepQmI0oxoI)
Gillian Sanders Schmidler, Duke University. Slide Presentation on 2nd Panel on CEA. 2016
World Health Organization. Guide to Cost-Effectiveness Analysis. 2003


Example 4B. Generalized Cost Effectiveness Analysis (GCEA) with Quality Adjusted Stock Years (QASYs) (RCA5)
URLs
https://www.devtreks.org/greentreks/preview/carbon/resourcepack/Coffee Firm RCA Example 4B/1556/none

http://localhost:5000/greentreks/preview/carbon/resourcepack/Coffee Firm RCA Example 4/541/none

Resource Stock Assessment and Monitoring and Evaluation Assessment

http://localhost:5000/greentreks/preview/carbon/output/Coffee Firm RCA4B Both/2141223494/none

https://www.devtreks.org/greentreks/preview/carbon/output/Coffee Firm RCA4B Both/2141223480/none

A. Introduction
This example uses the RCA4 (LCC/LCB) algorithm to carry out a Cost Effectiveness Analysis (CEA) of conservation technologies designed to improve private and public social performance at country or sector scale. Rather than using the health care industry’s Quality Adjusted Life Years (QALYs) or Disability Adjusted Years (DALYs) to measure health care outcome performance, this example demonstrates how to use the more generic Quality Adjusted Stock Years (QASYs) for measuring the more generic public service outcome performance.
The following techniques will be demonstrated in this example:
1. Generalized Cost Effectiveness Analysis (GCEA) of Sustainability: The Hutubessy et al (2003) and WHO (2003) references discuss the importance of applying CEA techniques to allocate resources efficiently in the health care sectors of countries. In effect, the overriding purpose behind the RCA framework, except RCA assesses the sustainability of technologies in all sectors of economies. 
2. Harmonized Indicator Systems: WHO’s standardized, harmonized, international, Performance Indicator system (WHO, 2015) provides the initial “ideal standards system” that will be used in this example. That system, which is treated as an industry-specific supplement to a multi-sector Performance Indicator System (i.e. SDG), will be supplemented by another industry-specific multi-capital supplement, in this example, for assessing environmental and socioeconomic performance. The health care industry’s Societal Perspective (2nd Panel on CEA, 2016) will be changed to Public Capital Perspective. Their Health Care Perspective is replaced by a Natural Capital Care Perspective. A resultant Social Performance Assessment completed using the RCA1 algorithm will be harmonized with one or more EU budgets completed using the RCA4 algorithm for CEA and Budget Impact Analysis. 
3. Cost-Utility Analysis using QASY Outcomes (1*): This example replaces the single capital stock characteristic, health, used in QALYs and DALYs, with the multi-stock characteristic, quality of life, used in QASYs. SPA1 explains that the fundamental goal of the RCA framework is to increase human quality of life by protecting and improving the services generated by the 7 public capital stocks. Appendix 1, Cost Effectiveness Analysis 3.0, to this example explains the relation of QASYs to QALYs/DALYs in CEA. 
4. Scenarios and Stakeholder Tradeoff Assessments: Scenarios will demonstrate CEA tradeoff analysis for different technologies and “societal perspectives”. The health care industry’s Impacts Inventory (2nd Panel on CEA, 2016) demonstrated in Appendix 2, will be adapted for public capital impacts.
5. Structured Abstract Format: This example demonstrates how to apply the 2nd Panel on CEA’s (2016) recommendations for reporting. Appendix 3. Report Checklist, illustrates including the checklist with the analysis (Smidler, 2016). Although not applied in this example, Antioch et al (2017) discuss the use of similar checklists to verify the quality of health care economic evaluations.

The CEA references used with this Example come from the health care industry because they apply CEA very practically, seriously, and with full decision support context (albeit in their own silo). And because they supply the initial model that other public service agents and community service organizations should be following to reduce societal risks, such as climate change, in transparent and affordable ways. Chapters 7, 8, 9, and 12 in WHO (2016), explain national costing, budgeting, M&E, and intersectoral (i.e. multi-sectoral and multi-capital) assessment, and should be read prior to this example (i.e. because that’s how DevTreks should be used).

A: Objective

This country’s citizens and businesses want biodiversity protected, climate change mitigated, and social discord-based migration handled fairly. During their scoping analysis, their scientists, with assistance from citizen and business stakeholders, formulated 4 Performance Measures to achieve these objectives. The objective of this study is to assess the cost effectiveness of these approaches, decide whether to apply one or more of the actions, and if enacted, monitor and evaluate their performance to ensure actual risks get reduced.

An objective, science-based, social network (i.e. EU Resource Conservation Value Accounting Network) has developed an Indicator Threshold system that their clubs can follow when developing resource conservation accounting and financial reports. The network started by harmonizing the immense number of indicator systems, assessment approaches, and indicator reporting instruments used by its individual member countries, industries, and affiliated international organizations (as discussed by Schroter et al, 2016) . 
They decided to adopt the Performance Indicator system recommended by the international health care community and summarized in the following 2 images (WHO, 2015). The images demonstrate using a “results chain” applied using a surmised “explanatory mixed methods” approach, measured using 100 standard Performance Indicators, enforced with “international norms of good reporting behavior”, and harmonized across countries, to improve health care reporting and performance.



They then adapted the WHO system to carry out all other capital stock assessments in the EU, including this Example’s Public Capital Stock Assessment, as follows:
1. Public Capital Stock Indicator System: The WHO Indicator system is treated as an industry-specific supplement to a more general Public Capital Stock Indicator System. SPA1 introduced the SDG, Sendai DRR, GSSB, SASB, and EMAS Indicator systems as examples of multi-sector Sustainability Assessment Indicator systems. The WHO’s health care system is part of a more generic system focused on Public Capital Stocks (Kieny et al, 2017 and WHO 2017 demonstrate using the SDG, Chapter 12 in WHO 2016 provides an overview of “intersectoral” assessment), and for this example, supplemented with a Natural Capital Care system. This supplemental system addresses issues identified as highly important to the network’s citizens and businesses, including their concerns with protection of biodiversity and mitigation and adaptation to climate change. An additional socioeconomic concern, balanced migration, has been added to reinforce the importance of multi-capital stock assessment.
2. Public Capital Service Outcome Measurement: The RCA framework requires materiality impact measurements dealing with costs, benefits, and performance (3*). As further explained in Appendix 1, Cost Effectiveness Analysis 3.0, the common metrics used by the health care industry to measure outcomes and impacts, QALYs and DALYs, are replaced by the more generic, QASY (or Example 12’s Subjective Well Beings).
3. Indicator Thresholds. Example 1’s SAFA Indicator Thresholds will be adapted for this Indicator System.
4. Discounted Indicators. Several of the CEA QALY references demonstrate using a real discount rate of 3% to discount both costs and QALYs. The CEA techniques introduced in Example 4A have been upgraded to also discount QASYs. 
5. Index-based Results Chain. This study employs a mixed methods qualitative explanatory path approach, applied using a results chain, for the analysis. They use the Outcome Indicator in that pathway as “ecosystem services” because of the importance their “natural capital care industry” places on the natural capital stressors. Furthermore, rather than Example 1’s 1:1 relation between Performance Indicators, a 1:1 relationship is defined only for the Performance Indexes. [For convenience, this release uses generic Indicators, such as Indicator 1 and Indicator 2 in the Indexes. Given that this example’s main point is to understand how to carry out CEAs, only the final Impacts Indicator dataset is used with the RCA1 algorithm.]
6. Public Capital Perspective, Natural Capital Care Perspective, Stakeholder Group A Perspective, and Impacts Inventory: The 2nd Panel on CEA’s (2016) recommendations to include a Health Care Perspective, Societal Perspective, Payer Perspective, and Impact Inventory in the final analysis, are adapted for this example’s primary stressors. The Societal Perspective is replaced by a Public Capital Perspective. The Health Care Perspective is replaced by a Natural Capital Care Perspective. The Payer Perspective is replaced by a Stakeholder Group A Perspective.
7. Decision Making with CEA QASY Thresholds: The American Heart Association (Anderson et al. 2014) recommends applying the following image’s CEA Thresholds in clinical guideline recommendations. IPSOR (2017) discusses alternative CEA Thresholds. This Example adapts the same Thresholds, but replaces the term QALY with the term QASY. These 2 publications provide a good overview of the “why and how” being addressed by practitioners of resource stock conservation (i.e. who work in community service organizations, such as health clinics and RCA Districts).

B: Intervention

This country, or club, chose to use the following General Scenario for the purpose of building a disaster risk reduction and resiliency plan. The plan helps them understand the primary risks that may impact the country’s 7 capitals, and the quality of life of their businesses and citizens, over their planning horizon. They use the plan and their Social Performance Score to identify needed investments in mitigation and adaptation performance measures. These measures are summarized in the 4 Scenarios listed below. They use a full M&E system, with short term Social Monitoring Assessments, to monitor and evaluate how well the selected portfolio actually reduces risks.

General Scenario
Threatened Quality of Life: High GHG result in 1.5 degree temperature increase with higher incidence of biodiversity loss, droughts, severe heat waves, crop and livestock production risks, air pollution, floods, migration, and social discord. 
Targeted Stakeholder Groups: see Section C. Target Populations
Targeted Social Performance Stressors: Habitat Change, Climate Change, Overexploitation, Invasive Alien Species, Pollution and Nutrient Enrichment, and Migration (refer to the EEA references in SPA1, and this example’s ecosystem services references, such as Schroter et al, 2016)
Mitigation and Adaptation Actions: Portfolio 1 consists of a) …, b)…, and c)…. 

Scenario 1 or Performance Measure 1: Benchmark, or No Mitigation and Adaptation, Performance: 
Scenario 2 or Performance Measure 2: Stakeholder Group A with Mitigation and Adaptation Portfolio A: 
Scenario 3 or Performance Measure 3:  Stakeholder Group A with Mitigation and Adaptation Portfolio B: 
Scenario 4 or Performance Measure 4: Stakeholder Group A with Mitigation and Adaptation Portfolio C: 
The term “Performance Measure” is also used because Appendix 1 explains that the health care industry refers to these actions, or Alternatives, as “performance measures” used for clinical practice guidelines, and recommend using CEA to assess their efficiency (Anderson et al, 2014).

C: Target Populations

The populations targeted by the 4 proposed Performance Measures include:
EU member states; trading partners; this country’s citizens and businesses; Supply chain buyers and end product consumers concerned that companies proactively address biodiversity loss, climate change, and migration in their business practices; Companies concerned about investors, supply chain buyers, and end product consumers who want independent verification of company compliance with socially sound business practices; Developing country citizens migrating because of social discord; European citizens concerned about social discord-caused migration.

Example 4 discusses the need for additional population algorithms to fully address population and stakeholder metrics in GCEA (i.e. refer to the 2nd Panel on CEA or WHO references). Examples 5 through 8 in the Social Performance Analysis 3 reference addresses this need.

D: Perspectives

The analysis bases its costs and performance estimates based on the following perspectives:
a. Natural Capital Care Perspective: IPSOR (2017) describes this perspective in terms of the 2nd Panel on CEA recommended health care perspective:

“The health care sector perspective includes “formal health care sector (medical) costs borne by third party payers or paid for out-of-pocket by patients.” This includes “current and future health costs, related and unrelated to the condition under consideration.”   Notably, it does not include patient time costs, or the future benefits and costs of other types of consumption associated with increased longevity.  In contrast, the more narrowly-construed “payer perspective” does not include patient out-of-pocket costs because they are not borne by payers.” 

This example adapts the health care perspective to a natural capital care perspective. In practice, this example demonstrates applying this technique by using a natural capital care budget that focuses mostly on ecosystem service outcomes and impacts.

b. Public Capital Perspective: IPSOR (2017) describes this perspective in terms of the 2nd Panel on CEA recommended societal perspective:

“Societal perspective is very broad, adding time costs and effects on future productivity and consumption as well as relevant non-health impacts in other sectors, such as education and criminal justice”. 

This example adapts this societal perspective to a public capital perspective. In practice, this example demonstrates applying this technique by using a public capital budget that focuses mostly on broad social welfare outcome impacts, particularly related to migration.

c. Stakeholder Group A Perspective: Westrich (2016) and Sorenson et al (2017) criticize several health care value frameworks currently being developed in the health care sector for not being “patient-centric”, which in terms of CEA, translates in to not putting priority on patient perspectives. ICER (2017) demonstrates how to institutionalize patient engagement in health care economic evaluations. The latter organization endorses allowing the patients, or their representatives, to complete this type of perspective. WHO (2017) provides international perspective on the importance of this perspective. Jacobs et al (2016) discusses the importance of this perspective in natural capital care assessments.

In practice, this example uses a household budget based on Stakeholder Group A together with a Social Assessment focusing on impacts to that targeted group, as demonstrated in Example 3B. The RCA3 algorithm, applied using S-LCA, is used for this purpose and treated as 1 Performance Monitoring Assessment observation that feeds into the other 2 perspectives, or Impact Evaluations, via M&E.
Neuman et al (2016) describes the Appendix B. Impact Inventory that is used with this example in the following statement. Examples of summary “Impact Inventories” can be found in the Impact TEXT datasets used throughout this reference.

“Our panel further recommends inclusion of an “impact inventory,” a structured table listing the health and non–health-related effects of an intervention that should be considered in a societal reference-case analysis. When interventions have substantial effects beyond the formal health care sector (e.g., effects on economic productivity, social services, legal or criminal justice, education, housing, or the environment), such an inventory allows analysts to clarify those consequences for decision makers.”

E: Time Horizon

This country employs a 200 year planning horizon, similar to GHG emission life, for societal planning. 

F: Discount Rates

Costs and QASYs have been discounted using a real rate of 3% and a nominal rate of 5%. QASYs, or Example 4A’s Performance Indicators, are discounted, and/or escalated, exactly the same as LCCs and LCBs.

Because the main purpose of this example is to introduce a “big picture” view of applied GCEA, none of the LCC/LCB escalation options introduced in Example 4 have been used. However, the health care industry is quite concerned about health care affordability given probable future price escalations. The LCC/LCB-style price and QASY escalations will need to evolve as these industries reach consensus on how to deal with projected future price escalations. Especially important in special interest group-dominated countries characterized by a small number of companies exercising market domination. The potentially disastrous future effects of this example’s stressors require similar evolution in escalation techniques.

G: Costing Year
The selected Performance Measure will be applied at the start of 2018. For the purposes of eyeball metrics, costs and QASYs have not been discounted for this example. The full lifetime of the natural capital stressor Indicators, such as GHG that lasts more than 100 years, requires careful use of the LCC/LCB techniques.
H: Study Design (4*)

This study employed a mixed methods qualitative explanatory path approach, applied using a WHO-style results chain, to assess outcome performance and impact consequence. The nature of this study’s stressors meant that ecosystem services, and more general public services, had to be fully addressed because of the importance their “natural and institutional capital care industries” place on the natural capital, and institutional capital, stressors. In practice, the Outcome Indicators used in this example’s results chain are used for that purpose. Migration is considered an institutional capital stressor because sound QOL development institutions may thwart the stress (i.e. the S-LCA references in Example 3B have a specific category for migration).

Example 1, Section G, Impact Evaluation, explains how results chains relate Performance Assessments, Impact Evaluations, and Monitoring and Evaluation systems. In summary, the Monitoring and Evaluation system links short term Performance Assessments to long term Impact Evaluations. Appendix 1 further explains how QASYs fit into the metrics.

Example 4A, Footnote 1, explains that DevTreks’ role is practical application, not perfection. This example applies that reality in a simplistic manner. This example’s Social Performance Assessment uses the RCA1 subalgorithm as a meta-data summary derived from COSA-like Impact Evaluations (i.e. QOL surveys), which in turn, derive from RCA2-like Performance Monitoring Assessments. 

The overall M&E system will not be demonstrated in this example. However, the following table (Schroter et al, 2016) hints at how to apply a more thorough M&E system. The table demonstrates how the EU currently assesses trends for this study’s ecosystem [and socioeconomic] stressors. The table shows that trends have been assessed for 9 separate resource stock states and 6 primary pressures. The EU may use a combination of trial-based experiments and simulation-based models to assess these trends. When possible, scientists conduct field experiments, and, with assistance from trained volunteers, conduct field measurements to further verify the trends. That initial data, which can be stored using the RCA2 Trends subalgorithm, is then added to EU-wide simulation models that can be used for EU-wide GIS mapping. 


In effect, RCA2-style and RCA3-style instruments are used for the M&E’s short term Performance Assessments and an RCA1-style instrument is used for the long term Impact Evaluations. The COSA references used throughout this tutorial demonstrate that, in practice, more advanced algorithms are used to conduct Impact Evaluations (i.e. as introduced in the SPA3 reference). Antioch et al (2017) also discuss advanced algorithms, such as Markov cohort simulations and Reference-level modeling, employed in the health sector.

I: Data Sources

The EEA has EU-wide responsibility for collecting the data used in these analyses. The references demonstrate that individual countries collect and maintain supplemental data using their environmental agencies.

J. Outcome Measurement

Appendix 1. Cost Effectiveness 3.0, verifies that QASYs are used to measure outcome performance. Appendix 2. Impact Inventory, summarizes the full impacts being measured in this analysis.

In line with the CEA recommendations (via Kim et al, 2016) this study used the following approach for outcome measurement: “Incremental cost-effectiveness ratios (ICERs) and net monetary benefits (NMBs) were used as primary outcome measures. For the NMB, we used $100,000 per QALY as a willingness-to-pay (WTP) threshold in the base-case analysis.”

The 2nd Panel on CEA discusses supplemental outcome measurements to include in these analyses, including intermediate health outcomes, disaggregated results, and measures of robustness.

K: Results of Base Case Analysis

The following table illustrates the stylized Indicator Threshold system applied by this network. All of the numbers used in this analysis are stylized because either data does not exist, or it’s the role of complete social networks and clubs to figure the numbers out (Liquete et al, 2016, demonstrate the type of “real” data used in these analyses). They use the SAFA system for scoring, but translate the results to QALY-style 0-1 ratings before being analyzed. In practice, the health care industry’s techniques for either HRQol-QALY or expert-DALY must be adapted for this QASY scoring system.



The following table illustrates harmonizing this Performance Indicator system with an EU budget. The Indicator system was developed by EU scientists with active feedback from concerned, or more accurately, informed, citizens and businesses. WHO (2016) uses the term “Performance Budgeting” to describe tying government budget revenues and expenditures to Performance Indicators and Indexes in this manner.



The goal of full Impact Evaluation is to use full explanatory paths to understand cause and effect. Given that the goal of this example is to understand GCEA, Indicator 1 to 3 in the results chain are not completed for the 3 Perspectives. Instead, Indicator 1 illustrates the completion of the RCA1 probability distributions for Impacts alone, Indicator 2 illustrates an EU budget for these interventions, and Indicator 3 demonstrates the GCEA. For the sake of brevity, tables are not shown for the Public Capital and Stakeholder A Perspectives.

The following Social Performance Scoring system is completed for 1 ecosystem, or 1 location. Similar Scores are developed for the remaining ecosystems. As explained in Section H: Study Design, an RCA2-style instrument uses the same Indicators and Indexes for short term M&E. This instrument strictly serves as a stylized Impact Evaluation that evaluates Scenarios 1 to 4. 

Natural Capital Care Perspective
Indicator 1. RCA1 Social Performance Assessment MathResult


Indicator 2. EU Budget MathResult


Indicator 3. GCEA MathResult


Natural Capital Care Perspective ICER 
The previous table’s MathResults were edited in a spreadsheet to produce the following table. The table shows that even stylized analyses need careful data input (and that the multiplicative law of mathematics works).


Given that this example’s primary purpose is to illustrate adapting respected health care industry valuation techniques to the RCA Framework, this example chose not to change the stylized data used to generate these ICERs. The data still supports “eyeball” evidence that this algorithm works. Instead, the table’s costs and QASY’s were changed to show a constant incremental increase in performance, as illustrated in the following table.



Public Capital Care Perspective ICER
The use of stylized data doesn’t actually contribute much to this introduction to GCEA. This will be completed when better data can be found.

Stakeholder Group A Perspective ICER
This will be completed when better data can be found.

The goal of this country’s conservation efforts, and their overall Social Performance Assessment, is to increase the quality of life for their stakeholders. In order to do so, they must understand the interactions, linkages, and cause-effect relationships, between the 9 resource stock states and the 6 pressures. Their resultant knowledge of the tradeoffs that must be made between services, mitigation actions and impacts, and stakeholder values, assists them to achieve their societal performance goals.

The following image demonstrates how UNEP (2011) uses MCDA to understand tradeoffs better in the natural capital care sector, specifically to address climate change. This example adapts this technique for explanatory mixed methods assessments. Specifically, outputs are replaced by QASY measurements taken from Impact Indicators. Inputs are replaced by Cost measurements derived from the EU budget. 


The following image displays the resultant cost effectiveness plane for the 3 scenarios in this example. The stylized data results in a linear plane suggesting that, without budget constraints, consideration of CEA Thresholds, or the Uncertainty Indexes, Scenario AC should be chosen.



L: Results of Uncertainty Analysis

The following table comes from Example 4A and illustrates the uncertainty metrics available with this algorithm. Kim et al (2016) describe this type of uncertainty analysis in statements such as: “Results were robust across a range of WTP thresholds and across plausible ranges of remission rates and health-related quality of life (HRQoL) in the AUD state. However, substantial uncertainty in final estimates indicates [a] large value of future research on this topic.” Of course, this reference corrects the last 8 words as follows: “large value of applied IT and effective research and on this topic.”



The results of the RCA4 algorithm supported completion of the following Reference Case Cost Effectiveness Results table (5*). For this example, 2 uncertainty “Indexes” have been developed from the previous table to support Appendix A’s “qualitative communication of performance” (i.e. Uncertainty Index 3 = established but incomplete). Similar tables are produced from the Low and High Estimates.



These results show that all scenarios generate substantial Net Monetary Benefits (NMB), but Scenario AA has the lowest incremental cost per incremental QASY and is considered most efficient. Once the Uncertainty Indexes are considered, Scenario AC’s costs and performance are substantially more certain than the other Scenarios. Given AC’s substantial NBM, regardless of the negative Incremental NMB, Scenario AC should receive strong consideration for funding from decision makers.

Jacobs et al (2016) provide an overview of the importance of the Stakeholders perspective in natural capital care assessments. A strong focus on those perspectives ensure that, at the very least, these assessments introduce decision makers to “socially acceptable paths forward”. The authors also warn of the danger to equity these approaches pose when “some social actors [capture] more power” in the overall decision making process. A particular danger in special interest group dominated countries.

The following image (EPA, 2016) illustrates additional decision support that can be derived using the Uncertainty Index-style metrics shown in the previous table. The USEPA applied a mixed methods evaluation at city-scale for this analysis.



M: Limitations
The following image (WHO, 2017) demonstrates how the international health care sector sets priorities for improving health systems performance.



These are the same dimensions needed to improve the public service “delivery models” summarized in this reference. Given that the conventional institutions in some countries appear incapable of understanding, let alone applying, these common sense dimensions, social networks and clubs may need to take independent action.

Besides institutional failure, possibly originating in special interest group-dominated institutions, the CEA references discuss further limitations that must be addressed in these analyses. They include the importance of counterfactual evidence, alternative ways to devise social weights to address equity, the use of international units, patient population heterogeneity versus patient population averages, and the need for additional sensitivity analysis (i.e. weights, discount rates, life of project). This reference would be remiss without pointing out the importance of putting priority on funding IT that applies “effective research” as contrasted to “conventional research”.

N: Conclusions

Hutubessy et al (2003) describe why GCEA is applied at country and sector scales:

“Most country applications focus on local and marginal improvements in technical efficiency. The term allocative efficiency, on the other hand, is typically used in health economics to refer to the distribution of resources among different programmes or interventions in order to achieve the maximum possible socially desired outcome for the available resources. By definition, addressing issues of allocative efficiency in health requires a broader, sectoral approach to evaluation”

WHO (Example 4, 2016, chapters 7 and 8) explain the use of cost estimates and budgets further and use the following definitions to define efficiency in the health care sector of countries.

“Allocative efficiency. This concerns the “what” – i.e. the health service package that is being provided, and whether changing the composition of services within the package (subsidized by public funds) would bring more value for money. Here, cost-effectiveness analysis is a useful tool to assess efficiency. In the case of a budget reduction for health, important decisions would need to be made whether to restrict access and/ or increase co-payment for some services and/ or populations and if so, which ones.”

“Technical efficiency. This relates to “how” resources are used, and whether the same set of services could be delivered more efficiently. Potential strategies may include shifting tasks from one type of health worker cadre to another, changing purchasing strategies for drugs and medicines in order to obtain lower prices, and shifting from inpatient to outpatient care where this can be safely and effectively done.”

This “allocative efficiency” and “technical efficiency” explanations for GCEA at country or sector scale are further explained in the Performance Analysis 1 reference. Example 4B demonstrates that these techniques, with additional social network work, can also be used to reduce serious societal risks, including climate change, biodiversity loss, migration, and civil rights protection. This “mainstreaming a new culture of valuation” (Jacobs et al, 2016) may help people to improve their lives and livelihoods.

M: Compliance Labelling

The following DevTreks logo illustrates how to relate this example to compliance standards. This logo, or compliance label, is used when complete GCEA and M&E systems have been fully implemented at country or sector scale to assess public service improvement technologies. The label signifies that countries or industries fully comply with using transparent and affordable resource conservation actions to reduce serious societal risks, such as climate change. Citizens hold their public officials accountable for failure to comply. Consumers hold their industry executives accountable for failure to comply. Supply chain buyers hold their producers accountable for failure to comply. Investors hold their companies accountable for failure to comply. In practice, social networks work through the ISEAL Alliance, producer organizations, and local communities, for similar purposes.


Footnotes
1. The author recently spent several hours reviewing the Second Panel on Cost Effectiveness in Health and Medicine’s 2016 youtube presentation summarizing their recent publication on CEA in the health care sector (i.e. they charge fees for the book which has limited usefulness in the public goods context of DevTreks). First impressions were that their single-capital recommendations for Societal Perspectives, Impact Inventories, and multi-stakeholder impact analysis, could be addressed more directly using the multi-capital and multi-sector approaches introduced in this tutorial. In fact, the author believed that the health care industry “missed the boat” and should have started with a clear mandate for a multi-capital approach (i.e. for a more complete Societal Perspective and a better QALY that could be used in multiple sectors). Closer inspection of their Working Examples and other applied uses found that their recommendations serve practical, if partial, purposes that could be adapted to the RCA’s multi-capital framework and this Example’s multi-sector CEA techniques (i.e. or WHO’s, 2017, health systems performance need). 
2. The author acknowledges that as this stage of development, QASYs are “quasi-right” –they need more work, but Appendix 1 suggests that the work appears to be under development by several national and international organizations (even if for related purposes). The danger of social engineering hubris (i.e. rationing of health care) with this type of algorithm is avoided by focusing strongly on the health care sector’s Health Technology Assessment approach and DevTreks’ Conservation Technology Assessment approach. Society is not being engineered –technologies, including policies and management practices, are being evaluated for social soundness.
3. Version 2.1.0 changed the name of the Resource Conservation Accounting Framework to Resource Conservation Value Accounting (RCA) Framework because of the similarity to the current value frameworks being developed in the health care industry (IPSOR 2017, ICER 2017, Westrich 2016, ACC-AHA 2014). The objective of these value frameworks is to ensure that money spent on products and services, including health care and ecosystem services, generates cost effective value (i.e. measured using CEA and M&E-based Performance Monitoring and Impact Evaluation in this reference). Readers should recognize that most of those frameworks would be handled, or “harmonized”, using 1 or more subalgorithms in DevTreks. Implying exactly what it sounds like it implies.
4. Although this example demonstrates applying a stylized “mixed methods explanatory approach”, specifically results chains, explained throughout this tutorial, Appendix 1 points out that the health care industry appears to be “leaning towards” accepting straight Multi-Criteria Decision Analysis (MCDA) frameworks, alone. Social networks need to keep an eye out for developments in that field, especially when “ease of use” has to be a primary initial step during the adoption of these techniques. Example 4A demonstrates a simplified application of that technique. As a side note, after spending a considerable amount of time on this example, it occurred to the author that he probably could have come up with the same approach several decades ago, but may have become “obfuscated” somewhere along the way.
5. See Footnote 1 in Example 4A. DevTreks always recommends using professional references when completing these types of analyses (i.e. even if, as in the case of the 2nd Panel on Cost Effectiveness Analysis’ 2016 book, they charge fees).
References

Health Care QALY/DALY/QASY CEA (Additional QALY/DALY CEA References can be found in Example 4A.)

Jeffrey L. Anderson, Paul A. Heidenreich, Paul G. Barnett, Mark A. Creager, Gregg C. Fonarow, Raymond J. Gibbons, Jonathan L. Halperin, Mark A. Hlatky, Alice K. Jacobs, Daniel B. Mark, Frederick A. Masoudi, Eric D. Peterson and Leslee J. Shaw. Association Task Force on Performance Measures and Task Force on Practice Guidelines Performance Measures: A Report of the American College of Cardiology/American Heart ACC/AHA Statement on Cost/Value Methodology in Clinical Practice. 2014
Kathryn M. Antioch, Michael F. Drummond, Louis W. Niessen and Hindrik Vondeling. International lessons in new methods for grading and integrating cost effectiveness evidence into clinical practice guidelines.  Cost Eff Resour Alloc (2017) 15:1
European Consortium in Healthcare Outcomes and Cost-Benefit Research (ECHOOUTCOME). European Guidelines for Cost-Effectiveness Assessments of Health Technologies. 2013
Institute for Clinical and Economic Review (ICER). Overview of the ICER value assessment framework and update for 2017-2019
International Society for Pharmacoeconomics and Outcomes Research (IPSOR). A Health Economics Approach to US Value Assessment Frameworks.  DRAFT Special Task Force Report, July 7, 2017 [DevTreks focus on technology development, rather than academic reporting, explains the use of draft references.]
Marie Paule Kieny, Henk Bekedam, Delanyo Dovlo, James Fitzgerald, Jarno Habicht, Graham Harrison, Hans Kluge, Vivian Lin, Natela Menab, Zafar Mirza, Sameen Siddiqi & Phyllida Travis. Strengthening health systems for universal health coverage and sustainable development. Bulletin of the World Health Organization; Type: Perspectives Article ID: BLT.17.187476, 2017
Kimberly Westrich. Current Landscape: Value Assessment Frameworks. National Pharmaceutical Council. 2016
Corinna Sorenson, Gabriela Lavezzari, Gregory Daniel, Randy Burkholder, Marc Boutin, Edmund Pezalla, Gillian Sanders, Mark McClellan. Advancing Value Assessment in the United States: A Multistakeholder Perspective. Value in Health 20 (2017) 299 – 307
World Health Organization. 2015 Global Reference List. 100 Core Health Indicators. 2015

World Health Organization. Strategizing national health in the 21st century: a handbook. 2016. Chapters 7, 8, and 9 cover cost estimating, budgeting, and M&E. Chapter 12 introduces multi-sectoral and multi-capital health care performance analysis.
 
World Health Organization. Healthy systems for universal health coverage - a joint vision for healthy lives. Universal Health Coverage (UHC) 2030. 2017

Natural Capital Care QASY CEA [although these references are thorough, none present complete raw datasets that can be used with this example]

International Institute for Tropical Agriculture (IITA) and Committee on Sustainable Agriculture (COSA). Impacts of Certification on Organized Small Coffee Farmers in Kenya. Baseline, 2016.
Jäppinen, J.-P. & Heliölä, J. (eds.) 2015: Towards a sustainable and genuinely green economy. The value and social significance of ecosystem services in Finland (TEEB for Finland). Synthesis and roadmap. The Finnish Environment 1en/2015. The Finnish Ministry of Environment, Helsinki. 144 p.

Jacobs, Sanders , Nicolas Dendoncker , Berta Martín-López , David Nicholas Barton , Erik Gomez-Baggethun , Fanny Boeraeve, Francesca L. McGrath , Kati Vierikko , Davide Geneletti , Katharina J. Sevecke , Nathalie Pipart , Eeva Primmer , Peter Mederly , Stefan Schmidt, Alexandra Aragã, Himlal Baral , Rosalind H. Bark , Tania Briceno , Delphine Brogna , Pedro Cabral , Rik De Vreese , Camino Liquete , Hannah Mueller, Kelvin S.-H. Peh , Anna Phelan, Alexander R. Rincón , Shannon H. Rogers, Francis Turkelboom, Wouter Van Reeth, Boris T. van Zanten, Hilde Karine Wam, Carla-Leanne Washbourne. A new valuation school: Integrating diverse values of nature in resource and land use decisions. Ecosystem Services 22 (2016) 213–220
Camino Liquete, Angel Udias, Giulio Conte, Bruna Grizzetti, Fabio Masi. Integrated valuation of a nature-based solution for water pollution control. Highlighting hidden bene?ts. Ecosystem Services 22 (2016) 392-401
Mattias Schroter, Christian Albert, Alexander Marques, Wolke Tobon, Sandra Lavorel, Joachim Maes, Claire Brown, Stefan Klotz, Aletta Bonn. National Ecosystem Assessments in Europe: A Review. BioScience • October 2016 / Vol. 66 No. 10

Santos-Martín F., García Llorente M.; Quintas-Soriano C., Zorrilla-Miras P., Martín-López B., Loureiro M., Benayas J.; Montes M.. Spanish National Ecosystem Assessment: Socio-economic valuation of ecosystem services in Spain. Synthesis of the key findings. Biodiversity Foundation of the Spanish Ministry of Agriculture, Food and Environment. Madrid, Spain 68 pp. ISBN: 978-84-608-8776-8. 2016

Science for Environment Policy Ecosystem Service and the Environment. In-depth Report 11 produced for the European Commission, DG Environment by the Science Communication Unit, UWE, Bristol. Available at: http://ec.europa.eu/science-environment-policy. 2015

Stevens, M., Demolder, H., Jacobs, S., Michels, H., Schneiders, A., Simoens, I., Spanhove, T., Van Gossum, P., Van Reeth, W., Peymen, J. (Eds.). Flanders Regional Ecosystem Assessment: State and trends of ecosystems and their services in Flanders. Synthesis. Communications of the Research Institute for Nature and Forest, INBO.M.2015.7842756, Brussels. 2015

U.S. Environmental Protection Agency, National Center for Environmental Assessment Office of Research and Development. Evaluating Urban Resilience to Climate Change: A Multi-Sector Approach. EPA/600/R-15/312 DO NOT CITE OR QUOTE External Review Draft, June 2016  
United Nations Environment Programme (UNEP). A Practical Framework for Planning Pro-Development Climate Policy. 2011


Appendix 1 to Example 4B. Cost Effectiveness Analysis 3.0 (Quality Adjusted Stock Years, or QASYs (1*))

Version 2.2.0: This release added Appendix C to the SDG Plan reference which explains the use of Subjective Well Being Valuations, or QASYs. The SDG Plan reference can be found in the Social Performance Analysis tutorial.

Ireland HIQA (2010) defines the use of a primary CEA technique used in the health care industry: 

“A cost-utility analysis is the preferred evaluation type for the reference case. It is considered the gold standard method for conducting economic evaluations and is recommended by many expert and consensus groups. The preferred outcome measure to be used in the reference case is the quality-adjusted-life-year (QALY)”.  

IPSOR (2017) uses the following example to clarify the principal terms used in this Appendix, including cost utility analysis, QALYs, QALY CEA thresholds, and net monetary benefits (NMB):

“As an example, suppose that patients with lung cancer can expect to live an average of 4 years.  
Suppose also that they value each year spent with lung cancer as equal to 6 months of life spent in perfect health. Thus, they experience 2 QALYs.  Now suppose that a new drug is introduced that extends life expectancy to 4.5 years.  Suppose further that it reduces some of the disabilities and comorbidities associated with lung cancer such that patients now value one year spent with lung cancer as being equal to 8 months of life spent in perfect health. When treated with this drug, therefore, patients experience 3 QALYs, and the incremental benefit of the drug is equal to 1 additional QALY. 

With the incremental QALY gain in hand, the next challenge lies in obtaining a monetary value that society, payers, consumers or patients place on each QALY. For example, if each QALY is worth $150,000, the lung cancer drug in our example produces $150,000 worth of incremental benefit.  If that drug costs $125,000, we would then conclude its net value or “net monetary benefit” is $25,000 (=$150,000-$125,000).”

In support of this reference’s use of generic, multi-capital, social performance assessments, the single sector, single capital, single service, health outcome measurements, QALY and DALY, are replaced by the multi-sector, multi-capital, multi-service, quality of life outcome measurement, QASY (Quality Adjusted Stock Year). The following table demonstrates that, strained grammar aside, almost every characteristic, and definition, applied to QALYs can be applied to QASYs. Quotes that use citations in the following table come directly from the Wikipedia definition for QALY. Essentially, every use of the term “health” is replaced by the term “quality of life” (i.e. the basis for QALYs to begin with). The term “health care intervention” is replaced by the term “public service intervention” (i.e. the basis for health care interventions to begin with). 

Outcome Characteristic
QALY, Quality Adjusted Life Year
QASY, Quality Adjusted Stock Year
Relation to Capital Stocks
Health is considered a single capital stock characteristic of human capital stocks.
Quality of life is considered a multi capital stock characteristic of human, social, physical, institutional, economic, natural, and cultural, stocks.
Definition
“generic measure of disease burden, including both the quality and the quantity of life lived”
generic measure of quality of life experience, including both the quality and the quantity of ‘quality of life’ lived
QOL Basis
Quality of Life caused by personal experience of health state
Quality of Life caused by personal experience of public services state
Typical Use
“economic evaluation to assess the value for money [spent on] medical interventions”
economic evaluation to assess the value for money spent on public service improvement interventions
Scoring system
“multiplies the utility value associated with a given state of health by the years lived in that state”
multiplies the utility value associated with a given state of quality of life by the years lived in that state
Weighting or Utility Multiplier: Time Trade Off
“Respondents are asked to choose between remaining in a state of ill health for a period of time, or being restored to perfect health but having a shorter life expectancy.”
Respondents are asked to choose between remaining in a state of poor quality of life for a period of time, or being restored to perfect quality of life but having a shorter life expectancy.
Weighting or Utility Multiplier: Standard Gamble
“Respondents are asked to choose between remaining in a state of ill health for a period of time or choosing a medical intervention which has a chance of either restoring them to perfect health, or killing them.”
Respondents are asked to choose between remaining in a state of poor quality of life for a period of time or choosing a public service improvement intervention which has a chance of either restoring them to a perfect quality of life, or permanently destroying their quality of life.
Weighting or Utility Multiplier: Visual Analog Scale
“Respondents are asked to rate a state of ill health on a scale from 0 to 100, with 0 representing being dead and 100 representing perfect health. This method has the advantage of being the easiest to ask, but is the most subjective.”
Respondents are asked to rate a state of poor quality of life on a scale from 0 to 100, with 0 representing a permanently  destroyed quality of life and 100 representing perfect quality of life. This method has the advantage of being the easiest to ask, but is the most subjective.
Weighting or Utility Multiplier: Quality of Life (QOL) Survey Instruments
“standard descriptive systems such as the EuroQol Group's EQ-5D questionnaire, which categorises health states according to five dimensions: mobility, self-care, usual activities (e.g. work, study, homework or leisure activities), pain/discomfort and anxiety/depression”
Standard descriptive systems such as the Sustainable Food Lab’s (2016) small scale Agricultural Sustainability Assessment which categorizes quality of life states according to 7 impact areas: livelihood and well-being, gender, environmental stewardship, farm production, access to services, trading relations, and next generation farmers.
Score = 0
“[Nonexistent quality of life caused by] death resulting from personal health state”
Nonexistent quality of life caused by permanently destroyed quality of life resulting from personal quality of life state
Score = 1
“Excellent quality of life with no problems caused by experience of personal health state”
Excellent quality of life with no problems caused by experience of personal quality of life state
Score = 0. 5
“1 year of life lived in a situation with utility 0.5 (e.g. bedridden, 1 year × 0.5 Utility) is assigned 0.5 QALYs. Similarly, half a year lived in perfect health is equivalent to 0.5 QALYs (0.5 years × 1 Utility)”
1 year of life lived in a situation with utility 0.5 (e.g. impoverished, 1 year × 0.5 Utility) is assigned 0.5 QALYs. Similarly, half a year lived with perfect quality of life is equivalent to 0.5 QALYs (0.5 years × 1 Utility)
Score = 0 Example 1
Persons who are dying from bad health care services in sections of Syria and South Sudan
Persons who are dying from sectarian violence in sections of Syria and South Sudan
Score =  0 Example 2
Ethiopian pastoralists dying from insufficient quantity and quality of health care services
Ethiopian pastoralists who are dying of hunger due to poor ecosystem services caused by poverty, drought, population increase, and land use degradation

The following table confirms that, in practice, QASYs are similar to a multi-capital DALY. They expand QALY’s emphasis on life expectancy to include the RCA Framework’s multi-capital emphasis on quality of life.

Outcome Characteristic
DALY, Disability Adjusted Life Year
QASY, Quality Adjusted Stock Year
Relation to Capital Stocks
Disability is considered a single capital stock characteristic of human capital stocks.
Quality of life is considered a multi capital stock characteristic of human, social, physical, institutional, economic, natural, and cultural, stocks.
Definition
DALYs capture both mortality in terms of years of life lost (YLL) and morbidity in terms of years of life with disability (YLD)
QASYs capture both mortality in terms of years of life with destroyed quality of life (YLL) and mobidity in terms of years of life with impaired quality of life (YLD)

In regards to “cost-utility analysis”, or any utility-based economic measurement approach, QALYs appear to have wide acceptance because people have intimate knowledge of the worth they place on their personal health. In an economic sense, they know how much they are “willing to pay” to move from one health state to another. Formally, IPSOR (2017) describes this as “net value reflects the willingness to pay for the improvement in well-being minus the opportunity cost of resources used to produce that improvement”. The instruments used to measure QALYs, such as QOL Surveys and Weighting and Utility Scales, capture a population’s perception of how much they value their health, as the state of their health changes from health care interventions.

This reference argues that people also have intimate knowledge of the worth they place on their personal quality of life. They risk life and limb to migrate because they know how much they are “willing to pay” to move from one quality of life state to another. The reporting instruments introduced and referenced in Examples 1 to 4 guide people through recognition of the full worth of several multi-capital “impact areas” that directly impact their quality of life. Those impact areas serve the exact same purpose as the health care states categorized in the underlying QOL survey instruments used for measuring QALYs. The instruments used to measure QASYs, such as Social Performance Assessments or newly designed Weighting and Utility Scales, when applied to populations, capture people’s perception of how much they value their personal quality of life, as it changes from public service improvement interventions.

The health care industry (via Anderson et al, 2014) appears to agree with this “quality of life”, or QASY (or multi-capital DALY), valuation approach in the following statement:

“Another area of uncertainty is the incorporation of quality of life into value. Clearly, a treatment that improves quality of life at a reasonable cost has some value even if it does not improve life expectancy. Combining quality and length of life provides a more accurate estimate of the benefit of any intervention or program.”

The question appears to be whether or not “mixed methods qualitative explanatory paths” as applied in population-based, multi-capital, multi-sector, assessment techniques, such as Multi Criteria Decision Analysis, and the final outcome metric or Social Performance Score, QASY, have the same legitimacy as related single capital assessment techniques used to quantify QALYs and DALYs. 

IPSOR (2017) believes that standard MCDA approaches alone, without QASYs, “elicit from the decision maker the tradeoff values to incorporate issues that cannot or have not been included in CEA or CBA. These models provide a unified single dimensional measure of value of alternative choices using a multi-attribute metric that combines the preference weights specified by “the decision maker” and the performance of alternative “candidates” along each of the dimensions of value.”

They qualify the use of MCDA approaches by endorsing MCDA as useful for capturing additional elements of value that CEA cannot capture, but reaffirm the centrality of QALY-based CEAs. Antioch et al (2017) concur with the IPSOR recommendations for MCDA. Although ICER (2017) does not endorse MCDA, they do endorse using qualitative, mixed methods-like, “contextual considerations” when conducting CEA.

Neither IPSOR, nor Antioch, fully address QASYs, or similar multi-capital QALY alternatives. Their extended and augmented QALYs use multi-dimensional, but not multi-capital, approaches. They also do not address the relation of “impact areas” and “health states” as a basis for quality of life measurements. It’s not clear whether the health care industry, with the exception of WHO, has “missed”, or “dismissed”, the advantages of “mixed methods explanatory approaches”, measured using the M&E-based Performance Monitoring Assessments and Impact Evaluations, explained by references used throughout this tutorial.

The final scientific answer to QASY’s legitimacy goes beyond the scope of this reference, but this reference sees answers starting to be supplied by organizations such as WHO, UNEP, IPSOR, and COSA (2*). In the natural capital sector, Jacob et al (2016) use the term “Integrated Valuation” to describe how these types of valuation approaches are starting to be applied by a “changing valuation culture”.

WHO (2015), in fact, argues the merit of using qualitative explanatory mixed methods, specifically “results chains”, to measure health systems performance. The following image, and their images displayed in Section A, demonstrate that their Performance Assessment approach is similar to the SAFA, and other certification organization, approaches introduced in Examples 1 to 4. Their final outcomes, and their 100 Indicators, match the final impacts measured using this reference’s “impact pathways” and their related Indicators.

Can their [RCA] Framework also serve as the basis of “personal willingness to pay” to move from one quality of life state to another? Although WHO is reluctant to extend their framework to “actions that influence people’s personal behavior”, organizations such as COSA, suggest that, when based upon comprehensive, population-based, longer term, Impact Evaluations, possibly. The references suggest the following requirements for QASYs to serve the same purpose as QALYs and DALYs.

1. First, the Impact Evaluation has to measure how well specific stakeholders’ quality of life have been, or will be, changed by an intervention. Environmental, Economic, Social, and Corporate Governance sustainability must boil down to improvements in stakeholder quality of life. The SPA1 makes that case. Giovannucci (UNFSS, 2016) phrases this as follows: “Ultimately, the motivation for all standards is to improve the lives and livelihoods of farmers and the sustainability of their communities”.

2. Second, the Impact Evaluation’s quality of life (QOL) outcomes must be correctly identified and attributed to specific actions (i.e. clinical guideline performance measures) at population scale. Once attribution is known, those actions and impacts can serve as the basis for measuring “people’s personal behavior”, including their “willingness to pay” to achieve the known (QOL) outcomes. 

3. Third, QOL-based, short term Performance Assessments must be based, initially, upon the results of the high quality, long term, Impact Evaluations. The Impact Evaluations help define the “impact areas” (i.e. including health care states) used in the Performance Assessments Indexes. Absent Impact Evaluations, the Sustainable Food Lab (2016) explains how to set up Item 4’s M&E systems to feed in to Impact Evaluations.

4. Fourth, the Performance Assessment results feedback, via formal Monitoring and Evaluation systems and adaptive learning, to improve subsequent Impact Evaluations, which in turn feedback ….

The agricultural sustainability sector’s (i.e. COSA) pursuit of “outcome [or impact] attribution” appears to parallel the health care industry’s pursuit of “performance measure attribution”. Anderson et al (2014) summarize this pursuit in the following statement. Note that their “observational studies” must encompass the sustainability sectors’ qualitative mixed methods explanatory approaches.

“Once the evidence from randomized clinical trials and observational studies is summarized into clinical practice guidelines, the ACC/AHA Task Force on Performance Measures evaluates those recommendations with the strongest evidence to consider which should become a clinical performance measure. 
…
The writing committee believes that it is important to consider both the cost-effectiveness and total cost burden of potential performance measures before selection. Although these may change over time, explicitly quantifying the cost-effectiveness of treatments at the time that performance measures are created is aligned with the Institute of Medicine (IOM) goal for a more efficient healthcare system and will minimize the likelihood that unintended economic consequences for society and hospitals emerge from adopting a measure.”

In terms of institutionalizing the QOL survey and Weighting and Utility Scale instruments for QASY measurement, UNEP (2011) demonstrates how to apply standardized MCDA instruments to support decisions related to multi-capital and multi-sector stocks at international scale. USEPA (2016) demonstrates how to apply mixed methods evaluation techniques for the same purpose at city scale. The following image (Antioch et al, 2017) summarizes current efforts to institutionalize both MCDA and economic evaluation in the health care sector. 

Once QASY’s, and/or MCDAs, have been institutionalized, the Anderson et al (2014) statement can then be adapted to support Performance Measures dealing with climate change, biodiversity loss, migration, civil rights, and other multi-capital and multi-sector assessment purposes. As an example applied in the natural capital care sector, Liquete et al (2016) demonstrate applying MCDA to value ecosystem services. A multi-sector task force, “RCA Task Force on Performance Measures”, provides the leadership and “clinical guidance” (i.e. see Chapter 12 in WHO 2016 and ICER 2017). Suggesting, once again, that social networks and clubs need to get busy.

The use of QASYs appears consistent with the ECHOOUTCOME (2013) recommendations for improving health outcome measurement, including their statement: “Alternative methodologies for assessing cost-effectiveness should be explored on a case-by-case basis”. In the case of this RCA Framework, and most of its principle references, algorithms need to emphasize “multis over singles”, including outcome metrics, if serious societal risks, such as climate change, are to be reduced.

The ECHOOUTCOME (2013) recommendation, “Cost-Effectiveness Analyses Should Be Expressed as Costs per Relevant Clinical Outcome”, appear similar to Example 4A’s technique of using the final “outcome or impact” Indicator from an “impact pathway” in the same manner as “Clinical Outcome”, but absent the relation to personal utility-based, or economic, valuation approaches. 
The institutional consequences of fully adopting QASYs involve the same institutional improvements recommended throughout this reference. One reason for optimism, besides the source code, is that the use of QASYs and/or MCDAs, QALYs, or DALYs, may boil down to a matter of expanded perception of how to apply widely known, but only partially applied, techniques.



Appendix 2 to Example 4B. Impacts Checklist (from Smidler, 2016)
Impact TEXT datasets can be supplemented with this type of checklist that has been customized for the industry being analyzed.



Appendix 3 to Example 4B. Report Checklist (from Smidler, 2016)




Example 4C. GCEA and LCIA (RCA5)
URLs 
https://www.devtreks.org/greentreks/preview/carbon/resourcepack/Coffee Firm RCA Example 4C/1557/none
http://localhost:5000/greentreks/preview/carbon/resourcepack/Coffee Firm RCA Example 4/541/none
https://www.devtreks.org/greentreks/preview/carbon/output/Coffee Firm RCA4C, CEA/2141223483/none
http://localhost:5000/greentreks/preview/carbon/output/Coffee Firm RCA4C Stock, CEA/2141223498/none
A. Introduction or Goal and Scope
This example combines LCIA data from Example 3B’s S-LCA with Example 4’s coffee crop budget. The initial budgets and LCIAs should be completed jointly to support the LCIA and CEA analysis introduced in this example. 

B. Indicator Thresholds, or System Boundary and Resource Inventory

Example 3, 3A, and 3B, explain that Operating Budgets directly support the Resource Inventory Phase of LCIAs and Capital Budgets support LCIA Scenario Analyses. The following data comes from the Score.MathResults for Example 3B’s Conventional Coffee Production. The column, factor7, stores the Production Processes needed to complete Hotspots Analysis. These Production Processes were taken directly from the Labels used in Example 4’s coffee crop budget Categorical Indexes (i.e. EDF = Irrigation Water Management). The column, factor6, confirms that these CIs are associated with the life cycle stage, crop production. The column, factor4, represents the Elementary Flows, or Environmental Damages and S-LCA Socioeconomic SubCategories. 



The following partial budget took Example 4’s coffee crop budget and removed any crop operations, or Categorical Indexes, that weren’t used in the previous image’s Hotspots Analysis. The previous image’s stylized QTMost, QTLow, and QTUp, certainty, and unit of measurement, columns were then transferred to the crop budget’s CategoricalIndex.factor1, factor2, factor 3, factor4, and factor5 properties. These columns measure normalized and weighted Elementary Flows and S-LCA-related subcategories.



As explained in Example 3, when the LCIA includes multiple Damage Categories for each crop operation, such as Climate Change and Water Use Impacts, the Damages must be allocated, in some manner to the operation (or vice versa). One option is to use the LCIA’s LocationalIndex.QTMost data, and base the allocation on the sum of the percent contribution of each child Categorical Index to the LI (see the last column in the first image). The CEA’s CategorialIndex.QTMost then measures Cost per Unit normalized, weighted, and allocated, LCIA Locational Damage Index. With this technique, the initial budgets should be supply chain budgets so that all life cycle stage damages can be evaluated.

The quantity of data generated by the LCIA techniques supports alternative techniques as well, such as completing a new budget that allocates crop operation budget data to each separate LCIA Categorical Index.

C. Indicators and Life Cycle Impact Assessment CEA

The following Indicator.MathResults show the resultant CEA Analysis. In this example, The Categorical Index columns, QTMost, QTLow, and QTUp, measure Costs per unit Categorical Damage. Most analyses will not be this simple, because, as discussed in the last section, each crop operation will have multiple Damage Impact Categories.



D. Communication and Interpretation and Decisions

Example 3B illustrates using this data to conduct more thorough Hotspots Analysis that accounts for different stakeholder groups, life cycle stages, and production technologies. Example 4B demonstrates how to use Reference Case Cost Effectiveness Results to communicate the final results to decision makers. The Reference Case’s Stakeholder Perspectives, in particular, demonstrate complementary LCIA-S-LCA-CEA techniques.

The complementary use of LCC/LCB budgets, CEA, and LCIA, provide other types of decision support as well. UNSETACd explains the results of O-LCIA can be “allocated”, based on economic factors such as price or revenue, to product categories. They use the following image to confirm that the results of these types of P-LCIA allocations allow more fine-tuned understanding of the actual environmental impacts associated with specific company products. Although Example 4’s Crop Revenue Categorical Impact was removed from this example, it may be possible to use those Output and Revenue categories, with the children Indicators’ product yields and revenues, for these types of product allocations.



References
Same as Examples 3, 4, 4A, and 4B


Additional Examples
The next batch of examples can be found in the Social Performance Analysis 3 reference.

  DevTreks –social budgeting that improves lives and livelihoods		

1