Evaluating effectiveness

ASI periodically conducts independent outcome and impact evaluations on specific topics of critical importance to implementing our Monitoring & Evaluation program. These outcome or impact evaluations are undertaken by independent third-parties to document results and evaluate the effectiveness of ASI’s standards and certification program.

Below, we provide the evaluations that have been completed, are ongoing, and are in the planning stage.

Completed publications

Analysis of Implementation of Key Governance and Social Criteria from ASI Certified Entities

Study prepared for the Aluminium Stewardship Initiative (ASI) by: Dr. Leopold von Carlowitz

  • Consultant, Law, Rights, Resources
  • Associate, MSP Institute – Multi-Stakeholder Processes for Sustainable Development

Completion date: 25 September 2021

Findings and Recommendations

The following recommendations are made to ASI with respect to the general improvement of data quality and reporting standards; with respect to conceptual issues relating to particular criteria; and with respect to training and capacity building.

General improvement of data quality and reporting standards:

  • Consider improving data capture, analysis and comparison by introducing more specific reporting requirements based on specific key terms, activities and points to consider contained in the Guidance to the Performance Standard. Special focus should be placed on the elements, design and implementation of a responsible sourcing policy (criterion 2.4), of adequate complaint resolution mechanisms (criterion 3.4) and of policies and processes to respect the rights of Indigenous peoples (criteria 9.3 and 9.4).
  • Consider developing reporting schemes that differentiate performance geographically and by supply chain activities in a broader Certification scope.

Conceptual issues for specific criteria:

  • Criterion 2.4: Determine whether or not an effective responsible sourcing policy should include due diligence measures for suppliers beyond the first tier.
  • Criterion 3.4: Clarify and explain the meaning of the principles for complaint resolution mechanisms, i.e. accessible, transparent, understandable, culturally sensitive, gender sensitive and adequate to address stakeholder complaints.
  • Criterion 9.3: Reflect and confirm to what extent general CSR-measures building capacities for Indigenous peoples count for implementation of the criterion, if they are not directly linked to Certified operations of the Entity.
  • Criterion 9.3: Determine ASI’s stance towards the existence of Indigenous peoples in China.
  • Criteria 9.3 and 9.4: Develop more detailed points to consider for determining how close Indigenous peoples must be situated to Entities’ operations so that their concerns are to be considered for the fulfillment of the criteria. 2

Training and capacity-building

  • Criterion 3.4: Train auditors with respect to the principles for an adequate complaint resolution process, including training on cultural and gender sensitivity.
  • Criterion 9.4: Train auditors as to the particulars of FPIC processes, including methodological issues and procedural requirements.
  • Criteria 9.3 and 9.4: Strengthen the capacity of IPAF to give advice on issues relating to Indigenous peoples and to network with Indigenous peoples’ organizations in critical countries with oppressed Indigenous communities and national minorities.

ASI Management Response

This study provides very useful insights into the consistency of implementation of Principle 8 (Biodiversity), and in particular criterion 8.1, 8.2 and 8.5, across its certified Entities. Principle 8 addresses specific topics such as biodiversity assessment, biodiversity management, alien species, “No Go” and rehabilitation. Where possible, the research analysed, by location and supply chain activity, the quality of the implementation of these criteria.

The Secretariat welcomes the recommendations from this evaluation, and they directly informed the ASI Standards Revision 2020-2022 and ASI Performance Standard Principle 8 – Biodiversity. The second draft documents went out for public consultation in January 2022. The report findings have also been discussed with the ASI Standards Committee during their 20 October 2021 meeting: https://aluminium-stewardship.org/wp-content/uploads/2021/12/ASI-SC-Teleconference-Minutes-20Oct21.pdf

In regards to the specific recommendations:

Recommendation 1: The Secretariat has made the decision that this recommendation was too specific to incorporate into the Assurance Manual and would be better incorporated on an ‘as required’ basis in the pre-audit assessments that ASI is now doing for the audit plan and when providing feedback to the auditor (also see recommendation 4). 3

Recommendation 2: The decision has been made that the ‘exemption option’ was not to be implemented at this time as ASI can not provide exemption for all of Provision 8 if criterion 8.1 was rated as Low Risk. There are other criteria in Provision 8 that should apply regardless of biodiversity risk (such as alien/weed species and rehabilitation/closure). This recommendation was therefore intentionally not enacted at this time. Entities certainly can (and ASI’s assurance systems offers) ‘not applicability’ for those directly-related criteria to 8.1 (i.e. 8.2a, b, and c).

Recommendation 3: Similar to recommendation 1, the Secretariat has made the decision that this was too specific to incorporate into the Assurance Manual and applies to high-risk sites only. However if biodiversity is flagged as a High-Risk issue in the pre-audit assessment, ASI reports to the audit team prior to the audit that detailed evidence needs to be included in the ‘Observations and Findings’ section of the audit report. The ASI Assurance team also reviews the detailed findings and looks for appropriate discussion by the auditor during the oversight assessment.

Recommendation 4: From January 2022, ASI requires a pre-audit assessment of audit plans/schedules/interviews for upstream Entities (initial Certification Audits) to better assess and clarify what is expected from Auditors when preparing and conducting the audit. The term ‘biodiversity outcome’ has been deleted from criterion 8.2c, and the ASI Performance Standard Guidance provides additional information and references to explain what is required to meet criterion 8.2c.

ASI’s auditor training modules will be updated when the revised ASI Standards are launched in May 2022, and the biodiversity component, and in particular recommendation 1 and 3, will be part of that (to be completed in early 2023).

Analysis of Implementation of Biodiversity Assessment and Management by ASI Certified Entities

A Report by: Stuart Anstee and Associates
Completion date: 13 August 2021

Conclusion & Recommendations

The Performance Standard audit reports, combined with company websites and sustainability reports provide a reasonable, but by no means complete, picture of the consistency of implementation of the ASI Performance Standard Criterion 8.1, 8.2 and 8.5. The information assessed indicates that ASI Entities are implementing the key elements required by the biodiversity criteria. That is, they are assessing the biodiversity risk and materiality of their impacts and then developing Biodiversity Action Plans to mitigate those impacts. It also appears that Entities and their auditors have a sound understanding of the concept of ‘area of influence’ and use it actively in the risk assessment process.

A key requirement of Criterion 8.1 is to assess the risk and materiality of an Entity’s impact on

biodiversity. Whilst it is not mandatory to report the results of this assessment in a public forum, it seems appropriate from an ASI assurance standpoint for the audit report to detail the results of the Entity’s risk assessment. A review of the audit findings for Criterion 8.1 found that documentation of an Entity’s biodiversity assessment result was not universal. While it is possible to determine which Entities have low or no material impact on biodiversity, it was not possible to determine the detail and level of materiality of those Entities that were assessed as having an impact on biodiversity.

Recommendation 1: Require auditors to provide full and complete details on the outcomes of the

Entity’s biodiversity risk assessment results as part of the Performance Standard audit report to ASI.

It is also clear that biodiversity risk assessment undertaken by Entities has not identified biodiversity as a material issue for many Entities with downstream activities. The ASI should look at the value of lessening the focus on these Entities so that a greater focus can be applied to those Entities that do have material risks and impacts on biodiversity. 2

Recommendation 2: Consider implementing a process whereby Entities that have been shown to have no material impact on biodiversity as part of Criterion 8.1 be given an exemption for the other criteria in Principle 8 of the Performance Standard.

While the information available in the audit summaries and public reports and websites provided a high-level understanding of the Standards implementation, it is not able to resolve how well elements such as biodiversity target setting, and the use of the mitigation hierarchy are being used by the Entities. ASI should determine if there is a need for auditors to provide more detail in their reports on elements such as target setting and the use of the mitigation hierarchy. This will enable ASI to gain a clearer understanding as to how these elements are being used by Entities to manage biodiversity risks and impact in a way that is consistent with principle 8 of the Performance Standard.

Recommendation 3: Require auditors to provide full and complete details on biodiversity target

setting and the use of the mitigation hierarchy as part of the performance standard audit report to ASI.

The review found that the communication of biodiversity outcomes through public websites and

reports is limited. In addition, there appears to be a level of confusion at both the Entity and auditor level on what constitutes a biodiversity outcome. The ASI should consider clarifying its definition of biodiversity outcome. Specifically, it should consider if there is a need to differentiate between a process outcome and a performance outcome and what the preferred outcome is when working to achieve compliance with Criterion 8.2c.

Recommendation 4: The ASI should develop and provide a clear and unambiguous definition of the term biodiversity outcome to improve clarity on what constitutes conformance with criterion 8.2c.

ASI Management Response

This study provides very useful insights into the consistency of implementation of Principle 8 (Biodiversity), and in particular criterion 8.1, 8.2 and 8.5, across its certified Entities. Principle 8 addresses specific topics such as biodiversity assessment, biodiversity management, alien species, “No Go” and rehabilitation. Where possible, the research analysed, by location and supply chain activity, the quality of the implementation of these criteria.

The Secretariat welcomes the recommendations from this evaluation, and they directly informed the ASI Standards Revision 2020-2022 and ASI Performance Standard Principle 8 – Biodiversity. The second draft documents went out for public consultation in January 2022. The report findings have also been discussed with the ASI Standards Committee during their 20 October 2021 meeting: https://aluminium-stewardship.org/wp-content/uploads/2021/12/ASI-SC-Teleconference-Minutes-20Oct21.pdf

In regards to the specific recommendations:

Recommendation 1: The Secretariat has made the decision that this recommendation was too specific to incorporate into the Assurance Manual and would be better incorporated on an ‘as required’ basis in the pre-audit assessments that ASI is now doing for the audit plan and when providing feedback to the auditor (also see recommendation 4). 3

Recommendation 2: The decision has been made that the ‘exemption option’ was not to be implemented at this time as ASI can not provide exemption for all of Provision 8 if criterion 8.1 was rated as Low Risk. There are other criteria in Provision 8 that should apply regardless of biodiversity risk (such as alien/weed species and rehabilitation/closure). This recommendation was therefore intentionally not enacted at this time. Entities certainly can (and ASI’s assurance systems offers) ‘not applicability’ for those directly-related criteria to 8.1 (i.e. 8.2a, b, and c).

Recommendation 3: Similar to recommendation 1, the Secretariat has made the decision that this was too specific to incorporate into the Assurance Manual and applies to high-risk sites only. However if biodiversity is flagged as a High-Risk issue in the pre-audit assessment, ASI reports to the audit team prior to the audit that detailed evidence needs to be included in the ‘Observations and Findings’ section of the audit report. The ASI Assurance team also reviews the detailed findings and looks for appropriate discussion by the auditor during the oversight assessment.

Recommendation 4: From January 2022, ASI requires a pre-audit assessment of audit plans/schedules/interviews for upstream Entities (initial Certification Audits) to better assess and clarify what is expected from Auditors when preparing and conducting the audit. The term ‘biodiversity outcome’ has been deleted from criterion 8.2c, and the ASI Performance Standard Guidance provides additional information and references to explain what is required to meet criterion 8.2c.

ASI’s auditor training modules will be updated when the revised ASI Standards are launched in May 2022, and the biodiversity component, and in particular recommendation 1 and 3, will be part of that (to be completed in early 2023).

Analysis of implementation of GHG Emissions reporting from ASI Certified Entities: March 2020-March 2021 update

Report prepared by Atmolite Consulting Pty Ltd Varsity Lakes, Gold Coast QLD 4227, Australia

Completion date: 12 October 2021

Recommendations

Recommendations that could be adopted by ASI to further improve the quality, transparency and benchmarking of data disclosures, include:

a) Encouraging disclosures relevant to facility or activity, which allows aggregation to ASI certification scope level (not aggregated data that can include non-certified production sites or multiple processes);

b) Encouraging disclosures of not only electricity use, but total energy use and energy use per energy carrier (fuels, electricity, etc.) which is a more relevant proxy for GHG emissions and carbon footprint (particularly for non-Smelter processes); and

c) Standardising the scope and units of disclosure, both totals and intensities (per unit of production), preferably indicating the specific numerators and denominators used when intensities are reported. These could be implemented through use of standardised data reporting templates, such as those already employed by the International Aluminium Institute (IAI) and other industry associations.

Finally, as per the IAI’s Beyond 2 Degrees Scenario (B2DS) to 2050 (IAI, 2021), the ultimate goal is decarbonisation of the entire aluminium sector, and particularly important for the primary aluminium smelting sector. Success will require all aluminium smelters to shift down the current emissions curve, particularly those currently using carbon-intensive sources of power. ASI could play a part in incentivising this by providing certification pathways for all smelters on the emissions curve, whilst ensuring clear, significant and time-bound emission reductions are demonstrated.

ASI Management Response

This follow-up study provides very useful insights in the energy use and GHG emissions reporting and disclosures of ASI Entities, energy use and GHG emissions comparison across ASI aluminium supply chain activities, from bauxite mining to downstream activities. ASI welcomes the findings and recommendations from the report, and will use these in the following ways:

  • As with the 2020 study, the findings and recommendations from this evaluation will directly inform the ASI Standards Revision 2020-2022 and ASI Performance Standard Principle 5 – GHG Emissions. The second draft documents will go out for public consultation in Q1 2022. The report findings will also be discussed with the ASI Standards Committee and GHG Working Group.
  • In addition to the recommendations from this report, ASI has proposed in the draft ASI Performance Standard V3.0 (draft 1.0 – March 2021) that publicly disclosed GHG data to be independently verified. For an overview of ASI’s Standards Revision process and new requirements for Principle 5, please refer to the ‘ASI Standards – 2021 Consultation Round I – Overview’ document.
  • ASI agrees with the authors that the current ‘2030 smelter emissions threshold’ of 8 t CO2e/t Al provides less incentive for smelters in the upper 50% of the emissions curve to consider ASI certification. To engage more smelters and provide further incentives, ASI has therefore included alternative pathways in the ASI Performance Standard V3.0 (draft 1.0 – March 2021) to certification for smelters above the current 8t CO2e/t threshold, whilst still requiring clear, significant and time-bound reductions in GHG emissions.
  • This evaluation will also inform the development of GHG-specific member and auditor training modules which will be published as part of the ASI Standards Revision process. The modules aim to support the implementation and the evaluation of the revised ASI Performance Standard Principle 5 Criteria, and will help Entities point to relevant tools and resources for GHG accounting and reporting.
  • The comparison of publicly disclosed data from ASI Entities against data from CRU’s Emissions Analysis Tool, for primary aluminium activities, provides for the valuable checking of data and is a tangible outcome of ASI’s recent MoU with CRU. Over the coming months, ASI Certified Entities’ CoC certification status, inclusion of rolling mills and the challenges of differing GHG inventory scopes will be added to the CRU Emissions Analysis Tool.
  • In conjunction with the CRU Emissions Analysis Tool, the ASI Secretariat will also use the analysis contained in this report to inform oversight of audit reports for GHG disclosures. The study provides a ‘year on year’ snapshot of Certified Entity data being publicly disclosed, enables comparison between similar Entity types. In addition, the CRU Emissions Analysis Tool enables the Secretariat to compare the Entity’s publicly disclosed data against the best available emissions intensity estimate published in the CRU Tool. Upon identification of any material discrepancy, the ASI Secretariat will liaise with the auditors and/or Entity to address any errors or inaccuracies in the public data.

Data Collation and Validation of Greenhouse Gas Emissions (GHGs) from ASI Members

Report prepared by Atmolite Consulting Pty Ltd Varsity Lakes, Gold Coast QLD 4227, Australia

Completion date: 14 November 2020

Findings

This report summaries a review of published Greenhouse Gas (GHG) Emissions and energy use data from certified ASI Entities, and was conducted by Atmolite Consulting Pty Ltd. Where possible, totals and intensities (per unit product) were obtained for energy use and GHG emissions (Scope 1, 2 and where available Scope 3). The review covered 51 ASI Entities, from 34 ASI members. Of these, 15 Entities were engaged in aluminium smelting.

The findings here represent a snapshot of GHG data disclosures as of July 2020, with granularity at the regional, country and supply chain activity level. The data collected was from the most recent year for each Entity (2018 or 2019), and where available, up to 5 years’ historical data to examine trends over time.

Note that this report uses generic labels (region and numeral) to refer to specific ASI Entities; for example, ‘SA-1’ refers to Entity #1 in South America.

Data challenges

All data acquired for this exercise were collected from publicly available sources. However, complete data sets – for both emissions and energy use – were not found for a large number of ASI Entities, despite reasonable efforts to source these. Challenges in obtaining data are described in the report, with a ‘data quality rating’ provided for each Entity. For example, no data were found for five out of 51 Entities, and data disclosure was rated ‘poor’ for 12 Entities. In light of these data challenges, it is recommended that ASI and its GHG Working Group consider the creation and adoption of standard data templates, or consistent reporting requirements, for Entities disclosing emissions and energy use.

Given the gaps in data completeness, absent emissions and/or energy values were calculated using other published data – for example, conversions of totals to intensities if production figures are known. This allowed for more meaningful comparisons across ASI’s membership. Furthermore, where data specific to an ASI Entity was not available, data from their parent / corporate Entities (if applicable) was used to ‘fill the gaps’. Data issues for specific Entities are described in the report’s appendices.

Many ASI Entities report aggregated data across multiple supply chain activities, rather than individually. To facilitate comparison by activities, the aluminium supply chain was redefined into five categories: Bauxite, Alumina, Smelting, ‘Remelt-to-Downstream’ and Downstream. The resulting energy and emissions profiles (using intensity units, GJ and t CO2e per t product) across the ASI membership were then examined by supply chain, region and country.

Trends in energy intensity

As expected, the Smelting supply chain activity exhibits the highest energy intensity, with a median of ~60 GJ/t product, compared to 5-7 GJ/t product for the Remelt and Downstream activities. Comparison with Bauxite and Alumina activities was not possible, given the lack of data. In Smelting, variability in energy intensity is noted in the Gulf Cooperation Council (GCC) region (different accounting basis) whilst Entities in West Europe exhibit the lowest energy intensity. Data Collation & Validation of GHGs from ASI Members, as of July 2020 iv Trends in GHG emissions intensity (Scope 1+2) Consistent with the energy profile, Smelting is responsible for the highest emissions across the supply chain activities within ASI’s membership, with a median and average of 3.1 and 5.7 t CO2e/t Al, respectively. Smelting also has the most significant variation compared to other supply chain activities, ranging from 1.6 to 17.8 t CO2e/t Al. This is driven by the range of electricity sources – from renewable hydro to carbon-intensive coal. Nine of 11 reporting Smelting Entities meet the current ASI Performance Standard target of 8 t CO2e/t Al. Seven Entities (mainly hydro powered) are at 4 t CO2e/t Al or less, situated in West Europe, East/Central Europe, North America South America, or Oceania. GCC smelting Entities (mainly natural gas powered) are at or just above the 8 t CO2e/t Al level. Despite representing >50% of global smelting capacity, China was only represented by a single ASI Entity with the 17.8 t CO2e/t Al data point, due to coal being the energy source. Significant changes in the energy mix are needed if smelters in this region are to meet the 8 t CO2e/t Al target over the next ten years. Some potential trends in China are discussed in this report.

ASI’s Bauxite and Alumina supply chain activities exhibit lower GHG emissions, at below 0.1 t CO2e/t bauxite and with a median of 0.7 t CO2e/t alumina, respectively. The median value for Alumina is perhaps lower than might be expected from the International Aluminium Institute’s (IAI) modelling of the emissions from the global alumina supply chain activity. The Remelt-to-Downstream and Downstream activities – representing over 50% of ASI Entities – reported median intensities of 0.8 and 0.5 t CO2e/t Al, respectively.

In terms of trends in emissions or energy use over time, most Entities are reasonably stable or reducing, with some exceptions due to production increases or changes in investments/divestments.

Impact of ASI membership

An observed positive impact of ASI’s membership has been the adoption of emissions, energy and general sustainability disclosures for companies and regions (particularly developing nations) that did not previously exhibit this practice.

Given the global Smelting activity’s significant contribution to GHG emissions, it may be advantageous for ASI to provide alternative pathways for engagement and membership (if not certification) for smelting Entities that currently source their energy from coal (or other carbon-intensive sources). This could focus on Entities that cannot practically meet the 8 t CO2e/t Al emissions target but can still demonstrate material reductions over time.

ASI Management Response

This outcome evaluation provides useful insights into the challenges facing ASI’s Members and Entities in adopting a consistent, accurate and complete approach to GHG emissions reporting and public disclosure. ASI has considered the findings of this Report with respect to its current review process of the ASI Performance Standard, including the criteria relating to Principle 5 – Greenhouse Gas Emissions.

  • The ASI GHG Working Group and ASI Standards Committee have both also reviewed the findings of this Report, and considered its main outcomes in the development of revised draft criteria.
  • During 2021, the findings and recommendation from this evaluation will also directly inform the revision of existing ASI Performance Standard Guidance and development of new Guidance (as required) to better support Members and Entities in effectively implementing the revised requirements of Principle 5 of the Standard when the updated Standard is released in early 2022.
  • The outcome evaluation also offers the ASI Secretariat additional baseline and comparative data to support the audit oversight process, when assessing Entity performance against select criterion of Principle 5.
  • ASI also expects this evaluation to inform the development and facilitation of GHG-specific auditor training modules as part of the revised Standard implementation process.

In-progress studies

ASI Certification benefits and drivers study

The ASI Certification Program was launched in December 2017.  As of 18 August 2021, 96 certificates have been issued against the ASI Performance Standard (PS) and 36 certificates have been issued against the ASI Chain of Custody (CoC) Standard.

After more than 3 years of implementing its Certification Program, ASI is conducting an internal research project in Q3 and Q4 2021 to understand what the drivers, benefits and challenges are for different types of companies seeking ASI PS and CoC Certification.

Planned studies

A written report including the executive summary, methodology, aggregate results, and conclusions will be uploaded on this page.