Intelligence | The Responsibility100 Index

Methodology

1.0 Summary

The Responsibility100 Index is a comparative ranking of the FTSE 100 companies based on measurements of their contribution to the UN’s Sustainable Development Goals for 2030. It uses data from a range of sources; including non-governmental organisation research, annual and sustainability reporting, web-based searching, and material from Companies House and regulatory bodies. The indicators that comprise the Index have been selected to measure contributions to the SDGs according to a variety of qualitative and quantitative measures and to provide as comprehensive a portrait of each company given the data available in order to establish comparability.

1.1 What does the methodology report show?

This document is a detailed explanation of the sources, definitions, indicators and weightings that form The Responsibility100 Index. It is intended to clarify our methodology and to outline the conceptual basis upon which we have put the data to use. It also addresses the aims of the investigation, and the ground for the presentation of the data in the form of a comparative index. In addition it contains a full table showing our indicators, the pillars into which they are organised, their respective weightings and the rationale for their inclusion in the Index. We comment on the quality of the data-set that underpins The Responsibility100 Index indicators and its impact on our reporting.

We hope that this paper will help to demonstrate the conceptual framework upon which the Index is based, and provide a transparent explanation of our process and thinking. The purpose of a comparative index is to accelerate a ‘race to the top’ by highlighting successful practices, as well as to put pressure on companies that are underperforming to adapt or improve.

Built into the Index is the recognition that the future of sustainable development relies upon cooperation, transparency and meaningful (rather than cosmetic) action. Our methodology has been developed with great care and in consultation with many outside authorities and experts.

However: it cannot be stressed enough that the Index is still in beta mode. We want to improve and enhance it in any way we can, and hope that companies, other stakeholders and Tortoise members will help us do this by responding to this initial launch with their comments, thoughts and observations.

1.2 Why is The Responsibility100 methodology important?

The United Nations ratified the Sustainable Development Goals in September 2015 as part of the 2030 Sustainable Development Agenda. The overall aim of that agenda is to promote cooperative action to combat the interrelated challenges of climate change, species extinction, social injustice, inequality, intolerance and many others besides. The goals can be seen as part of an ‘indivisible whole’”.

The drafting and ratification of the 2030 Agenda for Sustainable Development was hailed as a considerable geo-political achievement. But the very breadth (and, in some cases, studied vagueness) of these goals also makes their application much more challenging. Who, in truth, would declare themselves to be against human rights? The question is what measurable action those in power – economic, political or otherwise – are taking to enforce them.

Since the goals’ publication in 2015, many publications and guidelines have been produced with a view to fleshing them out. Some of this work has been promising. But it has been hindered by the patchiness of collated data and argument over which metrics should be applied to judge the achievement of this or that SDG. The difficulty of comparing rhetoric with reality has been conspicuous.

Building a complete and dynamic portrait of the contributions that companies are making towards the 2030 Agenda is especially challenging. Companies are inherently complex entities, often leaving a remarkably intricate and widespread ‘footprint’ in relation to the SDGs.

The general character of the goals is matched by the intricacy of measuring them in the corporate sphere. What impact, for instance, does the supply chain of a particular company have upon its ethical performance? What impact do the portfolio investments made by businesses have? How best to include the role played by subsidiaries? How many less tangible contributions to global well-being are businesses making through – say – job security, adult learning opportunities or other initiatives? The quest for the best possible metrics remains open to debate – which we welcome.

Though the SDGs were set for the nations of the world – rather than the business sector in any specific sense – it should be self-evident that their achievement will be impossible without the active cooperation and participation of companies. It should be no less self-evident that the big corporations listed in the FTSE100 have a particularly central role.

1.3 What is the aim of Responsibility100?

The Responsibility100 Index is driven by the three core values of transparency, accountability and responsibility. We believe that we are providing one of the most accurate and comprehensive analysis of the SDGs to date. That said – and this cannot be stressed enough – we warmly welcome feedback on our methodology.

To be clear: this is not an abstract academic exercise, though we stand by its intellectual rigour. We want to accelerate a race to the top amongst the companies of the FTSE 100 – and those outside it. We aspire to build a tool that is of real consequence and value: constantly being refined, constantly improving as a spur to greater social responsibility.

2.0 Structure of the Index

The Responsibility100 Index is organised around pillars, sub-pillars and indicators. Our two main pillars are that of ‘People’ and ‘Planet’. This broad categorisation reflects the fact that most of the Sustainable Development Goals can be defined as predominantly connected to ecological factors and processes – ‘Planet’ – or social and economic factors and processes – ‘People’. Of course, the division is far from absolute. But it is a useful starting point.

The Index includes several thematic sub-categories; Poverty, Climate, Waste, Water, Gender Equality, Investments, Good Business, Charitable Support, Human Rights, Fines & Violations (both ‘People’ and ‘Planet’ related), Memberships (both ‘People’ and ‘Planet’ related) and Self-Reporting (both ‘People’ and ‘Planet’ related); into which the range of indicators is organised.

2.1 Walk and Talk

Each of the indicators in The Responsibility100 Index framework has been identified as either an ‘action’-related or ‘commitment’-related measure. These measures – for which we use the shorthand ‘Walk’ and ‘Talk’ respectively – are a way of comparing and contrasting the level of contributions by companies that require material effort and real output to their rhetoric and presentational claims.

Again, it is important not to be automatically dismissive of ‘talk’. We want to encourage reporting and transparency of all kinds.

For example: companies are rewarded for the quality of their Sustainability Reports – a ‘talk’-related indicator – and this variable contributes to their overall score according to our weightings. Similarly, companies are rewarded for the level of reduction they have achieved in CO2 emissions – a ‘Walk’-related indicator.

The difference between total scores for ‘Walk’ and ‘Talk’ indicators can be expressed as a delta which describes the gap between action and commitment for each company. It is important to stress that this element of the Index’s structure is not fundamental to the ranking, or scoring, it is an investigative device to present a picture that we believe is important.

2.2 Scoring

The total score for a company is calculated as the sum of the indicator value multiplied by the respective weight designated to it. Therefore, all of the indicators regardless of being commitment or action add to the overall score, however they do so in varying amounts.

As each indicator has a weighted value and we have assigned these indicators to groups, e.g, we have designated some of our indicators as commitment, we can create a score for those respective groups by summing the weighted values of the indicators in that group. This method forms the basis for a company’s commitment – or walk – score and its action – or talk – score.

3.0 Factors affecting the ranking

The Responsibility100 Index produces relative scoring, comparing the companies of the FTSE 100; and whilst all aspects of the methodology have been configured to improve the relevance and accuracy of this scoring, some factors affecting the overall ranking must be explained and qualified.

3.1 Missing values

There are a number of missing values across the Index data-set. This has several implications. In order to avoid adding arbitrary influences to the scoring we adopted two basic principles: the absence of data on some indicators is assumed to mean that the company in question is not engaged in a particular activity and that – accordingly – it should not be credited or penalised.

Dig deeper and we encounter a dilemma: in some cases, there may be missing data for a given company that – had it been available – might have scored negatively. Let us say that there is no available data for the ‘Number of Employment Tribunal Cases Upheld’ in the case of a particular company – and that this absence contributes ‘nil’ to the company’s overall score. It is possible that instances of misconduct have simply not been recorded – and that, in this respect at least, the company gets a free pass.

This is one of the principal reasons that we are launching in beta. We want to nudge companies to be more open and honest – and also for those who have reason to believe that they are not being transparent to let us know. The Index is an organic process, not a tablet in stone.

3.2 Imputed values

In some instances, missing values have been replaced with an imputed median value – where a value was needed to establish an average or sum against which other values could be compared. This is a standard practice in data analysis. Those companies that dislike imputation need only be more transparent to avoid its use in the case of their scoring.

3.3 Temporal coverage

The data underlying The Responsibility100 Index refers to a range of years, rather than the single year of the edition, 2019. The inclusion of data is partly due to availability. Depending on the latest year available in which a given variable could be assessed. In order to maintain a wide ranging set of data-points, and improve the analytical usefulness of the data-set, some data is included from 2016 onwards. Please see the full indicator table in this report for more detailed information on the temporal range of each indicator.

We have sought to minimise the number of instances where companies are compared on the basis of data from different years; however, due to the varying cycles of reporting, and the irregular schedule on which some companies and other organisations disclose information this cannot be avoided completely.

3.4 Normalisation

The majority of the variables have been normalised to a [0,1] range so they can be easily compared when ranking. This was required as we had a range of different data types including binary, continuous, ordinal and proportion variables.

xnew = (x – xmin) / (xmax – xmin)

The remaining variables were also normalised to comparable scales, but due to the distribution of the data it was sometimes deemed necessary to transform the values, e.g square/square root transformations, before normalising.

A few of the variables we gathered had both positive and negative values, such as the ‘Gender Pay Gap Improvement’. These were treated similarly, the positive and negative values being independently normalised around 0 to ranges of [0,1] and [-1,0].

3.5 Currency adjustments

Average annual value of the currency was used where conversions were necessary. This means that the rate of exchange, between GBP (£) and USD ($), was calculated using the pair of annual average rates for a given year. Given the maximal variation in currency value across each year we have calculated the maximum potential distortion to a given value from the annual average-based conversion that we have applied across all data for each year.

Year Year High Year Low Year Average Max. Distortion from Annual Average to Annual High Max. Distortion from Annual Average to Annual Low
2019* 1.33 1.2 1.27801 6.10% 4.07%
2018 1.43 1.25 1.34355 6.96% 6.43%
2017 1.36 1.2 1.282692 6.45% 6.03%
2016 1.49 1.21 1.376045 12.07% 8.28%

*correct up to September of 2019

The magnitude of potential distortions to the value of data points such as fines, and charitable donations is indicated by the percentage values shown above. This is an implicit result of the detail with which certain sources that inform our data-set are reported. Often, information about fines and violations is not labelled down to the specific date of occurence. We also contend that the annual average rate of charitable giving is an appropriate tool. All fines, and donations are treated equally within the framework.

4.0 How is the data weighted?

The Responsibility100 Index uses a four-tier weighting system: three weights, each with equal impact on the overall weighting of the indicator, and a fourth reliability factor.

The first tier is weighted for engagement, the second for relevance, the third for impact, and the fourth factor is determined according to the quality – reliability and comprehensiveness – of the data.

4.1 Weighting for engagement

Each indicator is weighted according to the level of engagement that it demonstrates, specifically on progress towards the Sustainable Development Goals. Measurement of engagement is challenging; and the measurability of meaningful effects is often poor when considered on a large scale. In the case of the FTSE 100 certain activities are thoroughly reported, and widely considered to be important, yet their effect on the wider world remains difficult to assess. As part of The Responsibility100 methodology we have rated the indicators through a comparative engagement assessment based on our judgements, expert opinion and our extensive literature review. We have also referred constantly and directly to the UN Framework for the Sustainable Development Agenda and its constituent indicators; some of which we have directly proxied.

Each indicator received a five-point engagement score; based on a judgement of the indicators as either ‘relatively low, medium or high engagement’. Our conceptual scheme for this assessment asked whether the process being indicated required a significant degree of financial or procedural commitment – indicators that did so we weighted more heavily. Also featuring in this assessment was the question of whether the indicator demonstrates a bare minimum level of engagement; as our data-set is being used to create a comparative index, our assessments of impact are all relative. In this sense, the impactfulness of a factor is greater if it is an exemplar activity. We make the argument, therefore, that all companies should meet minimum standards, whether they are regulatory or conventional. By extension we aim to reward more engaged, and relatively impactful activities with higher weighting.

In reference to a range of scientific development and sustainability literature; we have considered the extent to which the Sustainable Development Goals can be viewed as either ‘means’ or ‘ends’. Specifically, some of the Goals are framed with a view to  reaching static end points, e.g. ‘No Poverty’ – whereas others are objectives that call for improvement by degree towards a putative final objective. So – for instance – ‘reduced Inequalities’ is a means of reaching the other ‘end’ goals.

The engagement weighting for each indicator also reflects the extent to which the process or outcome being measured is a meaningful step towards reaching the ‘ends’ of the Sustainable Development Goals, and therefore reach the 2030 Agenda. This aim is looking increasingly doubtful, and on a global scale more concerted recognition and action is needed. The engagement scoring in The Responsibility100 Index is intended to reflect the degree to which companies are accelerating the ‘means’ towards reaching the expressed ‘ends’.

4.2 Weighting for relevance

Each indicator is weighted according to an assessment of its relevance to the Sustainable Development Goals. Whilst we have developed our own thinking on the merits of the Goals, and considered their potential flaws, we have – as a general principle – developed our methodology in accordance with the Agenda.

As such, it is necessary to judge each indicator with respect to its relevance to the specific framework set out by the UN, and to make certain that we are indeed analysing and comparing companies on the basis of relevant and significant action towards the Goals.

Each indicator is identified as being orientated towards one particular Sustainable Development Goal of the 17. It subsequently received a three-point relevance score; the score was based on an assessment of whether the indicator was either; ‘in the spirit of’ the Sustainable Development Goals i.e. that it was linked through implicit meaning. For example, ‘CEO Remuneration’ is included as an indicator as it is ‘in the spirit of’ Sustainable Development Goal 10.0 ‘Reduced Inequalities’. It is, however, given relatively little weight, reducing its overall impact on the total score for a given company. This is because it does not feature as a specific indicator within the UN Framework, nor is it directly referenced as a significant factor in achieving the overall Goal.

Those indicators that are referenced by extension receive a higher relevance score. For example, ‘Presence of Plastic Initiatives’ is included as an indicator, and weighted according to the UN Framework section Goals and Targets for the 2030 Agenda for Sustainable Development – ‘14.1 By 2025, prevent and significantly reduce marine pollution of all kinds, in particular from land-based activities, including marine debris and nutrient pollution’.

The highest relevance scores are attributed to those indicators that directly referenced in the UN Framework – that is, verbatim reference to either a target or indicator within a given Goal represents high relevance. For example, ‘Sustainability Report Relevance’ is included as an indicator, and weighted according to the UN Framework Section Goals and Targets for the 2030 Agenda for Sustainable Development – 12.6 ‘Encourage companies, especially large and transnational companies, to adopt sustainable practices and to integrate sustainability information into their reporting cycle’.

4.3 Weighting for impact

Each indicator is also weighted according to an assessment of the Sustainable Development Goals themselves, and their own relative degree of impact on the future. There is deeply complex and unresolved debate surrounding the Goals, and indeed the overall conception of the risks facing humanity. Sustainability challenges represent the most multifaceted, and urgent, of those currently being addressed by our institutions. The Sustainable Development Goals do not privilege any single goal with more significance than another; each of the 17 is notionally equal. We have, through a process of expert consultation, decided to apply to the weightings a consideration of the impact and urgency of the Sustainable Development Goals, with respect to the role that companies in the FTSE 100 may play in the achievement of the 2030 Agenda.

Redefinition of the term ‘sustainable development’, since the 1987 report from the UN World Commission on Environment and Development, has increasingly gravitated towards recognition that we require ‘development that meets the needs of the present while safeguarding Earth’s life-support system, on which the welfare of current and future generations depends’. This is to say that the most urgent requirement, given the existential threat it poses to our ‘life support systems’, is action on alleviating climate change. Global temperature rise, and consequent degradation of global ecosystems, does indeed threaten our civilisations on an unprecedented scale. We also recognise that these ecosystems form the basis of all human activity and that the environment that we inhabit is the vital context for all other aspects of human life.

We also view the recognition and protection of human rights, by governments, business, communities and individuals as of huge impact and urgency. The integrity of our societies is based upon our ability to recognise and protect common humanity; dignity, compassion and respect for life are all values that must be upheld. Recognition of the climate emergency and its primacy is wholly consistent with this value judgment.

The same process of conceptual assessment and consultation was made for each of our sub-pillars, and for the Sustainable Development Goals themselves to arrive as a theoretical hierarchy of impact and urgency against which it was possible to assign weights to our full range of metrics.

Each indicator was given a five-point score; based on a sliding scale of subjective assessment of the Goals to which the indicators were orientated. Defined conceptually as either; necessary, essential or existential, weighting were assigned to each sub-pillar, and constituent indicators.

4.4 Weighting for data quality

Each indicator is weighted for data quality; giving it a final five-point score, which acts as a factor by which the other weighting tiers are affected. The main considerations behind this aspect of the weighting are: the comprehensiveness of the data-set – the number of companies for which data is available.

4.5 Example weighting case

For example, ‘Percentage Operational Renewable Energy Usage’ is weighted as a 4.0 for impactfulness, acknowledging the financial and procedural commitment necessary to make changes toward renewable energy resource use. It is weighted as a 5.0 for relevance, in reference to Sustainable Development Goal & Target Section 7.2 ‘By 2030, increase substantially the share of renewable energy in the global energy mix’. It is assigned 5.0 for urgency, consistent with the view that divestment from fossil fuels, and switching to renewable sources of energy is a crucial process in reducing emissions of greenhouse gas. Finally, it is weighted as a 3.0 for data reliability; as the overall data-set is not comprehensive, this weighting adjust down the overall effect on the rankings and reflects the fact the reporting of this indicator is inconsistent. Whilst a majority of companies reporting the relevant data, there were too many missing values to reliably impute the remainder of the data-set for the indicator. To compensate for this lack of representation the data quality reduces the overall significance of the values for a company’s overall score in the comparative ranking.

4.6 What is the impact of the weightings?

Each layer of the weighting system for The Responsibility100 Index adds to the accuracy, completeness and explanatory value of the comparative rankings. It is intended to account for the fact that contributions to the Sustainable Development Goals take many different forms, and have varying degrees of impact on the 2030 Agenda.

We recognise, however, that our weightings are based on subjective assumptions, and judgements applied in order to improve the coherence of the data. These subjective judgements affect the composite scoring for each company and therefore their position in the rankings.

5.0 Sensitivity analysis & testing

Our weighting system is one of many different potential approaches. It is therefore important that we test the sensitivity of the data to a range of those potential options. In this beta release of the Index we are using our base weighting approach rather than a randomised weighting. This is because our base weights are, we believe, more closely aligned with the real world, and the editorial values that Tortoise strives to uphold.

In future, we will be reiterating the Index’s findings according to both an Equally Weighted approach and a Randomised Weighted approach. The intention will be to make sure that the weighting system is playing a secondary role to the values within the data-set itself. We expect to find a significant degree of consistency between the rank-ordered comparisons, or ranking pairings, of companies across several different weighting approaches. The results of this sensitivity analysis will be published in subsequent reporting.

6.0 Ongoing investigation

The Beta release date of the Responsibility100 index is 23rd September 2019. The Index was pre-released to the FTSE100 7 days before to review any incorrect values.

To allow the Index to be built in time for the pre-release date, there was a cut-off for new data sources of 3 days before the release. This means that if a company published any new data after this time then it is not included in this iteration of the Index.

The Responsibility100 is not a one off Index however, and will receive minor updates once every quarter when the FTSE reshuffle happens and will receive a full data refresh and review every 6 months.

Bibliography

Camacho, Luis. “Sustainable Development Goals: Kinds, Connections and Expectations.” Journal of Global Ethics 11, no. 1 (February 2015): 18–23. https://doi.org/10.1080/17449626.2015.1010097.

Filho, Walter Leal, Ulisses Azeiteiro, Fátima Alves, Paul Pace, Mark Mifsud, Luciana Brandli, Sandra S. Caeiro, and Antje Disterheft. “Reinvigorating the Sustainable Development Research Agenda: the Role of the Sustainable Development Goals (SDG).” International Journal of Sustainable Development & World Ecology 25, no. 2 (2017): 131–42. https://doi.org/10.1080/13504509.2017.1342103.

Griggs, D, Stafford-Smith, M, Gaffney, O, Rockström, Öhman, M C., Shyamsundar, P, Steffen, W, Glaser, G, Kanie, N, Noble I. ‘Sustainable development goals for people and planet’. (2013) Nature Vol. 495: 305-307.

Hák, Tomáš, Svatava Janoušková, and Bedřich Moldan. “Sustainable Development Goals: A Need for Relevant Indicators.” Ecological Indicators 60 (2016): 565–73. https://doi.org/10.1016/j.ecolind.2015.08.003.

Leeuw, Sander Van Der, Arnim Wiek, John Harlow, and James Buizer. “How Much Time Do We Have? Urgency and Rhetoric in Sustainability Science.” Sustainability Science 7, no. S1 (2012): 115–20. https://doi.org/10.1007/s11625-011-0153-1.

Nilsson, M, Griggs, D, Visbeck, M. ‘Policy: Map the interactions between the Sustainable Development Goals’. (2016) Nature Vol. 534. 320-322.

Scheyvens, Regina, Glenn Banks, and Emma Hughes. “The Private Sector and the SDGs: The Need to Move Beyond ‘Business as Usual.’” Sustainable Development 24, no. 6 (2016): 371–82. https://doi.org/10.1002/sd.1623.

Spaiser, Viktoria, Shyam Ranganathan, Ranjula Bali Swain, and David Sumpter. “The Sustainable Development Oxymoron: Quantifying and Modelling the Incompatibility of Sustainable Development Goals.” SSRN Electronic Journal, 2016. https://doi.org/10.2139/ssrn.2766875.

Sign up for the slow newsletter

Fancy getting a selection of stories, hand-picked by our editors and members, delivered to you each week?