Methodology Development

Ranking Digital Rights (RDR) produces a Corporate Accountability Index that ranks the world’s largest internet, mobile, and telecommunications companies’ disclosed commitments and policies affecting users’ freedom of expression and privacy. The Index is a standard-setting tool aimed at encouraging companies to abide by international principles and standards safeguarding freedom of expression and privacy.

The standards the Index uses to measure companies build on more than decade of work by the human rights, privacy and security communities. These standards include the UN Guiding Principles on Business and Human Rights, which affirm that just as governments have a duty to protect human rights, companies also have a responsibility to respect human rights. The Index also builds on the Global Network Initiative principles and implementation guidelines, which address ICT companies’ specific responsibilities towards freedom of expression and privacy in the face of government demands to restrict content or hand over user information. It further draws on a body of emerging global standards and norms around data protection, security, and access to information. The data and analysis produced by the Index informs the work of human rights advocates, policymakers, and responsible investors and is used by companies to improve their own policies and practices.

The Index methodology was developed over three years of research, testing, consultation, and revision in consultation with stakeholders across several project phases. Since its inception, the project has engaged closely with researchers around the globe. For the methodology development, the pilot study, and the inaugural Index we also partnered with Sustainalytics, a leading provider of ESG (environmental, social, and governance) research to investors.

In 2015, RDR launched its inaugural Index, which ranked 16 internet and telecommunications companies.

For the 2017 Index, RDR expanded the ranking to 22 companies, which included all of the companies ranked in 2015 plus an additional six companies. In addition to internet and telecommunications companies, the Index was expanded with new types of services, including those that produce software and devices that we call “mobile ecosystems.” As a result, the RDR team further revised the 2017 methodology based on a detailed review of the raw data from the 2015 Index as well as consultations with stakeholders from civil society, academia, the investor community, and the companies themselves.

The 2018 Index applies the same methodology to evaluate the same 22 companies as in the 2017 Index. This will enable us to produce comparative analyses of each company’s performance and to track overall trends.

We believe that rigorous research, stakeholder consultation, and transparency about the methodology development process are vital if a ranking is to be credible, rigorous, and effective.

Below is a detailed description of this process.

Theory and strategy: Initial research and consultation for the Index started in the second half of 2012. For more about the core concepts related to business and human rights, ICT and human rights, and the project’s “theory of change,” please see our Theory and Strategy overview. We also carried out research on best practices for corporate rankings in general as can be seen from our 2013 report on Elements of Successful Ranking Systems.

Consultations: Two stakeholder meetings in late 2012 and Spring of 2013 helped shape the ranking’s focus and framed research questions we needed to answer before developing the methodology. The project team has continued to consult with stakeholders at a range of conferences as well as in smaller bilateral or private group meetings.

Case study research: Initial case study research conducted in the second half of 2013 and early 2014 helped us answer key questions before starting to draft the methodology. Our Phase 1 Case Study Research report describes what we learned from that research, followed by summaries of each case study’s findings and recommendations.

Human Rights risk scenarios: Parallel to the case study research we carried out consultations with civil society groups, activists, and journalists on a set of scenarios describing risks that they may face when using ICTs. Our Human Rights Risk Scenarios document raises key questions about company policies and practices in relation to those user risks.

Methodology drafting and consultation

The 2015 Corporate Accountability Index methodology was developed across the following phases:

  • Phase 1 Draft Methodology v1 (February 2014): This is the first iteration of RDR’s methodology, based on case study research and stakeholder consultation.
  • Phase 1 Draft Methodology v2 (May 2014): This is the second iteration of RDR’s methodology. We incorporated insights from our case study research as well as feedback from a variety of stakeholders to create this version. We invited stakeholders and the public to comment on this version.
  • Phase 1 Pilot Methodology (October 2014): This is the third iteration of RDR’s methodology. We updated the methodology based on feedback we received on version two. This is the methodology tested in the Phase 1 pilot.

Pilot study: In October 2014 in partnership with Sustainalytics we launched a Pilot Study, testing the methodology on 12 companies. The research was completed in January 2015. Following consultations with stakeholders on the draft, a revised version of the pilot study report was published. A blog post describing the study is here and the full report can be downloaded here.

  • 2015 Index Indicators. These are the indicators used for the 2015 Corporate Accountability Index, the product of revisions based on what we learned from the pilot. Also see the list of companies and a description of the research process.
  • 2015 Methodology. This document provides further explanation about how we applied the indicators to examine 16 companies.

Following the release of the 2015 Index results, the RDR team reviewed the raw data and worked in consultation with stakeholders to revise and improve the 2017 Index methodology. This occurred over the following stages:

The 2018 Index applies the same methodology as the 2017 Index.

[last updated 25 July 2017]