Methodology Development

Ranking Digital Rights (RDR) produces a Corporate Accountability Index that ranks the world’s largest ICT companies’ public commitments to users’ freedom of expression and privacy rights. The Index is a standard-setting tool aimed at encouraging companies to abide by international principles and standards safeguarding freedom of expression and privacy.

The standards the Index uses to measure companies build on more than decade of work by the human rights, privacy and security communities. These standards include the UN Guiding Principles on Business and Human Rights, which affirm that just as governments have a duty to protect human rights, companies also have a responsibility to respect human rights. The Index also builds on the Global Network Initiative principles and implementation guidelines, which address ICT companies’ specific responsibilities towards freedom of expression and privacy in the face of government demands to restrict content or hand over user information. It further draws on a body of emerging global standards and norms around data protection, security, and access to information. The data and analysis produced by the Index informs the work of human rights advocates, policymakers, and responsible investors and is used by companies to improve their own policies and practices.

In 2015, RDR launched its inaugural Index, which ranked 16 Internet and telecommunications companies. For the 2017 Index, RDR has expanded the ranking to cover additional companies and services, including those that produce software and devices that create what we call mobile ecosystems.” As a result, we expanded the methodology, adding new indicators and elements to account for the potential threats to users’ freedom of expression and privacy that can arise from use of networked devices and software. The RDR team also further refined the 2017 methodology based on a detailed review of the raw data from the 2015 Index as well as consultations with stakeholders from civil society, academia, the investor community, and the companies themselves.

We believe that rigorous research, stakeholder consultation, and transparency about the methodology development process are vital if a ranking is to be credible, rigorous, and effective.

Below is a detailed description of this process.

Theory and strategy: Initial research and consultation for the Index started in the second half of 2012. For more about the core concepts related to business and human rights, ICT and human rights, and the project’s “theory of change,” please see our Theory and Strategy overview. We also carried out research on best practices for corporate rankings in general as can be seen from our 2013 report on Elements of Successful Ranking Systems.

Consultations: Two stakeholder meetings in late 2012 and Spring of 2013 helped shape the ranking’s focus and framed research questions we needed to answer before developing the methodology. The project team has continued to consult with stakeholders at a range of conferences as well as in smaller bilateral or private group meetings.

Case study research: Initial case study research conducted in the second half of 2013 and early 2014 helped us answer key questions before starting to draft the methodology. Our Phase 1 Case Study Research report describes what we learned from that research, followed by summaries of each case study’s findings and recommendations.

Human Rights risk scenarios: Parallel to the case study research we carried out consultations with civil society groups, activists, and journalists on a set of scenarios describing risks that they may face when using ICTs. Our Human Rights Risk Scenarios document raises key questions about company policies and practices in relation to those user risks.

 

Methodology drafting and consultation

The 2015 Corporate Accountability Index methodology was developed across the following phases:

  • Phase 1 Draft Methodology v1 (February 2014): This is the first iteration of RDR’s methodology, based on case study research and stakeholder consultation.
  • Phase 1 Draft Methodology v2 (May 2014): This is the second iteration of RDR’s methodology. We incorporated insights from our case study research as well as feedback from a variety of stakeholders to create this version. We invited stakeholders and the public to comment on this version.
  • Phase 1 Pilot Methodology (October 2014): This is the third iteration of RDR’s methodology. We updated the methodology based on feedback we received on version two. This is the methodology tested in the Phase 1 pilot.

Pilot study: In October 2014 in partnership with Sustainalytics we launched a Pilot Study, testing the methodology on 12 companies. The research was completed in January 2015. Following consultations with stakeholders on the draft, a revised version of the pilot study report was published. A blog post describing the study is here and the full report can be downloaded here.

Based on what we learned from the pilot study, we revised the methodology, indicators, and research process for the inaugural 2015 Corporate Accountability Index:

  • 2015 Methodology. This document provides further explanation about how we applied the indicators to examine 16 companies.

Following the release of the 2015 Index results, the RDR team reviewed the raw data and worked in consultation with stakeholders to revise and improve the 2017 Index methodology. This occurred over the following stages:

  • The RDR team incorporated feedback received during the public consultation phase to finalize the 2017 Index methodology.

Last updated: Sept 14, 2016