Index methodology and scoring
Ranking Digital Rights (RDR) produces a Corporate Accountability Index that ranks the world's most powerful internet, mobile, and telecommunications companies' disclosed commitments and policies affecting users' freedom of expression and privacy. The Index is a standard-setting tool aimed at encouraging companies to abide by international human rights principles and standards for safeguarding freedom of expression and privacy.
The Index methodology was developed over three years of research, testing, consultation, and revision. Since inception the project has worked closely with researchers around the globe. For methodology development, pilot study, and the inaugural Index we also partnered with Sustainalytics, a leading provider of ESG (environmental, social, and governance) research to investors.
For the 2017 Index, we expanded the ranking to cover additional types of companies and services, including those that produce software and devices that create what we call "mobile ecosystems." As a result, we also expanded the methodology, which included a period of public consultation, adding new indicators and elements to account for the potential threats to users' freedom of expression and privacy that can arise from use of networked devices and software.
The 2017 Index consists of 35 indicators in three main categories: Governance, Freedom of Expression, and Privacy. Each category contains indicators measuring company disclosure for that category; each indicator is comprised of a series of elements that measure company disclosure for that indicator.
View or download the full methodology here.
Index categories :
- Governance (G): This category contains six indicators measuring company disclosure of commitments to freedom of expression and privacy principles along with measures taken to implement those commitments across the company's global operations.
- Freedom of Expression (F): This category contains 11 indicators measuring company disclosure of policies that affect users' freedom of expression.
- Privacy (P):This category contains 18 indicators measuring company disclosure of policies and practices that affect users' privacy rights.
The 2017 Index evaluates 22 companies. While every company examined has attributes that make it unique, for the purpose of our evaluation and scoring we divided the 22 companies into two groups.
Internet and mobile: These company types were evaluated together because Google is both an internet company and is also a mobile ecosystem company, and Apple also offers services such as iMessage and iCloud. We did not evaluate hardware attributes of devices, focusing our assessment on disclosures pertaining to the newest devices offered by those companies and their operating systems. The freedom of expression and privacy issues faced by mobile cloud data and operating systems overlap with the issues faced by traditional internet services. Additional elements relevant only to mobile ecosystems were added to some indicators. For each company we examined up to four services, as follows:
- Apple (U.S.): iOS mobile ecosystem, iMessage, iCloud
- Baidu (China): Baidu Search, Baidu Cloud, Baidu PostBar
- Facebook (U.S.): Facebook, Instagram, WhatsApp, Messenger
- Mail.Ru (Russia): VKontakte, Mail.Ru email, Mail.Ru Agent
- Microsoft (U.S.): Bing, Outlook.com, Skype
- Kakao (South Korea): Daum Search, Daum Mail, KakaoTalk
- Google (U.S.): Search, Gmail, YouTube, Android mobile ecosystem
- Samsung (South Korea): Samsung implementation of Android
- Tencent (China): QZone, QQ, WeChat
- Twitter (U.S.): Twitter, Vine, :Periscope
- Yahoo (U.S.): Yahoo Mail, Flickr, Tumblr
- Yandex (Russia): Yandex Mail, Yandex Search, Yandex Disk
Telecommunications companies - For these companies we evaluated global group-level policies for relevant indicators plus the home-country operating subsidiary's pre-paid and post-paid mobile service, and fixed-line broadband service where offered, as follows:
- Ammérica Móvil (Mexico): Telcel
- AT&T (U.S.): AT&T Mobile, AT&T Broadband
- Axiata (Malaysia): Celcom
- Bharti Airtel (India): India Airtel Mobile, India Airtel Broadband
- Etisalat (UAE): Etisalat UAE Mobile, Etisalat UAE Broadband
- MTN (South Africa): MTN South Africa Mobile
- Ooredoo (Qatar): Ooredoo Qatar Mobile, Ooredoo Qatar Broadband
- Orange (France): Orange France Mobile, Orange France Broadband
- Telefónica (Spain): Movistar Mobile, Movistar Broadband
- Vodafone (UK): Vodafone UK Mobile, Vodafone UK Broadband
All companies evaluated in the Index are multinational corporations listed on a major stock exchange. The following factors influenced company selection:
User Base: The companies in the Index have a significant footprint in the areas where they operate. The telecommunications companies have a substantial user base in their home markets, and the internet companies have a large number of global users as identified by established global traffic rankings such as Alexa. The policies and practices of the selected companies, and their potential to improve, thus affects a large percentage of the world's 3.7 billion internet users.
Geographic reach and distribution: The Index includes companies that are headquartered in North America, Europe, Africa, Asia, and the Middle East, and collectively, the companies in the Index have users in many regions around the world.
Relevance to users' freedom of expression and privacy rights: Most of the companies in the Index operate in or have a significant user base in countries where human rights are not universally respected. This is based on relevant research from such organizations as Freedom House, the Web Foundation, and Reporters Without Borders as well as stakeholder feedback.
The following factors guided the selection of services:
Telecommunications services: These operators provide a breadth of services. To keep the scope of the Index manageable while still evaluating services that directly affect freedom of expression and privacy, the Index focused on: 1) post-paid and pre-paid mobile services, including the reasonable expected mobile offerings of voice, text, and data services; and, 2) fixed-line broadband, in cases where it was available in the company's home operating market. Only consumer services were included.
Internet services: Two or three discrete services were selected based on their comparability across companies, the size of their user base, and their ability to paint a fuller picture of the overall company approach to freedom of expression and privacy. This enabled researchers to discern whether company commitments, policies, and practices applied to the entire corporate entity or only to specific services.
Mobile ecosystems: In 2016 most of the world's mobile devices were running either Apple's iOS operating system, or some version of Google's Android mobile operating system. Thus we evaluated Apple's iOS ecosystem plus two different variants of the Android ecosystem: Android on devices controlled directly by Google (the Nexus smartphone and Pixel tablet product lines), and Android on devices controlled by Samsung, which in 2016 held the largest worldwide market share for Android devices.
Research for the 2017 Index was conducted from September 1, 2016 through January 13, 2017. New information published by companies after that date was not evaluated. RDR works with a network of international researchers to collect data on each company, and to evaluate company policies in the language of the company's operating market.
RDR's external research team in 2017 consisted of 28 researchers from 19 countries. The research process for the 2017 Index consisted of several steps involving rigorous cross-checking and internal and external review.
Researchers evaluated company disclosure of the overarching "parent, or "group," level as well as well as those of selected services and/or local operating companies (depending on company structure). This enabled the research team to develop as complete an understanding as possible about the level at which companies disclose or apply their policies.
For internet and mobile ecosystem companies, the parent company typically delivered the services. In some cases the service was also a subsidiary. However, the structure of these companies was generally such that the subsidiary only delivered one service, which made it straightforward to understand the scope of policy disclosure.
For telecommunications companies, with the exception of AT&T, the parent company did not directly provide consumer services, so researchers also examined a subsidiary or operating company based in the home market to ensure the Index captured operational policies alongside corporate commitments. Given AT&T's external presentation of its group-level and U.S. operating company as an integrated unit, we evaluated the group-level policies for AT&T.
The evaluation includes an assessment of disclosure for every element of each indicator, based on one of the following possible answers:
- "Yes" - Company disclosure meets the element requirement.
- "Partial" - Company disclosure has met some but not all aspects of the element, or the disclosure is not comprehensive enough to satisfy the full scope of the what the element is asking for.
- "No disclosure found" - Researchers were not able to find information provided by the company on their website that answers the element question.
- "No" - Company disclosure exists, but it specifically does not disclose to users what the element is asking. This is distinct from the option of "no disclosure found," although both result in no credit.
- "N/A" - Not applicable. This element does not apply to the company or service. Elements marked as N/A will not be counted for or against a company in the scoring process.
Companies receive a cumulative score of their performance across all Index categories, and results show how companies performed by each category and indicator. Scores for the Freedom of Expression and Privacy categories are calculated by averaging scores for each individual service. Scores for the Governance category indicators include parent- and operating-level performance (depending on company type).
- Yes = 100
- Partial = 50
- No = 0
- No disclosure found = 0
- N/A excluded from the score and averages
Governance category scoring
- G1 and G5:
- Internet and mobile ecosystem companies: scores were based on the "group" level scores;
- Telecommunications companies: scores based on average "group" and operating company scores
- G2, G3, G4:
- Internet and mobile ecosystem companies: scores based on average of "group"-level and services scores;
- Telecommunications companies: Average of group, operating, and services scores
- Internet and mobile ecosystem companies: Average of service-level scores.
- Telecommunications companies: Average of service-level scores.
Indicator and element scoring
Telecommunications companies were evaluated on 32 of the 35 indicators; internet and mobile ecosystem companies were evaluated on 33 of the 35 indicators. Some elements within indicators were not applicable to certain services.
The following list identifies which indicators or elements were N/A for certain companies or services:
- F3, Element 2: N/A for search engines
- F3, Elements 4-5: N/A for pre-paid and post-paid mobile services,
- F5-F7: N/A for e-mail services
- F6, Element 2: N/A for search engines
- F7, Element 2:N/A for search engines
- F8, Element 1: N/A for telecommunications companies
- F8, Elements 1 & 4: N/A for search engines
- F8, Elements 1-3: N/A for email services
- F9: N/A for internet companies and mobile ecosystems
- F10: N/A for internet companies and mobile ecosystems
- F11: N/A for post-paid mobile and fixed-line internet services; search engines
- P9: N/A for telecommunications companies
- P14, Elements 5, 6, 9: N/A for internet companies
- P16: N/A for telecommunications companies
- P16, Elements 3-4: N/A for internet services without private messaging functions
- P17: N/A for telecommunications companies; search engines
The following elements apply only to mobile ecosystems:
- P1, Element 4
- P2, Element 5
- P3, Elements 4-5
- P4, Elements 5-6
- P6, Elements 6-7
- P7, Element 5
- P8, Element 5
- P14, Elements 4, 7-8