The Ranking Digital Rights Corporate Accountability Index methodology was developed over three years of research, testing, consultation, and revision. Since its inception in 2013, the project has engaged closely with researchers around the globe. For methodology development, pilot study, and the inaugural Index we also partnered with Sustainalytics, a leading provider of ESG (environmental, social, and governance) research to investors.
The first Corporate Accountability Index was launched in November 2015, applying the methodology to rank 16 internet and telecommunications companies.
For the 2017 Index, launched in March 2017, we expanded the ranking to cover additional types of companies and services, including those that produce software and devices that create what we call “mobile ecosystems.” As a result, we also expanded the methodology, adding new indicators and elements to account for the potential threats to users’ freedom of expression and privacy that can arise from use of networked devices and software.
The 2018 Index applies the same methodology to evaluate the same 22 companies as in the 2017 Index.[108] This enabled us to produce comparative analyses of each company’s performance and to track overall trends.
We encourage stakeholders to read more about our methodology development: https://rankingdigitalrights.org/methodology-development/
To view or download the full 2018 methodology, visit: https://rankingdigitalrights.org/2018-indicators/
The 2018 Index evaluates 10 telecommunications companies and 12 internet and mobile ecosystem companies.
All companies evaluated in the Index are multinational corporations listed on a major stock exchange. The following factors influenced company selection:
User base: The companies in the Index have a significant footprint in the areas where they operate. The telecommunications companies have a substantial user base in their home markets, and the internet companies have a large number of global users as identified by established global traffic rankings such as Alexa. The policies and practices of the selected companies, and their potential to improve, thus affect a large percentage of the world’s 4.2 billion internet users.[109]
Geographic reach and distribution: The Index includes companies that are headquartered in North America, Europe, Africa, Asia, and the Middle East, and collectively, the companies in the Index have users in many regions around the world.
Relevance to users’ freedom of expression and privacy rights: Most of the companies in the Index operate in, or have a significant user base in, countries where human rights are not universally respected. This is based on relevant research from such organizations as Freedom House, the Web Foundation, and Reporters Without Borders, as well as stakeholder feedback.
The following factors guided the selection of services:
Telecommunications services: These operators provide a breadth of services. To keep the scope of the Index manageable while still evaluating services that directly affect freedom of expression and privacy, the Index focused on: 1) post-paid and pre-paid mobile services, including the reasonable expected mobile offerings of voice, text, and data services; and, 2) fixed-line broadband, in cases where it was available in the company’s home operating market. Only consumer services were included.
Internet services: Two or three discrete services were selected based on their comparability across companies, the size of their user base, and their ability to paint a fuller picture of the overall company approach to freedom of expression and privacy. This enabled researchers to discern whether company commitments, policies, and practices applied to the entire corporate entity or only to specific services.
Mobile ecosystems: In 2016 most of the world’s mobile devices were running either Apple’s iOS operating system, or some version of Google’s Android mobile operating system. Thus we evaluated Apple’s iOS ecosystem plus two different variants of the Android ecosystem: Android on devices controlled directly by Google (the Nexus smartphone and Pixel tablet product lines), and Android on devices controlled by Samsung, which in 2016 held the largest worldwide market share for Android devices.
For a full list of company services evaluated in the Index, see Section 1.2.
The Index considered company disclosure on several levels—at the parent company level, the operating company level (for telecommunications companies), and the service level. This enabled the research team to develop as complete an understanding as possible about the level at which companies disclose or apply their policies.
For internet and mobile ecosystem companies, the parent company typically delivered the services. In some cases, the service was also a subsidiary. However, the structure of these companies was generally such that the subsidiary only delivered one service, which made it straightforward to understand the scope of policy disclosure.
For telecommunications companies, with the exception of AT&T, the parent company did not directly provide consumer services, so researchers also examined a subsidiary or operating company based in the home market to ensure the Index captured operational policies alongside corporate commitments. Given AT&T’s external presentation of its group-level and U.S. operating company as an integrated unit, we evaluated the group-level policies for AT&T.
RDR works with a network of international researchers to collect data on each company, and to evaluate company policies in the language of the company’s operating market. RDR’s external research team for the 2018 Index consisted of 28 researchers from or based in 18 countries. A list of our partners and contributors can be found at: https://rankingdigitalrights.org/who/affiliates/.
The research process for the 2018 Index consisted of several steps involving rigorous cross-checking and internal and external review, as follows:
Step 1: Data Collection. A primary research team collected data for each company and provided a preliminary assessment of company performance across all indicators.
Step 2: Secondary Review. A second team of researchers conducted a fact-check of the assessment provided by primary researchers in Step 1.
Step 3: Review and Reconciliation. RDR research staff examined the results from Steps 1 and 2 and resolved any differences that arose.
Step 4: First Horizontal Review. Research staff cross-checked the indicators to ensure they had been evaluated consistently for each company.
Step 5: Company Feedback. Initial results were sent to companies for comment and feedback. All feedback received from companies by the agreed upon deadline was reviewed by RDR staff who made decisions about score changes or adjustments.
Step 6: Second Horizontal Review. Research staff conducted a second horizontal review, cross-checking the indicators for consistency and quality control.
Step 7: Final Scoring. The RDR team calculated final scores.
Proactive and open stakeholder engagement has been a critical component of the Index’s methodology. We communicated with companies throughout the research process.
Open dialogue and communication: Before the research began, we contacted all 22 companies and informed them that they were included in this year’s Index, describing our research process and timeline. Following several stages of research and review, we shared each company’s initial results with them. We invited companies to provide written feedback as well as additional source documents. The research team conducted conference calls or meetings with companies that requested them to discuss the initial findings as well as broader questions about the Index and its methodology.
Incorporating company feedback into the Index: While engagement with the companies was critical to understand company positions and ensure the research reviewed relevant disclosure, the Index evaluates information that companies disclose publicly. Therefore we did not consider a score change unless companies identified publicly available documentation that supported a change. Absent that, the research team reviewed company feedback and considered it as context for potential inclusion in the narrative report, but did not use it for scoring purposes.
Research for the 2018 Index was based on company policies that were active between January 13, 2017 and January 12, 2018. New information published by companies after January 12, 2018 was not evaluated.
2017 Index score adjustments: Some company scores from 2017 were adjusted for comparison with the 2018 evaluation. Scores were adjusted at the element level, in accordance with clarified evaluation standards that were applied in the 2018 Index, or to include information not located during the 2017 Index cycle, or as a result of a re-assessment of the company’s disclosure. These adjustments did not produce changes to any company position in the 2017 rankings or to any of the key findings highlighted in the 2017 Index. Each score adjustment, including a detailed explanation of the reason for each change, is recorded in each company’s final dataset, which is publicly available for download here.
How companies are scored: The Index evaluates company disclosure of the overarching “parent,” or “group,” level, as well as those of selected services and/or local operating companies (depending on company structure). Each indicator has a list of elements, and companies receive credit (full, partial, or no credit) for each element they fulfill. The evaluation includes an assessment of disclosure for every element of each indicator, based on one of the following possible answers:
“Yes”/ full disclosure — Company disclosure meets the element requirement.
“Partial”— Company disclosure has met some, but not all, aspects of the element, or the disclosure is not comprehensive enough to satisfy the full scope of what the element is asking for.
“No disclosure found” — Researchers were not able to find information provided by the company on their website that answers the element question.
“No” — Company disclosure exists, but it does not disclose to users what the element is asking. This is distinct from the option of “no disclosure found,” although both result in no credit.
“N/A” — Not applicable. This element does not apply to the company or service. Elements marked as N/A will not be counted for or against a company in the scoring process.
Points
Yes/full disclosure = 100
Partial = 50
No = 0
No disclosure found = 0
N/A excluded from the score and averages
Companies receive a cumulative score of their performance across all Index categories, and results show how companies performed by each category and indicator. Scores for the Freedom of Expression and Privacy categories are calculated by averaging scores for each individual service. Scores for the Governance category indicators include group-, operating-, and service(s)-level performance (depending on indicator and company type, see below).
Governance category scoring
G1 and G5:
Internet and mobile ecosystem companies: scores were based on the “group” level scores.
Telecommunications companies: scores based on average “group” and operating company scores.
G2, G3, G4:
Internet and mobile ecosystem companies: scores based on average of “group”-level and services scores.
Telecommunications companies: average of group, operating, and services scores.
G6:
Internet and mobile ecosystem companies: average of service-level scores.
Telecommunications companies: average of service-level scores.
Indicator and element scoring
Telecommunications companies were evaluated on 32 of the 35 indicators; internet and mobile ecosystem companies were evaluated on 33 of the 35 indicators. Some elements within indicators were not applicable to certain services.
The following list identifies which indicators or elements were N/A for certain companies or services:
F3, Element 2: N/A for search engines
F3, Elements 4-5: N/A for pre-paid and post-paid mobile services, Cloud services, email services, and messaging services.
F5-F7: N/A for email services
F6, Element 2: N/A for search engines
F7, Element 2: N/A for search engines
F6, Element 3: N/A for messaging services
F8, Element 1: N/A for telecommunications companies
F8, Elements 1 & 4: N/A for search engines
F8, Elements 1-3: N/A for email services
F9: N/A for internet and mobile ecosystem companies
F10: N/A for internet and mobile ecosystem companies
F11: N/A for post-paid mobile and fixed-line internet services; search engines
P9: N/A for telecommunications companies
P14, Elements 5, 6, 9: N/A for internet companies and Google and Apple mobile ecosystems
P14, Elements 4, 7, 8: N/A for internet companies and telecommunications companies
P16: N/A for telecommunications companies
P16, Elements 3-4: N/A for internet services without private messaging functions
P17: N/A for telecommunications companies; search engines
The following elements apply only to mobile ecosystems:
P1, Element 4
P2, Element 5
P3, Elements 4-5
P4, Elements 5-6
P6, Elements 6-7
P7, Element 5
P8, Element 5
P14, Elements 4, 7-8
For more information about RDR’s methodology development, see: https://rankingdigitalrights.org/methodology-development/.
The 2015 Index can be viewed here: https://rankingdigitalrights.org/index2015/.
The 2017 Index can be viewed here: https://rankingdigitalrights.org/index2017/.
For more details about differences between the 2015 and 2017 methodology, see: https://rankingdigitalrights.org/2016/09/15/rdr-launches-2017-research/.
For more information about the project please see our “frequently asked questions” page: https://rankingdigitalrights.org/who/frequently-asked-questions/.
Figure 1. The 2018 Corporate Accountability Index ranking
Figure 2. Year-on-year score changes (2017 to 2018)
Figure 3. Governance scores
Figure 4. Comprehensiveness of human rights impact assessments (G4)
Figure 5. How transparent are companies about their internal security measures (P13-P15)?
Figure 6. How transparent are companies about policies for responding to data breaches (P15)?
Figure 7: How transparent are companies about their security oversight processes (P13)?
Figure 8. How transparent are companies about their policies for addressing security vulnerabilities (P14)?
Figure 9. How transparent are internet and mobile ecosystem companies about how they handle user information?
Figure 10. How transparent are internet and mobile ecosystem companies about what user data they share and with whom (P4)?
Figure 11: How transparent are internet and mobile ecosystem companies about the purpose for collecting and sharing user information (P5)?
Figure 12: How transparent are internet and mobile ecosystem companies about options users have to control their own data (P7)?
Figure 13: How transparent are internet and mobile ecosystem companies about tracking users across the internet (P9)?
Figure 14. How transparent are internet and mobile ecosystem companies about policing content (F3-F8)?
Figure 15. How transparent are internet and mobile ecosystem companies about their rules and how they are enforced (F3, F4)?
Figure 16. How transparent are internet and mobile ecosystem companies about handling external demands to censor content and restrict user accounts (F5-F7)?
Figure 17. Year-on-year score changes (2017 to 2018), telecommunications companies
Figure 18. How transparent are telecommunications companies about blocking content and access (F3-F10)?
Figure 19. How transparent are telecommunications companies about policies for responding to government shutdown orders (F10)?
Figure 20. How transparent are telecommunications companies about handling external demands to censor content and restrict accounts (F5-F7)?
Figure 21: How transparent are telecommunications companies about their rules and how they are enforced (F3, F4)?
Figure 22. Access to and notification about privacy policies (telecommunications companies)
Figure 23. How transparent are telecommunications companies about government and private requests for user information (P10, P11, P12)?
Figure 24. How transparent are telecommunications companies about their handling of user information (P3-P8)?
[108] “2018 Companies,” Ranking Digital Rights, https://rankingdigitalrights.org/2018-companies/.
[109] Figures as of December 31, 2017. "World Internet Users Statistics and 2018 World Population Stats," Internet World Stats, accessed March 19, 2018, https://www.internetworldstats.com/stats.htm.
