Version 2 of the Phase 1 Methodology needs your feedback!

After releasing a first draft in February, we sought feedback from companies, human rights advocates, technologists, investors, and other specialists. After analyzing all of the feedback received and conducting further research, we have now released Version 2 for public comment.

This public comment period will last until July 7th, after which we will produce Version 3 for use in a pilot study. The pilot study will enable us to refine the methodology before implementing a full ranking system in 2015. Please click here to review Version 2 of the Phase 1 Methodology. There are options to submit comments either publicly or privately – or both – as you prefer.

Knight News Challenge

Knight News Challenge screenshot from March 18, 2014

The theme of this year’s Knight foundation News Challenge is: “How can we strengthen the Internet for free expression and innovation?”

We believe that ranking ICT companies on freedom of expression and privacy criteria will help to do just that.

Please support our proposal! Check out our entry, then clicking on the pink “applaud” button. If you have time, please post a comment and let us know what you think – or feel free to ask questions.

By Research Coordinator Allon Bar and Research Intern Hae-in Lim

Members of the RDR team debuted our preliminary “methodology elements” at RightsCon, a conference hosted by Access on human rights in the tech sector from March 3 to 5 in San Francisco. RightsCon proved to be the ideal place to strengthen our engagement with advocates, companies, policy makers, and investors—all of whom we envision as future users of the ranking.

A decidedly global affair with attendees hailing from more than 65 countries, RightsCon featured a diverse range of workshops and panels on topics from digital security and activism to corporate accountability mechanisms, transparency reporting and terms of service, The gathering served as an important reminder that numerous non-western companies serve countries with large and growing digital populations, underscoring why it is important for Ranking Digital Rights to take into account the concerns and perspectives of advocacy groups from all around the world.

Ranking Digital Rights hosted three sessions at RightsCon: the first in-person meeting of RDR researchers from around the world, a closed-door meeting with company representatives, and a public workshop open to all conference attendees that was attended primarily by activists. At the public workshop we presented the first discussion draft of our methodology and received feedback from four pre-selected respondents, Charles Mok, Sana Saleem, Michael Connor and Phil Bloomer, before opening up the floor up to questions and comments.

RDR was praised for its global scope and also as a possible tool to educate the companies themselves. Participants with prior experience in publishing rankings cautioned about the need to be transparent and credible, as well as the overall narrative of the project. One participant advised that our criteria should not only reflect company conduct (such as soliciting user feedback) but also measure its outcomes (such as implementing feedback received). Potential challenges raised included timeliness of an annual ranking of companies whose technologies and services change constantly and “survey fatigue” on the part of companies that are already being ranked and benchmarked on other issues like sustainability and supply chain labor rights.  The importance of advocacy partnerships with on-the-ground groups around the world was emphasized over and over again. Such partnerships will be central in ensuring that the data produced by the ranking will be used in a practical and effective way.

Developing a credible ranking methodology certainly is an undertaking that requires patience, commitment, and public dialogue. RightsCon was a wonderful opportunity to debut our methodology and receive initial feedback. Please keep an eye out for an updated version that will be posted for public consultation in the coming weeks!

Thanks to case study research conducted by our research partners and coordinated by a hard working team, we have developed a “discussion draft” of potential methodology elements to be used in ranking 50 Internet and telecommunications companies on freedom of expression and privacy criteria.

Right now we are proposing to ask 50 questions about these 50 companies. Answers to the questions will be scored. Companies will be ranked according to their total scores, likely with sub-scores for “privacy” and “freedom of expression” since we anticipate some companies will be stronger in one or the other and it will be meaningful to show those differences.

We welcome comments from anybody who might be affected by such a ranking. We would also like to hear from people who might want to use it in different ways: as users of technology, as investors, as activists, as policymakers, or as journalists.

There is still much work to be done before the methodology is ready to implement in full.  After presenting it at RightCon next week and receiving initial feedback, we will try to work out some of the unresolved issues, make another set of revisions, then post the revised version online for a six week public comment period. We will publish a final Phase 1 Pilot Methodology along with the case studies that helped us develop it by mid-2014. In the second half of 2014 we will conduct a pilot study looking at approximately 10 of the companies listed in the draft. The results of the pilot will inform further revisions, as long as a set of advocacy, engagement, and media strategies.

If all goes as planned we will then implement the full “Phase 1” ranking for Internet and telecommunications companies in 2015.  “Phase 2” adding device, equipment, and software companies will follow in 2016.

Please click here to download the full PDF document. Comments are welcome publicly via this blog’s comments section, or privately via e-mail to Rebecca MacKinnon (mackinnon AT newamerica DOT org).

Our friends at Access have organized an exciting conference called RightsCon, a three-day gathering in San Francisco next week “where human rights experts, investors, corporate leaders, engineers, activists, and government representatives from around the world work to advance solutions to human rights challenges by concentrating on the possibilities within the technology sector.”

The Ranking Digital Rights team will hold a workshop at 10:30am on Wednesday March 5th in a room called “The MiniHub”. If you are coming to RightCon, please join us. (For the full conference schedule click here.)

In this brainstorming workshop, members of our  team will present our draft methodology for an annual ranking of the world’s most powerful Internet and telecommunications companies. We will invite RightsCon participants to help brainstorm on the best way to use the ranking to hold companies accountable.