Version 2, Phase 1 Draft Methodology: Ready for your feedback

Version 2 of the Phase 1 Methodology needs your feedback!

After releasing a first draft in February, we sought feedback from companies, human rights advocates, technologists, investors, and other specialists. After analyzing all of the feedback received and conducting further research, we have now released Version 2 for public comment.

This public comment period will last until July 7th, after which we will produce Version 3 for use in a pilot study. The pilot study will enable us to refine the methodology before implementing a full ranking system in 2015. Please click here to review Version 2 of the Phase 1 Methodology. There are options to submit comments either publicly or privately – or both – as you prefer.


We’re hiring: Program Associate

We are looking to hire a full-time Program Associate based at the New America Foundation in Washington, DC. The position involves working closely with the project Director on a range of responsibilities including project research, writing, coordination and logistics. For more information about the job and to apply, please click here


RDR wins the Knight News Challenge!!!

Ranking Digital Rights is honored to announce that we are one of nine winners of the 2014 John S. and James L. Knight Foundation’s Knight News Challenge. This year’s challenge focuses on breakthrough ideas that strengthen the Internet for freedom of expression and innovation.

One of our sister projects at New America’s Open Technology Institute, Measurement Lab (M-Lab), is also a Knight News Challenge winner. We look forward to collaborating with them and the other impressive projects run by brilliant people committed to keeping the Internet open and free.

The News Challenge award supports significant elements of RDR’s work through the end of 2015. Specifically, in the second half of 2014 it will support a pilot study in which we will test out our Phase 1 methodology on ten or so Internet and telecom companies around the world. In 2015, it will support implementation of the full public ranking of up to 50 companies.

It goes without saying that this award would not have been possible without the hard work and dedication of RDR’s institutional partners and researchers. Nor would it have been possible without the generosity of our existing funders who made that work possible.

We sincerely thank the Knight Foundation which supports transformational ideas that promote quality journalism, advance media innovation, engage communities and foster the arts. The foundation believes that democracy thrives when people and communities are informed and engaged. Ranking Digital Rights contributes to that mission by engaging and informing the public on what ICT sector companies are (or aren’t) doing to respect users’ rights, generating data that that the consumers, the media, investors, and policymakers can use to hold the world’s most powerful companies accountable to international human rights standards.


RDR featured on London School of Economics “Measuring Business and Human Rights” blog

The following post is featured on the London School of Economics “Measuring Business and Human Rights” blog:

Ranking Digital Rights: How can and should ICT sector companies respect Internet users’ rights to freedom of expression and privacy?

Vodafone’s blockbuster Law Enforcement Disclosure report, published last week, reveals greater detail than any telecommunications company has previously shared about the extent and nature of government surveillance demands all over the world.

Vodafone is certainly not alone: the problem is rampant across the entire sector. Norway’s Telenor isunder pressure from Thailand’s new military leaders who just seized power in a coup to help monitor and censor any content that might “lead to unrest.” Human Rights Watch recently questioned the French company, Orange, about its operations in Ethiopia whose government jails bloggers for political critiques.

Censorship is also a serious and growing problem for the ICT sector. On the 25th anniversary of China’s Tiananmen Square massacre on Wednesday, LinkedIn blocked mentions of the tragedy for its users in China. Last month, Twitter came under fire from free speech activists for agreeing to censor several tweets in Pakistan at the government’s request. Earlier this year, The Atlantic reported that “the Syrian opposition is disappearing from Facebook” – and not by choice.

Clearly, the policies and practices of Internet and telecommunications companies have real impact for the free expression and privacy of people around the world. Are they living up to their responsibilities? Are they doing everything they can to respect the rights of their users?

Some companies are trying – to varying degrees, publishing “transparency reports,” signing up for assessment processes through membership the Global Network Initiative, and making joint commitments as part of the Telecom Industry Dialogue. Others are doing little more than public relations window-dressing, while yet others are making little or no discernible effort to respect their users’ digital rights.

Meanwhile, investors have begun to ask questions about the materiality of companies’ policies and practices related to freedom of expression and privacy. One concrete example is the addition of freedom of expression and privacy criteria to recommended SEC reporting standards by the Sustainability Accounting Standards Board.

As Internet users, or as investors who care about social value as well as financial returns, what should we be asking of these companies? How do we benchmark and compare companies’ policies and practices affecting free expression and privacy? What should be considered “best practice” in a world where governments are making unreasonable demands of companies, whose staff risk jail or worse in many cases for non-compliance?

The Ranking Digital Rights project is working on answers to those questions, developing a system rank the world’s most powerful ICT sector companies on free expression and privacy criteria. We have just released a draft methodology on which we are now inviting public comment until July 7th. After further revision followed by a pilot study, we aim to start ranking up to 50 Internet and telecommunications companies in 2015. (We will add up to 50 more device, software, and equipment companies in 2016.)

The project is modeled after other efforts by investors, universities, NGOs and international organizations that measure companies on other human rights, social responsibility and sustainability criteria – from conflict minerals to labor practices to carbon disclosure. Many rankings efforts such as the Access to Medicines Index and the Corporate Equality Index have had real impact on corporate practices.

Thus we believe that if the methodology is well constructed, a ranking focused on the policies and practices of ICT sector companies affecting free expression and privacy can have a substantial, measurable impact on the extent to which companies respect and protect Internet users’ rights.

The current draft methodology is the product of more than a year’s worth of research and stakeholder consultation. The first step came with a stakeholder consultation in the Fall of 2012 to ascertain whether there was sufficient interest among investors, advocates, and technologists to proceed with the project. After some initial funds had been secured and partnerships forged, an April 2013 workshop at the University of Pennsylvania brought together a group of researchers from around the world, technologists, experts in business and human rights, and experts on rankings. Out of that meeting came a set of draft criteria in the summer of 2013: an initial list of questions that stakeholders believe are relevant to understanding how and whether Internet and telecommunications companies are making genuine efforts to respect Internet users’ freedom of expression and privacy. We then used the draft criteria as the basis for a set of case studies examining companies in the United States, Europe, Brazil, India, China, and Russia. The results of the case study in turn enabled us to make key decisions about the methodology’s scope and focus, and to publish a first draft in February. We then carried out another round of consultations with companies, investors, technologists, experts on business and human rights, and experts on rankings. After absorbing their feedback and carrying out further research, we were able to publish Version 2 of the draft methodology late last month.

Public consultation on the current draft runs through July 7th, after which we will make another round of revisions and produce Version 3. That version will be used as a basis for a pilot study focusing on up to 10 of 50 companies we are likely to rank in 2015. This pilot study will enable us to improve the methodology and make final decisions about scoring and weighting for the full ranking to be implemented in 2015. It will also enable us to identify adoption and advocacy strategies for investors and civil society, so that we can ensure that the ranking is produced in a manner that is as useful to these stakeholder groups as possible.

But first, in order to make sure that our methodology is as solid and credible as possible, it is important that we get feedback on our latest draft from experts on digital privacy and freedom of expression, anybody who might want to use our data when it comes out, as well as companies who may be candidates for ranking.

If you think you might be one of those people – or if you just care about these issues and want to weigh in – please click here, read the methodology, and help us improve it.


Support RDR in the Knight News Challenge!!

Knight News Challenge

Knight News Challenge screenshot from March 18, 2014

The theme of this year’s Knight foundation News Challenge is: “How can we strengthen the Internet for free expression and innovation?”

We believe that ranking ICT companies on freedom of expression and privacy criteria will help to do just that.

Please support our proposal! Check out our entry, then clicking on the pink “applaud” button. If you have time, please post a comment and let us know what you think – or feel free to ask questions.


RDR @ RightsCon

By Research Coordinator Allon Bar and Research Intern Hae-in Lim

Members of the RDR team debuted our preliminary “methodology elements” at RightsCon, a conference hosted by Access on human rights in the tech sector from March 3 to 5 in San Francisco. RightsCon proved to be the ideal place to strengthen our engagement with advocates, companies, policy makers, and investors—all of whom we envision as future users of the ranking.

A decidedly global affair with attendees hailing from more than 65 countries, RightsCon featured a diverse range of workshops and panels on topics from digital security and activism to corporate accountability mechanisms, transparency reporting and terms of service, The gathering served as an important reminder that numerous non-western companies serve countries with large and growing digital populations, underscoring why it is important for Ranking Digital Rights to take into account the concerns and perspectives of advocacy groups from all around the world.

Ranking Digital Rights hosted three sessions at RightsCon: the first in-person meeting of RDR researchers from around the world, a closed-door meeting with company representatives, and a public workshop open to all conference attendees that was attended primarily by activists. At the public workshop we presented the first discussion draft of our methodology and received feedback from four pre-selected respondents, Charles Mok, Sana Saleem, Michael Connor and Phil Bloomer, before opening up the floor up to questions and comments.

RDR was praised for its global scope and also as a possible tool to educate the companies themselves. Participants with prior experience in publishing rankings cautioned about the need to be transparent and credible, as well as the overall narrative of the project. One participant advised that our criteria should not only reflect company conduct (such as soliciting user feedback) but also measure its outcomes (such as implementing feedback received). Potential challenges raised included timeliness of an annual ranking of companies whose technologies and services change constantly and “survey fatigue” on the part of companies that are already being ranked and benchmarked on other issues like sustainability and supply chain labor rights.  The importance of advocacy partnerships with on-the-ground groups around the world was emphasized over and over again. Such partnerships will be central in ensuring that the data produced by the ranking will be used in a practical and effective way.

Developing a credible ranking methodology certainly is an undertaking that requires patience, commitment, and public dialogue. RightsCon was a wonderful opportunity to debut our methodology and receive initial feedback. Please keep an eye out for an updated version that will be posted for public consultation in the coming weeks!


Phase 1 Methodology discussion draft

Thanks to case study research conducted by our research partners and coordinated by a hard working team, we have developed a “discussion draft” of potential methodology elements to be used in ranking 50 Internet and telecommunications companies on freedom of expression and privacy criteria.

Right now we are proposing to ask 50 questions about these 50 companies. Answers to the questions will be scored. Companies will be ranked according to their total scores, likely with sub-scores for “privacy” and “freedom of expression” since we anticipate some companies will be stronger in one or the other and it will be meaningful to show those differences.

We welcome comments from anybody who might be affected by such a ranking. We would also like to hear from people who might want to use it in different ways: as users of technology, as investors, as activists, as policymakers, or as journalists.

There is still much work to be done before the methodology is ready to implement in full.  After presenting it at RightCon next week and receiving initial feedback, we will try to work out some of the unresolved issues, make another set of revisions, then post the revised version online for a six week public comment period. We will publish a final Phase 1 Pilot Methodology along with the case studies that helped us develop it by mid-2014. In the second half of 2014 we will conduct a pilot study looking at approximately 10 of the companies listed in the draft. The results of the pilot will inform further revisions, as long as a set of advocacy, engagement, and media strategies.

If all goes as planned we will then implement the full “Phase 1″ ranking for Internet and telecommunications companies in 2015.  “Phase 2″ adding device, equipment, and software companies will follow in 2016.

Please click here to download the full PDF document. Comments are welcome publicly via this blog’s comments section, or privately via e-mail to Rebecca MacKinnon (mackinnon AT newamerica DOT org).


RDR at Rightscon next week

Our friends at Access have organized an exciting conference called RightsCon, a three-day gathering in San Francisco next week “where human rights experts, investors, corporate leaders, engineers, activists, and government representatives from around the world work to advance solutions to human rights challenges by concentrating on the possibilities within the technology sector.”

The Ranking Digital Rights team will hold a workshop at 10:30am on Wednesday March 5th in a room called “The MiniHub”. If you are coming to RightCon, please join us. (For the full conference schedule click here.)

In this brainstorming workshop, members of our  team will present our draft methodology for an annual ranking of the world’s most powerful Internet and telecommunications companies. We will invite RightsCon participants to help brainstorm on the best way to use the ranking to hold companies accountable.


Progress report: From case studies to a draft methodology

In the five months since I last posted a public update, our research partners in the United States, Brazil, the UK, Germany, Hungary, Russia, Egypt, China, and India have been hard at work conducting case study research on the Phase 1 Draft Criteria. We are all grateful to  human rights specialist and research coordinator Allon Bar, and research coordinator Richard Danbury, for their indispensable work in coordinating and conducting research as well as conducting outreach with civil society and corporate stake-holders. (Click here to learn more about Allon and Richard.)

Based on the results and recommendations made by our case study researchers (whose work will be published later this spring), we are now working to produce a discussion draft of the full methodology that will eventually be applied to evaluate and rank up to 50 major Internet and telecommunications companies across the world.

We plan to publish that discussion draft online for public comment in early March, both on this website as well as on the websites of allied organizations. The draft will also be presented at workshops at conferences such as Rightscon in San Francisco and Cyber-dialogue in Toronto. We will arrange further opportunities to for online and in-person feedback by stakeholders (civil society groups, investors, and companies) on the methodology as well.

In April and May we will revise the methodology based on the feedback we have received. The revised version will be published in May alongside the edited case studies, plus other research materials that will provide context and background for those wanting to understand how the methodology was developed.

Before our methodology is  applied to dozens of the world’s most powerful ICT sector companies in a public ranking, it is important to prepare the field so that those we are seeking to serve (civil society advocates and investors) and influence (companies and policymakers) will be in a position to use and act upon the data that we aim to generate on an annual basis.

To that end, we continue to re-evaluate and revise our work plan and timeline for 2014 and 2015.  In the Summer and Fall of 2014 we are likely to conduct a pilot study applying the methodology we are now developing, with the aim of rolling out the full annual ranking in 2015.  The precise nature and scope of that pilot study will be shaped by the results of the methodology consultation phase that we are about to launch.

None of the above would be possible without the continued support of our funders. We continue to reach out to other potential funders and  partners whose support would be compatible with maintaining the integrity and independence of our work. Suggestions are always welcome!


September progress report

Much has happened since my last blog post of early July, announcing publication of our draft criteria and launch of case study research. Here is an overview:

Funding:

We are deeply grateful to the MacArthur Foundation for a substantial 2-year investment in our work for 2014 and 2015. While the grant does not cover all project costs, MacArthur’s commitment lays the base to continue our work through the next two years in some way or another. The ultimate scope of the ranking, how much engagement we are able to conduct, and how well it is promoted among other things, depends on the extent to which we can raise other funds. Please also see our lists of 2013 funders and project partners without whose support and commitment we would never have gotten to this point.

Case Study Research and Methodology Development:

In July we launched case studies focused on Internet companies in India, Russia, China, and the U.S. In August and September we began the process of developing and launching case studies focused on telecommunications companies, including: a comparative case study looking at Deutsche Telekom in Germany and its subsidiary T-mobile Hungary; a comparative study of several telcos operating in Brazil, with an added examination of Telefonica based in Spain, whose subsidiaries include Vivo in Brazil; an examination of two Indian telcos Bharti Airtel and BSNL. A U.S. telco case study is also getting off the ground and we are examining whether further case studies are needed on other European telcos as part of this methodology-development stage. The purpose of the case study research is to test out the draft criteria on selected companies in several different types of jurisdictions, in order to gain a better understanding of how the draft criteria play out across a range of companies in a range of jurisdictions. Most importantly we hope to be able to answer to the following questions:

  • What criteria should apply to all companies everywhere regardless of size, maturity, or jurisdiction? (In other words what things do we believe that no company anywhere has an excuse not to be doing – even in hostile legal and political environments?)
  • What criteria can only reasonably be met after company has reached a certain size?
  • What criteria can only be met under certain legal/political/regulatory conditions?
  • How should we focus the methodology – what is most important and urgent to measure companies on and what is better addressed by government-focused advocacy? (The answer to this will be informed heavily by the answers to the first three questions.)
  • What can actually be meaningfully measured and compared and what simply cannot?
  • On what basis should we select the companies we will evaluate in Phase 1?
  • In particular countries and parts of the industry, what impact is such a ranking likely to have on company behavior?

It is too early to report on the final results or conclusions as research is still ongoing, but we can say that most researchers are telling us that in the next iteration, the criteria should be more streamlined and focused in order to be effective. That means we won’t be able to include everything on everybody’s laundry list of what an ideal company ought to be doing on all fronts; we will have to prioritize and focus on what is most urgent, important, and measurable.

We are also learning important things about the relationship between what companies say and commit to publicly, what they actually do in practice, and what their practices mean concretely for users’ free expression and privacy. While there is clearly a relationship between companies’ statements and public commitments and practice, that relationship is different in different contexts. Companies that make similar commitments can still have a wide range of actual practice that plays out with users in a range of different ways. We need to make sure our methodology reflects a nuanced understanding of this reality, and emphasizes the right things.

If all goes as planned we hope to have final drafts of the case study write-ups completed in December. The goal is to publish them in late January or early February, alongside a final draft of our proposed ranking methodology which will be the product of what we learned from our case study research. The methodology will then be subjected to a period of public consultation and stakeholder engagement in early 2014. Funds permitting we hope to then make a final revision and begin applying it in Q2 2014 with an aim to produce our first Phase 1 ranking report in Q4 2014. Please see the project timeline for more details.

Company engagement: 

The case study research includes company interviews. We have encountered a range of reactions from companies: from highly enthusiastic, to curious, to neutral but willing to talk, to negative, to hostile, to indifferent, to radio silence. On the positive end of the spectrum, we have had some truly enlightening and energizing meetings and calls. We have learned a tremendous amount from companies that have been willing to talk with us about the specifics of the draft criteria in the context of their own products, services, and operations. We continue to reach out to other companies with the message that if we can talk to them now during the case study research phase, the final draft methodology – and ultimately the ranking – is more likely to take their concerns and perspectives on board. We have developed an FAQ for companies.

Civil society engagement:

It is vital that our ranking should promote best practices by ICT companies that will better minimize digital threats to human rights defenders and journalists under threat. One of the things we intend to do this Fall is to revise our human rights risks scenarios (which haven’t been revised much since April) through consultation with groups working with human rights defenders and journalists, and to make sure that these scenarios play a strong role in the methodology development process.

Also note that we have been actively reaching out to all the organizations and projects that we know of whose work is remotely related to supporting the emergence of better standards and practices around digital rights, to maximize synergies and minimize duplication.

Investor engagement:

We are exploring whether and how we might work with an organization that provides research data to responsible investors on environmental, social, human rights, and governance factors.

Academic relationships:

Our case study research network includes academics from around the world. Our partnership with the University of Pennsylvania has yielded a course on “Human Rights, Corporate Responsibility and ICT” at Penn Law, two public events, and a workshop. If you are in or near Philadelphia on October 17th please join us for an event called “Scholarship After Snowden. This project is committed to supporting our academic partners – whose research contributions are so vital – in their efforts to beef up teaching and research on issues related to digital rights.

Staffing

I remain the only full-time staff member of the project, so you can blame me for all of the project’s shortcomings. Click here to see the long list of people without whom the project would be nothing and nowhere. While we still cannot afford to offer a one-year full-time contract to a person with the kind of experience we need, we do plan to hire a 6-month consultant, ideally based in Europe, to support methodology development and stakeholder engagement. The job description is posted on the New America Foundation website.