Facebook, Google Plus, Uncensored Search etc without a VPN

GreatFire.org's mission is to bring transparency to online censorship in China and to that end we are continuously making our own improvements to how we monitor blocked searches and key words. This past week we launched a new version of our web site where we now provide real-time searches so concerned netizens can keep up to date with information which is being censored via the Great Firewall of China.

We are also fortunate in that we have a core of super users who are constantly and consistently testing our technology to keep abreast of censorship in China. After we launched the new version of the site, we received this feedback from one of our super users via Twitter:

@percyalpha:

Congrats on ur update. But it seems connection reset for https is not detected by ur server. e.g https://en.greatfire.org/https/www.youtube.com

After receiving this message we logged on to Facebook, checked out Google Plus and did some searches on Google, specifically for freedom and 六四. What's so special about that? These websites and searches are blocked in China but we were able to access them without using a VPN or proxy, the tools people commonly use to get around this censorship.

How did we do it?

We launched a new version of the website last week, and among other things it detects types of blocking including the infamous Connection Reset. For some reason, however, it did not report Connection Reset for a test of YouTube on March 17 - nor any other type of censorship. All the data suggested that the encrypted version of YouTube was not blocked in China on that day. Could that be right?

Notice that I said the encrypted version. That means https://www.youtube.com, and not http://www.youtube.com. One single letter makes all the difference when it comes to Internet communication. When browsing a URL that starts with https://, all the traffic between you and the server is encrypted and therefore out of reach of any content-based filtering such as what's practiced by the Great Firewall of China (GFW). It means that the GFW cannot interfere with the connection based on it's content, such as the URL, any data sent in forms, or the response data. It can, however, interfere in other ways:

1. DNS Poisoning

In short, this means that the IP address returned for the requested domain is incorrect. This happened when our system tested YouTube again today (but not yesterday - which is why it seemed to be accessible). More on DNS Poisoning in our FAQ.

2. IP Blocking

This means that the actual IP address of the server where the website is hosted is blocked. All communication with this server is subsequently made impossible. However, this type of filtering is not very effective nowadays, because IP addresses tend to change frequently. Major websites use load-balancing and globally distributed hosting meaning that they have a range of IP addresses at their disposal. For example, if you go to google.com, and then again half an hour later, it may well direct you to different servers. The content looks the same, so you don't notice. You also don't care, because it's technical mumbo jumbo - unless, that is, you live in China or some other authoritarian country with major Internet censorship.

So what's the "So what?"

Let's put together what we have so far:

  1. If the connection is encrypted (ie, the URL starts with https://), the Great Firewall cannot know what the content is and cannot block the request based on it's content.
  2. It can however use other types of censorship, such as DNS Poisoning and IP Blocking.
  3. If we can get around these other types of censorship, we can get around the Great Firewall!

DNS Poisoning is probably the least sophisticated type of Internet censorship tool that the Chinese and other strong-armed governments have at their disposal. To get around this type of censorship, all you need to do is to use an alternative DNS solution. Here's the juicy stuff:

Solution 1: Use IP addresses directly

Instead of using a DNS service, which can be and is manipulated by Chinese ISPs, you can enter the IP address directly. For example, instead of browsing to https://encrypted.google.com (which is blocked) you can browse to https://74.125.235.100 (which is, at the time this is written, not blocked). This is the encrypted version of Google, which means that you can search for anything, including the two examples mentioned at the beginning of this post, or any of the other 200+ searches that are otherwise blocked in China.

Another example is https://199.59.150.7 - at the time this is written, we're able to access Twitter on this URL.

  • Pros
    • You can access any blocked website if it has an encrypted version (a URL that starts with https://) and it's IP is not blocked.
  • Cons
    • You have to know the current IP address of the website (or one of them, there is often a range). IP addresses of major websites change frequently. 
    • You have to enter IP addresses instead of domains every time you want to visit a blocked website.
    • Your browser will compain about the SSL cerficicate being invalid, because you're not accessing the website on it's real URL. You have to manually confirm an exception to access the website.
    • Some websites, such as Facebook, will redirect you to the domain if you try to acccess them directly through their IP.

Solution 2: Manually add IP addresses to your hosts file

What if you could tell your computer to remember that certain domains should resolve to certain IP addresses so that in the future you do not need to remember different IP addresses? Guess what - you can. Every operating system (that I know of) maintains something called a hosts file which is the local cache for DNS entries. Whenever you lookup a domain, such as www.facebook.com, your computer will first check this file and if there's an entry, it will use it. Normally it isn't used for anything, but it can be. All you need to do is to edit your hosts file (here's a tutorial on how to do it) and add entries for the websites you want to unblock. For example, to access Facebook, I added the following:

69.171.224.11 facebook.com www.facebook.com
184.31.178.110  s-static.ak.facebook.com
 
Then save the file, and go to https://www.facebook.com in your browser (remember the https://).
  • Pros
    • You can access any blocked website if it has an encrypted version (a URL that starts with https://) and it's IP is not blocked.
  • Cons
    • You have to know the current IP address of the website (or one of them, there is often a range). IP addresses of major websites change frequently.

(Non) Solution 3: Use a third-party DNS service

What if there was some alternative DNS service that you could use instead of trusting your ISP? Google provides just such a service called Public DNS. Moreover, it's not blocked in China. However, there is a major weakness to this approach. The DNS request to the third-party provider is not encrypted and so it can easily be blocked or tampered with by censors along the way, such as the GFW. We tried accessing some major websites that are blocked using Google's Public DNS and were unable to do so.

Solution 4: Use an encrypted DNS service

This is where it becomes really exciting. There's an application called DNSCrypt which provides a third-party DNS service (OpenDNS) over an encrypted protocol. In essence, this is a keyturn, free solution for getting around major parts of the online censorship in China. All you need to do is to 1) download and install the software (the download site is not blocked, so far) and 2) Make sure to use encrypted versions of blocked websites (eg https://twitter.com instead of http://twitter.com). Unfortunately, DNSCrypt only works with Mac OS so far.

  • Pros
    • You can access any blocked website if it has an encrypted version (a URL that starts with https://) and it's IP is not blocked.
  • Cons
    • It's for Mac only (so far).
    • You have to trust OpenDNS.

What about speed?

When you use a VPN or proxy to access blocked websites, all traffic is rerouted through your third-party servers. One drawback of this is speed. For example, if you are in China, your VPN server is in the US and the website you are browsing is in Singapore, every request you make is routed first to the US, then to Singapore, then back to the US and finally back to China. All this means that it's bound to be slower than if you could access it directly. Using the techniques described in this post, you can do just that: access blocked websites directly from China. It should be faster.

Overview

Let's put it all together to compare, and add in a general VPN solution for reference.

 IP Address DirectlyHosts FileGoogle Public DNSDNSCryptVPN
Access blocked websitesYYYYY
Free of chargeYYYYN
Direct access to blocked website, avoiding third-party reroutingYYYYN
Works for unencrypted websitesNNNNY
Works if IP is blockedNNNNY
Does not require you to know IP addressesNNYYY
Does not require you to enter IP addresses instead of domainsNYYYY
Does not cause invalid SSL CertificatesNYYYY
Access websites that redirect you to their domainNYYYY
Prevents DNS PoisoningYYNYY
Available for all operating systemsYYYNY

Other notes

More and more websites are offering encrypted versions, and several are switching to HTTPS by default, including Google as well as our own GreatFire.org. This is changing the game for governments around the world that are trying to censor the Internet.

YouTube offers an encrypted version of its website but unfortunately the actual streaming of videos is not encrypted. This means that by using the techniques described in this post you can access YouTube (and it's fast!) but the actual viewing of videos won't work.

There's a Firefox and Chrome extension called HTTPS Everywhere which automatically redirects you to HTTPS versions of major sites.

What do you think?

  • Are you able to use these techniques to get around online censorship?
  • Do you know of other encrypted DNS services that work on other operating systems?
  • Are there other ways to use these tools?
  • How do you think this will affect future development of the Internet, attempts to censor it, and ways around the censorship?

Comment is free.

Comments

More Blog Posts

Subscribe to our mailing list
Show content from Blog | Google+ | Twitter | All. Subscribe to our blog using RSS.

Wed, Mar 19, 2014

Bing Bests Baidu Censorship

Abstract

Independent research from Xia Chu has shown that, in addition to non-China content, Bing censors a vast amount of content that is hosted inside China and which is not censored by China-based internet companies like Baidu. After communicating our issues with Microsoft, Bing removed certain censorship rules (kudos to Bing), but much work remains to be done.

We recently called for Microsoft to release its transparency report for Bing (as have others - full disclosure, Rebecca sits on our advisory board).  Microsoft has yet to respond to this request. But Xia’s independent research of Bing’s China censorship policy could be regarded as a de facto transparency report for the search engine.

In this thorough study, the results of which we have verified, Xia examined Bing's SERP (search engine results page) for over 30,000 sensitive and nonsensitive query terms, and launched these queries from both inside and outside of China. Comparing and examining these results, plus querying with special search operators, reveals unprecedented detail on Bing's China filtering practices.

The main findings from Xia’s research include:

  • Bing has a list of “forbidden” terms where no results are shown. 139 such terms have been identified.

  • Bing has a blacklist of websites that it never shows to China users. 329 such websites are identified. (5 have been lifted after our communication with Microsoft.)

Thu, Feb 13, 2014

Setting Bing's Broken Record Straight

We can also now trace complicit Bing Chinese censorship back to 2009 as highlighted by Nicholas Kristof. It looks like Microsoft has indeed changed its censorship mechanism after our research made headlines this week. But Bing is still seriously flawed on two fronts: its algorithm favors pro-Chinese government websites by default on all search terms in simplified Chinese and their front end mistakenly delivers explicit censorship of search results on some search terms for users from all over the world.

Wed, Feb 12, 2014

No error here: Microsoft deploying Chinese censorship on global scale

Microsoft says: “The results themselves are and were unaltered outside of China”. This is simply not true.

Tue, Feb 11, 2014

Bing practicing Chinese censorship globally

Our latest research indicates that Microsoft’s search engine Bing is censoring English and Chinese language search on its home page in order to exclude certain results. We have also noticed that Bing is practicing subtle censorship with search results. In both instances, Bing is filtering out links and stories that the Chinese authorities would deem damaging.

Thu, Jan 23, 2014

Massive blocking of foreign media in China

After Tuesday’s report Leaked Records Reveal Offshore Holdings of China’s Elite by ICIJ, China blocked a number of major newspaper websites. All websites below were blocked after publishing copies of the original report. They're all listed as the publishing partners for “Chinaleaks” stories on ICIJ's website. The Great Firewall rarely blocks non-Chinese websites. Many of them have published the Chinese version of the report which probably explains the unusual development.

Newspaper

Main Language

Article

http://www.icij.org

English

Chinese

http://www.theguardian.com

English

Subscribe to our blog using RSS.

Comments

Good news regarding DNSCrypt - it is also available for Linux, and works well. I used the guide at http://www.ab9il.net/crypto/dnscrypt.html and was getting through the GFW for many more websites than before. Still need a VPN for Facebook, though.

Add new comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
By submitting this form, you accept the Mollom privacy policy.