Independent research from Xia Chu has shown that, in addition to non-China content, Bing censors a vast amount of content that is hosted inside China and which is not censored by China-based internet companies like Baidu. After communicating our issues with Microsoft, Bing removed certain censorship rules (kudos to Bing), but much work remains to be done.
We recently called for Microsoft to release its transparency report for Bing (as have others - full disclosure, Rebecca sits on our advisory board). Microsoft has yet to respond to this request. But Xia’s independent research of Bing’s China censorship policy could be regarded as a de facto transparency report for the search engine.
In this thorough study, the results of which we have verified, Xia examined Bing's SERP (search engine results page) for over 30,000 sensitive and nonsensitive query terms, and launched these queries from both inside and outside of China. Comparing and examining these results, plus querying with special search operators, reveals unprecedented detail on Bing's China filtering practices.
The main findings from Xia’s research include:
Bing has a list of “forbidden” terms where no results are shown. 139 such terms have been identified.
Bing has a blacklist of websites that it never shows to China users. 329 such websites are identified. (5 have been lifted after our communication with Microsoft.)
We can also now trace complicit Bing Chinese censorship back to 2009 as highlighted by Nicholas Kristof. It looks like Microsoft has indeed changed its censorship mechanism after our research made headlines this week. But Bing is still seriously flawed on two fronts: its algorithm favors pro-Chinese government websites by default on all search terms in simplified Chinese and their front end mistakenly delivers explicit censorship of search results on some search terms for users from all over the world.
Our latest research indicates that Microsoft’s search engine Bing is censoring English and Chinese language search on its home page in order to exclude certain results. We have also noticed that Bing is practicing subtle censorship with search results. In both instances, Bing is filtering out links and stories that the Chinese authorities would deem damaging.
After Tuesday’s report Leaked Records Reveal Offshore Holdings of China’s Elite by ICIJ, China blocked a number of major newspaper websites. All websites below were blocked after publishing copies of the original report. They're all listed as the publishing partners for “Chinaleaks” stories on ICIJ's website. The Great Firewall rarely blocks non-Chinese websites. Many of them have published the Chinese version of the report which probably explains the unusual development.
Yesterday we witnessed one of the largest Internet outages ever in China. We have three theories about why this outage may have occurred - two related to the Falun Gong but our third theory is that the Chinese authorities set out to attack our unblockable mirror websites.
From 15:30 to 16:30 (China time) on January 21, DNS lookup to any domain would incorrectly resolve to 188.8.131.52. Websites inside and outside of China were affected. Even Baidu and Sina were inaccessible. Only software using IP directly (e.g. QQ, VPNs) worked during that time. Attempts to visit any website redirected to http://184.108.40.206, which didn’t respond during that time. The overwhelming traffic to this IP likely crashed the server.