Every Analytics Service is Broken

In a previous post, we introduced the concept of analytics poisoning. In short, analytics poisoning is a specific degradation of ones ability to disentangle legitimate and illegitimate traffic. Depending on the web property being attacked, this could have impacts as minimal as simply misreporting traffic to one’s personal website, to impacts as significant as mass ad/referral fraud, compromised strategic decision-making based on faulty data, and so forth. At best, analytics poisoning is a nuisance. At worst, it is an existential threat to the continued operation of many different online services. In previous work, we showed how, with relative ease, IPM could demonstrate gaping sociotechnical security holes on Google Analytics - we demonstrated hundreds of concurrent visitors, all fake, generated by our Vulnerability Engine.

In this demonstration, we revisit Google Analytics and again show that no changes have been made, and that anyone depending on Google Analytics for an accurate view of their traffic can easily be mislead by extremely simple attacks. We expanded the scope of this work as well, and looked at many other competitors in the web analytics space and found that not a single company that tracks web analytics protects their customers against automated traffic. Below, we provide proof of these results for Mixpanel, Oribi, Woopra, Quantcast and Indicative, as well as an updated confirmation for Google Analytics. Jointly, these companies are worth billions in total valuation, and track the majority of the web’s traffic.

What follows is proof that they are all unreliable and readily report false information to their customers. For all of these results, we estimate the cost to an attacker to effectively poison these results to be between $5 and $250 for 100,000 traffic pings. Lower-end attacks will fail to obfuscate simple-to-mitigate characteristics such as all traffic coming from the same IP - higher end attacks, however, will re-route traffic through proxy networks, effectively masking the originator of such attacks, and produce realistic-looking geographically-dispersed results. The majority of these services cost orders of magnitude more than the cost of attacks, which is indicative that under the right circumstances, it is financially sensible to spend the money on attacks in order to degrade these services.

Mixpanel

We observed Mixpanel’s dashboard for a morning and found that fully 100% of the traffic we sent to our demonstration site was treated as legitimate, despite it being sourced from selenium-based browsers. No matter how complicated or simplistic our visitations, no attempt was made to detect, block, mitigate, or otherwise discount the traffic.

Oribi

Oribi costs, at minimum, about $500 per month to provide traffic statistics. We found no evidence of any attempt to block bad traffic through their service.

Woopra

Woopra encourages their users to spend $1,000 a month on their service. As with all others, this service faithfully reports our illegitimate traffic.

Quantcast

Despite taking a day to finally start reporting data, there appears to be no measure of attempting to delete fraudulent traffic on Quantcast.

Indicative

Indicative faithfully reports our fraudulent traffic. They advertise a professional plan which costs $950 per month

Google Analytics, Round 2

Google Analytics dashboard overview - several days prior to the test, and then currently running test results.