Lies, Damned Lies and Marketing

Lies, Damned Lies, And Marketing Claims

I’ve been reading a little book by Michael Brooks called The Maths That Made Us. Actually re-reading it. Something I never do, so many books still to read. But in Bangkok, having no new books in English to read, I’ve been going through each of my small library one by one. It’s an interesting book, some bits quite insightful, others cursory or banal. In a section in algebra, and covering quadratics, cubics and quartics, it mentions the original paper by Sergey Brin and Lawrence Page when they were at Stanford in 1998. The key point in this part of the book was the difference between linear and nonlinear algebra. Having built search engines for robots to navigate, I’m familiar with the exponential problem of powers of n complexity. I have always been intrigued at how a search engine like Google for instance can achieve such amazing web search results so quickly, suggesting it searches using a linear algorithm. Intrigued but not enough to find out how. Brin and Page’s paper was the one that spelled it all out. So I now have a copy of it on my laptop, and when intrigue builds, I’ll read it. In the meantime, like a reader of novels who races to the last page to see how everyone dies, I checked out the conclusions. I do this with all research papers. This time I was in awe of the altruistic prescience of this bit in an appendix:

“Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users. For example, in our prototype search engine one of the top results for cellular phone is “The Effect of Cellular Phone Use Upon Driver Attention”, a study which explains in great detail the distractions and risk associated with conversing on a cell phone while driving. This search result came up first because of its high importance as judged by the PageRank algorithm, an approximation of citation importance on the web [Page, 98]. It is clear that a search engine which was taking money for showing cellular phone ads would have difficulty justifying the page that our system returned to its paying advertisers. For this type of reason and historical experience with other media [Bagdikian 83], we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers.

Since it is very difficult even for experts to evaluate search engines, search engine bias is particularly insidious. A good example was OpenText, which was reported to be selling companies the right to be listed at the top of the search results for particular queries [Marchiori 97]. This type of bias is much more insidious than advertising, because it is not clear who “deserves” to be there, and who is willing to pay money to be listed. This business model resulted in an uproar, and OpenText has ceased to be a viable search engine. But less blatant bias are likely to be tolerated by the market. For example, a search engine could add a small factor to search results from “friendly” companies, and subtract a factor from results from competitors. This type of bias is very difficult to detect but could still have a significant effect on the market. Furthermore, advertising income often provides an incentive to provide poor quality search results. For example, we noticed a major search engine would not return a large airline’s homepage when the airline’s name was given as a query. It so happened that the airline had placed an expensive ad, linked to the query that was its name. A better search engine would not have required this ad, and possibly resulted in the loss of the revenue from the airline to the search engine. In general, it could be argued from the consumer point of view that the better the search engine is, the fewer advertisements will be needed for the consumer to find what they want. This of course erodes the advertising supported business model of the existing search engines. However, there will always be money from advertisers who want a customer to switch products, or have something that is genuinely new. But we believe the issue of advertising causes enough mixed incentives that it is crucial to have a competitive search engine that is transparent and in the academic realm.”

The Anatomy of a Large-Scale Hypertextual Web Search Engine, Sergey Brin and Lawrence Page, Computer Science Department, Stanford University, Stanford, CA 94305, USA and