I first discovered the internet nearly two decades back. It was an era when the internet speed was slow and the charges were extremely high. One could end up paying as much as Rs 120 per hour at an internet café. In fact, the first few times I logged on to the web, I wondered what was the fuss all about.
It is worth remembering here that I am talking about an era when even the humble sms was yet to make an appearance and the mobile phone rates were extremely expensive, with one having to pay for both incoming as well as outgoing calls. It was also an era when people largely surfed the internet from internet cafes. Of course, all that has now moved to the smart phone and home WiFi connections.
After a few sessions at internet cafés, I was told that there are websites known as search engines which allow you to search for stuff on the internet. One such website was called Ask Jeeves and there were others like Lycos and Alta Vista. While, all this sounded interesting, rarely did these websites throw up what one was searching for.
As Tim Harford writes in Fifty Things That Made the Modern Economy: “In 1998… if you typed ‘cars’ into Lycos—then a leading search engine—you’d get a results page filled with porn websites. Why? Owners of porn websites inserted many mentions of popular search terms like ‘cars’, perhaps in tiny text, or in white on white background. The Lycos algorithm saw many mentions of ‘cars’, and concluded that the page would be interesting to someone searching for ‘cars’.” It was easy to game the system. This is something that I personally experienced when I first started to use the internet regularly in 1999.
And then came Larry Page and Sergey Brin with Google. Their original idea was not come up with a search engine at all. In fact, they were trying to do something different. They were trying to build a system in order to measure how much credibility a research paper had. In academia, a research paper published in an academic journal is said to have credibility, if it is cited by other research papers. It has even more credibility if it is cited by research papers which are themselves cited many times by other research papers.
This led to the basic idea behind the Google search engine. As Harford writes: “Page and Brin realised that when you looked at a page on the nascent World Wide Web, you had no way of knowing which other pages linked to it. Web links are analogous to academic citations. If they could find a way to analyse all the links on the web, they could rank the credibility of each page in any given subject.”
And this idea essentially led to Google throwing up relevant search results unlike other search engines. The irony is that Page and Brin were not really sure of the potential of what they had built. As Duncan J Watts writes in Everything is Obvious – Once You Know the Answer, “In the late 1990s the founders of Google, Sergey Brin and Larry Page, tried to sell their company for $1.6 million.” The story goes that the buyer thought that Brin and Page were asking for too high a price and decided not to go ahead with the deal.
Thankfully, they didn’t. And now they are in a position where they have a natural monopoly. Why? As Harford writes: “Among the best ways to improve the usefulness of search results is to analyse which links were ultimately clicked by people who previously performed the same search, as well as what the user has searched for before. Google has far more of that than anyone else. That suggests it may continue to shape our access to knowledge for generations to come.”
The column originally appeared on August 16, 2017 in the Bangalore Mirror.