- February 14, 2011
In a controversial post that was countered by Matt Cutts of Google, Michael Arrington compared the modern search experience to what it was like before Google was created, claiming, “It’s a sh*t show of layer upon layer of SEO madness vying for my click”.
“Yes, search is very hard. But Silicon Valley is really good at doing hard things,” claims Arrington.
Here at Infochimps, we have never shied away from difficult problems. We are a company of 12 people handling data stores that companies 100 times our size never see. Soon, Infochimps will launch hundreds of more data sets and data APIs. What do data APIs have to do with Google not being effective though?
In the post, Arrington describes that when he knows what he is looking for such as a vacation, he goes straight to vacation website such as TripAdvisor or Gogobot. Sites like TripAdvisor or Yelp start with data though, either through mass aggregation or subscription to expensive services. Aggregation is not easy though. It equates to hours of scraping, cleaning, parsing, and updating data. There are scraping scripts that could potentially break. It also means potentially thousands of dollars of hosting costs to house and process that data. Search is not a time effective or cheap exercise.
If you can launch an app and plug into a data API in minutes, suddenly creating targeted vertical search engines becomes easy and more affordable. You are in essence sharing data set with others, which makes creating intelligent algorithms with that data a lot easier and more affordable.
There will always be a place for a major search engine like Google. Vertical search engines are content rich and can become “the search within the search” though. We are doing what we can to make those more accessible to everyone, so hopefully we can make Mr. Arrington and others happy in their quest to find things.
By the way, we solve difficult problems here in Austin too. ;-)