The Web search arena just became a little more crowded as Amazon.com has announced that it is opening programmer access to its Web crawler, Alexa, for a small fee.
Heretofore, Alexa has been known more for traffic rankings than searches, but the newly unveiled Alexa Web Search Platform will give developers access to online tools for searching, indexing, publishing, computing, and storing Internet data.
"Today, Alexa is releasing the Alexa Web Search Platform Beta, effectively opening up the Alexa Web Crawl and ushering in a new era where anybody can create new search services without having to invest millions of dollars in crawl, storage, processing, search, and server technology," read a post by Geoffrey Mack, product manager at Alexa, on the Alexa Web site.
The price for each feature offered by the service is $1. Developers can rent CPUs by the hour, store 1 gigabyte of data per year, process 50 gigabytes of internal data, upload or download 1 gigabyte of data, or publish up to 4,000 new searches using Alexa's self-service publication system, all for $1 each.
Plenty of Information
Alexa has been crawling all publicly available sites on the Internet since 1996, gathering information to create what it calls snapshots of the Web. The collected data is then used to create services, such as traffic rankings.
Alexa claims to gather approximately 1.6 terabytes, or 1,600 gigabytes, of online content every day. That works out to four to five billion pages per month. It takes the company roughly two months to complete a snapshot of the Web for which Alexa will have gathered some 4.5 billion pages from more than 16 million sites.
"Since 1996, Alexa has been crawling and storing the Web at millions of pages per day," Mack's blog post continued. "Alexa has also been building out the infrastructure to store and analyze the data and serve it to toolbars, browsers, and Web sites worldwide. Now, all of that infrastructure is yours to use via the Alexa Web Search Platform."
Niche Searches Take Off
According to a Yankee Group study, Web searches rank as the number-two online activity in the U.S. The firm's data shows tremendous growth in the products and services that are being developed around searches, said Jennifer Simpson, a Yankee Group research associate. In addition, the success of Google's public offering created a lot of buzz among investors and venture capitalists.
"The main core search used to be the Web search, the general Web search that Google does on its main interface, but what we've seen develop are specific searches such as blog searches [and] multimedia searches that are now popular looking for both video and music online," Simpson said.
Amazon's decision to make Alexa available to developers is an interesting development, she said, as it allows them "to develop the technology without having all the funds."
Search and Employ
While it is not clear how Amazon will work with programmers using the Alexa service, chances are good that the company will "try and get those developers, once their search engine is ready, to come work for [Amazon]," Simpson said. "It is a new development to allow developers access to the data and indexing. [Amazon is] allowing developers to use their crawling and data to create new searches. The end product will be new forms of searches."
For Amazon, which is not a portal in the traditional sense like Yahoo and MSN, facilitating the development of niche search engines moves them into the portal space. One of Amazon's competitors, eBay, has taken a divergent path, offering portal-like services such as Voice over Internet Protocol (VoIP) and instant messaging.
"What it does for Amazon is kind of interesting," Simpson said. "With eBay's purchase of Skype this year, that was their move into portal-like services. It seems that Amazon is interested in the search end of the portal. It is interesting that these two companies are taking divergent paths in terms of how they are developing their portal businesses."