July 27, 2007 5:41 PM PDT
Crawl the Web with Wales' open-source search tool
Friday morning, at the O'Reilly Open Source conference in Portland, Ore., Jimmy Wales, founder of Wikipedia and Wikia, announced the release of an open-source Web crawling site called Grub.
Grub was acquired from LookSmart under the open-source project Wikia. The platform, now available for downloading and testing, is built on users donating their personal computer power. It's meant to operate through open protocol and community collaborative added functions combined with the wiki.
Last year, Wales claimed that Internet search as we know it is broken. Grub is one of his attempts to gather open-source technologies to organize free content on the Web.
(GRUB)??? Why two grubs? This makes no sense. At
least it isn't weevil, or we'd all be making
Master and Commander jokes right about now...
A distributed crawler makes sense only, when the index is distributed too.