YACY is a distributed Web crawler and also a caching HTTP/HTTPS proxy. Pages that pass through the proxy are indexed and can be searched using a built-in HTTP server. YACY peers connect each other and form a P2P-based index exchange network based on distributed hash tables. Explicit web crawls can be done locally or collaboratively, forming a global search and distributed indexing engine for the Web. It also provides URL filtering with blacklist sharing among other proxy peers, individual Web/servlet page hosting, a file sharing zone, and a database engine. This has been added to the search engines section of the 2005 Internet MiniGuides.
posted by Marcus Zillman |
4:15 AM
