HarvestMan is a multithreaded off-line browser.It has many features for customizing offline browsing through URL filters, depth-fetching, fetch levels, domain filters, file limits, thread limits, download depth, directory checking, and robot exclusion protocol. It is useful to download an entire Web site or certain files from a Web site to the hard disk for offline browsing later. It features an XML project file, and support for the HTTP/HTTPS and FTP protocols. It works transparently across proxies and can also crawl intranets. This has been added to the tool section of Research Resources Subject Tracer™ Information Blog.
posted by Marcus Zillman |
4:15 AM
