Nutch
| Lucene Nutch Logo | |
|---|---|
Screenshot | |
| Developer(s) | Apache Software Foundation |
| Stable release | 1.0.0 / March 23, 2009 |
| Written in | Java |
| Operating system | Cross-platform |
| Development status | Active |
| Type | Search Engine |
| License | Apache License 2.0 |
| Website | http://lucene.apache.org/nutch/ |
Nutch is an effort to build an open source search engine based on Lucene Java for the search and index component.
Features
It is coded completely in the Java programming language, but data is written in language-independent formats.
Nutch has a highly modular architecture allowing developers to create plugins for the following activities: media-type parsing, data retrieval, querying and clustering.
The fetcher ("robot" or "web crawler") has been written from scratch solely for this project.
History
Nutch originated with Doug Cutting (creator of both Lucene and Hadoop) and Mike Cafarella.
In June 2003, there was a successful 100 million page demo system. To meet the multimachine processing needs of the crawl and index tasks, the Nutch project has also implemented a MapReduce facility and a distributed file system. These two facilities have been spun out into their own subproject called Hadoop.
As of June 2005, Nutch has graduated from the Apache Incubator, and is now a subproject of Lucene.
Scalability
IBM Research studied the performance[1] of Nutch/Lucene as part of its Commercial Scale Out (CSO) project [2]. Their findings were that a scale-out system, such as Nutch/Lucene, could achieve a performance level on a cluster of blades that was not achievable on any scale-up computer such as the Power5.
Related projects
- Hadoop - Java framework that supports distributed applications running on large clusters
- nutchWAX - Uses Nutch to search a web archive
- Sixearch - An unstructured peer network application, which provides a complementary way for users to actively and collaboratively share their own document collections.
- SEO Tips - Collections of seo tips to promote your websites.
Search engines built with Nutch
References
Bibliography
- Shoberg, J (October 26, 2006). Building Search Applications with Lucene and Nutch (1st ed.). Apress. pp. 350. ISBN 978-1590596876. http://www.apress.com/book/view/9781590596876.
External links
- Official website
- Official wiki
- Building Nutch: Open Source Search (2004) - ACM Queue vol. 2, no. 2
- An article about Nutch (2003) - Search Engine Watch
- Another article about Nutch (2003) - Tech News World
- Official page of the Hadoop project
| ||||||||||||||||||||||||||
ca:Nutch de:Nutch es:Nutch fr:Nutch ko:너치 it:Nutch nl:Nutch tr:Nutch
If you like SEOmastering Site, you can support it by - BTC: bc1qppjcl3c2cyjazy6lepmrv3fh6ke9mxs7zpfky0 , TRC20 and more...
