Inductive Bias

On last Sunday the Apache Mahout project published the 0.4 release. Nearly every piece of the code has been refactored and improved since the last 0.3 release. The release was timed to happen exactly before Apache Con NA in Atlanta. As such it was published on October 31st - the Halloween release, sort-of.

Especially mentionable are the following improvements:

• Model refactoring and CLI changes to improve integration and consistency
  
• Map/Reduce job to compute the pairwise similarities of the rows of a matrix using a customizable similarity measure
  
• Map/Reduce job to compute the item-item-similarities for item-based collaborative filtering
  
• More support for distributed operations on very large matrices
  
• Easier access to Mahout operations via the command line
  
• New vector encoding framework for high speed vectorization without a pre-built dictionary
  
• Additional elements of supervised model evaluation framework
  
• Promoted several pieces of old Colt framework to tested status (QR decomposition, in particular)
  
• Can now save random forests and use it to classify new data
  

New features and algorithms include:

• New ClusterEvaluator and CDbwClusterEvaluator offer new ways to evaluate clustering effectiveness
  
• New Spectral Clustering and MinHash Clustering (still experimental)
  
• New VectorModelClassifier allows any set of clusters to be used for classification
  
• RecommenderJob has been evolved to a fully distributed item-based recommender
  
• Distributed Lanczos SVD implementation
  
• New HMM based sequence classification from GSoC (currently as sequential version only and still experimental)
  
• Sequential logistic regression training framework
  
• New SGD classifier
  
• Experimental new type of NB classifier, and feature reduction options for existing one
  

There were many, many more small fixes, improvements, refactorings and cleanup. Go check out the new release, give the new features a try and report back to us on the user mailing list.

This week, Apache Mahout 0.3 was released. First of all thanks to all committers and contributors who made that possible: Thanks for all your hard work on making the code even faster and integrating even more algorithms.

To the highlights:

• New: math and collections modules based on the high performance Colt library
  
• Faster Frequent Pattern Growth(FPGrowth) using FP-bonsai pruning
  
• Parallel Dirichlet process clustering (model-based clustering algorithm)
  
• Parallel co-occurrence based recommender
  
• Parallel text document to vector conversion using LLR based ngram generation
  
• Parallel Lanczos SVD(Singular Value Decomposition) solver
  
• Shell scripts for easier running of algorithms, utilities and examples
  
  
  
  ... and much much more: code cleanup, many bug fixes and performance improvements. Check out the new release and watch for further news on Apache Mahout to come in the next days and weeks.
  
  
  Details on what's included can be found in the release notes.
  
  Downloads are available from the Apache Mirrors

Apache Mahout 0.2 has been released and is now available for public download at http://www.apache.org/dyn/closer.cgi/lucene/mahout

Up to date maven artifacts can be found in the Apache repository at
https://repository.apache.org/content/repositories/releases/org/apache/mahout/


Apache Mahout is a subproject of Apache Lucene with the goal of delivering scalable machine learning algorithm implementations under the Apache license. http://www.apache.org/licenses/LICENSE-2.0

Mahout is a machine learning library meant to scale: Scale in terms of community to support anyone interested in using machine learning. Scale in terms of business by providing the library under a commercially friendly, free software license. Scale in terms of computation to the size of data we manage today.

Built on top of the powerful map/reduce paradigm of the Apache Hadoop project, Mahout lets you solve popular machine learning problem settings like clustering, collaborative filtering and classification
over Terabytes of data over thousands of computers.

Implemented with scalability in mind the latest release brings many performance optimizations so that even in a single node setup the library performs well.

The complete changelist can be found here:

http://issues.apache.org/jira/browse/MAHOUT/fixforversion/12313278

New Mahout 0.2 features include

• Major performance enhancements in Collaborative Filtering, Classification and Clustering
  
• New: Latent Dirichlet Allocation(LDA) implementation for topic modelling
  
• New: Frequent Itemset Mining for mining top-k patterns from a list of transactions
  
• New: Decision Forests implementation for Decision Tree classification (In Memory & Partial Data)
  
• New: HBase storage support for Naive Bayes model building and classification
  
• New: Generation of vectors from Text documents for use with Mahout Algorithms
  
• Performance improvements in various Vector implementations
  
• Tons of bug fixes and code cleanup
  

Getting started: New to Mahout?

• Download Mahout at http://www.apache.org/dyn/closer.cgi/lucene/mahout
  
• Check out the Quick start: http://cwiki.apache.org/MAHOUT
  
• Read the Mahout Wiki: http://cwiki.apache.org/MAHOUT
  
• Join the community by subscribing to mahout-user@lucene.apache.org
  
• Give back: http://www.apache.org/foundation/getinvolved.html
  
• Consider adding yourself to the power by Wiki page: http://cwiki.apache.org/MAHOUT/poweredby.html
  

For more information on Apache Mahout, see http://lucene.apache.org/mahout

A very BIG Thank You to all those who made this release happen!