Mahout clustering

I have been exploring the mahout clustering packages. Some brief description of the codes

org.apache.mahout.clustering.kmeans.KMeansDriver is the main entry point. The run method setup the clustering job by specifying the input path to data points, path to initial k clusters, distance measure class used.

In the KMeansDriver, it also setup the settings in the Configuration as follows:

conf.set(KMeansConfigKeys.CLUSTER_PATH_KEY, clustersIn.toString());
conf.set(KMeansConfigKeys.DISTANCE_MEASURE_KEY, measureClass);
conf.set(KMeansConfigKeys.CLUSTER_CONVERGENCE_KEY, convergenceDelta);

KMeansConfigKeys is an interface which defines the config keys for the settings.

org.apache.mahout.clustering.kmeans.KMeansMapper loads the initial predefined clusters via the setup method. In the map method, it reads the VectorWritable and passes it to the KMeansCluster to do the clustering, comparing against the predefined clusters via the following call.

this.clusterer.emitPointToNearestCluster(point.get(), this.clusters, context);

In the emitPointToNeareastCluster, it finds the nearest cluster by comparing the distance to each cluster in the list. It then writes the cluster identifier and wraps the point in the ClusterObservations to HDFS.

context.write(new Text(nearestCluster.getIdentifier()), new ClusterObservations(1, point, point.times(point)));

In the KMeansReducer, it aggregates the list of ClusterObservations into a Cluster before writing it the Cluster id as key and Cluster as the value.

In the setup method, KMeansReducer loads the predefined initial cluster from HDFS path and creates the ClusterMap of key as cluster id and value as Cluster itself.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s