How to Perform Hierarchical Cluster Analysis using R Programming?

Cluster analysis or clustering is a technique to find subgroups of data points within a data set. The data points belonging to the same subgroup have similar features or properties. Clustering is an unsupervised machine learning approach and has a wide variety of applications such as market research, pattern recognition, recommendation systems, and so on. The most common algorithms used for clustering are K-means clustering and Hierarchical cluster analysis. In this article, we will learn about hierarchical cluster analysis and its implementation in R programming.

Hierarchical cluster analysis (also known as hierarchical clustering) is a clustering technique where clusters have a hierarchy or a predetermined order. Hierarchical clustering can be represented by a tree-like structure called a Dendrogram. There are two types of hierarchical clustering:

• Agglomerative hierarchical clustering: This is a bottom-up approach where each data point starts in its own cluster and as one moves up the hierarchy, similar pairs of clusters are merged.
• Divisive hierarchical clustering: This is a top-down approach where all data points start in one cluster and as one moves down the hierarchy, clusters are split recursively.

To measure the similarity or dissimilarity between a pair of data points, we use distance measures (Euclidean distance, Manhattan distance, etc.). However, to find the dissimilarity between two clusters of observations, we use agglomeration methods. The most common agglomeration methods are:

• Complete linkage clustering: It computes all pairwise dissimilarities between the observations in two clusters, and considers the longest (maximum) distance between two points as the distance between two clusters.
• Single linkage clustering: It computes all pairwise dissimilarities between the observations in two clusters, and considers the shortest (minimum) distance as the distance between two clusters.
• Average linkage clustering: It computes all pairwise dissimilarities between the observations in two clusters, and considers the average distance as the distance between two clusters.

Performing Hierarchical Cluster Analysis using R

For computing hierarchical clustering in R, the commonly used functions are as follows:

• hclust in the stats package and agnes in the cluster package for agglomerative hierarchical clustering.
• diana in the cluster package for divisive hierarchical clustering.

We will use the Iris flower data set from the datasets package in our implementation. We will use sepal width, sepal length, petal width, and petal length column as our data points. First, we load and normalize the data. Then the dissimilarity values are computed with dist function and these values are fed to clustering functions for performing hierarchical clustering.

R

 `# Load required packages` `library``(datasets) ``# contains iris dataset` `library``(cluster)  ``# clustering algorithms` `library``(factoextra) ``# visualization` `library``(purrr) ``# to use map_dbl() function`   `# Load and preprocess the dataset` `df <- iris[, 1:4]` `df <- ``na.omit``(df)` `df <- ``scale``(df)`   `# Dissimilarity matrix` `d <- ``dist``(df, method = ``"euclidean"``)`

Agglomerative hierarchical clustering implementation

The dissimilarity matrix obtained is fed to hclust. The method parameter of hclust specifies the agglomeration method to be used (i.e. complete, average, single). We can then plot the dendrogram.

R

 `# Hierarchical clustering using Complete Linkage` `hc1 <- ``hclust``(d, method = ``"complete"` `)`   `# Plot the obtained dendrogram` `plot``(hc1, cex = 0.6, hang = -1)`

Output:

Observe that in the above dendrogram, a leaf corresponds to one observation and as we move up the tree, similar observations are fused at a higher height. The height of the dendrogram determines the clusters. In order to identify the clusters, we can cut the dendrogram with cutree. Then visualize the result in a scatter plot using fviz_cluster function from the factoextra package.

R

 `# Cut tree into 3 groups` `sub_grps <- ``cutree``(hc1, k = 3)`   `# Visualize the result in a scatter plot` `fviz_cluster``(``list``(data = df, cluster = sub_grps))`

Output:

We can also provide a border to the dendrogram around the 3 clusters as shown below.

R

 `# Plot the obtained dendrogram with ` `# rectangle borders for k clusters` `plot``(hc1, cex = 0.6, hang = -1)` `rect.hclust``(hc1, k = 3, border = 2:4)`

Output:

Alternatively, we can use the agnes function to perform the hierarchical clustering. Unlike hclust, the agnes function gives the agglomerative coefficient, which measures the amount of clustering structure found (values closer to 1 suggest strong clustering structure).

R

 `# agglomeration methods to assess` `m <- ``c``(``"average"``, ``"single"``, ``"complete"``)` `names``(m) <- ``c``(``"average"``, ``"single"``, ``"complete"``)`   `# function to compute hierarchical ` `# clustering coefficient` `ac <- ``function``(x) {` `  ``agnes``(df, method = x)\$ac` `}`   `map_dbl``(m, ac)`

Output:

``` average    single  complete
0.9035705 0.8023794 0.9438858 ```

Complete linkage gives a stronger clustering structure. So, we use this agglomeration method to perform hierarchical clustering with agnes function as shown below.

R

 `# Hierarchical clustering ` `hc2 <- ``agnes``(df, method = ``"complete"``)`   `# Plot the obtained dendrogram` `pltree``(hc2, cex = 0.6, hang = -1, ` `       ``main = ``"Dendrogram of agnes"``)`

Output:

Divisive clustering implementation

The function diana which works similar to agnes allows us to perform divisive hierarchical clustering. However, there is no method to provide.

R

 `# Compute divisive hierarchical clustering` `hc3 <- ``diana``(df)`   `# Divise coefficient` `hc3\$dc`   `# Plot obtained dendrogram` `pltree``(hc3, cex = 0.6, hang = -1, ` `       ``main = ``"Dendrogram of diana"``)`

Output:

`[1] 0.9397208`

Whether you're preparing for your first job interview or aiming to upskill in this ever-evolving tech landscape, GeeksforGeeks Courses are your key to success. We provide top-quality content at affordable prices, all geared towards accelerating your growth in a time-bound manner. Join the millions we've already empowered, and we're here to do the same for you. Don't miss out - check it out now!

Previous
Next