ML | Fuzzy Clustering

Last Updated : 17 Sep, 2019

Prerequisite: Clustering in Machine Learning

What is clustering?
Clustering is an unsupervised machine learning technique which divides the given data into different clusters based on their distances (similarity) from each other.

The unsupervised k-means clustering algorithm gives the values of any point lying in some particular cluster to be either as 0 or 1 i.e., either true or false. But the fuzzy logic gives the fuzzy values of any particular data point to be lying in either of the clusters. Here, in fuzzy c-means clustering, we find out the centroid of the data points and then calculate the distance of each data point from the given centroids until the clusters formed becomes constant.

Suppose the given data points are {(1, 3), (2, 5), (6, 8), (7, 9)}

The steps to perform algorithm are:

Step 1: Initialize the data points into desired number of clusters randomly.
Let’s assume there are 2 clusters in which the data is to be divided, initializing the data point randomly. Each data point lies in both the clusters with some membership value which can be assumed anything in the initial state.

The table below represents the values of the data points along with their membership (gamma) in each of the cluster.

Cluster    (1, 3)    (2, 5)    (4, 8)    (7, 9)
1)          0.8        0.7       0.2       0.1
2)          0.2        0.3       0.8       0.9

Step 2: Find out the centroid.
The formula for finding out the centroid (V) is:

$V_{ij} = ( \sum \limits_1^n ( \gamma_{ik}^m * x_k) / \sum \limits_1^n \gamma_{ik}^m$

Where, µ is fuzzy membership value of the data point, m is the fuzziness parameter (generally taken as 2), and xk is the data point.
Here,

V11  = (0.82 *1 + 0.72 * 2 + 0.22 * 4 + 0.12 * 7) / ( (0.82 + 0.72  + 0.22  + 0.12 ) = 1.568
V12  = (0.82 *3 + 0.72 * 5 + 0.22 * 8 + 0.12 * 9) / ( (0.82 + 0.72  + 0.22  + 0.12 ) = 4.051
V11  = (0.22 *1 + 0.32 * 2 + 0.82 * 4 + 0.92 * 7) / ( (0.22 + 0.32  + 0.82  + 0.92 ) = 5.35
V11  = (0.22 *3 + 0.32 * 5 + 0.82 * 8 + 0.92 * 9) / ( (0.22 + 0.32  + 0.82  + 0.92 ) = 8.215

Centroids are: (1.568, 4.051) and (5.35, 8.215)

Step 3: Find out the distance of each point from centroid.

D11 = ((1 - 1.568)² + (3 - 4.051)²)^0.5 = 1.2
D12 = ((1 - 5.35)² + (3 - 8.215)²)^0.5 = 6.79

Similarly, the distance of all other points is computed from both the centroids.

Step 4: Updating membership values.

$\gamma = \sum \limits_1^n {(d_{ki}^2 /d_{kj}^2)}^{1/m-1} ]^{-1}$

For point 1 new membership values are:

$\gamma_{11}$ = [{ [(1.2)² / (1.2)²] + [(1.2)² / (6.79)²]} ^ {(1 / (2 – 1))} ] ^-1 = 0.96

$\gamma_{12}$ = [{ [(6.79)² / (6.79)²] + [(6.79)² / (1.2)²]} ^ {(1 / (2 – 1))} ] ^-1 = 0.04
Alternatively,
$\gamma_{12} = 1- \gamma_{11} = 0.04$

Similarly, compute all other membership values, and update the matrix.

Step 5: Repeat the steps(2-4) until the constant values are obtained for the membership values or the difference is less than the tolerance value (a small value up to which the difference in values of two consequent updations is accepted).

Step 6: Defuzzify the obtained membership values.

Implementation: The fuzzy scikit learn library has a pre-defined function for fuzzy c-means which can be used in Python. For using fuzzy c-means you need to install the skfuzzy library.

pip install sklearn
pip install skfuzzy

My Personal Notes

Related Articles