Mathematical explanation of K-Nearest Neighbour
Last Updated :
23 May, 2023
KNN stands for K-nearest neighbour, it’s one of the Supervised learning algorithm mostly used for classification of data on the basis how it’s neighbour are classified. KNN stores all available cases and classifies new cases based on a similarity measure. K in KNN is a parameter that refers to the number of the nearest neighbours to include in the majority voting process.
How do we choose K?
Sqrt(n), where n is a total number of data points(if in case n is even we have to make the value odd by adding 1 or subtracting 1 that helps in select better)
When to use KNN?
We can use KNN when Dataset is labelled and noise-free and it’s must be small because KNN is a “Lazy learner”. Let’s understand KNN algorithm with the help of an example
NAME |
AGE |
GENDER |
CLASS OF SPORTS |
Ajay |
32 |
0 |
Football |
Mark |
40 |
0 |
Neither |
Sara |
16 |
1 |
Cricket |
Zaira |
34 |
1 |
Cricket |
Sachin |
55 |
0 |
Neither |
Rahul |
40 |
0 |
Cricket |
Pooja |
20 |
1 |
Neither |
Smith |
15 |
0 |
Cricket |
Laxmi |
55 |
1 |
Football |
Michael |
15 |
0 |
Football |
Here male is denoted with numeric value 0 and female with 1. Let’s find in which class of people Angelina will lie whose k factor is 3 and age is 5. So we have to find out the distance using
d=√((x2-x1)²+(y2-y1)²) to find the distance between any two points.
So let’s find out the distance between Ajay and Angelina using formula
d=√((age2-age1)²+(gender2-gender1)²)
d=√((5-32)²+(1-0)²)
d=√729+1
d=27.02
Similarly, we find out all distance one by one.
Distance between Angelina and |
Distance |
Ajay |
27.02 |
Mark |
35.01 |
Sara |
11.00 |
Zaira |
29.00 |
Sachin |
50.01 |
Rahul |
35.01 |
Pooja |
15.00 |
Smith |
10.05 |
Laxmi |
50.00 |
Michael |
10.05 |
So the value of k factor is 3 for Angelina. And the closest to 3 is 9,10,10.5 that is closest to Angelina are Zaira, Smith and Michael.
Zaira 9 cricket
Michael 10 cricket
smith 10.5 football
so according to KNN algorithm, Angelina will be in the class of people who like cricket. So this is how KNN algorithm works.
Share your thoughts in the comments
Please Login to comment...