Prerequisite: Support Vector Machines
Definition of a hyperplane and SVM classifier:
For a linearly separable dataset having n features (thereby needing n dimensions for representation), a hyperplane is basically an (n – 1) dimensional subspace used for separating the dataset into two sets, each set containing data points belonging to a different class. For example, for a dataset having two features X and Y (therefore lying in a 2-dimensional space), the separating hyperplane is a line (a 1-dimensional subspace). Similarly, for a dataset having 3-dimensions, we have a 2-dimensional separating hyperplane, and so on.
In machine learning, Support Vector Machine (SVM) is a non-probabilistic, linear, binary classifier used for classifying data by learning a hyperplane separating the data.
Classifying a non-linearly separable dataset using a SVM – a linear classifier:
As mentioned above SVM is a linear classifier which learns an (n – 1)-dimensional classifier for classification of data into two classes. However, it can be used for classifying a non-linear dataset. This can be done by projecting the dataset into a higher dimension in which it is linearly separable!
To get a better understanding, let’s consider circles dataset.
The dataset is clearly a non-linear dataset and consists of two features (say, X and Y).
In order to use SVM for classifying this data, introduce another feature Z = X2 + Y2 into the dataset. Thus, projecting the 2-dimensional data into 3-dimensional space. The first dimension representing the feature X, second representing Y and third representing Z (which, mathematically, is equal to the radius of the circle of which the point (x, y) is a part of). Now, clearly, for the data shown above, the ‘yellow’ data points belong to a circle of smaller radius and the ‘purple’ data points belong to a circle of larger radius. Thus, the data becomes linearly separable along the Z-axis.
Now, we can use SVM (or, for that matter, any other linear classifier) to learn a 2-dimensional separating hyperplane. This is how the hyperplane would look like:
Thus, using a linear classifier we can separate a non-linearly separable dataset.
A brief introduction to kernels in machine learning:
In machine learning, a trick known as “kernel trick” is used to learn a linear classifier to classify a non-linear dataset. It transforms the linearly inseparable data into a linearly separable one by projecting it into a higher dimension. A kernel function is applied on each data instance to map the original non-linear data points into some higher dimensional space in which they become linearly separable.
Attention reader! Don’t stop learning now. Get hold of all the important CS Theory concepts for SDE interviews with the CS Theory Course at a student-friendly price and become industry ready.
- SVM Hyperparameter Tuning using GridSearchCV | ML
- ML | Non-Linear SVM
- Major Kernel Functions in Support Vector Machine (SVM)
- Introduction to Support Vector Machines (SVM)
- Pyspark | Linear regression with Advanced Feature Dataset using Apache MLlib
- Python - Basics of Pandas using Iris Dataset
- Image Caption Generator using Deep Learning on Flickr8K dataset
- Applying Convolutional Neural Network on mnist dataset
- Importing Kaggle dataset into google colaboratory
- Different dataset forms in Social Networks
- Python - Removing Constant Features From the Dataset
- Multiclass classification using scikit-learn
- Python | Image Classification using keras
- ML | Cancer cell classification using Scikit-learn
- Sentiment Classification Using BERT
- Getting started with Classification
- Regression and Classification | Supervised Machine Learning
- Basic Concept of Classification (Data Mining)
- ML | Classification vs Regression
- ML | Why Logistic Regression in Classification ?
If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to firstname.lastname@example.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.