Introduction to Naive Bayes
Naive Bayes is among one of the very simple and powerful algorithms for classification based on Bayes Theorem with an assumption of independence among the predictors. The Naive Bayes classifier assumes that the presence of a feature in a class is not related to any other feature. Naive Bayes is a classification algorithm for binary and multi-class classification problems.
- Based on prior knowledge of conditions that may be related to an event, Bayes theorem describes the probability of the event
- conditional probability can be found this way
- Assume we have a Hypothesis(H) and evidence(E),
According to Bayes theorem, the relationship between the probability of Hypothesis before getting the evidence represented as P(H) and the probability of the hypothesis after getting the evidence represented as P(H|E) is:
P(H|E) = P(E|H)*P(H)/P(E)
- Prior probability = P(H) is the probability before getting the evidence
Posterior probability = P(H|E) is the probability after getting evidence
- In general,
P(class|data) = (P(data|class) * P(class)) / P(data)
Bayes Theorem Example
Assume we have to find the probability of the randomly picked card to be king given that it is a face card.
There are 4 Kings in a Deck of Cards which implies that P(King) = 4/52
as all the Kings are face Cards so P(Face|King) = 1
there are 3 Face Cards in a Suit of 13 cards and there are 4 Suits in total so P(Face) = 12/52
P(King|face) = P(face|king)*P(king)/P(face) = 1/3
Code : Implementing Naive Bayes algorithm from scratch using Python
Total number of examples are: 200 Out of these, training examples are: 140 Test examples are: 60 Accuracy of your model is: 71.2376788
Attention geek! Strengthen your foundations with the Python Programming Foundation Course and learn the basics.
To begin with, your interview preparations Enhance your Data Structures concepts with the Python DS Course.
- Naive Bayes Classifiers
- Applying Multinomial Naive Bayes to NLP Problems
- Complement Naive Bayes (CNB) Algorithm
- Implementation of Ridge Regression from Scratch using Python
- Implementation of Lasso Regression From Scratch using Python
- Linear Regression Implementation From Scratch using Python
- Implementation of K-Nearest Neighbors from Scratch using Python
- Implementation of Logistic Regression from Scratch using Python
- Implementation of neural network from scratch using NumPy
- ML - Neural Network Implementation in C++ From Scratch
- ANN - Implementation of Self Organizing Neural Network (SONN) from Scratch
- Bidirectional Associative Memory (BAM) Implementation from Scratch
- Implementation of Elastic Net Regression From Scratch
- Polynomial Regression ( From Scratch using Python )
- Grid Searching From Scratch using Python
- Python Django | Google authentication and Fetching mails from scratch
- Deep Neural net with forward and back propagation from scratch - Python
- Decision tree implementation using Python
- Python implementation of automatic Tic Tac Toe game using random number
- ML | Reinforcement Learning Algorithm : Python Implementation using Q-learning
If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to email@example.com. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.