Logistic Regression and Decision Tree classification are two of the most popular and basic classification algorithms being used today. None of the algorithms is better than the other and one’s superior performance is often credited to the nature of the data being worked upon.
We can compare the two algorithms on different categories –
|Criteria||Logistic Regression||Decision Tree Classification|
|Interpretability||Less interpretable||More interpretable|
|Decision Boundaries||Linear and single decision boundary||Bisects the space into smaller spaces|
|Ease of Decision Making||A decision threshold has to be set||Automatically handles decision making||Overfitting||Not prone to overfitting||Prone to overfitting|
|Robustness to noise||Robust to noise||Majorly affected by noise|
|Scalability||Requires a large enough training set||Can be trained on a small training set|
As a simple experiment, we run the two models on the same dataset and compare their performances.
Step 1: Importing the required libraries
Step 2: Reading and cleaning the Dataset
Step 3: Training and evaluating the Logisitc Regression model
Step 4: Training and evaluating the Decision Tree Classifier model
On comparing the scores, we can see that the logistic regression model performed better on the current dataset but this might not be the case always.
Attention geek! Strengthen your foundations with the Python Programming Foundation Course and learn the basics.
To begin with, your interview preparations Enhance your Data Structures concepts with the Python DS Course. And to begin with your Machine Learning Journey, join the Machine Learning – Basic Level Course