Holdout Method is the simplest sort of method to evaluate a classifier. In this method, the data set (a collection of data items or examples) is separated into two sets, called the Training set and Test set.
A classifier performs function of assigning data items in a given collection to a target category or class.
E-mails in our inbox being classified into spam and non-spam.
Classifier should be evaluated to find out, it’s accuracy, error rate, and error estimates. It can be done using various methods. One of most primitive methods in evaluation of classifier is ‘Holdout Method’.
In the holdout method, data set is partitioned, such that – maximum data belongs to training set and remaining data belongs to test set.
If there are 20 data items present, 12 are placed in training set and remaining 8 are placed in test set.
- After partitioning data set into two sets, training set is used to build a model/classifier.
- After construction of classifier, we use data items in test set, to test accuracy, error rate and error estimate of model/classifier.
However, it is vital to remember two statements with regard to holdout method. These are :
If maximum possible data items are placed in training set for construction of model/classifier, classifier’s error rates and estimates would be very low and accuracy would be high. This is sign of a good classifier/model.
A student ‘gfg’ is coached by a teacher. Teacher teaches her all possible topics which might appear for exam. Hence, she tends to commit very less mistakes in exam, thus performing well.
If more training data are used to construct a classifier, it qualifies any data used from test set, to test it (classifier).
If more number of data items are present in test set, such that they are used to test classifier built using training set. We can observe more accurate evaluation of classifier with respect to it’s accuracy, error rate and estimation.
A student ‘gfg’ is coached by a teacher. Teacher teaches her some topics, which might appear for the exam. If the student ‘gfg’ is given a number of exams on basis of this coaching, an accurate determination of student’s weak and strong points can be found out.
If more test data are used to evaluate constructed classifier, it’s error rate, error estimate and accuracy can be accurately determined.
During partitioning of whole data set into 2 parts i.e., training set and test set, if all data items belonging to class – GFG1, are placed in test set entirely, such that none of data items of class GFG1 are in training set. It is evident, that model/classifier built, is not trained using data items of class – GFG1.
Stratification is a technique, using which data items belonging to class – GFG1 are divided and placed into two data sets i.e training set and test set, equally. Such that, model/classifier is trained by data items belonging to class -GFG1.
All the four data items belonging to class – GFG1, here, are divided equally and placed, two data items each, into two data sets – training set and test set.
Attention reader! Don’t stop learning now. Get hold of all the important CS Theory concepts for SDE interviews with the CS Theory Course at a student-friendly price and become industry ready.
- Introduction to Data Science : Skills Required
- Software Engineering | Introduction to Software Engineering
- Software Engineering | MOCK (Introduction)
- Client-Server Software Development | Introduction to Common Object Request Broker Architecture (CORBA)
- Introduction of Software Design process | Set 2
- Introduction to Pervasive Computing
- Introduction To Malware Analysis
- Introduction of JIRA
- Introduction to Mercurial
- Introduction of Technical Report
- An Introduction to Software Development Design Principles
- Introduction of Unified Database Language (UDL)
- Introduction to Business Process Re-engineering
- Introduction of Test Artifacts
- Introduction to Concept Drift
- Refactoring - Introduction and Its Techniques
- Introduction to ERP
- Introduction to Exploratory Style of Software Development
- Introduction of Ethereum Testnests
- Introduction to Supply Chain Management
If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to email@example.com. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.