Validation vs. Test vs. Training Accuracy. Which One is Compared for Claiming Overfit?

Answer: You should compare the training accuracy with the validation accuracy to claim overfitting.

Here’s a comparison of Validation, Test, and Training Accuracy and which one to compare for claiming overfitting in a tabular form:

Aspect	Validation Accuracy	Test Accuracy	Training Accuracy
Purpose	Evaluate model performance on unseen validation data during training.	Assess the model’s generalization ability on independent test data.	Measure the model’s performance on the training dataset.
Dataset	Subset of the training data used to tune hyperparameters and prevent overfitting.	Independent dataset not seen during training, used to evaluate final model performance.	The dataset used for training the model.
Indication of Overfitting	If validation accuracy lags significantly behind training accuracy, it suggests overfitting.	Similar to validation accuracy, a significant gap in training accuracy can indicate overfitting.	If training accuracy is substantially higher than validation accuracy, it implies overfitting.

Conclusion:

To claim overfitting, compare the Training Accuracy with the Validation Accuracy. If the Training Accuracy significantly surpasses the Validation Accuracy, it suggests overfitting, indicating that the model may have learned to fit the training data too closely, leading to poor generalization performance on unseen data.