Drop rows containing specific value in PySpark dataframe
In this article, we are going to drop the rows with a specific value in pyspark dataframe.
Creating dataframe for demonstration:
Method 1: Using where() function
This function is used to check the condition and give the results. That means it drops the rows based on the values in the dataframe column
Example 1: Python program to drop rows with college = vrs.
Example 2: Python program to drop rows with ID=1
Method 2: Using filter() function
This function is used to check the condition and give the results, Which means it drops the rows based on the values in the dataframe column. Both are similar.
Example: Python code to drop row with name = ravi.
Attention geek! Strengthen your foundations with the Python Programming Foundation Course and learn the basics.
To begin with, your interview preparations Enhance your Data Structures concepts with the Python DS Course. And to begin with your Machine Learning Journey, join the Machine Learning – Basic Level Course