Python | Pandas Index.drop_duplicates()

Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric python packages. Pandas is one of those packages and makes importing and analyzing data much easier.

Pandas Index.drop_duplicates() function return Index with duplicate values removed. The function provides the flexibility to choose which duplicate value to be retained. We can drop all duplicate values from the list or leave the first/last occurrence of the duplicated values.

Syntax: Index.drop_duplicates(labels, errors=’raise’)

Parameters :
keep : {‘first’, ‘last’, False}, default ‘first’
-> ‘first’ : Drop duplicates except for the first occurrence.
-> ‘last’ : Drop duplicates except for the last occurrence.
-> False : Drop all duplicates.

Returns : deduplicated : Index



Example #1: Use Index.drop_duplicates() function to drop all the occurrences of the duplicate value except the first occurrence.

filter_none

edit
close

play_arrow

link
brightness_4
code

# importing pandas as pd
import pandas as pd
  
# Creating the Index
idx = pd.Index([10, 11, 5, 5, 22, 5, 3, 11])
  
# Print the Index
idx

chevron_right


Output :

Let’s drop all occurrences of duplicate value in the Index except the first occurrence.

filter_none

edit
close

play_arrow

link
brightness_4
code

# drop all duplicate occurrences of the
# labels and keep the first occurrence
idx.drop_duplicates(keep ='first')

chevron_right


Output :

As we can see in the output, the Index.drop_duplicate() function has dropped duplicate occurrence of the labels in the Index.
 
Example #2: Use Index.drop_duplicate() function to drop all duplicate occurrence of the label. Do not keep any duplicated values in the Index.

filter_none

edit
close

play_arrow

link
brightness_4
code

# importing pandas as pd
import pandas as pd
  
# Creating the Index
idx = pd.Index([10, 11, 5, 5, 22, 5, 3, 11])
  
# Print the Index
idx

chevron_right


Output :

Let’s drop all occurrences of duplicate value in the Index.

filter_none

edit
close

play_arrow

link
brightness_4
code

# drop all duplicate occurrences of the labels
idx.drop_duplicates(keep = False)

chevron_right


Output :

As we can see in the output all duplicate values have been dropped form the Index.




My Personal Notes arrow_drop_up

Check out this Author's contributed articles.

If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.

Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.


Article Tags :

Be the First to upvote.


Please write to us at contribute@geeksforgeeks.org to report any issue with the above content.