Python NLTK | nltk.tokenize.SpaceTokenizer()
Last Updated :
07 Jun, 2019
With the help of nltk.tokenize.SpaceTokenizer()
method, we are able to extract the tokens from string of words on the basis of space between them by using tokenize.SpaceTokenizer()
method.
Syntax : tokenize.SpaceTokenizer()
Return : Return the tokens of words.
Example #1 :
In this example we can see that by using tokenize.SpaceTokenizer()
method, we are able to extract the tokens from stream to words having space between them.
from nltk.tokenize import SpaceTokenizer
tk = SpaceTokenizer()
gfg = "Geeksfor Geeks.. .$$&* \nis\t for geeks"
geek = tk.tokenize(gfg)
print (geek)
|
Output :
[‘Geeksfor’, ‘Geeks..’, ‘.$$&*’, ‘\nis\t’, ‘for’, ‘geeks’]
Example #2 :
from nltk.tokenize import SpaceTokenizer
tk = SpaceTokenizer()
gfg = "The price\t of burger \nin BurgerKing is Rs.36.\n"
geek = tk.tokenize(gfg)
print (geek)
|
Output :
[‘The’, ‘price\t’, ‘of’, ‘burger’, ‘\nin’, ‘BurgerKing’, ‘is’, ‘Rs.36.\n’]
Share your thoughts in the comments
Please Login to comment...