With the help of nltk.tokenize.LineTokenizer()
method, we are able to extract the tokens from string of sentences in the form of single line by using tokenize.LineTokenizer()
method.
Syntax :
tokenize.LineTokenizer()
Return : Return the tokens of line from stream of sentences.
Example #1 :
In this example we can see that by using tokenize.LineTokenizer()
method, we are able to extract the tokens from stream of sentences into small lines.
# import LineTokenizer() method from nltk from nltk.tokenize import LineTokenizer
# Create a reference variable for Class LineTokenizer tk = LineTokenizer()
# Create a string input gfg = "GeeksforGeeks...$$&* \nis\n for geeks"
# Use tokenize method geek = tk.tokenize(gfg)
print (geek)
|
Output :
[‘GeeksforGeeks…$$&* ‘, ‘is’, ‘ for geeks’]
Example #2 :
# import LineTokenizer() method from nltk from nltk.tokenize import LineTokenizer
# Create a reference variable for Class LineTokenizer tk = LineTokenizer(blanklines = 'keep' )
# Create a string input gfg = "The price\n\n of burger \nin BurgerKing is Rs.36.\n"
# Use tokenize method geek = tk.tokenize(gfg)
print (geek)
|
Output :
[‘The price’, ”, ‘ of burger ‘, ‘in BurgerKing is Rs.36.’]