Python NLTK | nltk.tokenize.LineTokenizer

With the help of nltk.tokenize.LineTokenizer() method, we are able to extract the tokens from string of sentences in the form of single line by using tokenize.LineTokenizer() method.

Syntax : tokenize.LineTokenizer()
Return : Return the tokens of line from stream of sentences.

Example #1 :
In this example we can see that by using tokenize.LineTokenizer() method, we are able to extract the tokens from stream of sentences into small lines.

# import LineTokenizer() method from nltk
from nltk.tokenize import LineTokenizer

# Create a reference variable for Class LineTokenizer
tk = LineTokenizer()

# Create a string input
gfg = "GeeksforGeeks...$$&* \nis\n for geeks"

# Use tokenize method
geek = tk.tokenize(gfg)

print(geek)

Output :

[‘GeeksforGeeks…$$&* ‘, ‘is’, ‘ for geeks’]

Example #2 :

# import LineTokenizer() method from nltk
from nltk.tokenize import LineTokenizer

# Create a reference variable for Class LineTokenizer
tk = LineTokenizer(blanklines ='keep')

# Create a string input
gfg = "The price\n\n of burger \nin BurgerKing is Rs.36.\n"

# Use tokenize method
geek = tk.tokenize(gfg)

print(geek)

Output :

[‘The price’, ”, ‘ of burger ‘, ‘in BurgerKing is Rs.36.’]