With the help of
nltk.tokenize.LineTokenizer() method, we are able to extract the tokens from string of sentences in the form of single line by using
Return : Return the tokens of line from stream of sentences.
Example #1 :
In this example we can see that by using
tokenize.LineTokenizer() method, we are able to extract the tokens from stream of sentences into small lines.
[‘GeeksforGeeks…$$&* ‘, ‘is’, ‘ for geeks’]
Example #2 :
[‘The price’, ”, ‘ of burger ‘, ‘in BurgerKing is Rs.36.’]
- Python | NLTK nltk.tokenize.ConditionalFreqDist()
- Python NLTK | nltk.tokenize.TabTokenizer()
- Python NLTK | nltk.tokenizer.word_tokenize()
- Python NLTK | nltk.tokenize.SExprTokenizer()
- Python NLTK | nltk.tokenize.SpaceTokenizer()
- Python NLTK | nltk.tokenize.StanfordTokenizer()
- Python NLTK | nltk.TweetTokenizer()
- Python NLTK | nltk.tokenize.mwe()
- Python NLTK | nltk.WhitespaceTokenizer
- Python | Lemmatization with NLTK
- Python NLTK | tokenize.WordPunctTokenizer()
- Python NLTK | tokenize.regexp()
- Tokenize text using NLTK in python
- Python | Gender Identification by name using NLTK
- Python | Stemming words with NLTK
If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to email@example.com. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.