With the help of
nltk.tokenize.TabTokenizer() method, we are able to extract the tokens from string of words on the basis of tabs between them by using
Return : Return the tokens of words.
Example #1 :
In this example we can see that by using
tokenize.TabTokenizer() method, we are able to extract the tokens from stream to words having tabs between them.
[‘Geeksfor’, ‘Geeks..’, ‘.$$&* \nis’, ‘ for geeks’]
Example #2 :
[‘The price’, ‘ of burger ‘, ‘in BurgerKing is Rs.36.\n’]
- Python NLTK | nltk.tokenize.StanfordTokenizer()
- Python | NLTK nltk.tokenize.ConditionalFreqDist()
- Python NLTK | nltk.tokenize.SExprTokenizer()
- Python NLTK | nltk.tokenizer.word_tokenize()
- Python NLTK | nltk.tokenize.LineTokenizer
- Python NLTK | nltk.tokenize.SpaceTokenizer()
- Python NLTK | nltk.WhitespaceTokenizer
- Python NLTK | nltk.TweetTokenizer()
- Python NLTK | nltk.tokenize.mwe()
- Python | Lemmatization with NLTK
- Python NLTK | tokenize.regexp()
- Python | Gender Identification by name using NLTK
- Python NLTK | tokenize.WordPunctTokenizer()
- Tokenize text using NLTK in python
- Python | Stemming words with NLTK
If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to email@example.com. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.