With the help of
nltk.tokenize.SpaceTokenizer() method, we are able to extract the tokens from string of words on the basis of space between them by using
Return : Return the tokens of words.
Example #1 :
In this example we can see that by using
tokenize.SpaceTokenizer() method, we are able to extract the tokens from stream to words having space between them.
[‘Geeksfor’, ‘Geeks..’, ‘.$$&*’, ‘\nis\t’, ‘for’, ‘geeks’]
Example #2 :
[‘The’, ‘price\t’, ‘of’, ‘burger’, ‘\nin’, ‘BurgerKing’, ‘is’, ‘Rs.36.\n’]
- Python | NLTK nltk.tokenize.ConditionalFreqDist()
- Python NLTK | nltk.tokenizer.word_tokenize()
- Python NLTK | nltk.tokenize.SExprTokenizer()
- Python NLTK | nltk.tokenize.StanfordTokenizer()
- Python NLTK | nltk.tokenize.TabTokenizer()
- Python NLTK | nltk.tokenize.LineTokenizer
- Python NLTK | nltk.WhitespaceTokenizer
- Python NLTK | nltk.tokenize.mwe()
- Python NLTK | nltk.TweetTokenizer()
- Python | Lemmatization with NLTK
- Python | Stemming words with NLTK
- Tokenize text using NLTK in python
- Python | Gender Identification by name using NLTK
- Python NLTK | tokenize.regexp()
- Python NLTK | tokenize.WordPunctTokenizer()
If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to email@example.com. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.