Using the data from the treebank_chunk corpus let us evaluate the chunkers (prepared in the previous article).
Code #1 :
Accuracy of ClassifierChunker : 0.9721733155838022 Precision of ClassifierChunker : 0.9258838793383068 Recall of ClassifierChunker : 0.9359016393442623
Code #2 : Let’s compare the performance of conll_train
Accuracy of ClassifierChunker : 0.9264622074002153 Precision of ClassifierChunker : 0.8737924310910219 Recall of ClassifierChunker : 0.9007354620620346
the word can be passed through the tagger into our feature detector function, by creating nested 2-tuples of the form ((word, pos), iob), The chunk_trees2train_chunks() method produces these nested 2-tuples.
The following features are extracted:
- The current word and part-of-speech tag
- The previous word and IOB tag, part-of-speech tag
- The next word and part-of-speech tag
The ClassifierChunker class uses an internal ClassifierBasedTagger and prev_next_pos_iob() as its default feature_detector. The results from the tagger, which are in the same nested 2-tuple form, are then reformated into 3-tuples to return a final Tree using conlltags2tree().
Code #3 : different classifier builder
Accuracy of ClassifierChunker : 0.9743204362949285 Precision of ClassifierChunker : 0.9334423548650859 Recall of ClassifierChunker : 0.9357377049180328
ClassifierBasedTagger class defaults to using NaiveBayesClassifier.train as its classifier_builder. But any classifier can be used by overriding the classifier_builder keyword argument.
- NLP | Classifier-based Chunking | Set 1
- NLP | Chunking and chinking with RegEx
- NLP | Chunking Rules
- NLP | Chunking using Corpus Reader
- NLP | Distributed chunking with Execnet
- Python - Chunking text using Enchant
- NLP | Training Tagger Based Chunker | Set 1
- NLP | Training Tagger Based Chunker | Set 2
- Processing text using NLP | Basics
- Readability Index in Python(NLP)
- Feature Extraction Techniques - NLP
- Python | NLP analysis of Restaurant reviews
- Applying Multinomial Naive Bayes to NLP Problems
- NLP | Training Unigram Tagger
- NLP | Synsets for a word in WordNet
- NLP | Part of Speech - Default Tagging
- NLP | Word Collocations
- NLP | WuPalmer - WordNet Similarity
- NLP | Training a tokenizer and filtering stopwords in a sentence
- NLP | How tokenizing text, sentence, words works
If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to firstname.lastname@example.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.