Some of the corpora that we use are often deep trees of nested phrases. But working on such deep trees is a tedious job for training the chunker. As IOB tag parsing is not designed for nested chunks. So, in order to use these trees for chunker training, we must flatten them.
Well, POS (part of Speech) are actually part of the tree structure instead of being in the word. These are used with
Tree.pos() method, designed specifically for combining words with preterminal Tree labels such as part-of-speech tags.
Code #1 : Class for flattening the deep tree
Code #2 : Evaluating
Deep Tree : (S (NP-SBJ (NP (NNP Pierre) (NNP Vinken)) (,, ) (ADJP (NP (CD 61) (NNS years)) (JJ old)) (,, )) (VP (MD will) (VP (VB join) (NP (DT the) (NN board)) (PP-CLR (IN as) (NP (DT a) (JJ nonexecutive) (NN director))) (NP-TMP (NNP Nov.) (CD 29)))) (. .)) Flattened Tree : Tree('S', [Tree('NP', [('Pierre', 'NNP'), ('Vinken', 'NNP')]), (', ', ', '), Tree('NP', [('61', 'CD'), ('years', 'NNS')]), ('old', 'JJ'), (', ', ', '), ('will', 'MD'), ('join', 'VB'), Tree('NP', [('the', 'DT'), ('board', 'NN')]), ('as', 'IN'), Tree('NP', [('a', 'DT'), ('nonexecutive', 'JJ'), ('director', 'NN')]), Tree('NP-TMP', [('Nov.', 'NNP'), ('29', 'CD')]), ('.', '.')])
The result is a much flatter Tree that only includes NP phrases. Words that are not part of an NP phrase are separated
How it works ?
- flatten_deeptree() : returns a new Tree from the given tree by calling flatten_childtrees() on each of the given tree’s children.
- flatten_childtrees() : Recursively drills down into the Tree until it finds child trees whose height() is equal to or less than 3.
Code #3 : height()
Height : 2 Height : 3
- Python | Grouped Flattening of list
- Deep Q-Learning
- Introduction to Deep Learning
- Deep Learning with PyTorch | An Introduction
- Differential Privacy and Deep Learning
- Implementing Deep Q-Learning using Tensorflow
- ML | Natural Language Processing using Deep Learning
- Prediction of Wine type using Deep Learning
- Deep Learning | Introduction to Long Short Term Memory
- Avengers Endgame and Deep learning | Image Caption Generation using the Avengers EndGames Characters
- Introduction to Multi-Task Learning(MTL) for Deep Learning
- Artificial intelligence vs Machine Learning vs Deep Learning
- copy in Python (Deep Copy and Shallow Copy)
- Decision Tree
- Decision Tree Introduction with example
If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to firstname.lastname@example.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.