Although not, possibly region-of-speech labels was diminished to decide exactly how a phrase shall be chunked. Such as for instance, take into account the following two comments:
Those two phrases have the same region-of-message tags, yet , they are chunked in different ways. In the 1st sentence, the new character and you can rice is actually separate chunks, because involved thing from the second sentence, the device display , is just one amount. Clearly, we must use facts about the content regarding the text, and additionally merely the area-of-address tags, when we need to optimize chunking abilities.
A proven way we can be utilize details about the message out of terminology is to use an excellent classifier-built tagger so you’re able to amount the brand new sentence. For instance the n-gram chunker experienced in the previous point, that it classifier-based chunker are working because of the delegating IOB tags with the conditions when you look at the a phrase, and changing those individuals labels so you can chunks. Into the classifier-built tagger in itself, we’ll use the same approach that individuals utilized in six.1 to construct an associate-of-message tagger.
seven.4 Recursion from inside the Linguistic Build
The basic code for the classifier-based NP chunker is shown in 7.9. It consists of two classes. The first class is almost identical to the ConsecutivePosTagger class from 6.5. The only two differences are that it calls a different feature extractor and that it uses a MaxentClassifier rather than a NaiveBayesClassifier . The second class is basically a wrapper around the tagger class that turns it into a chunker. During training, this second class maps the chunk trees in the training corpus into tag sequences; in the parse() method, it converts the tag sequence provided by the tagger back into a chunk tree.
Really the only bit remaining to submit is the ability extractor. We start with determining a straightforward feature extractor and this simply will bring brand new part-of-address mark of your own current token. With this particular ability extractor, all of our classifier-based chunker is extremely just as the unigram chunker, as is mirrored in its results:
We could include a component to your early in the day area-of-speech level. Including this particular feature lets the newest classifier to help you design relationships anywhere between adjoining labels, and results in a chunker which is directly related to brand new bigram chunker.
2nd, we are going to was adding a component towards the current keyword, given that we hypothesized that term posts shall be employed for chunking. We find that this ability does indeed enhance the chunker’s overall performance, by from the 1.5 payment issues (and therefore corresponds to on the good 10% loss in the fresh new mistake speed).
Finally, we can try extending the feature extractor with a variety of additional features, such as lookahead features , paired features , and complex contextual features . This last feature, called tags-since-dt , creates a string describing the set of all part-of-speech tags that have been encountered since the most recent determiner.
Your Turn: Try adding different features to the feature extractor function npchunk_has , and see if you can further improve the performance of the 
NP chunker.
Strengthening Nested Structure which have Cascaded Chunkers
So far, our chunk structures have been relatively flat. Trees consist of tagged tokens, optionally grouped under a chunk node such as NP . However, it is possible to build chunk structures of arbitrary depth, simply by creating a multi-stage chunk grammar containing recursive rules. 7.10 has patterns for noun phrases, prepositional phrases, verb phrases, and sentences. This is a four-stage chunk grammar, and can be used to create structures having a depth of at most four.
Unfortunately this result misses the Vp headed by saw . It has other shortcomings too. Let’s see what happens when we apply this chunker to a sentence having deeper nesting. Notice that it fails to identify the Vice-president chunk starting at .