FLAIR – A Framework for NLP
What is FLAIR?
It is a simple framework for state-of-the-art NLP. It is a very powerful library which is developed by Zalando Research. The Flair framework is built on top of PyTorch.
Attention reader! Don’t stop learning now. Get hold of all the important Machine Learning Concepts with the Machine Learning Foundation Course at a student-friendly price and become industry ready.
What are the Features available in Flair?
- Flair supports a number of word embeddings used to perform NLP tasks such as FastText, ELMo, GloVe, BERT and its variants, XLM, and Byte Pair Embeddings including Flair Embedding.
- The Flair Embedding is based on the concept of contextual string embeddings which is used for Sequence Labelling.
- Using Flair you can also combine different word embeddings together to get better results.
- Flair supports a number of languages.
Contextual String Embeddings:
In this word embedding each of the letters in the words are sent to the Character Language Model and then the input representation is taken out from the forward and backward LSTMs.
The input representation for the word ‘Washington’ is been considered based on the context before the word ‘Washington’. The first and last character states of each word is taken in order to generate the word embeddings.
You can see that for the word ‘Washington’ the red mark is the forward LSTM output and the blue mark is the backward LSTM output. Both forward and backward contexts are concatenated to obtain the input representation of the word ‘Washington’.
After getting the input representation it is fed to the forward and backward LSTM to get the particular task that you are dealing with. In the diagram mentioned we are trying to get the NER.
Installation of Flair:
You should have PyTorch >=1.1 and Python >=3.6 installed. To install PyTorch on anaconda run the below command-
conda install -c pytorch pytorch
To install flair, run –
pip install flair
Working of Flair
1) Flair Datatypes:
Flair offers two types of objects. They are:
To get the number of tokens in a sentence:
2) NER Tags:
To predict tags for a given sentence we will use a pre-trained model as shown below:
3) Word Embeddings:
Word embeddings give embeddings for each word of the text. As discussed earlier Flair supports many word embeddings including its own Flair Embeddings. Here we will see how to implement some of them.
A) Classic Word Embeddings – This class of word embeddings are static. In this, each distinct word is given only one pre-computed embedding. Most of the common word embeddings lie in this category including the GloVe embedding.
Note: You can see here that the embeddings for the word ‘Geeks‘ are the same for both the occurrences.
B) Flair Embedding – This works on the concept of contextual string embeddings. It captures latent syntactic-semantic information. The word embeddings are contextualized by their surrounding words. It thus gives different embeddings for the same word depending on it’s surrounding text.
Note: Here we see that the embeddings for the word ‘Geeks’ are different for both the occurrences depending on the contextual information around them.
C) Stacked Embeddings – Using these embeddings you can combine different embeddings together. Let’s see how to combine GloVe, forward and backward Flair embeddings:
4) Document Embeddings:
, Unlike word embeddings, document embeddings give a single embedding for the entire text. The document embeddings offered in Flair are:
- A) Transformer Document Embeddings
- B) Sentence Transformer Document Embeddings
- C) Document RNN Embeddings
- D) Document Pool Embeddings
Let’s have a look at how the Document Pool Embeddings work-
Document Pool Embeddings — It is a very simple document embedding and it pooled over all the word embeddings and returns the average of all of them.
Similarly, you can use other Document embeddings as well.
5) Training a Text Classification Model using Flair:
We are going to use the ‘TREC_6’ dataset available in Flair. You can also use your own datasets as well. To train our model we will be using the Document RNN Embeddings which trains an RNN over all the word embeddings in a sentence. The word embeddings which we will be using are the GloVe and the forward flair embedding.
Results of training:
The accuracy of the model is around 95%.
Predictions: Now we can load the model and make predictions-
Now you would have got a rough idea of how to use the Flair library.