Processing text using NLP | Basics
In this article, we will be learning the steps followed to process the text data before using it to train the actual Machine Learning Model.
The following must be installed in the current working environment:
- NLTK Library: The NLTK library is a collection of libraries and programs written for processing of English language written in Python programming language.
- urllib library: This is a URL handling library for python.
- BeautifulSoup library: This is a library used for extracting data out of HTML and XML documents.
Once importing all the libraries, we need to extract the text. Text can be in string datatype or a file that we have to process.
For this article, we are using web scraping to read a webpage then we will be using get_text() function for changing it to str format.
Once the data extraction is done, the data is now ready to process. For that follow these steps :
1. Deletion of Punctuations and numerical text
2. Creating Tokens
3. Removing Stopwords
5. Joining the final tokens
To execute the above functions refer this code :