Load text in Tensorflow
In this article, we are going to see how to load the text in Tensorflow using Python.
Tensorflow is an open-source Machine Learning platform that helps to create production-ready Machine Learning pipelines. Using Tensorflow, one can easily manage large datasets and develop a Neural network model in a few lines of code. These large datasets may include audio, image, video, or text. In this article, we will focus on the text dataset.
How to load the text in Tensorflow?
Text is the most used form of data in today’s real-time world. Documentation, Media Posts, Social Media conversations, and Blog articles all come in the form of text. All the text comes in raw form to be used in Machine Learning models. Tensorflow provides utility support to load the text.
Let’s take an example to demonstrate on how to load and preprocess text.
Before we proceed let us first import the required modules and download the dataset.
We downloaded Stack Overflow question text data in the above code using Keras API. utils.get_file method takes in the origin URL which contains the actual data. By setting untar=True, the dataset is unzipped automatically and saved in the directory. A Machine Learning model is continually trained on training data, verified on validation data, and tested on testing data.
Tensorflow enables us to read or load text directly from the directory and moreover lets us split the dataset into train and validation, everything using the same method.
To create validation data and assign labels to the data, we shall now use the text_dataset_from_directory method that is used to load text from the directory.
Found 8000 files belonging to 4 classes. Using 6400 files for training. Found 8000 files belonging to 4 classes. Using 1600 files for validation.
This is how you can load the text in Tensorflow.
Please Login to comment...