ML | Text Summarization of links based on user query
Whenever a user searches for particular information on the internet, multiple results are returned which are explained in variety ways. It becomes difficult and time-consuming to understand information.
Let’s say for example when a user searches for “machine learning” on Google, number of results are returned. Results returned by Google related to “machine learning” have explained “machine learning” in different ways. It becomes difficult and time-consuming to understand the various definition of “machine learning”. Thus, given a busy schedule of people and an immense amount of information available on Internet, there is a need for automatic summarization of links based on user query.
Introduction to Text Summarization:
Text summarization is the process of creating a shorter version of the text with only vital information and thus, helps the user to understand the text in a shorter amount of time. The main advantage of text summarization lies in the fact that it reduces user’s time in searching the important details in the document.
There are two main approaches to summarizing text documents –
- Extractive Method: It involves selecting phrases and sentences from the original text and including it in the final summary.
Original Text : Python is a high-level, interpreted, interactive and object-oriented scripting language.Python is a great language for the beginner-level programmers.
Extractive Summary : Python is a high-level scripting language is great language for beginner-level programmers.
- Abstractive Method: The Abstractive method involves generating entirely new phrases and sentences to capture the meaning of source document.
Original Text : Python is a high-level, interpreted, interactive and object-oriented scripting language.Python is a great language for the beginner-level programmers
Abstractive Summary : Python is interpreted and interactive language and it is easy to learn.
As we compare the summaries of two methods, we find the abstractive method best for creating summaries. Summaries created by abstractive method is summary that we humans create. Although best, not much of advances have been made in the Abstractive method.
The problem of surfing can be solved by following steps:
- Allow user to enter query.(on web application or on app.)
- If the query is valid, search the query on google.
- Google will return multiple results related to query, extract all the links on the first page(because the links are highly relevant to user query)
- Scrape and clean the data from all links and store it in text file.
- Send the data to machine learning models to generate a summary(abstractive)