Multilingual Google Meet Summarizer – Python Project
At the start of 2020, we faced the largest crisis of the 21st century – The COVID-19 pandemic. Amidst the chaos, the generation eventually found a way to get the job done by introducing automation in every other aspect of life. After the hit of the pandemic, we have encountered a rise of 87% in video conferencing tools for daily communications. The communications ranging from online meetups, college lectures, business meets, almost everything got hosted over to the internet, which being virtual, encroached the chances of unfruitful communications. In fact, the data collected from employees of all domains, show that people often miss important points because they find taking minutes of those meetings a time-consuming, distracting, and really boring task, and over 37 billion dollars is wasted over these unproductive meetings. So there arises the need for automatic Text Summarisation.
Attention reader! Don’t stop learning now. Get hold of all the important Machine Learning Concepts with the Machine Learning Foundation Course at a student-friendly price and become industry ready.
The purpose of this project is to transcribe online meetings and summarize them by applying hardcore machine learning techniques, in order to produce minutes of the meeting and multilingual summarized audio to gain the user’s, a better understanding of the topic.
The features proposed by the chrome extension are –
- Meet Transcription
- Summarization using Extractive and Abstractive Models.
- Multilingual Audio generation.
Additional Features Offered –
- Summarization of NPTEL/ MOOC lectures
- As a Twitter Tweet Shortner.
Tools and Technologies
- Backend: Django, Django Rest API
- Database: SQLite
- ML Libraries: NLTK, Torch
Knowledge of Python, NLP Libraries, use of Rest API, and good work experience in Web Development using Reactjs.
- The user logs on to the website and enables the chrome extension.
- The chrome extension extracts the audio from the Google meet concerning every speaker and transcribes it via the chrome extension.
- This extracted transcript is further sent to the backend where hardcore Machine Learning techniques are applied for the text summarization.
- This ultra-processed text is then directed towards the translator to translate the extract into the user’s desired language
- This transcript can then be downloaded or even heard on the dashboard according to the user’s wish.
So this is how Multilingual Google Meet Summarizer contributes to the pandemic-driven automation.
Step B Step Implmentation
1. Chrome Extension
The main task of the chrome extension is to extract the google meet caption from DOM elements of Google Meet. It makes use of the caption container generated by the inbuilt feature of Google Meet – Turn on Captions. The Meet is Transcribed as Follows –
- With the help of selenium, the Xpath of the “Turn On Captions” button is traced out.
- The code automatically activates the Google Meet captions by auto-clicking the Captions button.
- Then we trace out the Xpath of the caption container and extract the auto-generated text rolling inside the container.
- Finally, we append the text in a string with the speaker’s name and timestamp.
The complete text is then sent to the backend for processing.
2. Frontend and Backend
- The first step will be to create an authentication system for users to be able to log in and save their meeting transcripts in the database.
- Create a user model in Django.
- Implement JWT(JSON Web Token) in Django using the djangorestframework_simplejwt app.
- Create respective REST Views for token generation and authentication.
- One can also use social authentication using JWT instead of email registration depending on preference.
- The next step will be to create a database for storing user meet transcript information. Create a standard SQL Relation on similar lines as given below.
- Transcript(transcript_id, owner_name, transcript_date, hostname, transcript_title, meet_duration, content).
- Additional fields and schemas can also be used depending on the CRUD functionalities to be provided to the user.
- Create REST API for saving a new Transcript in the database. This API will be used by the chrome extension which will send its transcript along with meta information like timestamp, hostname, etc.
The API processes the received information and stores it in the database.
- Create separate APIs for each NLP model which is to be used for summarization. The front end will send the transcript to this API. The API uses the NLP model and returns the summarized text to the frontend. The API simply runs the ML NLP Script which is described below. Create API for translation of the text into multiple languages.
3. ML Algorithm
Project Application in Real-Life
- Used in business virtual meetings
- Used by Students for getting concise notes from a lecture
- An aid to visually impaired
- Also as a Twitter Tweet Shortener