Open In App
Related Articles

Word Sense Disambiguation in Natural Language Processing

Improve
Improve
Improve
Like Article
Like
Save Article
Save
Report issue
Report

Word sense disambiguation (WSD)  in Natural Language Processing (NLP) is the problem of identifying which “sense” (meaning) of a word is activated by the use of the word in a particular context or scenario. In people, this appears to be a largely unconscious process. The challenge of correctly identifying words in NLP systems is common, and determining the specific usage of a word in a sentence has many applications. The application of Word Sense Disambiguation involves the area of Information Retrieval, Question Answering systems, Chat-bots, etc.

Word Sense Disambiguation (WSD) is a subtask of Natural Language Processing that deals with the problem of identifying the correct sense of a word in context. Many words in natural language have multiple meanings, and WSD aims to disambiguate the correct sense of a word in a particular context. For example, the word “bank” can have different meanings in the sentences “I deposited money in the bank” and “The boat went down the river bank”.

WSD is a challenging task because it requires understanding the context in which the word is used and the different senses in which the word can be used. Some common approaches to WSD include:

  1. Supervised learning: This involves training a machine learning model on a dataset of annotated examples, where each example contains a target word and its sense in a particular context. The model then learns to predict the correct sense of the target word in new contexts.
  2. Unsupervised learning: This involves clustering words that appear in similar contexts together, and then assigning senses to the resulting clusters. This approach does not require annotated data, but it is less accurate than supervised learning.
  3. Knowledge-based: This involves using a knowledge base, such as a dictionary or ontology, to map words to their different senses. This approach relies on the availability and accuracy of the knowledge base.
  4. Hybrid: This involves combining multiple approaches, such as supervised and knowledge-based methods, to improve accuracy.

WSD has many practical applications, including machine translation, information retrieval, and text-to-speech systems. Improvements in WSD can lead to more accurate and efficient natural language processing systems.

Word Sense Disambiguation (WSD) is a subfield of Natural Language Processing (NLP) that deals with determining the intended meaning of a word in a given context. It is the process of identifying the correct sense of a word from a set of possible senses, based on the context in which the word appears. WSD is important for natural language understanding and machine translation, as it can improve the accuracy of these tasks by providing more accurate word meanings. Some common approaches to WSD include using WordNet, supervised machine learning, and unsupervised methods such as clustering.

The noun ‘star’ has eight different meanings or senses. An idea can be mapped to each sense of the word. For example,

  • “He always wanted to be a Bollywood star.” The word ‘star’ can be described as “A famous and good singer, performer, sports player, actor, personality, etc.”
  • “The Milky Way galaxy contains between 200 and 400 billion stars”. In this, the word star means “a big ball of burning gas in space that we view as a point of light in the night sky.”

Difficulties in Word Sense Disambiguation

There are some difficulties faced by Word Sense Disambiguation (WSD).

  • Different Text-Corpus or Dictionary: One issue with word sense disambiguation is determining what the senses are because different dictionaries and thesauruses divide words into distinct senses. Some academics have proposed employing a specific lexicon and its set of senses to address this problem. In general, however, research findings based on broad sense distinctions have outperformed those based on limited ones. The majority of researchers are still working on fine-grained WSD.
  • PoS Tagging: Part-of-speech tagging and sense tagging have been shown to be very tightly coupled in any real test, with each potentially constraining the other. Both disambiguating and tagging with words are involved in WSM part-of-speech tagging. However, algorithms designed for one do not always work well for the other, owing to the fact that a word’s part of speech is mostly decided by the one to three words immediately adjacent to it, whereas a word’s sense can be determined by words further away. 

Sense Inventories for Word Sense Disambiguation

Sense Inventories are the collection of abbreviations and acronyms with their possible senses. Some of the examples used in Word Sense Disambiguation are: 

  • Princeton WordNet: is a vast lexicographic database of English and other languages that is manually curated. For WSD, this is the de facto standard inventory. Its well-organized Synsets, or clusters of contextual synonyms, are nodes in a network.
  • BabelNet: is a multilingual dictionary that covers both lexicographic and encyclopedic terminology. It was created by semi-automatically mapping numerous resources, including WordNet, multilingual versions of WordNet, and Wikipedia.
  • Wiktionary: a collaborative project aimed at creating a dictionary for each language separately, is another inventory that has recently gained popularity.

Approaches for Word Sense Disambiguation

There are many approaches to Word Sense Disambiguation. The three main approaches are given below: 

1. Supervised: The assumption behind supervised approaches is that the context can supply enough evidence to disambiguate words on its own (hence, world knowledge and reasoning are deemed unnecessary). 

Supervised methods for Word Sense Disambiguation (WSD) involve training a model using a labeled dataset of word senses. The model is then used to disambiguate the sense of a target word in new text. Some common techniques used in supervised WSD include:

  1. Decision list: A decision list is a set of rules that are used to assign a sense to a target word based on the context in which it appears.
  2. Neural Network: Neural networks such as feedforward networks, recurrent neural networks, and transformer networks are used to model the context-sense relationship.
  3. Support Vector Machines: SVM is a supervised machine learning algorithm used for classification and regression analysis.
  4. Naive Bayes: Naive Bayes is a probabilistic algorithm that uses Bayes’ theorem to classify text into predefined categories.
  5. Decision Trees: Decision Trees are a flowchart-like structure in which an internal node represents feature(or attribute), the branch represents a decision rule, and each leaf node represents the outcome.

Random Forest: Random Forest is an ensemble learning method for classification, regression, and other tasks that operate by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes.

  • Supervised WSD Exploiting Glosses: Textual definitions are a prominent source of information in sense inventories (also known as glosses). Definitions, which follow the format of traditional dictionaries, are a quick and easy way to clarify sense distinctions
  • Purely Data-Driven WSD: In this case, a token tagger is a popular baseline model that generates a probability distribution over all senses in the vocabulary for each word in a context.
  • Supervised WSD Exploiting Other Knowledge: Additional sources of knowledge, both internal and external to the knowledge base, are also beneficial to WSD models. Some researchers use BabelNet translations to fine-tune the output of any WSD system by comparing the output senses’ translations to the target’s translations provided by an NMT system. 

2. Unsupervised: The underlying assumption is that similar senses occur in similar contexts, and thus senses can be induced from the text by clustering word occurrences using some measure of similarity of context. Using fixed-size dense vectors (word embeddings) to represent words in context has become one of the most fundamental blocks in several NLP systems. Traditional word embedding approaches can still be utilized to improve WSD, despite the fact that they conflate words with many meanings into a single vector representation. Lexical databases (e.g., WordNet, ConceptNet, BabelNet) can also help unsupervised systems map words and their senses as dictionaries, in addition to word embedding techniques.

3. Knowledge-Based: It is built on the idea that words used in a text are related to one another, and that this relationship can be seen in the definitions of the words and their meanings. The pair of dictionary senses having the highest word overlap in their dictionary meanings are used to disambiguate two (or more) words.  Lesk Algorithm is the classical algorithm based on Knowledge-Based WSD. Lesk algorithm assumes that words in a given “neighborhood” (a portion of text) will have a similar theme. The dictionary definition of an uncertain word is compared to the terms in its neighborhood in a simplified version of the Lesk algorithm. 

Subtopics:

  1. Supervised methods for WSD
  2. Unsupervised methods for WSD
  3. Knowledge-based methods for WSD
  4. Distributional methods for WSD
  5. Hybrid methods for WSD
  6. Evaluation metrics for WSD
  7. Applications of WSD in NLP tasks such as machine translation, information retrieval, and text summarization.
  8. Limitations and challenges in WSD research
  9. Recent developments and future directions in WSD
  10. Annotation schemes and tools for WSD

Example:

For example, consider the word “bank” in the sentence “I deposited my money in the bank.” Without WSD, it would be difficult for a computer to determine whether the word “bank” refers to a financial institution or the edge of a river. However, with WSD, the computer can use context clues such as “deposited” and “money” to determine that the intended meaning of “bank” in this sentence is a financial institution. This will improve the accuracy of natural language understanding and machine translation, as the computer will understand that the sentence is talking about depositing money in a bank account, not at the edge of a river.



Last Updated : 21 Apr, 2023
Like Article
Save Article
Previous
Next
Share your thoughts in the comments
Similar Reads