JAWAHARLAL NEHRU TECHNOLOGICAL UNIVERSITY HYDERABAD B.Tech III Year NATURAL LANGUAGE PROCESSING Previous Question Paper (September - 2021 - R18)

Natural Language Processing (NLP) bridges the gap between human language and computer understanding. Its applications and challenges are outlined below:

Applications of NLP:

• Machine Translation: Automatically translating text from one language to another (e.g., Google Translate).
• Sentiment Analysis: Determining the emotional tone behind a series of words, often used in social media monitoring.
• Named Entity Recognition (NER): Identifying and classifying key entities like names, organizations, and locations in text.
• Question Answering Systems: Building automated systems that can answer user queries based on provided context or knowledge bases.
• Speech Recognition: Converting spoken language into written text.
• Text Summarization: Creating concise versions of long documents while retaining core information.

Challenges in NLP:

• Ambiguity: Words often have multiple meanings depending on context (lexical ambiguity) or sentences may have multiple interpretations (syntactic ambiguity).
• Context Dependency: Understanding the meaning of a word or phrase often requires knowledge of previous sentences or the broader situation.
• Sarcasm and Irony: Detecting non-literal meaning is difficult because the literal interpretation of words often contradicts the intended sentiment.
• Language Variation: Handling dialects, slang, informal grammar, and code-switching (mixing languages) poses significant hurdles.
• Data Sparsity: For many languages, there is a lack of large, annotated datasets required for training robust machine learning models.

The Finite State Transducer (FST) is a prominent morphological model used to map between the surface form of a word and its underlying lexical form (root/lemma and morphemes). An FST is an extension of a Finite State Automaton that maps a sequence of symbols to another sequence of symbols.

Key Components:

• States: Represent the progress of the analysis or generation process.
• Transitions: Labeled with input/output pairs (e.g., i:o), where 'i' is the surface character and 'o' is the lexical character.

Working Principle:

Morphological processing using FSTs typically involves two levels:

• Lexical Level: Represents the abstract morphemes (e.g., 'cat' + 'plural').
• Surface Level: Represents the actual orthographic form (e.g., 'cats').

The FST acts as a bidirectional machine. In Morphological Analysis, it takes the surface form as input and produces the lexical form. In Morphological Generation, it takes the lexical form as input and produces the surface form.

Consider the rule for pluralizing a noun ending in 'y' preceded by a consonant (e.g., 'fly' to 'flies'):

This model is highly efficient because FSTs can be composed. For instance, a lexicon FST can be composed with a spelling-rule FST to create a single, unified transducer that maps surface strings directly to their morphological analyses, significantly reducing computational overhead during parsing.