JAWAHARLAL NEHRU TECHNOLOGICAL UNIVERSITY HYDERABAD MCA 2020 INFORMATION RETRIEVAL SYSTEMS Previous Question Paper (October/ November - 2020 - R15)

An Inverted Index is the fundamental data structure used in information retrieval to map content to its location. It consists of a dictionary of unique terms (vocabulary) and, for each term, a postings list containing the identifiers of all documents in which the term appears.

The construction process involves four main steps:

1. Tokenization: Breaking the document text into individual words or tokens.
2. Linguistic Preprocessing: Applying normalization techniques such as lowercasing, stop-word removal, and stemming or lemmatization to standardize the terms.
3. Indexing: Creating a sorted list of unique terms and recording the document ID for every occurrence.
4. Postings List Creation: Consolidating the occurrences into a structure that maps each term to a list of document IDs, often sorted to facilitate efficient intersection operations.

Consider three documents:

Doc 1: "Information retrieval is fun"

Doc 2: "Retrieval systems are useful"

Doc 3: "Information systems are fun"

After preprocessing (lowercasing and removing stop words), the terms are:

When a query like "Information retrieval" is processed, the system performs a lookup in the dictionary for both terms. It retrieves the postings lists: {1, 3} for "information" and {1, 2} for "retrieval". By performing a set intersection of these lists, the system identifies that Document 1 contains both terms, allowing for rapid retrieval without scanning the entire collection.