JAWAHARLAL NEHRU TECHNOLOGICAL UNIVERSITY HYDERABAD B.Tech IV INFORMATION THEORY AND CODING Previous Question Paper (July/August - 2022 - R16)

Entropy is a measure of the average uncertainty or information content associated with a random variable. For a discrete random variable X with possible outcomes x_i and probabilities P(x_i), entropy H(X) is defined as:

It represents the average amount of information produced by a source per symbol. Average Information is synonymous with entropy, representing the expected value of the information content of the source symbols. Mutual Information, denoted as I(X;Y), measures the amount of information that can be obtained about one random variable X through the observation of another random variable Y. It quantifies the reduction in uncertainty of X given knowledge of Y, defined as:

If X and Y are independent, the mutual information is zero, indicating that Y provides no information about X.

Errors in digital communication systems occur when the received bit differs from the transmitted bit due to noise or interference. They are broadly categorized as follows:

• Single-bit Errors: Only one bit in a data unit (such as a byte or character) is changed from 0 to 1 or vice-versa. These are common in parallel transmission.
• Burst Errors: Two or more bits in the data unit have changed. The length of the burst is measured from the first corrupted bit to the last corrupted bit. These are more common in serial transmission and are often caused by impulse noise.
• Random Errors: These occur independently of one another, typically caused by thermal noise in the communication channel.