JAWAHARLAL NEHRU TECHNOLOGICAL UNIVERSITY HYDERABAD B.Tech IV NEURAL NETWORKS AND DEEP LEARNING Previous Question Paper (December - 2020 - R16)

A back propagation network is a multilayer feedforward neural network trained using the back propagation algorithm, which computes the gradient of the loss function with respect to the weights by applying the chain rule. Because of its ability to learn complex non-linear mappings, it is used in a wide variety of tasks:

• Pattern Recognition and Classification: Back propagation networks are widely used to classify input data into predefined categories. For example, they can be trained to recognize handwritten digits, identify objects in images, or classify emails as spam or legitimate based on feature vectors.
• Function Approximation: These networks are universal approximators, meaning they can model any continuous function given sufficient hidden units. This is applied in control systems, signal processing, and financial forecasting to predict future values based on historical trends.
• Data Compression and Dimensionality Reduction: By using an autoencoder architecture, where the network is trained to reconstruct its input at the output layer through a bottleneck hidden layer, back propagation networks can learn efficient, compressed representations of data.
• Speech and Natural Language Processing: These networks serve as the foundation for tasks like phoneme recognition in speech-to-text systems and sentiment analysis in text, where the network learns to map variable-length sequences to specific labels or outputs.
• Medical Diagnosis: Networks are trained on patient datasets, such as symptoms and test results, to assist in the early detection of diseases like cancer or diabetes by identifying non-linear correlations that might be missed by traditional statistical methods.
• Time Series Forecasting: By utilizing sliding window inputs, back propagation networks predict future states of a system, such as stock market prices, weather patterns, or energy demand, by learning the temporal dependencies within the data.

A Bidirectional Associative Memory (BAM) is a two-layer heteroassociative neural network that stores pairs of patterns (A, B) such that when one pattern is presented, the network retrieves the other. It operates as a feedback system where information flows back and forth between two layers.

The architecture consists of two layers of neurons, Layer X (with n neurons) and Layer Y (with m neurons). These layers are fully interconnected by a weight matrix W, where the connection from X to Y is represented by W, and the connection from Y to X is represented by the transpose W^T. There are no intra-layer connections.

The algorithm for BAM proceeds as follows:

1. Initialization: Define the weight matrix W using the sum of outer products of the pattern pairs (A_i, B_i) for i=1 to p: W = Σ (A_i^T * B_i).
2. Activation: Present an input vector X to the input layer X.
3. Forward Pass: Calculate the output of layer Y using the activation function f: Y = f(X * W).
4. Backward Pass: Use the calculated Y to update layer X: X_new = f(Y * W^T).
5. Iteration: Repeat the forward and backward passes until the vectors X and Y stabilize and no longer change between iterations.
6. Convergence: The network reaches a stable state (a local minimum of the energy function) where the input-output pair represents the stored association.

The energy function for BAM is defined as E = -X * W * Y^T. The network is guaranteed to converge to a stable state because the energy function decreases or remains constant during each state update, ensuring the system settles into a local minimum.