JAWAHARLAL NEHRU TECHNOLOGICAL UNIVERSITY HYDERABAD B.Tech IV AI TECHNIQUES IN ELECTRICAL ENGINEERING Previous Question Paper (July/August-2022 - R18)

Artificial Neural Networks (ANN) and Biological Neural Networks (BNN) differ significantly in their structural and functional implementation:

• Processing Element: In BNN, the neuron is the fundamental unit, processing information through electrochemical signals. In ANN, the processing element is a mathematical node (perceptron) that computes a weighted sum of inputs followed by a non-linear activation function.
• Connectivity: BNNs exhibit massive, highly complex, and dynamic connectivity with billions of synapses that change strength via synaptic plasticity. ANNs typically use fixed, layered architectures with weights representing connection strengths, which are adjusted during training.
• Learning Mechanism: Learning in BNN occurs through physical changes in synaptic efficacy and structural remodeling. In ANN, learning is a mathematical optimization process, such as gradient descent, aimed at minimizing a cost function.
• Speed and Parallelism: BNNs operate at low speeds (milliseconds) but achieve massive parallelism. ANNs operate at high speeds (nanoseconds) on digital hardware, though they often simulate parallelism through matrix operations.
• Energy Efficiency: BNNs are extremely energy-efficient, operating on approximately 20 watts. ANNs are computationally intensive and require significant electrical power to perform complex matrix multiplications.

Boltzmann learning is a stochastic learning rule used in Boltzmann Machines, which are recurrent neural networks with symmetric weights. The process is based on statistical mechanics and the concept of energy minimization.

The learning process involves two distinct phases to adjust the weights between units:

1. Positive Phase (Clamping): The visible units are clamped to a specific state representing the training data. The network is allowed to reach thermal equilibrium, and the correlation between the states of connected units is measured. This phase increases the probability of the network generating the training patterns.
2. Negative Phase (Free-running): The network is allowed to run freely without any external input clamped to the visible units. The network reaches equilibrium based on its internal weights, and the correlation between units is measured again. This phase decreases the probability of the network generating patterns not present in the training set.

The weight update rule is given by:

Where η is the learning rate, p_ij⁺ is the probability of units i and j being active together during the positive phase, and p_ij^- is the probability during the negative phase. By iteratively updating weights based on the difference between these correlations, the network learns to represent the underlying probability distribution of the input data.