JAWAHARLAL NEHRU TECHNOLOGICAL UNIVERSITY HYDERABAD

Course Objectives:

• To introduce the foundations of Artificial Neural Networks
• To acquire the knowledge on Deep Learning Concepts
• To learn various types of Artificial Neural Networks
• To gain knowledge to apply optimization strategies

Course Outcomes:

• Ability to understand the concepts of Neural Networks
• Ability to select the Learning Networks in modeling real world systems
• Ability to use an efficient algorithm for Deep Models
• Ability to apply optimization strategies for large scale applications

UNIT - I

Artificial Neural Networks Introduction, Basic models of ANN, important terminologies, Supervised Learning Networks, Perceptron Networks, Adaptive Linear Neuron, Back-propagation Network. Associative Memory Networks. Training Algorithms for pattern association, BAM and Hopfield Networks.

UNIT - II

Unsupervised Learning Network- Introduction, Fixed Weight Competitive Nets, Maxnet, Hamming Network, Kohonen Self-Organizing Feature Maps, Learning Vector Quantization, Counter Propagation Networks, Adaptive Resonance Theory Networks. Special Networks-Introduction to various networks.

UNIT - III

Introduction to Deep Learning, Historical Trends in Deep learning, Deep Feed - forward networks, Gradient-Based learning, Hidden Units, Architecture Design, Back-Propagation and Other Differentiation Algorithms

UNIT - IV

Regularization for Deep Learning: Parameter norm Penalties, Norm Penalties as Constrained Optimization, Regularization and Under-Constrained Problems, Dataset Augmentation, Noise Robustness, Semi-Supervised learning, Multi-task learning, Early Stopping, Parameter Typing and Parameter Sharing, Sparse Representations, Bagging and other Ensemble Methods, Dropout, Adversarial Training, Tangent Distance, tangent Prop and Manifold, Tangent Classifier

UNIT - V

Optimization for Train Deep Models: Challenges in Neural Network Optimization, Basic Algorithms, Parameter Initialization Strategies, Algorithms with Adaptive Learning Rates, Approximate Second- Order Methods, Optimization Strategies and Meta-Algorithms Applications: Large-Scale Deep Learning, Computer Vision, Speech Recognition, Natural Language Processing

TEXT BOOKS:

• 1. Deep Learning: An MIT Press Book By Ian Goodfellow and Yoshua Bengio and Aaron Courville
• 2. Neural Networks and Learning Machines, Simon Haykin, 3rd Edition, Pearson Prentice Hall.