JAWAHARLAL NEHRU TECHNOLOGICAL UNIVERSITY HYDERABAD

Prerequisite

Statistics and Linear Algebra

Course Objectives:

• 1. The student will be able to understand the mathematical formulation of patterns.
• 2. To study the various linear models.
• 3. Understand the basic classifiers.
• 4. Can able to distinguish different models.

Course Outcomes

On completion of this course student will be able to

• 1. Familiar the basics of pattern classes and functionality.
• 2. Construct the various linear models.
• 3. Use the different kernel methods.
• 4. Design the Markov and Mixed models.

UNIT - I

Introduction to Pattern recognition: Mathematical Formulation and Basic Functional Equation, Reduction of Dimensionality, Experiments in Pattern Classification, Backward Procedure for Both Feature Ordering- and Pattern Classification, Suboptimal Sequential Pattern Recognition, Nonparametric Design of Sequential Pattern Classifiers, Analysis of Optimal Performance and a Multiclass Generalization

UNIT - II

Linear Models: Linear Basis Function Models -Maximum likelihood and least squares, Geometry of least squares , Sequential learning, Regularized least squares, Multiple outputs , The Bias-Variance Decomposition, Bayesian Linear Regression -Parameter distribution, Predictive, Equivalent, Bayesian Model Comparison, Probabilistic Generative Models-Continuous inputs, Maximum likelihood solution, Discrete features, Exponential family, Probabilistic Discriminative Models -Fixed basis functions, Logistic regression, Iterative reweighted least squares, Multiclass logistic regression, Probit regression, Canonical link functions

UNIT - III

Kernel Methods: Constructing Kernels, Radial Basis Function Networks - Nadaraya-Watson model, Gaussian Processes -Linear regression revisited, Gaussian processes for regression, Learning the hyper parameters, Automatic relevance determination, Gaussian processes for classification, Laplace approximation, Connection to neural networks, Sparse Kernel Machines- Maximum Margin Classifiers, Overlapping class distributions, Relation to logistic regression, Multiclass SVMs, SVMs for regression, Computational learning theory, Relevance Vector Machines- RVM for regression, Analysis of sparsity, RVM for classification

UNIT - IV

Graphical Models: Bayesian Networks, Example: Polynomial regression, Generative models, Discrete variables, Linear-Gaussian models, Conditional Independence- Three example graphs, D-separation, Markov Random Fields -Conditional independence properties, Factorization properties, Illustration: Image de-noising, Relation to directed graphs, Inference in Graphical Models- Inference on a chain, Trees, Factor graphs, The sum-product algorithm, The max-sum algorithm, Exact inference in general graphs, Loopy belief propagation, Learning the graph structure.

UNIT - V

Mixture Models and EM algorithm: K-means Clustering-Image segmentation and compression, Mixtures of Gaussians-Maximum likelihood, EM for Gaussian mixtures, An Alternative View of EM- Gaussian mixtures revisited, Relation to K-means, Mixtures of Bernoulli distributions, EM for Bayesian linear regression, The EM Algorithm in General, Combining Models- Tree-based Models, Conditional Mixture Models- Mixtures of linear regression models, Mixtures of logistic models, Mixtures of experts.

TEXT BOOKS:

• 1. Sequential methods in Pattern Recognition and Machine Learning-K.S.Fu, Academic Press, volume no.52.
• 2. Pattern Recognition and Machine Learning- C. Bishop-Springer,2006.

REFERENCE BOOKS:

• 1. Pattern Classification- Richard o. Duda, Peter E. hart, David G. Stork, John Wiley& Sons, 2nd Ed., 2001.
• 2. The elements of Statistical Learning- Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, Springer, 2nd Ed., 2009. M.TECH.- I YEAR- I SEMESTER DIGITAL SYSTEMS & COMPUTER ELECTRONICS