
Lecture 4 of 87 · Math Foundations of Machine Learning
NPTEL IISc Bengaluru
Math Foundations of Machine Learning · 87 lectures
87 lectures
Overview of Function Approximation
Recap of Probability Theory - 1, Part 1
Recap of Probability Theory - 1, Part 2
Recap of Probability Theory - 1, Part 3
Recap of Probability Theory Part 2
Understanding a Chest X-Ray as Sample from Distribution
IID Assumption
Distribution Estimation
Challenge With ML
Tutorial 1: Introduction to Python Basics
Tutorial 2: Simple Problem solving in Probability Theory
Entropy
Kullback-Leibler (KL) Divergence
Minimization of KL Divergence
Example of ML Estimate
Risk Minimization Framework
Bayes Classifier
Tutorail 3: Risk Minimization Framework
MLE for Gaussian Distribution
MLE for Generalized Discrete Random Variable
Density Estimation for Mixed Distribution
Latent Variable Models
MLE for Latent Variable Models
Expectation Maximization Algorithm
Tutorial 4: Minmax Classifier
Tutorial 5: Neyman Pearson Classifier
Tutorial 6: Example of NP Classifier, ROC Curve
Tutorial 7A: MLE for Gaussian Distribution
Tutorial 7B: MLE for Generalized Discrete Distribution
Convergence of EM
EM for GMMs
MAP Estimate
Parzen Window
Nearest Neighbor Classifier
Tutorial 8: Computation of EM for GMMs
Tutorial 9: MAP Estimate
Ordinary Least Squares (OLS)
Generalized Least Squares (GLS)
Linear Models for Classification
Bias - Variance Decomposition and Analysis
Bias & Variance in Practice
Tutorial 10 Part A: Numerical Example on Bayes Classifier
Tutorial 10 Part B: Numerical Example on MLE and MAP Estimate
Regularization
Regularized ERM and MAP Estimate
Stochastic Gradient Descent as a Regularizer
Max-Margin Classifier and SVM
SVM Formulation
Dual Function in SVM
SVM for Non-Linear Seperable Case
SVM with Kernel Function
Neural Networks and Universal Approximation Theorem
ERM on Neural Networks and Error Backpropagation
Local Receptive Field and Parameter Sharing
Convolutional Neural Networks (CNNs) as Regularized MLP
Recurrent Neural Networks (RNNs)
Back Prapogation in RNNs and Vanishing Gradients Problem
LSTMs and GRUs
Tutorial 11: Pytorch - Tensors and Data Loaders
Tutorial 12: Pytorch - Building MLP and Auto Grad
Tutorial 13: Pytorch - Training the Model
Attention Part1
Attention Part2
Multi-Head Attention and Transformer Architecture
Positional Embeddings
Transfer Learning and Knowledge Distilation
SGD, RMS Prop, ADAM: Optimizers
Tutorial 14 Part 1: CNNs
Tutorial 14 Part 2: Transfer Learning using CNNs
Tutorial 15 Part 1: RNNs, LSTMs and GRUs
Tutorial 15 Part 2: Deep RNNs, LSTMs and GRUs
Decision Trees and Impurity Measures
Regression Trees
Ensemble Methods, Bagging and Boosting
Gradient Boosting Algorithm
Ada-Boosting
Cross Validation
Un-Supervised Learning
K-Means Clustering
PCA - Principal Component Analysis
NCE - Noise Contrastive Estimation
NCE, Info-NCE, SimCLR, JEPA
Introduction to Generative Models
GAN - Generative Adversarial Networks
Variational Auto Encoders: VAEs
Introduction to Large Language Models: LLMs
Introduction to Reinforcement Learning: RL