
Lecture 27 of 51 · Stochastic Approximation
Gugan Chandrashekhar Mallika Thoppe
Stochastic Approximation · 51 lectures
51 lectures
51: Review of Stochastic Approximation Concepts
50: Review of Probability Theory and Fundamental Inequalities
49: Analysis of Limiting Dynamics in Q-Learning with Function
48: Q-Learning with Linear Function Approximation under
47: Q-Learning with Linear Function Approximation — A Unified
46: Concluding the Asymptotic Analysis of Q-Learning
45: Asymptotic Behaviour of the Q-Learning Limit ODE — A
44: Asymptotic Analysis of Q-Learning Algorithm
43: Best Policy Algorithm for Q-Value Functions
42: Almost Sure Convergence Analysis of Temporal Difference
41: How Good Is the TD Solution? Fixed Point Analysis
40: Temporal Difference Algorithm Through the Lens
39: Introduction to Reinforcement Learning and Value Function
38: Almost Sure Convergence via Robbins–Siegmund Theorem
37: Almost Sure Convergence via Robbins–Siegmund Theorem
36: Convergence of Distributed Robust Mean Estimation Iterates
35: Well-Posedness of Differential Inclusions
34: Motivation for Stochastic Recursive Inclusion
33: Robust Distributed Learning under Adversaries
32: Distributed Estimation of the Mean of a Random Vector
31: Almost Sure Boundedness of Iterates: Theorem and Example
30: Stability Requirements in Stochastic Approximation
29: Lower Bounds and the Minimax Risk in Estimation
28: Convergence Rate for Linear Stochastic Approximation
27: Convergence Rate for Linear Stochastic Approximation
26: Extensions, Variants, and Applications of Stochastic
25: Proof of the Key Lemma - Part II
24: Proof of the Key Lemma - Part I
23: How Stochastic Approximation Iterates Track an ODE
22: Concluding the Convergence Proof
21: Almost Sure Convergence of Stochastic Approximation Iterates
20: Foundations of Stochastic Approximation
19: Stability of ODEs and Lyapunov Methods
18: Asymptotic Behaviour of Solutions to ODEs
17: Continuity of ODE Solutions with Respect to Initial
16: Existence and Uniqueness of Solutions to Ordinary
15: Revisiting the Martingale Convergence Theorem via Stopped
14: $L^2$ Martingales
13: Doob’s Forward Convergence Theorem for Supermartingales
12: Doob’s Uncrossing Lemma
11: Martingales: Definition and Examples
10: Properties of Conditional Expectation
9: Conditional Expectation: A Formal Introduction
8: Expectation of Random Variable as Lebesgue Integration
7: Random Variables as Measurable Maps
6: Probability Spaces- A Measure Theoretic Perspective
5: Policy Evaluation in Reinforcement Learning
4: An Introduction to Reinforcement Learning
3: Estimating change rates of webpages
2: Estimating the Mean of a Random Variable
1: Overview of Stochastic Approximation