
Lecture 9 of 63 · Reinforcement Learning
Balaraman Ravindran
Reinforcement Learning · 63 lectures
63 lectures
Tutorial 1 - Probability Basics 1
Tutorial 1 - Probability Basics 2
Tutorial 2 - Linear Algebra 1
Tutorial 2 - Linear Algebra 2
Introduction to RL
RL Framework and Applications
Introduction to Immediate RL
Bandit Optimalities
Value Function Based Methods
UCB 1
Concentration Bounds
UCB 1 Theorem
PAC Bounds
Median Elimination
Thompson Sampling
Policy Search
Lecture 17
Contextual Bandits
Full RL Introduction
Returns, Value functions and MDPs
MDP Modelling
Bellman Equation
Bellman Optimality Equation
Cauchy Sequence and Green’s Equation
Banach Fixed Point Theorem
Convergence Proof
Lpi Convergence
Value Iteration
Policy Iteration
Dynamic Programming
Monte Carlo
Control in Monte Carlo
Off Policy MC
UCT
Lecture 35
TD (0) Control
Q-Learning
Eligibility Traces
Backward View of Eligibility Traces
Eligibility Trace Control
Thompson Sampling (41)
Function Approximation
Linear Parameterization
State Aggregation Methods
Function Approximation and Eligibility Traces
LSTD and LSTDQ
LSPI and Fitted Q
DQN and Fitted Q Iteration
Policy Gradient Approach
Actor Critic and REINFORCE
REINFORCE (cont’d)
Policy Gradient with Function Approximation
Hierarchical Reinforcement Learning
Types of Optimality
Semi Markov Decision Processes
Options
Learning with Options
Hierarchical Abstract Machines
MAXQ
MAXQ Value Function Decomposition
Option Discovery
POMDP Introduction
Solving POMDP