Reinforcement learning is not always exciting. Sometimes you have to face the raw truth of it: most of the time, it won’t work at first. These algorithms have an enormous number of parameters to fine-tune, and the gap between “looks right on paper” and “actually learns” is wide.
I’ve been diving deep into policy gradient methods, specifically implementing the Vanilla Policy Gradient (VPG) algorithm with Generalized Advantage Estimation (GAE-λ). The video below is from one of my training runs.