How Each Method Weights Gradient Samples

Each dot is a sample with importance ratio ρ. Height = effective gradient weight f(ρ)·A. Adjust policy drift to see variance explode for f=ρ.