Anytime Tail Averaging

We propose a technique with a low constant memory cost to perform tail averaging with access to the average at every time step. We also show how one can improve the accuracy of that average at the cost of increased memory consumption.

• Machine Intelligence