Online statistical modeling in reinforcement learning

Abstract

Simulation against a model can greatly improve the learning rate of Reinforcement Learning. The Dyna algorithm uses both real experience and model learning to facilitate simulation. However, the model used in Dyna is fairly limited, yet still has some desirable properties. Examination of a few different known models can help bring to light ways of improving the Dyna model. Combining ideas from what is learned about these models should allow to a greatly improved model for Reinforcement Learning simulation.