Simple Random Search Provides a Competitive Approach to Reinforcement Learning. Mania, H., Guy, A., & Recht, B.
Simple Random Search Provides a Competitive Approach to Reinforcement Learning [link]Paper  abstract   bibtex   
A common belief in model-free reinforcement learning is that methods based on random search in the parameter space of policies exhibit significantly worse sample complexity than those that explore the space of actions. We dispel such beliefs by introducing a random search method for training static, linear policies for continuous control problems, matching state-of-the-art sample efficiency on the benchmark MuJoCo locomotion tasks. Our method also finds a nearly optimal controller for a challenging instance of the Linear Quadratic Regulator, a classical problem in control theory, when the dynamics are not known. Computationally, our random search algorithm is at least 15 times more efficient than the fastest competing model-free methods on these benchmarks. We take advantage of this computational efficiency to evaluate the performance of our method over hundreds of random seeds and many different hyperparameter configurations for each benchmark task. Our simulations highlight a high variability in performance in these benchmark tasks, suggesting that commonly used estimations of sample efficiency do not adequately evaluate the performance of RL algorithms.
@article{maniaSimpleRandomSearch2018,
  archivePrefix = {arXiv},
  eprinttype = {arxiv},
  eprint = {1803.07055},
  primaryClass = {cs, math, stat},
  title = {Simple Random Search Provides a Competitive Approach to Reinforcement Learning},
  url = {http://arxiv.org/abs/1803.07055},
  abstract = {A common belief in model-free reinforcement learning is that methods based on random search in the parameter space of policies exhibit significantly worse sample complexity than those that explore the space of actions. We dispel such beliefs by introducing a random search method for training static, linear policies for continuous control problems, matching state-of-the-art sample efficiency on the benchmark MuJoCo locomotion tasks. Our method also finds a nearly optimal controller for a challenging instance of the Linear Quadratic Regulator, a classical problem in control theory, when the dynamics are not known. Computationally, our random search algorithm is at least 15 times more efficient than the fastest competing model-free methods on these benchmarks. We take advantage of this computational efficiency to evaluate the performance of our method over hundreds of random seeds and many different hyperparameter configurations for each benchmark task. Our simulations highlight a high variability in performance in these benchmark tasks, suggesting that commonly used estimations of sample efficiency do not adequately evaluate the performance of RL algorithms.},
  urldate = {2019-04-16},
  date = {2018-03-19},
  keywords = {Statistics - Machine Learning,Computer Science - Artificial Intelligence,Computer Science - Machine Learning,Mathematics - Optimization and Control},
  author = {Mania, Horia and Guy, Aurelia and Recht, Benjamin},
  file = {/home/dimitri/Nextcloud/Zotero/storage/S5MWXRFU/Mania et al. - 2018 - Simple random search provides a competitive approa.pdf;/home/dimitri/Nextcloud/Zotero/storage/FHCCPQ7M/1803.html}
}
Downloads: 0
About
Documentation
Keywords
Search
Users
Terms and Conditions of Use
Privacy Policy
Sitemap
© BibBase.org 2021