@article{DBLP:journals/corr/SchulmanLMJA15,
author = {John Schulman and
Sergey Levine and
Philipp Moritz and
Michael I. Jordan and
Pieter Abbeel},
title = {Trust Region Policy Optimization},
journal = {CoRR},
volume = {abs/1502.05477},
year = {2015},
url = {http://arxiv.org/abs/1502.05477},
timestamp = {Wed, 07 Jun 2017 14:42:34 +0200},
biburl = {http://dblp.uni-trier.de/rec/bib/journals/corr/SchulmanLMJA15},
bibsource = {dblp computer science bibliography, http://dblp.org}
}
- Update for TensorFlow 1.3
- Fix some bugs. In file
main.py, Add
history["maxkl"] = []
A parallel implementation of Trust Region Policy Optimization (TRPO) on environments from OpenAI Gym.
Now includes hyperparaemter adaptation as well! For more info, check Kevin Frans' post on this project.
Kevin Frans is working towards the ideas at this openAI research request. The code is based off of this implementation.
Kevin Frans is currently working together with Danijar on writing an updated version of this preliminary paper, describing the multiple actors setup.
How to run:
# This just runs a simple training on Reacher-v1.
python main.py
# For the commands used to recreate results, check trials.txt
Parameters:
--task: what gym environment to run on
--timesteps_per_batch: how many timesteps for each policy iteration
--n_iter: number of iterations
--gamma: discount factor for future rewards_1
--max_kl: maximum KL divergence between new and old policy
--cg_damping: damp on the KL constraint (ratio of original gradient to use)
--num_threads: how many async threads to use
--monitor: whether to monitor progress for publishing results to gym or not
- TensorFlow >= 1.3
- Python 2.7