Skip to content

Commit 38c9fb0

Browse files
jsspencerderpson
jsspencer
authored and
derpson
committed
Correct link to kfac example.py training script.
PiperOrigin-RevId: 380577563
1 parent fa8c9be commit 38c9fb0

File tree

10 files changed

+1563
-1
lines changed

10 files changed

+1563
-1
lines changed

‎box_arrangement/README.md

+106
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
# Predicate tasks.
2+
3+
This package contains tasks associated with "Behavior Priors for Efficient
4+
Reiforcement Learning" (https://arxiv.org/abs/2010.14274), "Exploiting Hierarchy
5+
for Learning and Transfer in KL-Regularized RL" (https://arxiv.org/abs/2010.14274)
6+
and "Information asymmetry in KL-regularized RL"
7+
(https://arxiv.org/abs/1905.01240).
8+
This is research code, and has dependencies on more stable code that is
9+
available as part of [`dm_control`], in particular upon components in
10+
[`dm_control.locomotion`] and [`dm_control.manipulation`].
11+
12+
To get access to preconfigured python environments for the tasks, see the
13+
`task_examples.py` file. To use the MuJoCo interactive viewer (from dm_control)
14+
to load the environments, see `explore.py`.
15+
16+
<p float="left">
17+
<img src="tasks.png" height="200">
18+
</p>
19+
20+
## Installation instructions
21+
22+
1. Download [MuJoCo Pro](https://mujoco.org/) and extract the zip archive as
23+
`~/.mujoco/mujoco200_$PLATFORM` where `$PLATFORM` is one of `linux`,
24+
`macos`, or `win64`.
25+
26+
2. Ensure that a valid MuJoCo license key file is located at
27+
`~/.mujoco/mjkey.txt`.
28+
29+
3. Clone the `deepmind-research` repository:
30+
31+
```shell
32+
git clone https://github.com/deepmind/deepmind-research.git
33+
cd deepmind-research
34+
```
35+
36+
4. Create and activate a Python virtual environment:
37+
38+
```shell
39+
python3 -m virtualenv box_arrangement
40+
source box_arrangement/bin/activate
41+
```
42+
43+
5. Install the package:
44+
45+
```shell
46+
pip install ./box_arrangement
47+
```
48+
49+
## Quickstart
50+
51+
To instantiate and step through the go to one of K targets task:
52+
53+
```python
54+
from box_arrangement import task_examples
55+
import numpy as np
56+
57+
# Build an example environment.
58+
env = task_examples.go_to_k_targets()
59+
60+
# Get the `action_spec` describing the control inputs.
61+
action_spec = env.action_spec()
62+
63+
# Step through the environment for one episode with random actions.
64+
time_step = env.reset()
65+
while not time_step.last():
66+
action = np.random.uniform(action_spec.minimum, action_spec.maximum,
67+
size=action_spec.shape)
68+
time_step = env.step(action)
69+
print("reward = {}, discount = {}, observations = {}.".format(
70+
time_step.reward, time_step.discount, time_step.observation))
71+
```
72+
73+
The above code snippet can also be used for other tasks by replacing
74+
`go_to_k_targets` with one of (`move_box`, `move_box_or_gtt` and
75+
`move_box_and_gtt`).
76+
77+
## Visualization
78+
79+
[`dm_control.viewer`] can be used to visualize and interact with the
80+
environment. We provide the `explore.py` script specifically for this. If you
81+
followed our installation instructions above, this can be launched for the
82+
go to one of K targets task via:
83+
84+
```shell
85+
python3 -m box_arrangement.explore --task='go_to_target'
86+
```
87+
88+
## Citation
89+
90+
If you use the code or data in this package, please cite:
91+
92+
```
93+
@misc{tirumala2020behavior,
94+
title={Behavior Priors for Efficient Reinforcement Learning},
95+
author={Dhruva Tirumala and Alexandre Galashov and Hyeonwoo Noh and Leonard Hasenclever and Razvan Pascanu and Jonathan Schwarz and Guillaume Desjardins and Wojciech Marian Czarnecki and Arun Ahuja and Yee Whye Teh and Nicolas Heess},
96+
year={2020},
97+
eprint={2010.14274},
98+
archivePrefix={arXiv},
99+
primaryClass={cs.AI}
100+
}
101+
```
102+
103+
[`dm_control`]: https://github.com/deepmind/dm_control
104+
[`dm_control.locomotion`]: https://github.com/deepmind/dm_control/tree/master/dm_control/locomotion
105+
[`dm_control.manipulation`]: https://github.com/deepmind/dm_control/tree/master/dm_control/manipulation
106+
[`dm_control.viewer`]: https://github.com/deepmind/dm_control/tree/master/dm_control/viewer

‎box_arrangement/__init__.py

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright 2021 DeepMind Technologies Limited
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

‎box_arrangement/dmlab_assets.py

+82
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# Copyright 2018 Deepmind Technologies Limited.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""DeepMind Lab textures."""
16+
17+
from dm_control import composer
18+
from dm_control import mjcf
19+
from labmaze import assets as labmaze_assets
20+
21+
22+
class SkyBox(composer.Entity):
23+
"""Represents a texture asset for the sky box."""
24+
25+
def _build(self, style):
26+
labmaze_textures = labmaze_assets.get_sky_texture_paths(style)
27+
self._mjcf_root = mjcf.RootElement(model='dmlab_' + style)
28+
self._texture = self._mjcf_root.asset.add(
29+
'texture', type='skybox', name='texture',
30+
fileleft=labmaze_textures.left, fileright=labmaze_textures.right,
31+
fileup=labmaze_textures.up, filedown=labmaze_textures.down,
32+
filefront=labmaze_textures.front, fileback=labmaze_textures.back)
33+
34+
@property
35+
def mjcf_model(self):
36+
return self._mjcf_root
37+
38+
@property
39+
def texture(self):
40+
return self._texture
41+
42+
43+
class WallTextures(composer.Entity):
44+
"""Represents wall texture assets."""
45+
46+
def _build(self, style):
47+
labmaze_textures = labmaze_assets.get_wall_texture_paths(style)
48+
self._mjcf_root = mjcf.RootElement(model='dmlab_' + style)
49+
self._textures = []
50+
for texture_name, texture_path in labmaze_textures.items():
51+
self._textures.append(self._mjcf_root.asset.add(
52+
'texture', type='2d', name=texture_name,
53+
file=texture_path.format(texture_name)))
54+
55+
@property
56+
def mjcf_model(self):
57+
return self._mjcf_root
58+
59+
@property
60+
def textures(self):
61+
return self._textures
62+
63+
64+
class FloorTextures(composer.Entity):
65+
"""Represents floor texture assets."""
66+
67+
def _build(self, style):
68+
labmaze_textures = labmaze_assets.get_floor_texture_paths(style)
69+
self._mjcf_root = mjcf.RootElement(model='dmlab_' + style)
70+
self._textures = []
71+
for texture_name, texture_path in labmaze_textures.items():
72+
self._textures.append(self._mjcf_root.asset.add(
73+
'texture', type='2d', name=texture_name,
74+
file=texture_path.format(texture_name)))
75+
76+
@property
77+
def mjcf_model(self):
78+
return self._mjcf_root
79+
80+
@property
81+
def textures(self):
82+
return self._textures

‎box_arrangement/explore.py

+41
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Copyright 2020 Deepmind Technologies Limited.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""Simple script to launch viewer with an example environment."""
16+
17+
from absl import app
18+
from absl import flags
19+
from dm_control import viewer
20+
from box_arrangement import task_examples
21+
22+
FLAGS = flags.FLAGS
23+
flags.DEFINE_enum('task', 'go_to_target', [
24+
'go_to_target', 'move_box', 'move_box_or_go_to_target',
25+
'move_box_and_go_to_target'
26+
], 'The task to visualize.')
27+
28+
29+
TASKS = {
30+
'go_to_target': task_examples.go_to_k_targets,
31+
'move_box': task_examples.move_box,
32+
'move_box_or_go_to_target': task_examples.move_box_or_gtt,
33+
'move_box_and_go_to_target': task_examples.move_box_and_gtt,
34+
}
35+
36+
37+
def main(unused_argv):
38+
viewer.launch(environment_loader=TASKS[FLAGS.task])
39+
40+
if __name__ == '__main__':
41+
app.run(main)

0 commit comments

Comments
 (0)