Dynamic Obstacles Environment (#53)

* Added new environment: Dynamic Obstacles

* Added new environment: Dynamic Obstacles

* Added new environment: Dynamic Obstacles

* Added new environment: Dynamic Obstacles

* Updated Dynamic Obstacles environment

* Updated Dynamic Obstacles environment

* Updated Dynamic Obstacles environment

* Updated Dynamic Obstacles environment

* Updated Dynamic Obstacles environment

* Updated Dynamic Obstacles environment

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Updated Dynamic Obstacles environment

* Updated Dynamic Obstacles environment

* Updated Dynamic Obstacles environment

* Updated Dynamic Obstacles environment

* Delete vcs.xml

* Delete Project_Default.xml

* Update README.md

* Update README.md

* Updated Dynamic Obstacles environment

* Update README.md

* Updated Dynamic Obstacles environment

* Updated Dynamic Obstacles environment

* Update README.md

* Updated Dynamic Obstacles environment

* Update README.md

* Updated Dynamic Obstacles environment

* Update run_tests.py

* Finalized

README.md

@@ -19,7 +19,7 @@ Please use this bibtex if you want to cite this repository in your publications:
 
 ```
 @misc{gym_minigrid,
-  author = {Chevalier-Boisvert, Maxime and Willems, Lucas},
+  author = {Chevalier-Boisvert, Maxime and Willems, Lucas and Pal, Suman},
   title = {Minimalistic Gridworld Environment for OpenAI Gym},
   year = {2018},
   publisher = {GitHub},
@@ -28,7 +28,9 @@ Please use this bibtex if you want to cite this repository in your publications:
 }
 ```
 
-This environment has been built as part of work done at the [MILA](https://mila.quebec/en/).
+This environment has been built as part of work done at the [MILA](https://mila.quebec/en/). 
+
+The Dyamic Obstacle environment has been added as part of work done at [IAS in TU Darmstadt](https://www.ias.informatik.tu-darmstadt.de/) and the [University of Genoa](https://www.unige.it/en/) for mobile robot navigation with dynamic obstacles.
 
 ## Installation
 
@@ -140,6 +142,21 @@ The random variants of the environment have the agent starting at a random
 position for each episode, while the regular variants have the agent always
 starting in the corner opposite to the goal.
 
+### Dynamic obstacles environment
+Registered configurations:
+- `MiniGrid-Dynamic-Obstacles-5x5-v0`
+- `MiniGrid-Dynamic-Obstacles-Random-5x5-v0`
+- `MiniGrid-Dynamic-Obstacles-6x6-v0`
+- `MiniGrid-Dynamic-Obstacles-Random-6x6-v0`
+- `MiniGrid-Dynamic-Obstacles-8x8-v0`
+- `MiniGrid-Dynamic-Obstacles-16x16-v0`
+
+<p align="center">
+<img src="https://media.giphy.com/media/NUvVVRNTW4cG0NoQBr/giphy.gif">
+</p>
+
+This environment is an empty room with moving obstacles. The goal of the agent is to reach the green goal square without colliding with any obstacle. A large penalty is subtracted if the agent collides with an obstacle and the episode finishes. This environment is useful to test Dynamic Obstacle Avoidance for mobile robots with Reinforcement Learning in Partial Observability.
+
 ### Door & key environment
 
 Registered configurations:

gym_minigrid/envs/__init__.py

@@ -16,3 +16,4 @@ from gym_minigrid.envs.obstructedmaze import *
 from gym_minigrid.envs.memory import *
 from gym_minigrid.envs.fourrooms import *
 from gym_minigrid.envs.crossing import *
+from gym_minigrid.envs.dynamicobstacles import *

gym_minigrid/envs/dynamicobstacles.py

@@ -0,0 +1,132 @@
+from gym_minigrid.minigrid import *
+from gym_minigrid.register import register
+from operator import add
+
+
+class DynamicObstaclesEnv(MiniGridEnv):
+    """
+    Empty grid environment with moving obstacles
+    """
+
+    def __init__(
+            self,
+            size=8,
+            agent_start_pos=(1, 1),
+            agent_start_dir=0,
+            n_obstacles=4,
+            show_obstacles=True
+    ):
+        self.agent_start_pos = agent_start_pos
+        self.agent_start_dir = agent_start_dir
+        self.show_obstacles = show_obstacles
+
+        # Reduce obstacles if there are too many
+        if n_obstacles <= size/2 + 1:
+            self.n_obstacles = int(n_obstacles)
+        else:
+            self.n_obstacles = int(size/2)
+        super().__init__(
+            grid_size=size,
+            max_steps=4 * size * size,
+            # Set this to True for maximum speed
+            see_through_walls=True,
+        )
+        # Allow only 3 actions permitted: left, right, forward
+        self.action_space = spaces.Discrete(self.actions.forward + 1)
+        self.reward_range = (-1, 1)
+
+    def _gen_grid(self, width, height):
+        # Create an empty grid
+        self.grid = Grid(width, height)
+
+        # Generate the surrounding walls
+        self.grid.wall_rect(0, 0, width, height)
+
+        # Place a goal square in the bottom-right corner
+        self.grid.set(width - 2, height - 2, Goal())
+
+        # Place the agent
+        if self.agent_start_pos is not None:
+            self.start_pos = self.agent_start_pos
+            self.start_dir = self.agent_start_dir
+        else:
+            self.place_agent()
+
+        # Place obstacles
+        if self.show_obstacles:
+            self.obstacles = []
+            for i_obst in range(self.n_obstacles):
+                self.obstacles.append(Ball())
+                self.place_obj(self.obstacles[i_obst], max_tries=100)
+        self.mission = "get to the green goal square"
+
+    def step(self, action):
+        obs, reward, done, info = MiniGridEnv.step(self, action)
+
+        # Update obstacle positions
+        if self.show_obstacles:
+            for i_obst in range(len(self.obstacles)):
+                old_pos = self.obstacles[i_obst].cur_pos
+                top = tuple(map(add, old_pos, (-1, -1)))
+                self.place_obj(self.obstacles[i_obst], top=top, size=(3,3), max_tries=100)
+                self.grid.set(old_pos[0], old_pos[1], None)
+                if np.array_equal(self.obstacles[i_obst].cur_pos, self.agent_pos):
+                    reward = -1
+                    done = True
+        return obs, reward, done, info
+
+
+class DynamicObstaclesEnv5x5(DynamicObstaclesEnv):
+    def __init__(self):
+        super().__init__(size=5, n_obstacles=2)
+
+
+class DynamicObstaclesRandomEnv5x5(DynamicObstaclesEnv):
+    def __init__(self):
+        super().__init__(size=5, agent_start_pos=None, n_obstacles=2)
+
+
+class DynamicObstaclesEnv6x6(DynamicObstaclesEnv):
+    def __init__(self):
+        super().__init__(size=6, n_obstacles=3)
+
+
+class DynamicObstaclesRandomEnv6x6(DynamicObstaclesEnv):
+    def __init__(self):
+        super().__init__(size=6, agent_start_pos=None, n_obstacles=3)
+
+
+class DynamicObstaclesEnv16x16(DynamicObstaclesEnv):
+    def __init__(self):
+        super().__init__(size=16, n_obstacles=8)
+
+
+register(
+    id='MiniGrid-Dynamic-Obstacles-5x5-v0',
+    entry_point='gym_minigrid.envs:DynamicObstaclesEnv5x5'
+)
+
+register(
+    id='MiniGrid-Dynamic-Obstacles-Random-5x5-v0',
+    entry_point='gym_minigrid.envs:DynamicObstaclesRandomEnv5x5'
+)
+
+register(
+    id='MiniGrid-Dynamic-Obstacles-6x6-v0',
+    entry_point='gym_minigrid.envs:DynamicObstaclesEnv6x6'
+)
+
+register(
+    id='MiniGrid-Dynamic-Obstacles-Random-6x6-v0',
+    entry_point='gym_minigrid.envs:DynamicObstaclesRandomEnv6x6'
+)
+
+register(
+    id='MiniGrid-Dynamic-Obstacles-8x8-v0',
+    entry_point='gym_minigrid.envs:DynamicObstaclesEnv'
+)
+
+register(
+    id='MiniGrid-Dynamic-Obstacles-16x16-v0',
+    entry_point='gym_minigrid.envs:DynamicObstaclesEnv16x16'
+)