|
| | | | |
| | | |
| Stat |
Members: 3645
Articles: 2'504'928
Articles rated: 2609
25 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
On the Complexity of Exploration in Goal-Driven Navigation | | Maruan Al-Shedivat
; Lisa Lee
; Ruslan Salakhutdinov
; Eric Xing
; | | Date: |
16 Nov 2018 | | Abstract: | Building agents that can explore their environments intelligently is a
challenging open problem. In this paper, we make a step towards understanding
how a hierarchical design of the agent’s policy can affect its exploration
capabilities. First, we design EscapeRoom environments, where the agent must
figure out how to navigate to the exit by accomplishing a number of
intermediate tasks (emph{subgoals}), such as finding keys or opening doors.
Our environments are procedurally generated and vary in complexity, which can
be controlled by the number of subgoals and relationships between them. Next,
we propose to measure the complexity of each environment by constructing
dependency graphs between the goals and analytically computing emph{hitting
times} of a random walk in the graph. We empirically evaluate Proximal Policy
Optimization (PPO) with sparse and shaped rewards, a variation of policy
sketches, and a hierarchical version of PPO (called HiPPO) akin to h-DQN. We
show that analytically estimated emph{hitting time} in goal dependency graphs
is an informative metric of the environment complexity. We conjecture that the
result should hold for environments other than navigation. Finally, we show
that solving environments beyond certain level of complexity requires
hierarchical approaches. | | Source: | arXiv, 1811.6889 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER