|
| | | | |
| | | |
| Stat |
Members: 3645
Articles: 2'501'711
Articles rated: 2609
19 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
Scalable agent alignment via reward modeling: a research direction | | Jan Leike
; David Krueger
; Tom Everitt
; Miljan Martic
; Vishal Maini
; Shane Legg
; | | Date: |
19 Nov 2018 | | Abstract: | One obstacle to applying reinforcement learning algorithms to real-world
problems is the lack of suitable reward functions. Designing such reward
functions is difficult in part because the user only has an implicit
understanding of the task objective. This gives rise to the agent alignment
problem: how do we create agents that behave in accordance with the user’s
intentions? We outline a high-level research direction to solve the agent
alignment problem centered around reward modeling: learning a reward function
from interaction with the user and optimizing the learned reward function with
reinforcement learning. We discuss the key challenges we expect to face when
scaling reward modeling to complex and general domains, concrete approaches to
mitigate these challenges, and ways to establish trust in the resulting agents. | | Source: | arXiv, 1811.7871 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER