|
| | | | |
| | | |
| Stat |
Members: 3657
Articles: 2'599'751
Articles rated: 2609
14 October 2024 |
|
| | | | |
|
Article overview
| |
|
|
Predecessor Features | | Duncan Bailey
; Marcelo Mattar
; | | Date: |
1 Jun 2022 | | Abstract: | Any reinforcement learning system must be able to identify which past events
contributed to observed outcomes, a problem known as credit assignment. A
common solution to this problem is to use an eligibility trace to assign credit
to recency-weighted set of experienced events. However, in many realistic
tasks, the set of recently experienced events are only one of the many possible
action events that could have preceded the current outcome. This suggests that
reinforcement learning can be made more efficient by allowing credit assignment
to any viable preceding state, rather than only those most recently
experienced. Accordingly, we propose "Predecessor Features", an algorithm that
achieves this richer form of credit assignment. By maintaining a representation
that approximates the expected sum of past occupancies, our algorithm allows
temporal difference (TD) errors to be propagated accurately to a larger number
of predecessor states than conventional methods, greatly improving learning
speed. Our algorithm can also be naturally extended from tabular state
representation to feature representations allowing for increased performance on
a wide range of environments. We demonstrate several use cases for Predecessor
Features and contrast its performance with other similar approaches. | | Source: | arXiv, 2206.00303 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
|
| |
|
|
|
REGISTER