  
  
Stat 
Members: 3657 Articles: 2'599'751 Articles rated: 2609
15 October 2024 

   

Article overview
 

On Gapdependent Bounds for Offline Reinforcement Learning  Xinqi Wang
; Qiwen Cui
; Simon S. Du
;  Date: 
1 Jun 2022  Abstract:  This paper presents a systematic study on gapdependent sample complexity in
offline reinforcement learning. Prior work showed when the density ratio
between an optimal policy and the behavior policy is upper bounded (the optimal
policy coverage assumption), then the agent can achieve an
$Oleft(frac{1}{epsilon^2}
ight)$ rate, which is also minimax optimal. We
show under the optimal policy coverage assumption, the rate can be improved to
$Oleft(frac{1}{epsilon}
ight)$ when there is a positive suboptimality gap
in the optimal $Q$function. Furthermore, we show when the visitation
probabilities of the behavior policy are uniformly lower bounded for states
where an optimal policy’s visitation probabilities are positive (the uniform
optimal policy coverage assumption), the sample complexity of identifying an
optimal policy is independent of $frac{1}{epsilon}$. Lastly, we present
nearlymatching lower bounds to complement our gapdependent upper bounds.  Source:  arXiv, 2206.00177  Services:  Forum  Review  PDF  Favorites 


No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.

 


