|
| | | | |
| | | |
| Stat |
Members: 3657
Articles: 2'599'751
Articles rated: 2609
09 October 2024 |
|
| | | | |
|
Article overview
| |
|
|
Multi-block Min-max Bilevel Optimization with Applications in Multi-task Deep AUC Maximization | | Quanqi Hu
; Yongjian Zhong
; Tianbao Yang
; | | Date: |
1 Jun 2022 | | Abstract: | In this paper, we study multi-block min-max bilevel optimization problems,
where the upper level is non-convex strongly-concave minimax objective and the
lower level is a strongly convex objective, and there are multiple blocks of
dual variables and lower level problems. Due to the intertwined multi-block
min-max bilevel structure, the computational cost at each iteration could be
prohibitively high, especially with a large number of blocks. To tackle this
challenge, we present a single-loop randomized stochastic algorithm, which
requires updates for only a constant number of blocks at each iteration. Under
some mild assumptions on the problem, we establish its sample complexity of
$mathcal{O}(1/epsilon^4)$ for finding an $epsilon$-stationary point. This
matches the optimal complexity for solving stochastic nonconvex optimization
under a general unbiased stochastic oracle model. Moreover, we provide two
applications of the proposed method in multi-task deep AUC (area under ROC
curve) maximization and multi-task deep partial AUC maximization. Experimental
results validate our theory and demonstrate the effectiveness of our method on
problems with hundreds of tasks. | | Source: | arXiv, 2206.00260 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
|
| |
|
|
|
REGISTER