On Computational and Statistical Challenges of Learning with Multiple Objectives

Lisha Chen, Electrical, Computer, and Systems Engineering, Rensselaer Polytechnic Institute

1400 Wegmans Hall

Abstract: Multi-objective learning (MOL) problems often arise in emerging machine learning problems when there are multiple learning criteria, data modalities, or learning tasks. Different from single-objective learning, one of the critical challenges in MOL is the potential conflict among different objectives during the iterative optimization process. 

In this talk, I will first highlight the computational challenges of MOL, and introduce several stochastic dynamic weighting-based optimization algorithms that avoid conflicts among multiple objectives and achieve iteration complexity as conventional single-objective learning. In the second part, I will focus on the statistical challenges of MOL and study the generalization performance of the dynamic weighting based MOL algorithms and its interplay with optimization through the lens of algorithm stability. Notably, I will reveal that the key rationale behind the dynamic weighting based MOL algorithms – updating along conflict-avoidant direction – may hinder the algorithms from achieving the optimal population risk. I will further demonstrate the impact of the variability of dynamic weights on the three-way trade-off among optimization, generalization, and conflict avoidance that is unique in MOL.

Bio: Lisha Chen (https://lisha-chen.github.io/) is a Ph.D. candidate in the Department of Electrical, Computer, and Systems Engineering at Rensselaer Polytechnic Institute (RPI). She received her M.S. in Electrical Engineering from RPI in 2021, her B.S. from Huazhong University of Science and Technology in 2017. Her research focuses on theoretical foundations of multi-objective learning and meta learning, as well as their applications to computer vision and communication tasks. For those topics, she has co-authored a monograph in Foundations and Trends in Signal Processing and will serve as a tutorial speaker at AAAI 2024 and ICASSP 2024. 

Lisha Chen has received several awards, including the IEEE Signal Processing Society Scholarship and Rensselaer’s Founders Award of Excellence in 2023, Belsky Award for Computational Sciences and Engineering in 2021, the IBM-AIRC PhD Fellowship for four years to support her research since 2020. She is also the recipient of the National Scholarship from the Ministry of Education of China.