Collaborative Research: RI: Small: Robust Deep Learning with Big Imbalanced Data

合作研究：RI：小型：具有大不平衡数据的鲁棒深度学习

• 批准号: 2110546
• 负责人: Penghang Yin
• 金额: $ 23.35万
• 依托单位: SUNY at Albany
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2110546&HistoricalAwards=false
• 关键词: Collaborative; Research; RI; Small; Robust

This project promotes the progress of science and technology development by advancing artificial intelligence (AI) through innovations in scalable and robust computational methods. AI, especially deep learning, has brought transformative impact in industries and quantum leaps in the quality of a wide range of everyday technologies including face recognition, speech recognition and machine translation. However, in order to accelerate the democratization of AI there are still many challenges to be addressed including data issues and model issues. This project seeks to advance AI by addressing one critical issue related to data; i.e., data imbalance. This happens when the collected data for training AI models does not have enough instances representing some property the models are trying to learn. For example, molecules with a certain antibacterial property would be far fewer than all possible molecules making predictions of antibacterial properties challenging. The goal of this project is to develop algorithms with theoretical guarantees to make AI learn more effectively from the big imbalanced data. This project will also contribute to training future professionals in AI and machine learning, including training high school students and under-represented undergraduates. This project investigates a broad family of robust losses for deep learning. The research activities include (i) developing scalable offline stochastic algorithms for solving non-decomposable robust losses that are formulated into min-max, min-min formulations; (ii) developing efficient online stochastic algorithms for solving a family of distributionally robust optimization problems that are cast into compositional optimization problems; (iii) developing effective strategies for training deep neural networks by solving the considered non-decomposable robust losses; (iv) establishing the underlying theory including optimization and statistical convergence of the proposed algorithms. The algorithms are being evaluated on big imbalanced data such as images, graphs, texts.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目通过可扩展和强大的计算方法的创新来推进人工智能（AI），从而促进科学技术的进步。人工智能，尤其是深度学习，给各行各业带来了变革性影响，并为人脸识别、语音识别和机器翻译等各种日常技术的质量带来了质的飞跃。然而，为了加速人工智能的民主化，仍然有许多挑战需要解决，包括数据问题和模型问题。该项目旨在通过解决与数据相关的一个关键问题来推进人工智能的发展；即数据不平衡。当收集的用于训练 AI 模型的数据没有足够的实例来表示模型试图学习的某些属性时，就会发生这种情况。例如，具有某种抗菌特性的分子将远远少于所有可能的分子，这使得抗菌特性的预测具有挑战性。该项目的目标是开发有理论保证的算法，使人工智能更有效地从不平衡的大数据中学习。该项目还将有助于培训未来的人工智能和机器学习专业人士，包括培训高中生和代表性不足的本科生。该项目研究了深度学习的一系列稳健损失。研究活动包括（i）开发可扩展的离线随机算法，用于解决不可分解的鲁棒损失，这些损失被制定为最小-最大、最小-最小公式； (ii) 开发高效的在线随机算法来解决一系列分布稳健的优化问题，这些问题被转化为组合优化问题； (iii) 通过解决所考虑的不可分解鲁棒损失来制定训练深度神经网络的有效策略； (iv) 建立基础理论，包括所提出算法的优化和统计收敛。这些算法正在根据图像、图表、文本等大的不平衡数据进行评估。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Label Distributionally Robust Losses for Multi-class Classification: Consistency, Robustness and Adaptivity

为多类分类标记分布稳健损失：一致性、稳健性和适应性

• 发表时间: 2023-06
• 期刊: Proceedings of the 40th International Conference on Machine Learning
• 作者: Zhu, Dioxin;Ying, Yiming;Yang, Tianbao
• 通讯作者: Yang, Tianbao

AUC maximization in the era of big data and AI: A survey

大数据和人工智能时代 AUC 最大化：一项调查

• 发表时间: 2022-01
• 期刊: ACM computing surveys
• 影响因子: 16.6
• 作者: Yang, Tianbao;Ying, Yiming
• 通讯作者: Ying, Yiming

Minimax auc fairness: Efficient algorithm with provable convergence

Minimax auc 公平性：具有可证明收敛性的高效算法

• 发表时间: 2023-06
• 期刊: Proceedings of the AAAI Conference on Artificial Intelligence
• 作者: Yang, Zhenhuan;Ko, Yan Lok;Varshney, Kush;Ying, Yiming
• 通讯作者: Ying, Yiming

Stability and generalization for markov chain stochastic gradient methods

马尔可夫链随机梯度方法的稳定性和泛化

• 发表时间: 2022-12
• 期刊: Advances in neural information processing systems
• 作者: Wang, Puyu;Lei, Yunwen;Ying, Yiming;Zhou, Ding
• 通讯作者: Zhou, Ding

Generalization guarantee of SGD for pairwise learning

SGD 成对学习的泛化保证

• 发表时间: 2021-12
• 期刊: Advances in neural information processing systems
• 作者: Lei, Yunwen;Liu, Mingrui;Ying, Yiming
• 通讯作者: Ying, Yiming