Collaborative Online Optimization for Efficient Model-Based Learning

基于模型的高效学习的协作在线优化

基本信息

批准号：
1933878
负责人：
Shahin Shahrampour
金额：
$ 50万
依托单位：
Texas A&M Engineering Experiment Station
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-09-15 至 2021-07-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1933878&HistoricalAwards=false
关键词：
Collaborative Online Optimization Efficient Model

项目摘要

One of the grand challenges in Artificial Intelligence (AI) and Machine Learning (ML) is building intelligent systems that can learn from data in real time. To learn from streaming data, there is need for novel approaches in online optimization and prediction. Current methods assume sequential availability of gradients (or loss), posing a practical hurdle in implementation. We propose two approaches to address this gap using model-based learning. These approaches are aimed at respectively exploiting, a distributed computing architecture (to divide the required computational effort) or a communications network (to efficiently aggregate disparate data). The collaborative online optimization algorithms and theoretic extensions introduced in this work have a broad range of applications domains such as speech recognition and computer vision, autonomous vehicles, transportation, neuroscience, and business analytics.Most of classical ML algorithms have been developed under the assumption that data sets are already available in batch form. Transitioning from offline to online learning faces a major practical hurdle in many application domains where the closed-form of the objective function is unknown to the learner. When dealing with streaming data, this black-box property leads to a natural trade-off between delays (due to data or computation) and the speed and accuracy with which a model can be identified. A distributed computing architecture provides a way to reduce delays to obtain reasonably accurate models in the necessary timescale. We propose to study fast distributed asynchronous stochastic gradient approaches for online learning in which coordination between multiple workers (processors) interacting asynchronously is carefully engineered. Improved accuracy and speed may also be jointly achieved by a network of learners receiving different streams of data. Thus, we also consider decentralized models of online learning with multiple learning agents that communicate over a network. With the ability to share predictions or estimates with other agents in a network, the collective can aggregate disparate information in a way to outperform (in terms of accuracy and speed) any individually identified model. Finally, we consider the case in which data streams have graph structure. Streaming graph structure data arises in diverse application domains such as transportation networks, social networks and other networks found in biology, where the graph captures the correlation in data. The proposal includes the development of a new graduate course aimed at providing engineering students with working knowledge on state-of-the-art distributed online optimization techniques.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人工智能（AI）和机器学习（ML）的巨大挑战之一是构建可以实时从数据中学习的智能系统。要从流数据中学习，在线优化和预测中需要新颖的方法。当前方法假设梯度的顺序可用性（或损失），在实施方面构成了实用的障碍。我们建议使用基于模型的学习来解决这一差距的两种方法。这些方法旨在分别利用分布式计算体系结构（分配所需的计算工作）或通信网络（有效地汇总了不同的数据）。这项工作中介绍的协作在线优化算法和理论扩展具有广泛的应用领域，例如语音识别和计算机视觉，自动驾驶汽车，运输，神经科学和业务分析。数据集已经以批处理形式提供。从离线学习过渡到在线学习是许多应用领域的主要实用障碍，而学习者的封闭形式是学习者未知的。在处理流数据时，这种黑盒属性会导致延迟（由于数据或计算）与识别模型的速度和准确性之间的自然权衡。分布式计算体系结构提供了减少延迟以在必要时间范围内获得合理准确模型的方法。我们建议研究用于在线学习的快速分布异步随机梯度方法，在线学习中，对多个工人（处理器）之间的协调性进行了仔细的设计。通过接收不同数据流的学习者网络，也可以共同实现准确性和速度的提高。因此，我们还考虑了通过网络通信的多个学习代理的在线学习模型。凭借能够与网络中其他代理共享预测或估计，集体可以以跑赢大盘（在准确性和速度方面）的方式汇总不同的信息。最后，我们考虑数据流具有图结构的情况。流图结构数据出现在不同的应用领域，例如运输网络，社交网络和生物学中发现的其他网络，该图捕获了数据中的相关性。该提案包括开发新的研究生课程，旨在为工程专业的学生提供有关最先进的在线优化技术的工作知识。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子来评估的。优点和更广泛的影响审查标准。