CAREER: Foundations of Next-Generation Neural Architecture Search

职业：下一代神经架构搜索的基础

• 批准号: 2046613
• 负责人: Ameet Talwalkar
• 金额: $ 55万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-04-01 至 2026-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2046613&HistoricalAwards=false
• 关键词: CAREER; Foundations; Next; Generation; Neural

Deep learning has led to remarkable artificial intelligence breakthroughs on many important problems such as object recognition for autonomous vehicles, voice-activated assistants, and automated machine translation. At the heart of these breakthroughs is the design of complex, domain-specific deep neural network architectures. However, only a small set of highly-trained researchers are equipped with the resources and expertise to undertake this arduous, ad-hoc design process. Moreover, design efforts have been largely limited to applications in a handful of domains, most notably computer vision and natural language processing. While the burgeoning field of neural architecture search (NAS) aims to automate the design of neural network architectures, existing work on NAS has to date narrowly focused on these same well-studied domains. This project aims to develop, analyze, and implement novel methods that enable automated architecture design beyond these restricted domains. The project involves collaborations with practitioners in new domains to empower them to develop new architectures for their applications. The project will also include extensive educational efforts to create a new course that covers the complete lifecycle of machine learning workflows, including extensive treatment on the automated design and tuning of neural networks. The course material will be freely distributed to facilitate worldwide adoption and adapted to create a short course for high school students.The focus of this project is to develop principled, efficient, and automated Neural Architecture Search (NAS) capabilities to enable practitioners to seamlessly create novel architectures for new problems. To achieve this goal, the project proposes a fundamentally new NAS paradigm driven by the co-design of the two core components of NAS, namely architecture search spaces and methods to search through these spaces. The researchers will demonstrate the effectiveness of their proposed techniques across numerous domains, including those where expert-designed architectures do not exist. The technical problems being tackled blend ideas from optimization, learning theory, signal processing, and machine learning systems, and draw connections to the problems of compressed sensing, weak supervision, and meta-learning. The proposed NAS work will be transformational in unlocking the potential of novel deep learning applications in new domains, and will provide training opportunities for graduate students. Moreover, the project's proposed activities emphasize accessibility and broad dissemination via: foundational educational material disseminated to data scientists worldwide; growing and promoting diversity in the Machine Learning Systems research community; and widespread industry adoption via open-source activities, contributions to Carnegie Mellon University's Machine Learning blog, and a recurring podcast.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度学习在许多重要问题上带来了显着的人工智能突破，例如自动驾驶汽车的物体识别、语音激活助手和自动机器翻译。这些突破的核心是复杂的、特定领域的深度神经网络架构的设计。然而，只有一小部分训练有素的研究人员具备资源和专业知识来承担这一艰巨的临时设计过程。 此外，设计工作很大程度上局限于少数领域的应用，尤其是计算机视觉和自然语言处理。虽然新兴的神经架构搜索 (NAS) 领域旨在自动化神经网络架构的设计，但迄今为止 NAS 的现有工作仍主要集中在这些经过充分研究的领域。该项目旨在开发、分析和实施新颖的方法，使自动化架构设计能够超越这些限制领域。该项目涉及与新领域的从业者合作，使他们能够为其应用程序开发新的架构。该项目还将包括广泛的教育工作，以创建涵盖机器学习工作流程的完整生命周期的新课程，包括对神经网络的自动化设计和调整的广泛处理。该课程材料将免费分发，以促进全球采用，并改编为高中生短期课程。该项目的重点是开发有原则的、高效的、自动化的神经架构搜索（NAS）功能，使从业者能够无缝地创建针对新问题的新颖架构。为了实现这一目标，该项目提出了一种全新的 NAS 范式，由 NAS 的两个核心组件（即架构搜索空间和搜索这些空间的方法）的共同设计驱动。研究人员将展示他们提出的技术在多个领域的有效性，包括那些不存在专家设计的架构的领域。所解决的技术问题融合了优化、学习理论、信号处理和机器学习系统的思想，并与压缩感知、弱监督和元学习问题建立了联系。拟议的 NAS 工作将在释放新领域中新型深度学习应用的潜力方面具有变革性，并将为研究生提供培训机会。 此外，该项目拟议的活动强调可访问性和广泛传播，通过： 向全世界的数据科学家传播基础教育材料；发展和促进机器学习系统研究社区的多样性；通过开源活动、对卡内基梅隆大学机器学习博客的贡献以及定期播客的广泛行业采用。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。