Task-Aware Quantization in Data Science: Theory and Fast Algorithms

数据科学中的任务感知量化：理论和快速算法

基本信息

批准号：
2012546
负责人：
Rayan Saab
金额：
$ 25万
依托单位：
University of California-San Diego
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2020
资助国家：
美国
起止时间：
2020-08-01 至 2024-07-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2012546&HistoricalAwards=false
关键词：
Task Aware Quantization Data Science

项目摘要

Machine learning algorithms are ubiquitous, and their applications in data science are on the rise. This project focuses on developing computationally efficient algorithms in data-science applications where discretization, also known as quantization, plays a fundamental role. Here quantization is the process that replaces real numbers, like those obtained from sensor measurements, by elements in a finite set. This makes them amenable to efficient digital representation, storage, compression, and transmission. Applications of interest include deep learning, an area that has led to sensational breakthroughs in a stunning range of areas. One of its frontiers is building neural networks on hardware that can be put into handheld and wearable devices as well as those in smart homes. For that, neural networks must be efficiently quantized; a key goal of this project is to devise algorithms for this task. Another application concerns edge devices, such as sensors in a sensor network, which communicate and perform computations under severe power limitations. A goal of this project is to develop computationally efficient algorithms for quantizing and compressing their data to enable reducing power use. A third application involves recommender systems, which collect users’ discretized ratings of products and transform them into other product recommendations for others. The project provides training for graduate students through involvement in the research.This project focuses on developing computationally efficient quantization algorithms with provable error guarantees. It is motivated by three important application areas. First, in settings where the goal is discretizing the parameters of a function, as in the compression of deep neural networks, it seeks quantization algorithms to generate functionally equivalent networks that require many fewer bits to store. The second motivating area involves settings where inference tasks must be done on edge-devices, under communication and computation constraints, as in sensor-networks. Here, the focus is on computationally efficient measurement, quantization, and inference algorithms that entail minimal memory and power requirements. Third, in applications where signal recovery is the goal and measurements are inherently binary and expensive to collect, as in recommender systems, the focus is on devising and studying efficient adaptive algorithms for sequential selection of the measurements. This project, which aims to develop state of the art task-aware algorithms, entails developing and using tools from several areas of mathematics, including methods from geometric functional analysis and non-asymptotic random matrix theory. Connections with frame theory, compressed sensing, and noise-shaping quantization will also be established. In analyzing the algorithms, discrete geometry, optimization, and numerical analysis techniques will be developed and employed. To compare theoretical guarantees associated with this project with best possible ones, approximation theory will be essential.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习算法无处不在，它们在数据科学中的应用正在上升。该项目着重于在数据科学应用程序中开发计算有效的算法，而离散化（也称为量化）起着基本作用。这里的量化是通过有限集中元素替代实数的过程，就像从传感器测量中获得的过程一样。这使它们适合有效的数字表示，存储，压缩和传输。感兴趣的应用包括深度学习，该领域导致了令人惊叹的领域的轰动性突破。它的前沿之一是在硬件上建立神经网络，该网络可以放入手持式设备以及智能家居中。为此，必须有效地量化神经网络。该项目的关键目标是为此任务设计算法。另一个应用程序涉及边缘设备，例如传感器网络中的传感器，它们在严重的功率限制下进行通信和执行计算。该项目的一个目标是开发计算有效的算法，以量化和压缩数据以减少功率使用。第三个应用程序涉及推荐系统，该系统收集了用户离散的产品评级，并将其转换为其他产品的其他产品建议。该项目通过参与研究为研究生提供培训。本项目着重于开发具有可证明错误保证的计算高效量化算法。它是由三个重要的应用领域激励的。首先，在目标的设置中，该目标是分散函数参数（如深度神经网络的压缩），它寻求量化算法来生成功能等效的网络，这些网络需要更少的位数才能存储。第二个动机区域涉及设置，其中必须像传感器网络中的通信和计算约束下在边缘设备上完成推理任务。在这里，重点是计算高效的测量，量化和推理算法，这些算法需要最少的内存和功率要求。第三，在信号恢复是目标和测量的应用中，固有的二元且收集昂贵，就像在建议系统中一样，重点是设计和研究有效的自适应算法，以进行测量的顺序选择。该项目旨在开发最先进的任务感知算法，需要从几个数学领域开发和使用工具，包括几何功能分析和非偶然的随机矩阵理论的方法。还将建立与框架理论，压缩感应和噪声量化量化的连接。在分析仪中，将开发和使用离散的几何形状，优化和数值分析技术。为了将与该项目相关的理论保证与最佳的理论保证进行比较，近似理论将是必不可少的。该奖项反映了NSF的法定使命，并通过使用基金会的知识分子优点和更广泛的影响审查标准通过评估来诚实地获得支持。