Task-Aware Quantization in Data Science: Theory and Fast Algorithms

数据科学中的任务感知量化：理论和快速算法

基本信息

批准号：
2012546
负责人：
Rayan Saab
金额：
$ 25万
依托单位：
University of California-San Diego
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2020
资助国家：
美国
起止时间：
2020-08-01 至 2024-07-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2012546&HistoricalAwards=false
关键词：
Task Aware Quantization Data Science

项目摘要

Machine learning algorithms are ubiquitous, and their applications in data science are on the rise. This project focuses on developing computationally efficient algorithms in data-science applications where discretization, also known as quantization, plays a fundamental role. Here quantization is the process that replaces real numbers, like those obtained from sensor measurements, by elements in a finite set. This makes them amenable to efficient digital representation, storage, compression, and transmission. Applications of interest include deep learning, an area that has led to sensational breakthroughs in a stunning range of areas. One of its frontiers is building neural networks on hardware that can be put into handheld and wearable devices as well as those in smart homes. For that, neural networks must be efficiently quantized; a key goal of this project is to devise algorithms for this task. Another application concerns edge devices, such as sensors in a sensor network, which communicate and perform computations under severe power limitations. A goal of this project is to develop computationally efficient algorithms for quantizing and compressing their data to enable reducing power use. A third application involves recommender systems, which collect users’ discretized ratings of products and transform them into other product recommendations for others. The project provides training for graduate students through involvement in the research.This project focuses on developing computationally efficient quantization algorithms with provable error guarantees. It is motivated by three important application areas. First, in settings where the goal is discretizing the parameters of a function, as in the compression of deep neural networks, it seeks quantization algorithms to generate functionally equivalent networks that require many fewer bits to store. The second motivating area involves settings where inference tasks must be done on edge-devices, under communication and computation constraints, as in sensor-networks. Here, the focus is on computationally efficient measurement, quantization, and inference algorithms that entail minimal memory and power requirements. Third, in applications where signal recovery is the goal and measurements are inherently binary and expensive to collect, as in recommender systems, the focus is on devising and studying efficient adaptive algorithms for sequential selection of the measurements. This project, which aims to develop state of the art task-aware algorithms, entails developing and using tools from several areas of mathematics, including methods from geometric functional analysis and non-asymptotic random matrix theory. Connections with frame theory, compressed sensing, and noise-shaping quantization will also be established. In analyzing the algorithms, discrete geometry, optimization, and numerical analysis techniques will be developed and employed. To compare theoretical guarantees associated with this project with best possible ones, approximation theory will be essential.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习算法无处不在，其在数据科学中的应用正在不断增加，该项目专注于开发数据科学应用中的计算算法，其中离散化（也称为量化）发挥着基础作用，而高效量化是取代这一过程的过程。实数，如通过有限集合中的元素通过传感器测量获得的数据，这使得它们能够进行有效的数字表示、存储、压缩和传输，令人感兴趣的应用包括深度学习，这一领域已经取得了重大突破。其前沿领域之一是在可应用于手持设备和可穿戴设备以及智能家居的硬件上构建神经网络，因此必须对神经网络进行有效量化；另一个应用程序涉及边缘设备，例如传感器网络中的传感器，它们在严格的功率限制下进行通信和执行计算，该项目的目标是开发计算高效的算法来量化和压缩其数据。启用降低功率第三个应用涉及推荐系统，它收集用户对产品的离散评级，并将其转化为其他产品的推荐。该项目通过参与研究为研究生提供培训。该项目专注于开发具有计算效率的量化算法。它的动机是三个重要的应用领域，其目标是离散化函数的参数，例如在深度神经网络的压缩中，它寻求量化算法来生成需要更少的功能等效的网络。位第二个激励领域涉及推理任务必须在通信和计算限制下在边缘设备上完成的设置，就像在传感器网络中一样，这里的重点是需要最少的计算效率的测量、量化和推理算法。第三，在以信号恢复为目标且测量本质上是二进制且收集成本昂贵的应用中，例如在推荐系统中，重点是设计和研究用于测量的顺序选择的高效自适应算法。其目的是发展国家任务感知算法的艺术，需要开发和使用多个数学领域的工具，包括几何泛函分析和非渐近随机矩阵理论的方法，还将建立与框架理论、压缩感知和噪声整形量化的联系。在分析算法时，将开发和采用离散几何、优化和数值分析技术，为了将与该项目相关的理论保证与最佳可能的理论保证进行比较，逼近理论将是必不可少的。该奖项反映了 NSF 的法定使命，并被视为。值得通过使用基金会的智力优点和更广泛的影响审查标准进行评估来支持。