CIF: Small: ESTRELLA: Exploiting Structure in Tensors for Representation, Estimation, and Limits of Learning Algorithms

CIF：小：ESTRELLA：利用张量结构进行表示、估计和学习算法的限制

• 批准号: 1910110
• 负责人: Anand Sarwate
• 金额: $ 50万
• 依托单位: Rutgers University New Brunswick
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1910110&HistoricalAwards=false
• 关键词: CIF; Small; ESTRELLA; Exploiting; Structure

Tensors are multidimensional mathematical objects that generalize vectors (one dimensional) and matrices (two dimensional) to higher dimensions. Tensors, which can be written as multiway arrays, are ubiquitous in applications involving complex-structured data. The data themselves may be tensor-valued in some applications: for example, a grayscale video is a three-dimensional tensor with two spatial dimensions (horizontal and vertical) and one temporal dimension. Tensors can also be used in other applications to represent higher-order correlations between statistical variables; as an example, correlations among all triplets of variables correspond to a three-dimensional tensor. Although tensors have been used for decades in a variety of disciplines, statistical and signal processing methods using structured models for tensor data are less mature than their vector and matrix counterparts. This has immediate consequences for data science practitioners: they lack a theoretical framework for choosing a good model when working with tensor data. This project pursues a comprehensive theory for tensor data by focusing on a family of structured statistical models in which the number of parameters can be controlled in a principled manner. In particular, the project reaps the benefits of structured modeling of tensor data by quantifying the number of data samples needed to obtain a given parametrized structured tensor model and developing efficient algorithms for estimating the associated parameters. In the process, the project seeks also to simplify the measurement, storage, and statistical modeling of tensor-structured data. The outcomes of this project should impact many areas in which tensor data are being used, such as medical imaging, climate science, machine learning, computer vision, text and speech processing, and radar systems. Because of the wide-ranging uses of tensor data, this project also facilitates interactions between multiple research communities from statistics, engineering, and basic sciences.The project draws on the tight connection between tensor decompositions and structured matrix models in order to formulate the estimation of structured tensor models as nonconvex optimization problems over highly structured spaces of matrices. The work focuses on developing a fundamental understanding of structured models for tensor data along three research tracks: understanding the geometry of the resulting nonconvex problems, developing computationally efficient algorithms for solving the optimization problems, and quantifying the number of samples required to estimate the parameters within the structured model in a minimax sense. The first track develops a mathematical understanding of the nonconvex optimization problems that arise when using structured models for tensor data. These will inform the design strategies for effectively identifying a good structured model that fits the data. The second track entails the design of numerical methods and algorithms that implement these strategies to efficiently find models that best describe the tensor-valued data. The third and final track characterizes the fundamental limits of the proposed models and their relationship to the metrics of representation, reconstruction, and prediction errors.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

张量是将向量（一维）和矩阵（二维）推广到更高维度的多维数学对象。张量可以写成多路数组，在涉及复杂结构数据的应用中无处不在。在某些应用中，数据本身可能是张量值的：例如，灰度视频是具有两个空间维度（水平和垂直）和一个时间维度的三维张量。张量还可以在其他应用中用于表示统计变量之间的高阶相关性；例如，所有变量三元组之间的相关性对应于三维张量。尽管张量已在各个学科中使用了数十年，但使用张量数据结构化模型的统计和信号处理方法不如向量和矩阵对应方法成熟。这对数据科学从业者产生了直接影响：他们在处理张量数据时缺乏选择良好模型的理论框架。该项目通过关注一系列结构化统计模型来追求张量数据的综合理论，其中参数的数量可以以原则性的方式控制。特别是，该项目通过量化获得给定参数化结构化张量模型所需的数据样本数量并开发用于估计相关参数的有效算法，获得了张量数据结构化建模的好处。在此过程中，该项目还寻求简化张量结构数据的测量、存储和统计建模。该项目的成果应该会影响使用张量数据的许多领域，例如医学成像、气候科学、机器学习、计算机视觉、文本和语音处理以及雷达系统。由于张量数据的广泛用途，该项目还促进了统计、工程和基础科学等多个研究团体之间的互动。该项目利用张量分解和结构化矩阵模型之间的紧密联系，以制定对结构化张量模型作为高度结构化矩阵空间上的非凸优化问题。这项工作的重点是沿着三个研究方向发展对张量数据结构化模型的基本理解：理解由此产生的非凸问题的几何形状，开发计算有效的算法来解决优化问题，以及量化估计参数所需的样本数量。极小极大意义上的结构化模型。第一个轨道发展对使用张量数据的结构化模型时出现的非凸优化问题的数学理解。这些将为有效识别适合数据的良好结构化模型的设计策略提供信息。第二条路线需要设计数值方法和算法来实现这些策略，以有效地找到最能描述张量值数据的模型。第三个也是最后一个轨道描述了所提出模型的基本限制及其与表示、重建和预测误差指标的关系。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优点和更广泛的评估进行评估，被认为值得支持。影响审查标准。

项目成果

Learning Mixtures of Separable Dictionaries for Tensor Data: Analysis and Algorithms

张量数据的可分离字典的学习混合：分析和算法

• DOI: 10.1109/tsp.2019.2952046
• 发表时间: 2019-03-22
• 期刊: IEEE Transactions on Signal Processing
• 影响因子: 5.4
• 作者: Mohsen Ghassemi;Z. Shakeri;A. Sarwate;W. Bajwa
• 通讯作者: W. Bajwa

Structured Low-Rank Tensors for Generalized Linear Models

广义线性模型的结构化低阶张量

• DOI: 10.48550/arxiv.2308.02922
• 发表时间: 2023-08-05
• 期刊: ArXiv
• 作者: Batoul Taki;A. Sarwate;W. Bajwa
• 通讯作者: W. Bajwa

Learning Predictors from Multidimensional Data with Tensor Factorizations

使用张量分解从多维数据中学习预测器

• DOI: 10.14713/arestyrurj.v1i3.165
• 发表时间: 2021-10
• 期刊: Aresty Rutgers Undergraduate Research Journal
• 作者: Kwon, Soo Min;Sarwate, Anand D.
• 通讯作者: Sarwate, Anand D.

A Minimax Lower Bound for Low-Rank Matrix-Variate Logistic Regression

低秩矩阵变量逻辑回归的极小极大下界

• DOI: 10.1109/ieeeconf53345.2021.9723149
• 发表时间: 2021-05-31
• 期刊: 2021 55th Asilomar Conference on Signals, Systems, and Computers
• 作者: Batoul Taki;Mohsen Ghassemi;A. Sarwate;W. Bajwa
• 通讯作者: W. Bajwa

Tensor Regression Using Low-Rank and Sparse Tucker Decompositions

使用低秩和稀疏塔克分解的张量回归

• DOI: 10.1137/19m1299335
• 发表时间: 2020-01
• 期刊: SIAM Journal on Mathematics of Data Science
• 影响因子: 3.6
• 作者: Ahmed, Talal;Raja, Haroon;Bajwa, Waheed U.
• 通讯作者: Bajwa, Waheed U.