CIF: Small: Collaborative Research:Algorithms and Information-Theoretic Limits for Data-Limited Inference

CIF：小型：协作研究：数据有限推理的算法和信息论限制

• 批准号: 1117128
• 负责人: Aaron Wagner
• 金额: $ 24万
• 依托单位: Cornell University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-08-01 至 2014-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1117128&HistoricalAwards=false
• 关键词: CIF; Small; Collaborative; Research; Algorithms

It is an irony of our time that despite living in the 'information age' we are often data-limited. After decades of research, scientists still debate the causes and effects of climate change, and recent work has shown that a significant fraction of the most influential medical studies over the past 13 years have been subsequently found to be inaccurate, largely due to insufficient data. One reason for this apparent paradox is that modeling complex, real-world information sources requires rich probabilistic models that cannot be accurately learned even from very large data sets. On a deeper level, research inherently resides at the edge of the possible, and seeks to address questions that available data can only partially answer. It is therefore reasonable to expect that we will always be data-limited.This research involves developing new algorithms and performance bounds for data-limited inference. Prior work of the PIs has shown that, by taking an information-theoretic approach, one can develop new algorithms that are tailored specifically to the data-limited regime and perform better than was previously known, and in some cases are provably optimal. This project advances the goal of developing a general theory for data-limited inference by considering a suite of problems spanning multiple application areas, such as classification; determining whether two data sets were generated by the same distribution or by different distributions; distribution estimation from event timings; entropy estimation; and communication over complex and unknown channels. Whereas these problems have all been studied before in isolation, prior work of the PIs has shown it is fruitful to view them as instances of the same underlying problem: data-limited inference.

尽管生活在“信息时代”，我们的数据却常常受到限制，这对我们这个时代来说是一个讽刺。经过数十年的研究，科学家们仍在争论气候变化的原因和影响，最近的研究表明，过去 13 年中最具影响力的医学研究中有很大一部分后来被发现不准确，主要原因是数据不足。造成这种明显悖论的原因之一是，对复杂的现实世界信息源进行建模需要丰富的概率模型，即使从非常大的数据集中也无法准确学习这些模型。在更深层次上，研究本质上处于可能性的边缘，并寻求解决现有数据只能部分回答的问题。因此，我们有理由预期我们将始终受到数据限制。这项研究涉及开发新的算法和数据限制推理的性能界限。 PI 之前的工作表明，通过采用信息论方法，人们可以开发专门针对数据有限的情况定制的新算法，其性能比以前已知的更好，并且在某些情况下可证明是最佳的。该项目通过考虑跨越多个应用领域的一系列问题（例如分类），推进了开发数据有限推理的通用理论的目标。确定两个数据集是由相同分布还是不同分布生成的；根据事件时间进行分布估计；熵估计；以及通过复杂且未知的渠道进行通信。尽管这些问题之前都被单独研究过，但 PI 之前的工作表明，将它们视为同一根本问题（数据有限推理）的实例是富有成效的。