III: Small: Collaborative Research: Cost-Efficient Sampling and Estimation from Large-Scale Networks

III：小型：协作研究：大规模网络的经济高效采样和估计

• 批准号: 1910749
• 负责人: Do Young Eun
• 金额: $ 25万
• 依托单位: North Carolina State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1910749&HistoricalAwards=false
• 关键词: III; Small; Collaborative; Research; Cost

Sampling and estimating structural information from large-scale networks or graphs has been central to our understanding of the network dynamics and its rich set of applications. Markov Chain Monte Carlo (MCMC) has been the key enabler for a broader context of graph sampling, including estimating the properties of large graphs, sampling the corpus of documents indexed by search engines, sampling records from hidden databases behind Web forms, identifying subgraphs of certain characteristics and frequent graph pattern matching. Despite versatile applications of the MCMC methods and their customized algorithms for analyzing graph-structured data in various forms, there still exist critical challenges and limitations in the literature centered around the MCMC methods. One is the 'cost' consumption/constraints associated with the sampling operation, which limits the size of total samples obtained and negatively affects the accuracy of any estimator based on the obtained samples. Another limitation is that the recent advances in MCMC, especially built up on favorable non-reversible Markov chains, cannot be leveraged to the various large-graph sampling tasks, due to their required global knowledge of the underlying state space, lack of distribution implementation, unconstrained state space, as well as the simplified cost assumption. The goal of this research is to fully exploit the potentials of a set of crawling samplers by making the samplers adaptive and possibly interactive on a properly constructed graph domain, to transcend the current status-quo in the wide range of graph sampling tasks. Specifically, the project aims to: (i) build a theoretical framework to construct a suite of cost-efficient sampling policies by optimally balancing the tradeoff between the sample quality and quantity under challenged access environments with a given cost budget, (ii) design a class of adaptive random walks by fully exploiting the past information to achieve minimal temporal correlations over the obtained samples and by controlling the random walks collectively to enable maximal space exploration, and (iii) extend the standard MCMC toolkits toward faster and more cost-efficient exploration of feasible subgraphs/configurations and computing/optimization on a graph, along with extensive validations to create practical and usable solutions in reality. This research has a high potential impact on a vast range of multi-disciplinary applications, including sampling large-scale graphs for statistical inference and efficient estimation and randomized algorithms for combinatorial optimizations in various disciplines, where the standard MCMC methods have been dominant but also constrained our understanding.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

来自大规模网络或图形的采样和估算结构信息对于我们对网络动态及其丰富应用集集的理解至关重要。马尔可夫链蒙特卡洛（MCMC）一直是更广泛的图形采样背景的关键推动者，包括估计大图的属性，对搜索引擎索引的文档语料库进行采样，从网络表单背后的隐藏数据库中采样记录某些特征和频繁的图形模式匹配。尽管MCMC方法的多功能应用及其定制算法用于分析各种形式的图形结构数据，但仍存在围绕MCMC方法的文献中的关键挑战和局限性。一个是与采样操作相关的“成本”消耗/约束，该操作限制了获得的总样品的大小，并基于获得的样品对任何估计器的准确性产生负面影响。另一个限制是，MCMC的最新进展，尤其是建立在有利的非可逆马尔可夫链上的基础上，由于他们对基本国家空间所需的全球知识，缺乏分销的实施，缺乏分布的实施，缺乏分布的实施，无法利用各种大型绘制采样任务。不受约束的状态空间以及简化的成本假设。这项研究的目的是通过使采样器自适应和可能在正确构造的图形域上进行自适应和可能交互式来充分利用一组爬行采样器的电势，以在广泛的图形采样任务中超越当前状态。具体而言，该项目的目的是：（i）建立一个理论框架，通过在挑战的访问环境中最佳平衡样本质量和数量之间的折衷与给定的成本预算，（ii）设计A通过充分利用过去的信息来实现所获得的样本的最小时间相关性，并通过集体控制随机步行以实现最大的空间探索，而（iii）将标准的MCMC工具包扩展到更快，更具成本效率的探索，将自适应随机步行的类别类别用于实现最小的时间相关性。图表上可行的子图/配置和计算/优化的涵盖，以及广泛的验证，以创建现实中的实用和可用解决方案。这项研究对广泛的多学科应用具有很大的潜在影响，包括对统计推断和有效估计的大规模图进行抽样，以及在各种学科中进行组合优化的随机算法，其中标准MCMC方法占主导我们的理解。该奖项反映了NSF的法定任务，并通过使用基金会的知识分子优点和更广泛的影响审查标准来评估值得支持。

项目成果

Fiedler Vector Approximation via Interacting RandomWalks

通过交互随机游走进行费德勒矢量逼近

• DOI: 10.1145/3410048.3410107
• 发表时间: 2020
• 期刊: ACM SIGMETRICS Performance Evaluation Review
• 作者: Doshi, Vishwaraj;Young Eun, Do
• 通讯作者: Young Eun, Do

Trapping Malicious Crawlers in Social Networks

• DOI: 10.1145/3340531.3412004
• 发表时间: 2020-10
• 期刊: Proceedings of the 29th ACM International Conference on Information & Knowledge Management
• 作者: Shiju Li;Chul-Ho Lee;Do Young Eun

Minimizing File Transfer Time in Opportunistic Spectrum Access Model

• DOI: 10.1109/tmc.2022.3212926
• 发表时间: 2022-10
• 期刊: IEEE Transactions on Mobile Computing
• 影响因子: 7.9
• 作者: Jie Hu;Vishwaraj Doshi;Do Young Eun

Opportunistic Spectrum Access: Does Maximizing Throughput Minimize File Transfer Time?

• DOI: 10.23919/wiopt52861.2021.9589243
• 发表时间: 2021-09
• 期刊: 2021 19th International Symposium on Modeling and Optimization in Mobile, Ad hoc, and Wireless Networks (WiOpt)
• 作者: Jie Hu;Vishwaraj Doshi;Do Young Eun

Efficiency Ordering of Stochastic Gradient Descent

• DOI: 10.48550/arxiv.2209.07446
• 发表时间: 2022-09
• 期刊: ArXiv
• 作者: Jie Hu;Vishwaraj Doshi;Do Young Eun