Autonomic Data Management for Very Large Dataset Visualization

适用于超大型数据集可视化的自主数据管理

基本信息

批准号：
EP/D059674/1
负责人：
Min Chen
金额：
$ 1.96万
依托单位：
Swansea University
依托单位国家：
英国
项目类别：
Research Grant
财政年份：
2006
资助国家：
英国
起止时间：
2006 至无数据
项目状态：
已结题

来源：
https://gtr.ukri.org/projects?ref=EP%2FD059674%2F1
关键词：
Autonomic Data Management Very Large

项目摘要

Never before in history have we had such capability for generating, collecting and storing digital data. Data repositories at terabyte level are becoming commonplace in many applications, including bioinformatics, medicine, remote sensing and nano-technology. In some applications, such as network traffic visualization and video visualization, we are encountering the scenario that dynamic data streams are almost temporally unbounded. Many visualization tasks are evolving into visual data mining processes.The main scientific challenge to be tackled in this proposed collaborative project is how to manage visualization tasks that involve very large datasets which must reside on one or a few remote data servers for various reasons, including excessive space requirements, data sharing provision, dynamic data generation, and data security. Because of the rapid developments of data capture and data generation technologies (e.g., web-based data collection and Grid computing), the management of such visualization over a complex computing and communication infrastructure is becoming a bottleneck to interactive visual data mining. It is not possible to device a uniform approach to suit various requirements nor cost-effective to place the burden of complex data management upon users.One emerging strategy for developing complex computing infrastructure is autonomic computing, which seeks inspiration in self-adaptive biological systems and self-governing social and economic systems. In a recent survey coordinated by the PI, the concept of autonomic computing was first brought into the context of visualization, and it was suggested that the concept can be deployed in a visualization infrastructure in an evolutionary manner. It is desirable to investigate into an autonomic approach to the management of very large datasets in visualization, to design and prototype knowledge-based framework for capturing dynamic system information and users' experiences, and to develop adaptive algorithms that can utilize such information dynamically for data management in visualization.The aim of this research is to develop an autonomic approach to the management of very large dataset visualization. The joint research programme will be carried out primarily by the PI, Professor Min Chen, and his PhD student, Mr. David Chisnall, in collaboration with Professor Charles Hansen and his research team in University of Utah.The main resources for conducting this joint programme is in place with a current PhD studentship held by Mr. Chisnall (January 2004 - December 2007). Some work on O1, O2, and O3 has been carried out. We therefore do not seek any funding for staffing and equipment costs. The funding sought for this project is for covering travelling and subsistence cost, as face-to-face meetings and short-term visits are indispensable in such close collaboration involving intensive algorithm design and software development, as well as direct access to computing infrastructure, datasets, and scientific users in Utah.

在历史上，我们从未有过生成，收集和存储数字数据的能力。 Terabyte级别的数据存储库在许多应用中都变得司空见惯，包括生物信息学，医学，遥感和纳米技术。在某些应用程序中，例如网络流量可视化和视频可视化，我们遇到了动态数据流几乎是时间无限的情况。许多可视化任务正在发展为视觉数据挖掘过程。在此拟议的协作项目中要解决的主要科学挑战是如何管理涉及非常大的数据集的可视化任务，这些数据集必须在一个或几个远程数据服务器上，出于各种原因，包括过多的空间需求，数据共享提供，数据共享，动态数据和数据安全。由于数据捕获和数据生成技术的快速发展（例如，基于Web的数据收集和电网计算），因此在复杂的计算和通信基础架构上对这种可视化的管理正成为交互式视觉数据挖掘的瓶颈。不可能为适合各种要求或具有成本效益的统一方法设备，以将复杂的数据管理负担放在用户身上。开发复杂的计算基础架构的一种新兴策略是自主计算，它在自治生物系统和自治社会和经济系统中寻求灵感。在最新的PI协调的调查中，自主计算的概念首先引入了可视化的背景下，并建议可以以进化方式将该概念以可视化基础结构部署。需要研究一种在可视化中管理非常大的数据集的自主神经方法，以设计和原型的基于知识的框架来捕获动态系统信息和用户的经验，并可以在可视化中开发动态的数据管理这些信息，以开发该研究的数据管理。联合研究计划将主要由PI，Min Chen教授和他的博士生David Chisnall先生与Charles Hansen教授及其在犹他州大学的研究团队合作。进行此联合计划的主要资源与Chisnall先生（2004年1月至2004年12月至12月至12月）。在O1，O2和O3上进行了一些工作。因此，我们不会为人员配备和设备成本寻求任何资金。该项目寻求的资金是用于支付旅行和生存成本，因为在这种紧密的协作中，面对面的会议和短期访问是必不可少的，涉及密集算法设计和软件开发以及直接访问犹他州的计算基础架构，数据集和科学用户。