Mitigating Risks Associated with Big Data Solutions

降低与大数据解决方案相关的风险

• 批准号: RGPIN-2015-06075
• 负责人: Miranskyy, Andriy
• 金额: $ 1.31万
• 依托单位: Ryerson University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=708190
• 关键词: Mitigating; Risks; Associated; Big; Data

Today's ubiquitous communication and computing technologies generate magnitudes of data far beyond that available even a decade ago. The Big Data technology and services market is expected to grow at a compound annual rate of 27% from $9.8 billion in 2012 to $32.4 billion in 2017, leading to multi-fold increase in the amount of technical data generated. Modern organisations have unprecedented opportunities to gain new knowledge about their products, processes and services, which can be used to make projects more efficient, improve products and services, identify problems, take pre-emptive measures, and adapt business goals to new opportunities. Applications of new knowledge include predictive models which reduce, for example, hospital readmission rates, traffic congestion, and unnecessary power generation. Exploiting these opportunities requires addressing challenges posed by solutions for processing Big Data (BDS). These complex solutions have many dynamic components, such as distributed compute nodes, networks, databases, middleware, and business intelligence layers. Any component can fail its interactions with others, possibly leading to crashing failure of the solution or quality degradation (e.g., performance, reliability, security). Therefore, they are challenging to develop and maintain. Big Data environments lack methods, tools, processes and techniques to support the disciplined development and maintenance of BDS. Despite a significant body of knowledge on processing data generated by BDS, most techniques cannot process the volumes of operational data generated by BDS. My goal is to help improve testing and maintenance of the BDS by reaching the following objectives: 1) Build defect prediction models for the BDS to improve its General Testing and Maintenance process; 2) Develop trace analysis for the BDS to speed up root cause determination and the removal of redundant test cases. To reach the objectives, I will create novel scalable methods, techniques, and tools capable of processing the operational data. To the best of my knowledge, these objectives are novel and significant for practice. The anticipated results will help create a preliminary theory of automated problem determination techniques for BDS. Transfer of the results to Canadian industry will improve product quality and speed up defect detection and fixing, allowing developers to create more new functionality, and reduce maintenance effort and investment.

当今无处不在的通信和计算技术产生的数据量远远超过十年前。大数据技术和服务市场预计将以27%的复合年增长率增长，从2012年的98亿美元增长到2017年的324亿美元，导致生成的技术数据量成倍增长。现代组织拥有前所未有的机会来获得有关其产品、流程和服务的新知识，这些知识可用于提高项目效率、改进产品和服务、发现问题、采取先发制人的措施并使业务目标适应新的机遇。新知识的应用包括预测模型，例如可以减少医院再入院率、交通拥堵和不必要的发电等。 利用这些机会需要解决大数据 (BDS) 处理解决方案带来的挑战。这些复杂的解决方案具有许多动态组件，例如分布式计算节点、网络、数据库、中间件和商业智能层。任何组件都可能无法与其他组件交互，从而可能导致解决方案崩溃或质量下降（例如性能、可靠性、安全性）。因此，它们的开发和维护具有挑战性。 大数据环境缺乏支持北斗系统规范开发和维护的方法、工具、流程和技术。尽管在处理北斗系统生成的数据方面有大量的知识，但大多数技术无法处理北斗系统生成的大量操作数据。我的目标是通过实现以下目标来帮助改进北斗系统的测试和维护： 1）建立北斗系统的缺陷预测模型，以改进其总体测试和维护流程； 2) 为BDS开发跟踪分析，以加快根本原因确定和删除冗余测试用例。为了实现这些目标，我将创建能够处理操作数据的新颖的可扩展方法、技术和工具。 据我所知，这些目标是新颖的并且对于实践具有重要意义。预期结果将有助于创建北斗自动问题确定技术的初步理论。将结果转移到加拿大工业界将提高产品质量并加快缺陷检测和修复速度，使开发人员能够创建更多新功能，并减少维护工作和投资。