Mitigating Risks Associated with Big Data Solutions

降低与大数据解决方案相关的风险

• 批准号: RGPIN-2015-06075
• 负责人: Miranskyy, Andriy
• 金额: $ 1.31万
• 依托单位: Ryerson University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=676904
• 关键词: Mitigating; Risks; Associated; Big; Data

Today's ubiquitous communication and computing technologies generate magnitudes of data far beyond that available even a decade ago. The Big Data technology and services market is expected to grow at a compound annual rate of 27% from $9.8 billion in 2012 to $32.4 billion in 2017, leading to multi-fold increase in the amount of technical data generated. Modern organisations have unprecedented opportunities to gain new knowledge about their products, processes and services, which can be used to make projects more efficient, improve products and services, identify problems, take pre-emptive measures, and adapt business goals to new opportunities. Applications of new knowledge include predictive models which reduce, for example, hospital readmission rates, traffic congestion, and unnecessary power generation.*** Exploiting these opportunities requires addressing challenges posed by solutions for processing Big Data (BDS). These complex solutions have many dynamic components, such as distributed compute nodes, networks, databases, middleware, and business intelligence layers. Any component can fail its interactions with others, possibly leading to crashing failure of the solution or quality degradation (e.g., performance, reliability, security). Therefore, they are challenging to develop and maintain.*** Big Data environments lack methods, tools, processes and techniques to support the disciplined development and maintenance of BDS. Despite a significant body of knowledge on processing data generated by BDS, most techniques cannot process the volumes of operational data generated by BDS. My goal is to help improve testing and maintenance of the BDS by reaching the following objectives: 1) Build defect prediction models for the BDS to improve its General Testing and Maintenance process; 2) Develop trace analysis for the BDS to speed up root cause determination and the removal of redundant test cases. To reach the objectives, I will create novel scalable methods, techniques, and tools capable of processing the operational data.*** To the best of my knowledge, these objectives are novel and significant for practice. The anticipated results will help create a preliminary theory of automated problem determination techniques for BDS. Transfer of the results to Canadian industry will improve product quality and speed up defect detection and fixing, allowing developers to create more new functionality, and reduce maintenance effort and investment.********

当今无处不在的通信和计算技术产生的数据幅度远远超出了十年前。大数据技术和服务市场预计将从2012年的98亿美元增长到2017年的324亿美元，增长27％，从而导致生成的技术数据数量增加。现代组织有前所未有的机会，可以获得有关其产品，流程和服务的新知识，可以用来使项目提高效率，改善产品和服务，确定问题，采取先发制人的措施并将业务目标适应新的机会。新知识的应用包括预测模型，例如，减少医院再入院率，交通拥堵和不必要的发电。***利用这些机会需要解决解决方案处理大数据（BDS）所面临的挑战。这些复杂的解决方案具有许多动态组件，例如分布式计算节点，网络，数据库，中间件和商业智能层。任何组件都可能使其与他人的交互失败，可能导致解决方案或质量降解的崩溃失败（例如性能，可靠性，安全性）。因此，它们在开发和维护方面具有挑战性。尽管关于BDS生成的处理数据的知识很大，但大多数技术仍无法处理BDS生成的操作数据的数量。我的目标是通过达到以下目标来帮助改善BD的测试和维护：1）为BDS构建缺陷预测模型，以改善其一般测试和维护过程； 2）为BDS开发痕量分析，以加快根本原因确定并去除冗余测试用例。为了达到目标，我将创建能够处理操作数据的新型可扩展方法，技术和工具。***据我所知，这些目标是新颖的，对于实践而言是重要的。预期的结果将有助于创建BDS自动化问题确定技术的初步理论。将结果转移到加拿大行业将提高产品质量，并加快缺陷检测和修复，从而使开发人员能够创建更多的新功能，并减少维护工作和投资。**********