Large-Scale and Big Data Optimization

大规模、大数据优化

• 批准号: RGPIN-2017-06715
• 负责人: Jaumard, Brigitte
• 金额: $ 2.99万
• 依托单位: Concordia University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=711861
• 关键词: Large; Scale; Big; Data; Optimization; Large; Scale; Big; Data; Optimization

In today's digital world, with ever increasing amounts of data comes the need to solve optimization problems of unprecedented sizes. Machine learning, communication and social networks, logistics systems are some of the many prominent application domains where optimization problems arise with tens of thousands or millions of variables. Many optimization models and algorithms, while exhibiting great efficiency in modest dimensions, have great difficulties to scale for instances of this size and do not offer satisfactory solution. The primary and long-term objective of my research is to contribute to the design of novel optimization algorithms capable of working in very large-scale setting. I plan to investigate both exact and heuristic methods, and validate the findings on some particular applications in communication, logistics and social networks. For exact methods, the objective is to integrate knowledge based on both theoretical and empirical evidence from several disciplines, and explore the "what, why, how, and do" paradigm with an emphasis on (i) modelling aspects, (ii) combination of mathematical models, and (iii) parallelization techniques in order to take advantage of the heterogeneous environments combining multi-core processors, multi-threaded programming and GPU accelerators for very large scale optimization. While those environment were only available on mainframe computers, they are now available to computers that are easily accessible to the industry. For heuristic methods, focus will be on meta-heuristics, a wide class of solution methods that have been successfully applied to many optimization problems. However, they seem to have reached their limits to solve very large combinatorial problems such as those arising in cross-docking or network optimization. This is because meta-heuristics explore the solution space with ad-hoc methods, whose efficiency and computing time highly depend on the topology of the local optima which, except for some very particular problems, are very difficult to foresee. We plan to replace the ad-hoc exploration of the solution space with an informed exploration guided by machine learning. Comparison will be made with direct machine learning algorithms on practical problems arising in: (i) supply chain management and in particular with cross-docking, and (ii) network optimization and (iii) mechanism design in social networks. Data required by machine learning algorithms will be provided by ClearD and Ciena for the first two applications, and an organization/industrial partner needs to be identified for the third one. The results of my research will provide the industry (like ClearD and Ciena) information technology management tools for efficient and automated cross-docking/network management, not only to improve competitiveness but also to reduce energy consumption and therefore carbon footprint.

在当今的数字世界中，随着数据量的越来越多，需要解决前所未有的优化问题。机器学习，沟通和社交网络，物流系统是许多突出的应用领域中的一些，其中优化问题发生了数万或数百万变量。许多优化模型和算法在适度的维度上表现出很高的效率，但对于此尺寸的实例而言，很难扩展，并且没有提供令人满意的解决方案。我的研究的主要和长期目标是为能够在非常大规模的环境中工作的新型优化算法的设计做出贡献。我计划研究精确的和启发式方法，并验证有关交流，物流和社交网络中某些特定应用的发现。 对于确切的方法，目的是基于几个学科的理论和经验证据整合知识，并探索“什么，为什么，为什么，如何做”范式，重点是（i）建模方面，（i）建模，（ii）数学模型的组合，以及（iii）的跨度程序，以利用跨度环境的整合构成整合的整合构成组合的组合，这加速器进行非常大规模的优化。尽管这些环境仅在大型机计算机上可用，但现在它们可用于行业易于访问的计算机。 对于启发式方法，将重点放在元映体上，这是一种已成功应用于许多优化问题的广泛解决方案方法。但是，他们似乎已经达到了解决非常大的组合问题的限制，例如在交叉船上或网络优化中引起的问题。这是因为Meta-Heuristics使用临时方法探索解决方案空间，其效率和计算时间在很大程度上取决于局部Optima的拓扑，除了一些非常特殊的问题外，很难预见。我们计划通过以机器学习为指导的知情探索来替换解决方案空间的临时探索。将与直接机器学习算法进行比较，该算法在以下方面引起的实际问题：（i）供应链管理，尤其是交叉插件，以及（ii）社交网络中的网络优化和（iii）机制设计。 机器学习算法所需的数据将由Cleard和Ciena提供前两个应用程序，并且需要确定第三个组织/工业合作伙伴。 我的研究结果将为行业（例如Cleard和Ciena）提供高效和自动化的交叉插座/网络管理工具，不仅可以提高竞争力，而且还可以减少能源消耗和碳足迹。