Research and development of a cloud-based context-aware API for semantic scene understanding

基于云的上下文感知API的语义场景理解研究与开发

• 批准号: 558247-2020
• 负责人: Khan, Naimul
• 金额: $ 2.17万
• 依托单位: Ryerson University
• 依托单位国家: 加拿大
• 项目类别: Alliance Grants
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=706640
• 关键词: Research; development; cloud; based; context

Augmented Reality (AR) has become a promising tool for enhancing the quality of experience in digital media products. However, the technical tools available for AR lacks in providing the rich understanding required to design interactive digital experiences quickly and efficiently. In partnership with AWE Company Ltd, this research project will attempt at developing a context-aware Application Programming Interface (API) that can provide a semantic understanding of a user's surrounding by intelligently distributing the camera feed data from a user's device on to three separate sub-modules for semantic understanding: building facade segmentation, floor plan creation, and object localization. Through understanding the user's context (i.e. the requirement of the understanding of the aforementioned semantics), the proposed API can reduce the computational and data overhead required to run the neural network-based systems on the cloud. The proposed project will employ Distributed Deep Neural Network (DDNN) design principles to carefully distribute the computational task between the clients and the cloud, while building the semantic understanding stack on top of available technology tools such as ARCore/ARKit, so that AWE's software developers can easily integrate the proposed solution into their product development pipeline. The proposed API will result in a robust framework for quick distribution of digital twins projects that AWE plans to roll out for many of its clients within the next two years. The proposed research will help to position Canada as a leader in multimedia technologies, while the resultant technology transfer to Canadian industry will strengthen Canada's global competitiveness and create positive impacts to Canadian economy and society.

增强现实 (AR) 已成为增强数字媒体产品体验质量的有前景的工具。然而，可用于 AR 的技术工具缺乏提供快速有效地设计交互式数字体验所需的丰富理解。该研究项目将与 AWE Company Ltd 合作，尝试开发上下文感知应用程序编程接口 (API)，通过智能地将来自用户设备的摄像头馈送数据分配到三个独立的子系统上，提供对用户周围环境的语义理解。 -语义理解模块：建筑立面分割、平面图创建和对象定位。通过了解用户的上下文（即理解上述语义的要求），所提出的 API 可以减少在云上运行基于神经网络的系统所需的计算和数据开销。拟议的项目将采用分布式深度神经网络（DDNN）设计原理，在客户端和云端之间仔细分配计算任务，同时在 ARCore/ARKit 等可用技术工具之上构建语义理解堆栈，以便 AWE 的软件开发人员可以轻松地将建议的解决方案集成到他们的产品开发流程中。 拟议的 API 将形成一个强大的框架，用于快速分发数字孪生项目，AWE 计划在未来两年内为其许多客户推出该项目。拟议的研究将有助于使加拿大成为多媒体技术的领导者，而由此产生的向加拿大工业的技术转让将增强加拿大的全球竞争力，并对加拿大经济和社会产生积极影响。