Towards Highly Interactive Networked Multimedia Services with Crowd Intelligence

通过群体智能实现高度交互的网络多媒体服务

• 批准号: RGPIN-2019-04040
• 负责人: Liu, Jiangchuan
• 金额: $ 5.39万
• 依托单位: Simon Fraser University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=714624
• 关键词: Towards; Highly; Interactive; Networked; Multimedia

With the advances in social, mobile and cloud computing, crowdsourcing has become the norm in today's digital media production and sharing, which utilizes the collective effort of users in the online community. With no doubt, networked services in the future will focus on the user experience and participation with rich media from the crowd. The boundless coverage of socialized media sharing however presents unprecedented scale challenges. Highly diversified content types, origins, and distribution channels further impose complex interactions among the individual participants, particularly for advanced immersive applications such as interactive livecast/gamecast, multi-party media production, online gaming and augmented reality (AR). Their demands on quality and responsiveness are much higher (e.g., latency in the order of 100 ms, or even 10 ms for AR) than conventional IPTV or VoIP. These rich interactions, on the other hand, provide rich and low cost data that were previously unavailable. Leveraging advanced data analytics and learning tools, the hidden intelligence from the crowd can be unveiled toward better system design. For instance, inferring game progress from live chat messages and tracking viewport through gaze patterns for resource reservation and display optimization. Yet uncalibrated data in the wild can be highly noisy, not to mention the intrinsic heterogeneity and dynamics of the Internet. Our research is guided by the following critical question: How can massive digital media content be effectively generated and delivered among the crowd with intensive online interactions and with the intelligence therein? Our long-term objective is to systematically examine the challenges in advanced interactive and crowdsourced multimedia services and to develop integrated solutions that will accommodate and utilize the interactions. A series of issues are to be addressed within the solution framework: (1) Online analysis of crowdsourced interaction data; (2) Advanced computing and communication architecture, in particular, cloud-edge collaboration for delay-sensitive applications; (3) Ultra-low-latency and -energy data transport. Beyond simply putting the individual results together, the design of individual modules will be revisited during system integration. This iterative process will eventually lead to an optimized coherent design, making the future multimedia services agile, efficient, and intelligent, and, in the long run, benefiting emerging applications without human-in-the-loop, such as assisted/autonomous driving. My team's broad expertise with state-of-the-art computing and communication systems has prepared us for exploring these new research frontiers. We will train highly qualified personnel and will nurture extensive collaboration with academic researchers worldwide, as well as with local and international industrial partners to pursue technology transfers, so as to raise Canada's global profile in this priority area.

随着社交，移动和云计算的进步，众包已经成为当今数字媒体生产和共享的规范，该数字媒体生产和共享利用了在线社区中用户的集体工作。毫无疑问，将来的网络服务将重点关注用户体验，并与人群中丰富的媒体一起参与。然而，社交媒体共享的无限覆盖范围提出了前所未有的规模挑战。高度多样化的内容类型，起源和分销渠道进一步在个人参与者之间实现了复杂的互动，特别是对于高级沉浸式应用，例如交互式livecast/gamecast，多方媒体制作，在线游戏和增强现实（AR）。他们对质量和响应能力的需求要高得多（例如，AR的延迟为100毫秒，甚至10毫秒的延迟）比常规IPTV或VoIP高。 另一方面，这些丰富的互动提供了以前不可用的丰富和低成本数据。利用高级数据分析和学习工具，可以揭示出朝着更好的系统设计揭示的隐藏智能。例如，从实时聊天消息中推断游戏的进展以及通过凝视模式跟踪资源保留和显示优化的视口。然而，野外未校准的数据可能是嘈杂的，更不用说互联网的内在异质性和动态。 我们的研究以以下关键问题为指导：如何通过密集的在线互动以及与其中的情报有效地生成和交付大量数字媒体内容？我们的长期目标是系统地检查高级互动和众包多媒体服务的挑战，并开发将适应和利用交互的集成解决方案。解决方案框架中将解决一系列问题：（1）众包交互数据的在线分析； （2）高级计算和通信体系结构，尤其是针对延迟敏感应用程序的云边缘协作； （3）超低延迟和 - 能源数据传输。除了简单地将单个结果放在一起外，还将在系统集成期间重新审视单个模块的设计。这种迭代过程最终将导致优化的连贯设计，从而使未来的多媒体服务敏捷，高效且聪明，并且从长远来看，使新的应用程序受益于没有人类融合的新兴应用程序，例如辅助/自动驾驶。 我团队在最先进的计算和通信系统方面的广泛专业知识为我们做好了探索这些新研究前沿的准备。我们将培训高素质的人员，并将与全球学术研究人员以及本地和国际工业合作伙伴进行广泛的合作，以追求技术转移，以提高加拿大在此优先领域的全球知名度。