Modeling Diverse, Personalized and Expressive Animations for Virtual Characters through Motion Capture, Synthesis and Perception

通过动作捕捉、合成和感知为虚拟角色建模多样化、个性化和富有表现力的动画

• 批准号: RGPIN-2022-04920
• 负责人: Wang, Yingying
• 金额: $ 2.11万
• 依托单位: McMaster University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=750440
• 关键词: Modeling; Diverse; Personalized; Expressive; Animations

Character animation plays a key role in delivering motions for virtual characters in game development, robotics, Virtual Reality (VR) and Augmented Reality (AR) applications. Despite the large effort in modeling motions to be natural and realistic, previous research has been focused on generic motion content learning where stylistic features of how each individual performs the content are largely ignored. Without individual styles, different virtual characters move the same way when the same motion content is needed, which is far from satisfactory to create a diverse virtual world. One major challenge to stylize motions is the lack of large scale motion style databases, and thus no sufficient knowledge of how to effectively model and transfer styles. This proposal focuses on motion style learning, synthesis and transfer. In the long term, our goal is to find ultimate solutions to generate stylized motions with variations that match the diversity in the real-world. The short-term objectives are: we will first establish large scale motion style databases through motion capture technique; from the data, we develop methods to learn effective motion representations and explore generative models to conditionally generate motions with desired styles; we will further discover style transfer models to edit styles while keeping the original motion content. In the five-year period, we will specifically capture, learn and model three stylistic features: demographic styles belonging to different groups of people, e.g. age, gender, and race; personalized styles resulting from personalities, body build for different individuals; expressive styles demonstrating varied emotional and physical states for the same individual under different scenarios. We will set up our databases to cover these style variations, publicize the database for open access, and provide labelling, documentation and technical support to the public. Research findings and source code will be published, solving problems of extracting style features, generating motion styles in a controllable manner, and transferring styles to novel motions. Beyond the five-year term, we will continue adding more styles to our databases, to model a broader picture of motion styles. Our database can directly be used in animating diverse characters in AR/VR and game scenes, it also facilitates other researchers to model motion styles, and stimulates interdisciplinary research in psychology, kinesiology, and art etc. Research findings in motion style modeling supports intelligent applications such as action recognition, style recognition and person identification from motion input. Style synthesis and transfer technology can also be widely applied in the game, entertainment industry, AR/VR applications in education, media, and social networks. By creating virtual characters that authentically embody people in the real world, this work can promote Diversity, Equity and Inclusion of the virtual world.

角色动画在游戏开发、机器人、虚拟现实 (VR) 和增强现实 (AR) 应用中为虚拟角色提供动作方面发挥着关键作用。尽管在模拟自然和真实的动作方面付出了巨大的努力，但以前的研究一直集中在通用动作内容学习上，其中每个人如何执行内容的风格特征在很大程度上被忽略。如果没有个性化的风格，当需要相同的动作内容时，不同的虚拟角色会以相同的方式移动，这远远不能满足创建多样化的虚拟世界的要求。运动风格化的一个主要挑战是缺乏大规模的运动风格数据库，因此对如何有效地建模和转移风格缺乏足够的了解。该提案侧重于运动风格的学习、合成和迁移。从长远来看，我们的目标是找到最终的解决方案来生成风格化的运动，其变化与现实世界的多样性相匹配。短期目标是：首先通过动作捕捉技术建立大规模的动作风格数据库；根据数据，我们开发了学习有效运动表示的方法，并探索生成模型以有条件地生成具有所需样式的运动；我们将进一步发现风格转移模型来编辑风格，同时保留原始的运动内容。 在五年期间，我们将专门捕捉、学习和建模三种风格特征：属于不同人群的人口统计风格，例如年龄、性别和种族；因不同人的性格、体型而产生的个性化风格；表现风格表现出同一个人在不同场景下的不同情绪和身体状态。我们将建立数据库来涵盖这些风格变化，公开数据库以供开放访问，并向公众提供标签、文档和技术支持。研究成果和源代码将被发布，解决提取风格特征、以可控方式生成运动风格以及将风格转化为新运动的问题。在五年期限之后，我们将继续向我们的数据库添加更多风格，以对更广泛的运动风格进行建模。我们的数据库可以直接用于AR/VR和游戏场景中的各种角色动画，也方便其他研究人员对运动风格进行建模，并刺激心理学、运动学和艺术等领域的跨学科研究。运动风格建模的研究成果支持智能应用例如动作识别、风格识别和动作输入中的人物识别。风格合成和传输技术还可以广泛应用于游戏、娱乐行业、教育、媒体和社交网络中的AR/VR应用。通过创建真实体现现实世界中的人的虚拟角色，这项工作可以促进虚拟世界的多样性、公平性和包容性。