Research on singing rendering systems design based on an active auditory perception model

基于主动听觉感知模型的歌唱渲染系统设计研究

• 批准号: 14380165
• 负责人: KAWAHARA Hideki
• 金额: $ 9.15万
• 依托单位: Wakayama University
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (B)
• 财政年份: 2002
• 资助国家: 日本
• 起止时间: 2002 至 2004
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-14380165/
• 关键词: Speech analysis; Speech synthesis; Auditory morphing; Fundamental frequency; Radiation pattern; Paralinguistic information; Singing synthesis; Speech dynamics; 高品質音声合成; 歌唱システム; STRAIGHT; 音声変換; ビブラート; 表情づけ; 非言語情報; モーフィング; 韻律情報; 聴覚フィードバック /

The goal of this project is to investigate the source of reason why vocal music is attractive even without lyrics. This general goal was broken down to several sub-goal which consists of new research tool development and winning prize as the best artificial singing system at international contests. These goals were fulfilled even though the success introduced more questions than the answered questions. Firstly, the piece of chorus with artificially manipulated synthesized voices (that is an excerpt of a composition made by Toru Takemitsu titled "small sky") won the first prize among four synthetic singing systems at RENCON'04, the satellite event of the international conference on computer based entertainment systems (NIME'04) held in Shizuoka in 2004. The piece was made using a STRAIGHT based singing synthesis program. Secondly, the singing synthesis system is based on the auditory morphing algorithm invented for this research project. The morphing algorithm made a substantial impact … More on speech perception and music perception research and the algorithm is currently used in many research institutes worldwide. Thirdly, a new algorithm called "senza vibrato" was developed to made it possible to morph vibrato that is an essential ingredient of singing voice, and at the same time, is an obstacle that made morphing of singing voice very difficult. Fourthly, important experiences were obtained by performing actual investigations based on the "systematic downgrading strategy" that was proposed to characterize the current research project. Those accomplishments were reported at various international/domestic conferences and scientific journals. Those publications and the new research tools based on STRAIGHT made a research trend that is characterized by ecological views on auditory and speech perception. In conclusion, the project was a great success. However, it is important to note that even with all the accomplishments in this project, there still remains a huge gap between synthetic singers and human singers. There is a huge room for investigations to bridge this gap. The prospective research project may need to put attentions on methods for generalization from relatively small number of instances, because, based on experiences in this research project, it is generally impractical to provide sufficient number of singing voice instances to function the "systematic downgrading strategy" in its full extent. Less

该项目的目标是调查声乐即使没有歌词也有吸引力的根源，这个总体目标被分解为几个子目标，其中包括开发新的研究工具和赢得国际最佳人工歌唱系统奖。尽管成功提出的问题比回答的问题还多，但这些目标还是实现了。四个合成歌唱系统中的奖项RENCON'04是2004年在静冈举行的国际计算机娱乐系统会议（NIME'04）的卫星活动。该作品是使用基于STRAIGHT的歌唱合成程序制作的。其次，歌唱合成系统是基于听觉的。该研究项目发明的变形算法对语音感知和音乐感知研究产生了重大影响，该算法目前在全球许多研究机构中使用。第三，一种名为“senza”的新算法。 “颤音”的开发是为了使颤音变形成为可能，而颤音是歌声的重要组成部分，同时也是使歌声变形变得非常困难的障碍。第四，通过基于颤音的实际调查，获得了重要的经验。提出的表征当前研究项目的“系统降级策略”在各种国际/国内会议和科学期刊上都有报道，这些出版物和基于STRAIGHT的新研究工具形成了一种以生态观点为特征的研究趋势。总之，该项目取得了巨大成功，但值得注意的是，即使该项目取得了所有成就，合成歌手与人类歌手之间仍然存在巨大差距。未来的研究项目可能需要关注从相对较少的实例中进行概括的方法，因为根据本研究项目的经验，提供足够数量的歌声实例通常是不切实际的。发挥“系统降级”作用战略”的完整范围。更少

项目成果

Underlying principles of a high-quality speech manipulation system STRAIGHT and its application to speech segregation

• DOI: 10.1007/0-387-22794-6_11
• 发表时间: 2005-01-01
• 期刊: SPEECH SEPARATION BY HUMANS AND MACHINES
• 作者: Kawahara, H;Irino, T
• 通讯作者: Irino, T

Glottal closure instant synchronous sinusoidal model for high quality speech analysis/synthesis

用于高质量语音分析/合成的声门闭合即时同步正弦模型

• 发表时间: 2003
• 期刊: Proc.8th European Conference on Speech Communication and Technology (Eurospeech'03)
• 作者: Parham Zolfaghari

A new acoustic measurement and compensation method based on logarithmic transformation of the time axis and multi-location acquisition

基于时间轴对数变换和多点采集的声学测量与补偿新方法

• 发表时间: 2004
• 期刊: Proc.18th International Congress on Acoustics (ICA 2004) I
• 作者: Hirokazu Kameoka;Takuya Nishimoto;Shigeki Sagayama;Masanori Morise
• 通讯作者: Masanori Morise

Speech segregation using an event-synchronous auditory image and STRAIGHT

使用事件同步听觉图像和 STRAIGHT 进行语音分离

• 发表时间: 2005
• 期刊: Speech separation by human and machines
• 作者: Toshio Irino

"senza vibrato" : a key component for morphing singing

“senza vibrato”：变形歌唱的关键组成部分

• 发表时间: 2004
• 期刊: Proc.8th International Conference on Spoken Language Processing (ICSLP 2004) V
• 作者: A.Sugimoto;A.Nakayama;T.Matsuyama;Noriaki Sato;Hideki Kawahara
• 通讯作者: Hideki Kawahara