《计算机应用研究》|Application Research of Computers

汉语语音同步的三维口型动画研究

Chinese speech synchronized 3D lip animation

免费全文下载 (已被下载 次)  
获取PDF全文
作者 米辉辉,侯进,李克豹,甘凌云
机构 1.西南交通大学 信息科学与技术学院,成都 610031;2.南京大学 计算机软件新技术国家重点实验室,南京 210093
统计 摘要被查看 次,已被下载
文章编号 1001-3695(2015)04-1244-04
DOI 10.3969/j.issn.1001-3695.2015.04.068
摘要 针对汉语的发音习惯以及语音可视化技术中对口型动画自然、连续的要求,提出了一种基于肌肉模型与协同发音模型的与语音保持同步的口型动画的方法。首先,根据汉语发音时的口型视位特征将声、韵母音素归类,并用数据映射的方式合成与之对应的口型关键帧。通过分析输入的文本信息,合成与语音保持同步的三维人脸口型动画。为了解决汉语发音习惯的问题,设计了一种基于微分几何学描述的协同发音建模的方法,该方法通过分析相邻音子间视素的影响权重,可以产生符合汉语发音习惯的口型动画。最后,通过实验对比和分析,该方法产生的口型动画更为逼真,且符合汉语发音的习惯。
关键词 语音可视化;协同发音模型;口型动画;语音动画
基金项目 国家自然科学基金面上项目(61371165)
浙江大学CAD&CG国家重点实验室开放课题(A1416)
计算机软件新技术国家重点实验室开放课题基金资助项目(KFKT2013B22)
四川省动漫研究中心2012年度科研项目(DM201204)
本文URL http://www.arocmag.com/article/01-2015-04-068.html
英文标题 Chinese speech synchronized 3D lip animation
作者英文名 MI Hui-hui, HOU Jin, LI Ke-bao, GAN Ling-yun
机构英文名 1. School of Information Science & Technology, Southwest Jiaotong University, Chengdu 610031, China; 2. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210093, China
英文摘要 In order to meet the characteristics of Chinese pronunciation and satisfy the requirement of the speech visualization technology, namely the natural and continuous lip animation, this paper proposed a method of synchronized speech 3D lip animation based on muscle model and coarticulation model.Firstly, according to the mouth visemes characteristics of the Chinese pronounciation, consonant-vowel phonemes were grouped and Chinese visual phoneme key frames were synthesized by data mapping.And then, the method depended on the analysis of the input Chinese text to simulate the synchronized speech 3D facial animation.Moreover, a coarticulation model, which conformed to geometric description, was constructed where the pronunciation property of Chinese characteristics would be sufficiently considered.This method used the inter syllables weighting function of consonant-vowel to simulate the effect of coarticulation and lip animation that obeyed Chinese pronunciation habit.Finally, to testify the performance, it simulated the 3D facial lip animation.The results show that the synthesized lip animation is more natural and accord with the habits of Chinese pronunciation.
英文关键词 speech visualization; coarticulation model; lip animation; speech animation
参考文献 查看稿件参考文献
  [1] BREGLER C, COVELL M, SLANEY M. Video rewrite:driving visual speech with audio[C] //Proc of the 24th Annual Conference on Computer Graphics and Interactive Techniques. New York:ACM Press, 1997:353-360.
[2] BRAND M. Voice puppetry[C] //Proc of the 26th Annual Conference on Computer Graphics and Interactive Techniques. New York:ACM Press, 1999:21-28.
[3] LIU Jia, YOU Ming-yu, CHEN Chun, et al. Real-time speech-driven animation of expressive talking faces[J] . International Journal of General Systems, 2011, 40(4):439-455.
[4] MORO A, MUMOLO E, NOLICH M. Automatic 3D virtual cloning of a speaking human face[C] //Proc of ACM Workshop on Surreal Media and Virtual Cloning. 2010:45-50.
[5] BAGAI A, GANDHI H, GOYAL R, et al. Lip-reading using neural networks[J] . International Journal of Computer Science and Network Security, 2009, 9(4):108-111.
[6] TERRY L, LIVESCU K, PIERREHUMBERT J B, et al. Audio-visual anticipatory coarticulation modeling by human and machine[C] //Proc of the 11th Annual Conference on International Speech Communication Association, 2010:2682-2685.
[7] 杨逸, 侯进, 王献. 基于运动轨迹分析的3D唇舌肌肉控制模型[J] . 计算机应用研究, 2013, 30(7):2236-2240.
[8] 肖业清, 侯进, 王献. 基于正面照的自适应三维头部网格模型研究[J] . 计算机仿真, 2013, 30(7):412-417.
[9] 李皓, 陈艳艳, 唐朝京. 唇部子运动与权重函数表征的汉语动态视位[J] . 信号处理, 2012, 28(2):322-328.
[10] WATERS K. A muscle model for animation three-dimensional facial expression[J] . Computer Graphics, 1987, 22 (4):17-24.
收稿日期 2014/3/14
修回日期 2014/4/24
页码 1244-1247
中图分类号 TP391.9
文献标志码 A