SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation
Computer Vision and Pattern Recognition (CVPR), 2023
Biography
I am currently a research scientist in the Visual Computing Center of Tencent AI Lab. I received my M.S and Ph.D. degrees from the Department of Computer and Information Science, University of Macau, under the supervision of Dr. Chi-Man Pun in 2018 and 2021 respectively. I received my B.Sc degree in Computer Science from Xidian University.
Currently, I am working on designing novel applications for image/video generation, translation and editing (especially in the ACG area !!!) using AIGC related techniques.
[We are always hiring self-motivated students as research interns for ACG-related research topics, please drop me an email if you are interested.]
[We are always hiring self-motivated students as research interns for ACG-related research topics, please drop me an email if you are interested.]
News
- Six papers are accepted by CVPR-23. 2023.02
- One paper is accepted by AAAI-23. 2022.11
- One paper is conditionally accepted by SIGGRAPH Asia 2022 (Conference Track). 2022.08
- Two papers are accepted by ECCV 2022. 2022.07
- One paper is accepted by CVPR 2022. 2022.02
Activities
- Intern FrondEnd Engineer, Alibaba Group 2014.7-9
- Intern Algorithm Engineer, HIKVision Research Institute 2018.5-7
- Asian University Machine Learning Camp, Jeju National University 2018.9
- Research Intern, Huawei Noah's Ark Lab 2020.3-11
Research Highlight (🧑💻 Co-first Author, 📮 Corresponding Author)
Uformer: A General U-Shaped Transformer for Image Restoration
Zhendong Wang, Xiaodong Cun 📮, Jianmin Bao, Jianzhuang Liu, Wengang Zhou, Houqiang Li
Computer Vision and Pattern Recognition (CVPR), 2022
Selected Publications (🧑💻 Co-first Author, 📮 Corresponding Author) Clike to Show All 🔽
Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videos
ArXiv, 2023
FateZero: Fusing Attentions for Zero-shot Text-based Video Editing
ArXiv, 2023
Explicit Visual Prompting for Low-Level Structure Segmentations
Weihuang Liu, Xi Shen , Chi-Man Pun 📮, Xiaodong Cun 📮
Computer Vision and Pattern Recognition (CVPR), 2023
T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations
Jianrong Zhang 🧑💻, Yangsong Zhang 🧑💻, Xiaodong Cun, Shaoli Huang, Yong Zhang, Hongwei Zhao, Hongtao Lu, Xi Shen 📮
Computer Vision and Pattern Recognition (CVPR), 2023
3D GAN Inversion with Facial Symmetry Prior
Fei Yin, Yong Zhang, Xuan Wang, Tengfei Wang, Xiaoyu Li, Yuan Gong, Yanbo Fan, Xiaodong Cun, Ying Shan, Cengiz Oztireli, Yujiu Yang
Computer Vision and Pattern Recognition (CVPR), 2023
DPE: Disentanglement of Pose and Expression for General Video Portrait Editing
Youxin Pang, Yong Zhang, Weize Quan, Yanbo Fan, Xiaodong Cun, Ying Shan, Dong-ming Yan
Computer Vision and Pattern Recognition (CVPR), 2023
CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior
Computer Vision and Pattern Recognition (CVPR), 2023
SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation
Computer Vision and Pattern Recognition (CVPR), 2023
CoordFill: Efficient High-Resolution Image Inpainting via Parameterized Coordinate Querying
AAAI Conference on Artificial Intelligence (AAAI, Oral), 2023
VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wild
Kun Cheng 🧑💻, Xiaodong Cun 🧑💻📮, Yong Zhang, Menghan Xia, Fei Yin, Mingrui Zhu, Xuan Wang, Jue Wang, Nannan Wang
SIGGRAPH Asia (Conference Track), 2022
StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pretrained StyleGAN
Fei Yin, Yong Zhang, Xiaodong Cun, Mingdeng Cao, Yanbo Fan, Xuan Wang, Qingyan Bai, Baoyuan Wu, Jue Wang, Yujiu Yang
European Conference on Computer Vision (ECCV), 2022
Spatial-Separated Curve Rendering Network for Efficient and High-Resolution Image Harmonization
Jingtang Liang 🧑💻 , Xiaodong Cun 🧑💻 , Chi-Man Pun, Jue Wang
European Conference on Computer Vision (ECCV), 2022
Uformer: A General U-Shaped Transformer for Image Restoration
Zhendong Wang, Xiaodong Cun 📮, Jianmin Bao, Jianzhuang Liu, Wengang Zhou, Houqiang Li
Computer Vision and Pattern Recognition (CVPR), 2022
Learning Enriched Illuminants for Cross and Single Sensor Color Constancy
Xiaodong Cun 🧑💻, Zhendong Wang 🧑💻, Chi-Man Pun, Wengang Zhou, Jianzhuang Liu, Xu Jia, Houqiang Li
Preprint, 2021
Split then Refine: Sequential Attention-guided ResUNets for Blind Single Image Visible Watermark Removal
Xiaodong Cun, Chi-Man Pun
AAAI Conference on Artificial Intelligence (AAAI), 2021
Defocus Blur Detection via Depth Distillation
Xiaodong Cun, Chi-Man Pun
European Conference on Computer Vision (ECCV), 2020
Improving the Harmony of the Composite Image by Spatial-Separated Attention Module
Xiaodong Cun, Chi-Man Pun
IEEE Trans. on Image Processing (TIP), 2020
Towards Ghost-free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN
Xiaodong Cun, Chi-Man Pun, Cheng Shi
AAAI Conference on Artificial Intelligence (AAAI), 2020
Depth Assisted Full Resolution Network for Single Image based View Synthesis
SIGGRAPH Poster, 2018
/
IEEE CG&A, 2018
Academic Services
Reviewer of: PR, TPAMI, CVIU.
Misc
- 🎓 If you are looking for talent student on Machine Tranlation, please checkout my friend Xuebo Liu.
- 🎴 Level 19 (failed to level 20) in Slay the Spire. More than 10k death in Celeste.
- 🏃 Workout often & lose weight 10kg 7.5kg & 200+ hours in Fitness Boxing
- 🎮 Cloud player of DOTA 2, ancient player of Freestyle, just ordinary player.
- 👨💻 Out of fashion front-end engineer. Try this website template at Github.
- 💵 Lose the wallet of 0.2 BTC in 2013.
- ☕️ A big fan of Coffee and Tea.
- ❤️ Since 2019.4.13