Jiaxu Zhang | 张嘉旭

I am a Ph.D. student working with Prof. Deren Li and Prof. Zhigang Tu in LIESMARS at Wuhan University, China. Before that, I received my B.S. degree from Southeast University in 2020 and MEng from Wuhan University in 2023.
I worked as a research intern at Tencent from 2022 to 2024. Currently, I am interning at StepFun, collaborating with Dr. Gang Yu on AIGC research. My research interest lies in deep learning, computer vision and computer graphics, with a current focus on 3D/2D animation, motion generation, retargeting, and recognition.

Expected graduation in 2026, open to postdoc and research scientist positions.

Email  /  CV  /  Google Scholar  /  Github  /  WeChat


profile photo
News

  • [2024/07] 🎉 One paper gets accepted to ACM MM 2024.
  • [2024/04] 🕹️ I've released a repository, Freehand-Genshin-Diffusion, that transforms Genshin PVs into a freehand style using the Diffusion Model. Feel free to give it a try!
  • [2024/04] 🎉 One paper has been accepted by IEEE T-PAMI, which is an extension of our CVPR 2023 paper.
  • [2024/01] 🎉 One paper gets accepted to ICLR 2024.
  • [2023/06] 📌 I gave an oral presentation on Virtual Animation Technology at VALSE 2023.
  • [2023/02] 🎉 One paper gets accepted to CVPR 2023.

  • Research

    My research interests are broadly in 3D/2D Computer Vision and Computer Animation. My overarching research objective is to contribute to the development of lifelike, intelligent, and interactive virtual avatars and animations.

    MikuDance: Animating Character Art with Mixed Motion Dynamics
    Jiaxu Zhang, Xianfang Zeng, Xin Chen, Wei Zuo, Gang Yu*, Zhigang Tu*
    Arxiv, 2024
    project page / code (coming soon) / arxiv

    We propose MikuDance, a diffusion-based pipeline incorporating mixed motion dynamics to animate stylized character art.

    Freehand-Genshin-Diffusion

    A project for transforming Genshin PVs into a freehand style using Diffusion Model.

    I've been exploring 2D image animation recently. This project is purely for fun. Feel free to reach out and discuss this with me.

    A Modular Neural Motion Retargeting System Decoupling Skeleton and Shape Perception
    Jiaxu Zhang, Zhigang Tu*, Junwu Weng, Junsong Yuan, Bo Du
    IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2024
    code / arxiv

    M-R2ET is a modular neural motion retargeting system designed to transfer motion between characters with different structures but corresponding to homeomorphic graphs, meanwhile preserving motion semantics and perceiving shape geometries.

    Generative Motion Stylization of Cross-structure Characters within Canonical Motion Space
    Jiaxu Zhang, Xin Chen, Gang Yu, Zhigang Tu*
    Proceedings of the 32nd ACM International Conference on Multimedia (ACM MM), 2024
    arxiv

    We present MotionS, a generative motion stylization pipeline for synthesizing diverse and stylized motion on cross-structure source using cross-modality style prompts.

    TapMo: Shape-aware Motion Generation of Skeleton-free Characters
    Jiaxu Zhang#, Shaoli Huang#, Zhigang Tu*, Xin Chen, Xiaohang Zhan, Gang Yu, Ying Shan
    The Twelfth International Conference on Learning Representations (ICLR), 2024
    project page / code / arxiv

    TapMo is a text-based animation pipeline for generating motion in a wide variety of skeleton-free characters.

    Skinned Motion Retargeting with Residual Perception of Motion Semantics & Geometry
    Jiaxu Zhang, Junwu Weng, Di Kang, Fang Zhao, Shaoli Huang, Xuefei Zhe, Linchao Bao, Ying Shan, Jue Wang, Zhigang Tu*
    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
    project page / code / arxiv

    R2ET is a neural motion retargeting model that can preserve the source motion semantics and avoid interpenetration in the target motion.

    Zoom Transformer for Skeleton-based Group Activity Recognition
    Jiaxu Zhang, Yifan Jia, Wei Xie, and Zhigang Tu*
    IEEE Transactions on Circuits and Systems for Video Technology (T-CSVT), 2022
    code / arxiv

    We propose a novel Zoom Transformer to exploit both the low-level single-person motion information and the high-level multi-person interaction information in a uniform attention structure.

    Joint-bone Fusion Graph Convolutional Network for Semi-supervised Skeleton Action Recognition
    Zhigang Tu#, Jiaxu Zhang#*, Hongyan Li, Yujin Chen, and Junsong Yuan
    IEEE Transactions on Multimedia (T-MM), 2022
    code / arxiv

    we propose a semi-supervised skeleton-based action recognition method.


    Experience

    StepFun
    2024.05 - Present, Shanghai
    Research Intern for AIGC. Advisor: Dr. Gang Yu and Dr. Xianfang Zeng
    Wuhan University
    2020.09 - Present, Wuhan
    Ph.D Student in LIEMSARS.
    I received my Master Degree of Computer Technology in 2023.
    Research Advisor: Prof. Zhigang Tu
    Tencent
    2023.06 - 2024.04, Shanghai
    Research Intern in Tencent PCG. Advisor: Dr. Gang Yu and Dr. Xin Chen
    2022.07 - 2023.06, Shenzhen
    Research Intern in Tencent AI Lab. Advisor: Dr. Junwu Weng and Dr. Shaoli Huang
    Southeast University
    2016.09 - 2020.06, Nanjing
    I received my B.S Degree of Geographic Information Science in 2020. GPA: 3.88/4.0, Rank: 1/26.
    2018.11 - 2020.06, Nanjing
    Research assistant in Research Center of Complex Transportation Network (TLab).

    Awards and Honors

  • 2023: Lei Jun Excellence Scholarship (100,000RMB¥, Top 0.1‰)
  • 2023: Wang Zhizhuo Innovative Talent Award (8,000RMB¥, Top 1%)
  • 2022: National Scholarship (Highest Honor for Master students in China, 10,000RMB¥, Top 3%)
  • 2022: First-class Scholarship of Wuhan University (5,000RMB¥, Top 10%)
  • 2021: First-class Scholarship of Wuhan University (5,000RMB¥, Top 10%)
  • 2021: 1st Runner-up of ICCV 2021 MMVRAC Challenge (Track 2 and Track 3)
  • 2020: Outstanding graduates of Southeast University (Top 3%)
  • 2019: Meritorious Winner - Mathematical Contest In Modeling & Interdisciplinary Contest In Modeling, 2019
  • 2018: National Scholarship (Highest Honor for undergraduates in China, 8,000RMB¥, Top 3%)

  • This homepage is designed based on Jon Barron's website and deployed on Github Pages. Last updated: Jun. 17, 2024
    © 2024 Jiaxu Zhang