FrankenMotion Icon FrankenMotion: Part-level Human Motion Generation and Composition

Chuqiao Li1,2, Xianghui Xie1,2,3, Yong Cao1,2, Andreas Geiger1,2, Gerard Pons-Moll1,2,3
1University of Tübingen, 2Tübingen AI Center,
3Max Planck Institute for Informatics, Saarland Informatics Campus
Paper arXiv Code Data

FrankenMotion enables fine-grained control over individual body parts while maintaining global semantic coherence.

Abstract

TL;DR: We introduce the first framework for atomic, part-level motion control, powered by our new hierarchical Frankenstein dataset (39h) constructed via LLMs.

Human motion generation from text prompts has made remarkable progress in recent years. However, existing methods primarily rely on either sequence-level or action-level descriptions due to the absence of fine-grained, part-level motion annotations. This limits their controllability over individual body parts.

In this work, we construct a high-quality motion dataset with atomic, temporally-aware part-level text annotations, leveraging the reasoning capabilities of large language models (LLMs). Unlike prior datasets that either provide synchronized part captions with fixed time segments or rely solely on global sequence labels, our dataset captures asynchronous and semantically distinct part movements at fine temporal resolution.

Based on this dataset, we introduce a diffusion-based part-aware motion generation framework, namely FrankenMotion, where each body part is guided by its own temporally-structured textual prompt. This is, to our knowledge, the first work to provide atomic, temporally-aware part-level motion annotations and have a model that allows motion generation with both spatial (body part) and temporal (atomic action) control.

Experiments demonstrate that FrankenMotion outperforms all previous baseline models adapted and retrained for our setting, and our model can compose motions unseen during training.

🖼️ Frankenstein Gallery: The Frankenstein Dataset

Our work introduces the Frankenstein Dataset, the largest dataset providing hierarchical, temporally-aware annotations for 3D human motion. Generated automatically using our FrankenAgent, this dataset features high-quality, diverse motion annotations.

Sequence-Level

Top Text (Global)

Action-Level

Bottom Bar (Segments)

Part-Level

Colored Body Parts

Method Overview

FrankenMotion Architecture

Our model is a transformer-based diffusion model that can be input conditioned on a) sequence level prompt, b) action-level prompt and c) part-level prompt. After training with our paired data of motion and structured multi-granularity text annotations, it learns the essential motion elements and how to compose them into complex motions.

👑 Qualitative Comparison

To the best of our knowledge, no prior method is able to accomplish this complex task of hierarchical control. For fair comparison, we adapted state-of-the-art methods (UniMotion, DART, and STMC) to include part control and retrained them on our Frankenstein dataset.

🧪 Ablation Studies: The Value of Hierarchical Control

These ablations highlight the importance of our hierarchical conditioning. Notice how motion quality degrades as conditioning layers are removed.

BibTeX

@article{frankenmotion2026,
  title={FrankenMotion: Part-level Human Motion Generation and Composition},
  author={Li, Chuqiao and Xie, Xianghui and Cao, Yong and Geiger, Andreas and Pons-Moll, Gerard},
  journal={arxiv},
  year={2026}
}