Add model metadata, paper links, and sample usage

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +49 -3
README.md CHANGED
@@ -1,3 +1,49 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ library_name: transformers
4
+ pipeline_tag: other
5
+ tags:
6
+ - motion-generation
7
+ - human-motion
8
+ - multimodal
9
+ ---
10
+
11
+ # ReactMotion: Generating Reactive Listener Motions from Speaker Utterance
12
+
13
+ [**Project Page**](https://reactmotion.github.io) | [**Paper**](https://huggingface.co/papers/2603.15083) | [**GitHub**](https://github.com/awakening-ai/ReactMotion)
14
+
15
+ ReactMotion is a unified generative framework that jointly models text, audio, emotion, and motion to generate naturalistic listener body motions that appropriately respond to a speaker's utterance. It addresses the inherently non-deterministic nature of human reactions by training on the **ReactMotionNet** dataset with preference-based objectives.
16
+
17
+ ## Model Description
18
+ - **Task:** Generating reactive listener body motions from speaker utterance (text + audio + emotion).
19
+ - **Architecture:** Built on a T5-base backbone, processing multi-modal conditions to output motion token sequences.
20
+ - **Key Features:** Supports various conditioning modes (Text, Audio, Emotion) and utilizes a preference-oriented evaluation protocol.
21
+
22
+ ## Sample Usage
23
+
24
+ You can download the model weights using the `huggingface_hub` library as described in the official repository:
25
+
26
+ ```python
27
+ from huggingface_hub import snapshot_download
28
+
29
+ # Download the model to a local directory
30
+ snapshot_download(repo_id="awakening-ai/ReactMotion1.0", local_dir="models/ReactMotion1.0")
31
+ ```
32
+
33
+ For full inference instructions, including the motion VQ-VAE setup and Gradio demo, please refer to the [GitHub repository](https://github.com/awakening-ai/ReactMotion).
34
+
35
+ ## Citation
36
+
37
+ If you find this work useful for your research, please cite:
38
+
39
+ ```bibtex
40
+ @misc{luo2026reactmotiongeneratingreactivelistener,
41
+ title={ReactMotion: Generating Reactive Listener Motions from Speaker Utterance},
42
+ author={Cheng Luo and Bizhu Wu and Bing Li and Jianfeng Ren and Ruibin Bai and Rong Qu and Linlin Shen and Bernard Ghanem},
43
+ year={2026},
44
+ eprint={2603.15083},
45
+ archivePrefix={arXiv},
46
+ primaryClass={cs.CV},
47
+ url={https://arxiv.org/abs/2603.15083},
48
+ }
49
+ ```