Skip to content

Commit 031576d

Browse files
liuwenran刘文然
liuwenran
and
刘文然
authored
[Feature] Add animated drawings at projects (open-mmlab#1837)
* add file * add config * make it simple * add config * add main * fix lint * delete images * delete config * change version * version --------- Co-authored-by: 刘文然 <PJLAB\[email protected]>
1 parent 6903233 commit 031576d

File tree

10 files changed

+1451
-1
lines changed

10 files changed

+1451
-1
lines changed

‎.pre-commit-config.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ repos:
1818
- id: trailing-whitespace
1919
exclude: ^dicts/
2020
- id: check-yaml
21+
exclude: ^projects/animated_drawings/configs/retarget/
2122
- id: end-of-file-fixer
2223
exclude: ^dicts/
2324
- id: requirements-txt-fixer

‎mmagic/version.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Copyright (c) Open-MMLab. All rights reserved.
22

3-
__version__ = '1.0.0'
3+
__version__ = '1.0.1.dev0'
44

55

66
def parse_version_info(version_str):

‎projects/animated_drawings/README.md

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# Animated Drawings (SIGGRAPH'2023)
2+
3+
> [A Method for Animating Children's Drawings of The Human Figure](https://arxiv.org/abs/2303.12741)
4+
5+
> **Task**: Drawing
6+
7+
<!-- [ALGORITHM] -->
8+
9+
## Abstract
10+
11+
<!-- [ABSTRACT] -->
12+
13+
Children’s drawings have a wonderful inventiveness, creativity, and variety to them. We present a system that automatically animates children’s drawings of the human figure, is robust to the variance inherent in these depictions, and is simple and straightforward enough for anyone to use. We demonstrate the value and broad appeal of our approach by building and releasing the Animated Drawings Demo, a freely available public website that has been used by millions of people around the world. We present a set of experiments exploring the amount of training data needed for fine-tuning, as well as a perceptual study demonstrating the appeal of a novel twisted perspective retargeting technique. Finally, we introduce the Amateur Drawings Dataset, a first-of-its-kind annotated dataset, collected via the public demo, containing over 178,000 amateur drawings and corresponding user-accepted character bounding boxes, segmentation masks, and joint location annotations.
14+
15+
<!-- [IMAGE] -->
16+
17+
<div align=center >
18+
<img src="https://user-images.githubusercontent.com/6675724/219223438-2c93f9cb-d4b5-45e9-a433-149ed76affa6.gif" width="800"/>
19+
</div >
20+
21+
## Quick Start
22+
23+
### 1. Install Animated Drawings
24+
25+
```shell
26+
cd mmagic/projects/animated_drawings
27+
pip install -e git+https://github.com/facebookresearch/AnimatedDrawings.git#egg=animated_drawings
28+
```
29+
30+
### 2. Download resources
31+
32+
```shell
33+
cd mmagic/projects/animated_drawings
34+
mkdir resources
35+
# download image
36+
wget -O sakuragi.png https://user-images.githubusercontent.com/12782558/236157945-452fb9d0-e02e-4f36-8338-34f0ca0fe962.png
37+
# download mask image
38+
wget -O sakuragi_mask.png https://user-images.githubusercontent.com/12782558/236157965-539a5467-edae-40d0-a9da-7bb5906bcdc4.png
39+
# download background image
40+
wget -O basketball_playground.png https://user-images.githubusercontent.com/12782558/236190727-0e456482-2ae3-4304-9e6c-6ba7d319ea71.png
41+
```
42+
43+
### 3. Generate character
44+
45+
By running following codes, you will get texture and mask images for animated rendering in characters/slamdunk directory.
46+
47+
```shell
48+
cd mmagic/projects/animated_drawings
49+
python tools/generate_character.py
50+
```
51+
52+
### 3. Generate video
53+
54+
By running following codes, you will get a Sakuragi moving like a zombie.
55+
56+
```shell
57+
cd mmagic/projects/animated_drawings
58+
python tools/generate_video.py
59+
```
60+
61+
The output video will be saved at resources dir, and it looks like this:
62+
63+
<div align=center >
64+
<img src="https://user-images.githubusercontent.com/12782558/236162056-c9a65baa-89c4-4cb3-84da-7777f5f21757.gif" width="512"/>
65+
</div >
66+
67+
## Citation
68+
69+
```bibtex
70+
@misc{smith2023method,
71+
title={A Method for Animating Children's Drawings of the Human Figure},
72+
author={Harrison Jesse Smith and Qingyuan Zheng and Yifei Li and Somya Jain and Jessica K. Hodgins},
73+
year={2023},
74+
eprint={2303.12741},
75+
archivePrefix={arXiv},
76+
primaryClass={cs.CV}
77+
}
78+
```

‎projects/animated_drawings/bvh/zombie.bvh

Lines changed: 946 additions & 0 deletions
Large diffs are not rendered by default.

‎projects/animated_drawings/characters/slamdunk/char_cfg.yaml

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
height: 512
2+
skeleton:
3+
- loc:
4+
- 260
5+
- 230
6+
name: root
7+
parent: null
8+
- loc:
9+
- 260
10+
- 230
11+
name: hip
12+
parent: root
13+
- loc:
14+
- 257
15+
- 93
16+
name: torso
17+
parent: hip
18+
- loc:
19+
- 250
20+
- 54
21+
name: neck
22+
parent: torso
23+
- loc:
24+
- 213
25+
- 95
26+
name: right_shoulder
27+
parent: torso
28+
- loc:
29+
- 172
30+
- 157
31+
name: right_elbow
32+
parent: right_shoulder
33+
- loc:
34+
- 145
35+
- 228
36+
name: right_hand
37+
parent: right_elbow
38+
- loc:
39+
- 302
40+
- 88
41+
name: left_shoulder
42+
parent: torso
43+
- loc:
44+
- 342
45+
- 150
46+
name: left_elbow
47+
parent: left_shoulder
48+
- loc:
49+
- 373
50+
- 212
51+
name: left_hand
52+
parent: left_elbow
53+
- loc:
54+
- 231
55+
- 230
56+
name: right_hip
57+
parent: root
58+
- loc:
59+
- 212
60+
- 341
61+
name: right_knee
62+
parent: right_hip
63+
- loc:
64+
- 192
65+
- 454
66+
name: right_foot
67+
parent: right_knee
68+
- loc:
69+
- 290
70+
- 231
71+
name: left_hip
72+
parent: root
73+
- loc:
74+
- 303
75+
- 338
76+
name: left_knee
77+
parent: left_hip
78+
- loc:
79+
- 325
80+
- 452
81+
name: left_foot
82+
parent: left_knee
83+
width: 512

‎projects/animated_drawings/configs/motion/zombie.yaml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
filepath: bvh/zombie.bvh
2+
start_frame_idx: 0
3+
end_frame_idx: 320
4+
groundplane_joint: LeftFoot
5+
forward_perp_joint_vectors:
6+
- - LeftShoulder
7+
- RightShoulder
8+
- - LeftUpLeg
9+
- RightUpLeg
10+
scale: 0.025
11+
up: +z

‎projects/animated_drawings/configs/mvc/slamdunk.yaml

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
scene:
2+
ANIMATED_CHARACTERS:
3+
- character_cfg: characters/slamdunk/char_cfg.yaml
4+
motion_cfg: configs/motion/zombie.yaml
5+
retarget_cfg: configs/retarget/fair1_spf.yaml
6+
view:
7+
CAMERA_POS: [1.6, 1.7, 4.0]
8+
CAMERA_FWD: [0.0, 0.5, 2.0]
9+
BACKGROUND_IMAGE: resources/basketball_playground.png
10+
11+
controller:
12+
MODE: video_render
13+
OUTPUT_VIDEO_PATH: ./resources/slamdunk_video.mp4
14+
OUTPUT_VIDEO_CODEC: mp4v

‎projects/animated_drawings/configs/retarget/fair1_spf.yaml

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
char_starting_location: [0.0, 0.0, 0.0]
2+
bvh_projection_bodypart_groups:
3+
- bvh_joint_names:
4+
- RightShoulder
5+
- RightArm
6+
- RightForeArm
7+
- RightHand
8+
- RightHandEnd
9+
- LeftShoulder
10+
- LeftArm
11+
- LeftForeArm
12+
- LeftHand
13+
- LeftHandEnd
14+
method: sagittal
15+
name: Upper Limbs
16+
- bvh_joint_names:
17+
- RightUpLeg
18+
- RightLeg
19+
- RightFoot
20+
- RightToeBase
21+
- LeftUpLeg
22+
- LeftLeg
23+
- LeftFoot
24+
- LeftToeBase
25+
method: pca
26+
name: Lower Limbs
27+
- bvh_joint_names:
28+
- Hips
29+
- Spine
30+
- Spine1
31+
- Spine2
32+
- Spine3
33+
- Neck
34+
- Head
35+
method: frontal
36+
name: Trunk
37+
char_bodypart_groups:
38+
- bvh_depth_drivers:
39+
- Hips
40+
char_joints:
41+
- right_shoulder
42+
- left_shoulder
43+
- right_hip
44+
- left_hip
45+
- hip
46+
- torso
47+
- neck
48+
- bvh_depth_drivers:
49+
- LeftHand
50+
char_joints:
51+
- left_elbow
52+
- left_hand
53+
- bvh_depth_drivers:
54+
- RightHand
55+
char_joints:
56+
- right_elbow
57+
- right_hand
58+
- bvh_depth_drivers:
59+
- LeftFoot
60+
char_joints:
61+
- left_knee
62+
- left_foot
63+
- bvh_depth_drivers:
64+
- RightFoot
65+
char_joints:
66+
- right_knee
67+
- right_foot
68+
char_bvh_root_offset:
69+
bvh_projection_bodypart_group_for_offset: Lower Limbs
70+
bvh_joints:
71+
- - RightFoot
72+
- RightLeg
73+
- RightUpLeg
74+
- - LeftFoot
75+
- LeftLeg
76+
- LeftUpLeg
77+
char_joints:
78+
- - left_foot
79+
- left_knee
80+
- left_hip
81+
- - right_foot
82+
- right_knee
83+
- right_hip
84+
char_joint_bvh_joints_mapping:
85+
left_elbow: !!python/tuple
86+
- LeftArm
87+
- LeftForeArm
88+
left_foot: !!python/tuple
89+
- LeftLeg
90+
- LeftFoot
91+
left_hand: !!python/tuple
92+
- LeftForeArm
93+
- LeftHand
94+
left_knee: !!python/tuple
95+
- LeftUpLeg
96+
- LeftLeg
97+
torso: !!python/tuple
98+
- Hips
99+
- Spine3
100+
neck: !!python/tuple
101+
- Hips
102+
- Neck
103+
right_elbow: !!python/tuple
104+
- RightArm
105+
- RightForeArm
106+
right_foot: !!python/tuple
107+
- RightLeg
108+
- RightFoot
109+
right_hand: !!python/tuple
110+
- RightForeArm
111+
- RightHand
112+
right_knee: !!python/tuple
113+
- RightUpLeg
114+
- RightLeg
115+
char_runtime_checks:
116+
- - above
117+
- neck
118+
- right_shoulder
119+
- left_shoulder

0 commit comments

Comments
 (0)