Skullptor: High Fidelity 3D Head Reconstruction in Seconds with Multi-View Normal Prediction

Overview

Skullptor is a pipeline for rapid, high-fidelity 3D head reconstruction based on multi-view normal prediction. It includes a purpose-built view-aware Dense Prediction Transformer and a custom inverse rendering pipeline to produce accurate meshes from a small set of images.

Installation (Linux)

1. Clone the repository

   git clone https://github.com/ubisoft/ubisoft-laforge-Skullptor.git
   cd ubisoft-laforge-Skullptor

2. Clone dependencies into the expected paths (see VENDOR_PATHS in src/lf_skullptor/__init__.py):

   git clone https://github.com/microsoft/DAViD \
       src/lf_skullptor/_vendor/DAViD
   
   git clone https://github.com/Profactor/continuous-remeshing \
       src/lf_skullptor/_vendor/continuous_remeshing

3. Create a Python 3.10 Conda environment with required dependencies matching the system's gpu:

   conda create -y \
       --name skullptor \
       python=3.10 \
       cmake=3.24.3 \
       ninja=1.13.2 \
       libglib \
       libglvnd \
       xorg-libxext \
       xorg-libxrender \
       xorg-libsm

4. Within the Conda environment, install the project's Python dependencies

   (skullptor) python -m pip install -r requirements.txt

5. Download and install nvdiffrast v0.3.5

   (skullptor) wget -P /tmp https://github.com/NVlabs/nvdiffrast/archive/refs/tags/v0.3.5.tar.gz
   (skullptor) tar xvf /tmp/v0.3.5.tar.gz -C /tmp
   (skullptor) mv /tmp/nvdiffrast-* /opt/nvdiffrast
   (skullptor) python3 -m pip install /opt/nvdiffrast --no-build-isolation

6. Install OpenGL drivers for the system:

   apt-get update

   apt-get install -y \
       libgl1 \
       libglib2.0-0 \
       libsm6 \
       libxext6 \
       libxrender1

7. Install the project.

   (skullptor) python -m pip install -e .

8. Download models:

   DAViD models must be downloaded manually and placed in .data/david_models/:

   mkdir -p .data/david_models
   
   # Foreground segmentation model
   wget -P .data/david_models https://facesyntheticspubwedata.z6.web.core.windows.net/iccv-2025/models/foreground-segmentation-model-vitl16_384.onnx
   
   # Surface normal estimation model
   wget -P .data/david_models https://facesyntheticspubwedata.z6.web.core.windows.net/iccv-2025/models/normal-model-vitl16_384.onnx

9. Extract DAViD sub-modules from the downloaded ONNX model: (this make take a while)

(skullptor) python src/lf_skullptor/train/extract_modules.py 

This slices the DAViD onnx normal estimation model into the sub-modules expected by the pipeline.

Training (Optional)

Because the Skullptor model was trained on private data, we cannot redistribute it. If you want to train your own normal estimator using custom data, follow these steps.

Dataset structure

The training dataset must follow this layout:

<root_dir>/
└── <SubjectID>/
    └── <ROMID>/
        ├── textured/           # input RGB renders, one image per camera view
        ├── normals/            # ground truth normal maps, one image per camera view
        └── camera_params.npy   # camera parameters (see below)

Images inside textured/ and normals/ must be sorted in the same order and correspond to the same set of cameras. The number of images in both folders must match the number of cameras in camera_params.npy.

camera_params.npy is a NumPy file containing a dict with the key "mvs": a float array of shape (N, 4, 4) where N is the number of camera views and each (4, 4) matrix is the model-view (world-to-camera) extrinsic matrix.

Run training

Four parameters are required and must be passed on the command line as Hydra overrides:

Parameter	Description
run_name	A name for this training run, used when saving visualisations
dataset.root_dir	Absolute path to the dataset root (the folder containing subject sub-directories)
save_model_path	Absolute path where model checkpoints will be saved (must end in .pth)
tensorboard_log_dir	Directory where TensorBoard logs will be written

(skullptor) python src/lf_skullptor/train/train.py \
    run_name=my_experiment \
    dataset.root_dir=/path/to/dataset \
    save_model_path=/path/to/checkpoints/model.pth \
    tensorboard_log_dir=/path/to/logs

You can also modify these parameters directly through the configs/training.yaml. To monitor training with TensorBoard:

(skullptor) tensorboard --logdir /path/to/logs

Inference

If you didn't train your own model, you can still use the base DAViD as a backbone for surface normal estimation

Set model_type: david in the config file (skullptor_pipeline), or pass the flag at the command line.

python src/lf_skullptor/skullptor_entrypoint.py

The first pass will download the facer landmark detection model used to align the provided camera parameters with a common landmark topology.

Note: Results with the DAViD backbone are typically lower quality than a purpose-trained model because DAViD's normal estimation was not optimised for the Skullptor pipeline.

Output Structure

After processing, the outputs/ folder will contain:

• Normal predictions: Multi-view normal maps for each input image
• Final mesh: High-fidelity 3D head reconstruction