Skip to content

About

WACV 2027 | PlanarCityLoc: Breaking Silhouette Ambiguity in LoD Aerial Localization via 3D Planes

Resources

Stars

3 stars

Watchers

0 watching

Forks

Repository files navigation

Animated drone matching building planes

PlanarCityLoc: Breaking Silhouette Ambiguity in LoD Aerial Localization via 3D Planes

WACV 2027

Project page Paper on arXiv Video Poster Datasets License: CC BY-NC 4.0

Oussema Dhaouadi1,2,4,5,6, Shao Jie Hu Chen1,2, Jakob Gollreiter1,2,
Samir Jusufi1,3, Nikolas Rieger1,2,7, Caroline Wendlandt1,2,
Christian Wender1,8, Jingfeng Mao1,2,4, Guangming Wang1,4,
Zuria Bauer5, Marc Pollefeys5,9, Daniel Cremers2,6, Olaf Wysocki1,4

1 TUM.ai   2 Technical University of Munich   3 LMU Munich
4 University of Cambridge   5 ETH Zürich
6 Munich Center for Machine Learning
7 Konrad Zuse School of Excellence in Reliable AI (relAI)
8 BMW Group   9 Microsoft

PlanarCityLoc: silhouette alignment, 3D plane grounding, and conceptual pose-scoring curves

PlanarCityLoc estimates the 6-DoF pose of an aerial image in a textureless level-of-detail (LoD) city model. Silhouette alignment provides a coarse pose estimate. Explicit correspondences between image regions and 3D building planes then refine the camera pose.

🛰️ Method

  • Planar grounding: associate visible image regions with 3D building planes.
  • Geometry-aware search: use image masks, edges, and depth to initialize pose.
  • Training-free refinement: match facade and roof regions, then refine the pose with geometric constraints.

📊 Results

Joint recall requires both translation and rotation errors to be below the listed threshold.

Evaluation split Queries Recall @ 2 m / 2° Median translation Median rotation
UAVD4L-LoDv2 in-Traj 1,604 75.4% 1.28 m 0.68°
UAVD4L-LoDv2 out-of-Traj 2,192 80.6% 1.26 m 0.69°
Swiss-EPFLv2 in-Place 712 20.6% 3.70 m 1.90°
Swiss-EPFLv2 out-of-Place 379 48.5% 1.98 m 1.21°

🛠️ Setup

The reference environment uses Linux, Python 3.10, PyTorch 2.6, CUDA 12.4, and an NVIDIA GPU.

git clone https://github.com/tum-vision/PlanarCityLoc.git
cd PlanarCityLoc

conda create -n planarcityloc python=3.10 -y
conda activate planarcityloc
conda install -y -c nvidia cuda-nvcc=12.4.131 cuda-cudart-dev=12.4.127
conda install -y -c conda-forge cxx-compiler

pip install torch==2.6.0 torchvision==0.21.0 \
  --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt
pip install "setuptools<81" wheel ninja
pip install git+https://github.com/NVlabs/nvdiffrast.git@abb07ca0358f3a21c3942b50c54aa1eacd329af9 \
  --no-build-isolation
GIT_LFS_SKIP_SMUDGE=1 pip install --no-deps \
  git+https://github.com/microsoft/MoGe.git@07444410f1e33f402353b99d6ccd26bd31e469e8

Run commands from the repository root with PYTHONPATH=..

🗺️ Data and models

Benchmark assets and model weights are not hosted in this repository. Obtain them from their original providers, follow the providers' terms, and arrange the files as described in the dataset guide, which includes source links and attribution. The setup helper links locally available benchmark files, query images, and checkpoints:

bash scripts/setup_data_links.sh /path/to/extracted/data /path/to/checkpoints

After accepting the upstream SAM 3 model terms, prepare the compact building-prompt checkpoint with:

PYTHONPATH=. python scripts/prepare_sam3_fast.py

The script downloads the full checkpoint through your authorized Hugging Face account, or accepts a local checkpoint with --full-checkpoint /path/to/sam3.pt. MoGe-2 downloads automatically at the revision pinned in the configuration. See License and acknowledgements for component licenses.

🚀 Run

A small smoke run on UAVD4L-LoDv2 in-Traj:

PYTHONPATH=. python pipeline.py \
  --dataset uavd4l \
  --data_root data/UAVD4L-LoD/inTraj \
  --batchsize 8 --no-crop --n 3 \
  --out outputs/demo_intraj

For the reported full-split evaluation, use batch size 512 and a new output directory:

PYTHONPATH=. python pipeline.py \
  --dataset uavd4l \
  --data_root data/UAVD4L-LoD/inTraj \
  --batchsize 512 --no-crop \
  --out outputs/eval_uavd4l_intraj

Each run records its configuration, source hashes, inputs, and environment. To verify a run and recompute metrics from its saved poses:

PYTHONPATH=. python scripts/summarize_results.py outputs/eval_uavd4l_intraj

Full-split commands, dataset layout, and evaluation details are in docs/REPRODUCIBILITY.md. Component studies are described in docs/ABLATIONS.md.

The evaluation protocol is in configs/evaluation.json, with expected full-split results in docs/benchmark_results.json.

📝 Citation

@inproceedings{dhaouadi2027planarcityloc,
  title     = {{PlanarCityLoc}: Breaking Silhouette Ambiguity in {LoD} Aerial Localization via 3{D} Planes},
  author    = {Dhaouadi, Oussema and Hu Chen, Shao Jie and Gollreiter, Jakob and Jusufi, Samir and Rieger, Nikolas and Wendlandt, Caroline and Wender, Christian and Mao, Jingfeng and Wang, Guangming and Bauer, Zuria and Pollefeys, Marc and Cremers, Daniel and Wysocki, Olaf},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
  year      = {2027},
  url       = {https://arxiv.org/abs/2610.26729}
}

⚖️ License and acknowledgements

PlanarCityLoc code is released under CC BY-NC 4.0. Third-party software, models, and benchmark data retain their respective terms.

  • SAM 3: Copyright Meta Platforms, Inc. and affiliates. The bundled source and cached building prompt feature in assets/sam3_building_language_features.b85 are subject to the SAM License. Full checkpoints are obtained separately.
  • EoMT: Copyright Mobile Perception Systems Lab at TU/e. The bundled source retains its MIT License.
  • MoGe / MoGe-2: Installed from Microsoft's repository at the revision pinned above. MoGe is MIT-licensed, and its bundled DINOv2 component is Apache-2.0-licensed. See the upstream repository for complete notices.
  • nvdiffrast and packages in requirements.txt retain their upstream licenses and notices.

Model checkpoints and query images are not included and remain subject to their providers' terms. Dataset sources and attribution, including ©swisstopo for the Swiss geodata, are listed in the dataset guide.

About

WACV 2027 | PlanarCityLoc: Breaking Silhouette Ambiguity in LoD Aerial Localization via 3D Planes

Resources

Stars

3 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages