ContactDexNet: Multi-fingered Robotic Hand Grasping in Cluttered Environments through Hand-object Contact Semantic Mapping

Lei Zhang, Kaixin Bai, Guowen Huang, Zhenshan Bing, Zhaopeng Chen, Alois Knoll, Jianwei Zhang

University of Hamburg, Agile Robots, Technical University of Munich

Example of ContactDexNet dataset (previous FFHClutteredGrasping dataset)

Grasping Candidates and Contact Distance Map.	Cluttered Scene
Blue: Candidate under Collision, Green: Positive Candidate, Red: Negative Candidate

Contact Semantic Map	Affordance Map

Visualize example dataset

visualize example scene with cluttered grasping data of multi-fingered robotic hand with following modalities:

• collision score
• grasping quality
• contact distance map
• contact semantic map
• affordance information

cd example_dataset
# visualize cluttered scene with grasp candidates, and corresponding collision score, grasp qualities, contact distance and semantic information
python visualize_scene.py

# visualize the affordance information
python visualize_affordance.py

P.S: In visualization, the model of robotic hand is a simplied version.

BibTex

If you find our work helpful, please consider citing it:

@article{zhang2024multi,
  title={Multi-fingered robotic hand grasping in cluttered environments through hand-object contact semantic mapping},
  author={Zhang, Lei and Bai, Kaixin and Huang, Guowen and Bing, Zhenshan and Chen, Zhaopeng and Knoll, Alois and Zhang, Jianwei},
  journal={arXiv preprint arXiv:2404.08844},
  year={2024}
}

@misc{zhang2025contactdexnetmultifingeredrobotichand,
      title={ContactDexNet: Multi-fingered Robotic Hand Grasping in Cluttered Environments through Hand-object Contact Semantic Mapping}, 
      author={Lei Zhang and Kaixin Bai and Guowen Huang and Zhenshan Bing and Zhaopeng Chen and Alois Knoll and Jianwei Zhang},
      year={2025},
      eprint={2404.08844},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2404.08844}, 
}