[NeurIPS 2024 Spotlight✨] Official Codes for Bridge the Points: Graph-based Few-shot Segment Anything Semantically

Anqi Zhang¹,   Guangyu Gao^1*,   Jianbo Jiao²,   Chi Harold Liu¹,   Yunchao Wei³

¹School of Computer Science, Beijing Institute of Technology
²The MIx group, School of Computer Science, University of Birmingham
³WEI Lab, Institute of Information Science, Beijing Jiaotong University

Overview

A Graph-based approach that streamlines prompt selection, addresses mask ambiguity, improves generalization of SAM-based Few-shot Semantic Segmentation.

Advantages

• Training-Free
• External-Hyperparameter-Free
• No iterative mask generation
• Fast inference within 2s per image (NVIDIA RTX 2080Ti)
• New state-of-the-art performance
• Effective in various domains

Todo List

• Detailed discription
• Gradio Demo
• Colab Demo

Installation

See installation instructions.

Getting Started

See Preparing Datasets for GF-SAM.

See Getting Started with GF-SAM.

Visualization

Post Gating Process

Visualization on COCO-20ⁱ

One-shot Semantic Segmentation

One-shot Part Segmentation

One-shot Segmentation on Various Domains

Citation

@inproceedings{zhang2024bridge,
    title={Bridge the Points: Graph-based Few-shot Segment Anything Semantically},
    author={Zhang, Anqi and Gao, Guangyu and Jiao, Jianbo and Liu, Chi Harold and Wei, Yunchao},
    journal={NeurIPS},
    year={2024}
}

Acknowledgement

Our implementation is based on these repositories: SAM, DINOv2, SegGPT, HSNet, Matcher, SCCNet and detectron2. Thanks for their great work!