RJE

Knowledge Graph

Before running the system, you need to set up Freebase locally. Please follow the installation instructions.

Code

Our implementation is built upon the following open-source projects:

• PoG
• RDP

We sincerely appreciate the authors for making their code publicly available.

Datasets

Please refer to the RJE/data/datasets/README.md file for detailed instructions on how to download the datasets.

Installation

To install all required dependencies, run:

pip install -r requirements.txt

Running (Example: webqsp Dataset)

1. Train the Relation Retriever

Run the following script to train the relation retriever:

bash ./Retriever/train_retriever.sh

2. Train the Inference Path Ranker

Run the following script to train the path ranker:

bash train_webqsp_ranker.sh

3. Prepare for Running

Prepare necessary files with:

python prepare_running.py

4. Run the Model

To execute the main pipeline, run:

python rje.py \
  --dataset "webqsp" \
  --max_length 2048 \
  --select_num 30 \
  --path_num 10 \
  --LLM_type "gpt35" 

Evaluation

cd eval
python eval.py

Citation

If you find this work useful or are inspired by it, please cite our paper:

@inproceedings{lin2025rje,
  title={RJE: A Retrieval-Judgment-Exploration Framework for Efficient Knowledge Graph Question Answering with LLMs},
  author={Lin, Can and Jiang, Zhengwang and Zheng, Ling and Zhao, Qi and Zhang, Yuhang and Song, Qi and Zhou, Wangqiu},
  booktitle={Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  pages={17288--17305},
  year={2025}
}