Stability Constrained Reinforcement Learning for Real-Time Voltage Control in Distribution Systems

This repository contains source code necessary to reproduce the results presented in the paper with the same title. It also provides an OpenAI Gym environment for training various Reinforcement Learning algorithms in the IEEE-123 bus and IEEE-13 bus test case.
Authors: Jie Feng, Yuanyuan Shi, Guannan Qu, Steven H. Low, Anima Anandkumar, Adam Wierman
This paper is published in Transactions on Control of Network Systems (TCNS) (https://ieeexplore.ieee.org/abstract/document/10336939).

News

• Feb 2026: New related paper available on arXiv: Efficient Policy Adaptation for Voltage Control Under Unknown Topology Changes (https://arxiv.org/abs/2602.10355). This work will be presented at PSCC 2026.

hyper-parameters

We use a more conservative deadband for both liner and monotone neural network controller following Califronia standard. (3% instead of 5%)

Environment Setup Notes

• Current environment code in this repository uses an older pandapower interface:
  • pp_net = pp.converter.from_mpc(pp_model_pth, casename_mpc_file='case_mpc')
• For newer pandapower versions, use:
  • from pandapower.converter.matpower import from_mpc
  • pp_net = from_mpc(pp_model_pth, casename_mpc_file='case_mpc')
• Install numba in your environment for newer pandapower compatibility/performance:
  • pip install numba
• If your local setup is already built around the older pandapower API, keep the current code path unchanged.

How to train

python train_DDPG.py --algorithm safe-ddpg --env_name 13bus --status train
#customize your own algorithm, env_name and status
#env: 13bus,123bus,13bus3p
#algorithm: linear, safe-ddpg,ddpg
#status: train,test
#check points are available

Real World Data

The real world PV generation and load profile is available. The trajectory can be ploted with test_real_data.py The testing code creates a one-day trajectory (Sampling frequency 1 Hz). It will take a while to generate the plot.

Citation

If you find our code helpful, please cite our paper! :)

@ARTICLE{10336939,
  author={Feng, Jie and Shi, Yuanyuan and Qu, Guannan and Low, Steven H. and Anandkumar, Anima and Wierman, Adam},
  journal={IEEE Transactions on Control of Network Systems}, 
  title={Stability Constrained Reinforcement Learning for Decentralized Real-Time Voltage Control}, 
  year={2023},
  volume={},
  number={},
  pages={1-12},
  doi={10.1109/TCNS.2023.3338240}}

Related work:

@article{feng2026efficient,
  title={Efficient Policy Adaptation for Voltage Control Under Unknown Topology Changes},
  author={Feng, Jie and Shi, Yuanyuan and Deka, Deepjyoti},
  journal={arXiv preprint arXiv:2602.10355},
  year={2026}
}