Instance-Dependent Continuous-Time Reinforcement Learning via Maximum Likelihood Estimation

This repository contains code for Algorithm 3 of Instance-Dependent Continuous-Time Reinforcement Learning via Maximum Likelihood Estimation, accepted to ICML 2026.

The implementation is built on the continuous-time model-based reinforcement learning framework from here.

Overview

This code implements LAG-CT-MLE, a model-based continuous-time reinforcement learning algorithm using maximum likelihood estimation and a randomized measurement schedule. The repository is intentionally focused on the experiments needed for Algorithm 3.

Supported environments:

• Pendulum
• CartPole
• Acrobot

Installation

Create an environment and install dependencies:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Python 3.10 or newer is recommended. The code runs on CPU by default unless CUDA is available. You can also specify a device with --device.

Running

A sample running command for environment is:

python train_lag_ct_mle.py --env pendulum
python train_lag_ct_mle.py --env cartpole
python train_lag_ct_mle.py --env acrobot

Useful optional flags:

• --seed: random seed.
• --device: cpu, cuda, or another PyTorch device string.
• --preset quick: small smoke-test configuration.
• --plot: save rollout diagnostic plots.
• --run-name: custom output filename prefix.

Outputs

Each run saves a checkpoint to:

outputs/<run-name>.pkl

When --plot is enabled, diagnostic images are saved next to the checkpoint.

Repository Structure

train_lag_ct_mle.py  Main training script
ctrl/               LAG-CT-MLE training, policy, value, and dynamics modules
envs/               Continuous-time control environments
utils/              Shared neural-network and interpolation utilities

Acknowledgement

This codebase builds on the continuous-time model-based reinforcement learning framework of Yildiz, Heinonen, and Lahdesmaki. We thank the original authors for making their work available.

Citation

If you find this repository useful, please cite:

@inproceedings{
zhao2025instance,
title={Instance-Dependent Continuous-Time Reinforcement Learning via Maximum Likelihood Estimation},
author={Zhao, Runze and Yu, Yue and Wang, Ruhan and Huang, Chunfeng and Zhou, Dongruo},
booktitle={Forty-third International Conference on Machine Learning},
year={2026},
url={https://openreview.net/forum?id=aXvTMnyeLj}
}