Proposal of a New Online Controller for Grid2Op Based on Structural State Dynamics (DN / CCE)

[Zhenis-ref]

This is not a request to add a specific feature. Rather, I would like to share a research implementation that may be of interest to the Grid2Op community and to ask for your technical feedback.

Hello Grid2Op development team,
First of all, thank you for developing and maintaining Grid2Op. It has proven to be an excellent research platform for studying decision-making algorithms in power systems and for performing objective comparisons between different control approaches.
Over the past several weeks, we have been investigating a new online control strategy based on a structural representation of the power system state.
Instead of reinforcement learning, the controller continuously evaluates two characteristics of the current network state:

• external nonequilibrium of the system (ΔN);
• internal structural heterogeneity (ΔD).
  These quantities are then used by the "Combinatorial Compression Engine (CCE)" to select control actions through a limited number of predictive simulations.
  The current implementation combines three physical control channels:
• energy storage control;
• generator redispatch;
• network topology control using "set_bus" actions.
  All candidate actions are evaluated using a common structural risk metric and participate in the same online decision-making process.
  During local testing on the official deterministic "l2rpn_idf_2023" scenario (fixed seed), we obtained the following results:
• "Do Nothing baseline:" grid collapse after 9 steps;
• "DN / CCE controller:" successful completion of the entire scenario (300/300 steps).
  The controller evaluates approximately 32 candidate actions per decision step and operates entirely online, without reinforcement learning, offline training, or precomputed action databases.
  Unfortunately, the official CodaBench submission phase had already closed by the time this work was completed, so we were unable to evaluate the controller through the competition infrastructure.
  Nevertheless, we have published the complete implementation and would greatly appreciate your technical feedback on the proposed approach.
  Project repository: https://github.com/Zhenis-ref/DN-Controller-Grid2Op
  We would be particularly interested in your opinion regarding:
• the proposed online decision-making algorithm;
• the structural state representation used by the controller;
• the possibility of evaluating this approach on additional Grid2Op scenarios or benchmarks.
  If you have an opportunity to evaluate the controller using your own testing infrastructure or additional Grid2Op environments, we would greatly appreciate your independent assessment. Since the official competition has already concluded, an evaluation by the Grid2Op development team would be especially valuable. We would welcome any feedback, whether positive or critical, that could help us further improve the approach.
  Thank you once again for your time and for developing such a valuable platform for the research community.
  Best regards,
  Zhengis Tileubay
  Independent Researcher
  Kazakhstan

[BDonnot]

Hello,

What is possible right now is to particiapte to the next edition of the L2RPN competition here: https://www.codabench.org/competitions/14411/ This is the most "open" way to bencharmak all approches that might be used to solve the problem (and not only RL methods :-) )
There is also some prize money for the winners.

And you are "lucky" this competition is also based on the l2rpn_2023 environment, so you can probably reuse what you did and see how it compares to other approaches.

Longer term, if you think your project can be maintain on the long run, we can discuss the creation of a dedicated repository under the grid2op organization, for example "https://github.com/grid2op/dn_cce_agent.git" (this is an example of a name). Doing so comes with more visibility for your work but also with some constraints (it should remain open source, be maintained, etc.)

[Zhenis-ref]

Hello,

Thank you very much for your valuable feedback and for pointing me to the L2RPN 2026 Challenge.
It is an excellent and highly relevant benchmark, and I am now preparing my submission for the competition on CodaBench.
I also greatly appreciate your long-term suggestion regarding a dedicated repository within the Grid2Op organization. It would be a great honor to collaborate with the Grid2Op team if the project proves useful to the community.
My goal is to explore a scalable alternative to reinforcement-learning-based controllers for real-time power system control, particularly in situations where combinatorial complexity becomes a limiting factor.
I will submit the controller to the L2RPN 2026 Challenge and look forward to continuing our discussion as the project progresses.
Thank you again for your encouragement and support.

Best regards,
Zhengis Tileubay

[Zhenis-ref]

Hello,

I would like to briefly update you on what happened after your suggestion to test the DN/CCE controller in the L2RPN 2026 Challenge.
I participated in the competition and made several successive submissions of the controller. Unfortunately, I was not able to obtain a valid positive evaluation result on Codabench.
What is interesting is that the submissions did not fail during initialization. They were imported successfully, the Grid2Op environment and LightSim2Grid backend were initialized successfully, and the agent actually completed a significant number of episodes. The generated result archives contained complete data for many successfully processed episodes.
However, several substantially different versions of the submission repeatedly showed almost exactly the same behavior. The evaluation proceeded through a large number of episodes, then stopped; the directory corresponding to the final episode remained completely empty; and Codabench returned exactly the same final score of -210.
I made several changes to the implementation during these attempts, including simplifying the agent state and reducing possible memory-related overhead. Nevertheless, the overall behavior on Codabench remained unchanged. At the same time, the controller continued to operate correctly in our local Grid2Op tests.
Without access to the backend execution information, I do not want to speculate about the specific technical reason for this behavior. However, after several repeated attempts, it seems possible that the competition evaluation pipeline or its computational constraints may not be fully suited to the operating characteristics of this particular type of controller.
Unlike a conventional trained RL policy that primarily performs inference at each step, DN/CCE is an online search-based controller. It evaluates a limited set of possible actions through predictive Grid2Op simulations and uses the structural variables ΔN and ΔD to dynamically regulate this search. Its computational model is therefore somewhat different from that of a typical trained policy.
In any case, I consider the attempt to participate in the competition useful. It showed that simply submitting the controller to the existing competition infrastructure may not be the most suitable way to continue evaluating this approach.
For this reason, I would like to return to your earlier suggestion:
“Longer term, if you think your project can be maintain on the long run, we can discuss the creation of a dedicated repository under the grid2op organization.”
I remain very interested in this possibility, and I am prepared to maintain the project as a long-term open-source effort.
If your offer is still open, I would be very glad to begin discussing the creation of such a dedicated repository and how the DN/CCE controller should be structured for it.
I believe this could also provide a more suitable environment for evaluating the controller across different Grid2Op environments and benchmarks, without tying its further development exclusively to the constraints of a particular competition.
The goal is not to replace the existing RL or optimization approaches in Grid2Op, or to position DN/CCE against them, but rather to provide an additional independently reproducible approach to online control that researchers could test and compare with other methods.
If this is still of interest to the Grid2Op team, I would be glad to continue this work and adapt the existing repository according to any technical and maintenance requirements you consider appropriate.
Thank you again for your previous support and for giving me the opportunity to test this approach through the L2RPN competition.

Best regards,
Zhengis Tileubay

[BDonnot]

Hello

Sorry you could not make it to the next phase.

I dived into the codebase (of the competition) and found:

FAIL_SCORE_TIMEOUT = -130.0
FAIL_SCORE_INPUT = -200.0
FAIL_SCORE_JSON_MISSING = -210.0
FAIL_SCORE_ONE_SCEN_MISSING = -220.0

Your -210 score clearly show something went wrong with your submission.
The error is related to missing exported files. And I believe this is a codabench behavior. When a submission times out, no files are exported back to the "scoring program" that processes the submission and assign it a score.
To have all of that work, we made actually 2 time outs.
The hard time out for codabench (calibrated such that if you reach it, then your submission is necessarily too long). And the "agent timeout" which triggers when your submission manage to go to the end of all episodes and then check from the metadata that your agent has not overuse it's computation budget.
If you encounter a "hard time out" then no files are present hence the -200 or -210 or -220 errors.

That being said it does not mean anything about your agent in general. It means it's probably not fitted to this competition.

I'll spend some time to process all your message ASAP and if you like you can reach out to me by mail for the repo under the grid2op organization of you are still interested of course.

Best

Benjamin

Edit: I wrote the above message without having read yours first and I think we totally agree 😊