Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PEP: Dual Sign Convention #3519

Open
michaelbynum opened this issue Mar 11, 2025 · 6 comments · May be fixed by #3528
Open

PEP: Dual Sign Convention #3519

michaelbynum opened this issue Mar 11, 2025 · 6 comments · May be fixed by #3528

Comments

@michaelbynum
Copy link
Contributor

Summary

I propose the use of the following sign convention for duals in the new solver interfaces.

Given the problem

$$ \begin{align} \min & f(x) \\ \text{s.t.} \\ & c_i(x) = 0 && \forall i \in \mathcal{E} \\ & g_i(x) \leq 0 && \forall i \in \mathcal{U} \\ & h_i(x) \geq 0 && \forall i \in \mathcal{L} \end{align} $$

Define the Lagrangian as

$$ \begin{align} L(x, \lambda, \nu, \delta) = f(x) - \sum_{i \in \mathcal{E}} \lambda_i c_i(x) - \sum_{i \in \mathcal{U}} \nu_i g_i(x) - \sum_{i \in \mathcal{L}} \delta_i h_i(x) \end{align} $$

Then, the KKT conditions are [1]

$$ \begin{align} \nabla_x L(x, \lambda, \nu, \delta) = 0 \\ c(x) = 0 \\ g(x) \leq 0 \\ h(x) \geq 0 \\ \nu \leq 0 \\ \delta \geq 0 \\ \nu_i g_i(x) = 0 \\ \delta_i h_i(x) = 0 \end{align} $$

If a particular solver does not adopt this convention, then we will have to map things back and forth both when retrieving duals and when initializing duals.

This sign convention would have to be based on the (lower, body, upper) representation of constrains rather than the "raw" expression.

[1] Nocedal, Jorge, and Stephen J. Wright, eds. Numerical optimization. New York, NY: Springer New York, 1999.

@mrmundt
Copy link
Contributor

mrmundt commented Mar 11, 2025

This should actually be in #3496 , right?

@michaelbynum
Copy link
Contributor Author

I will start working on this. I'll put the above in our readthedocs docs and add tests to verify that all of the solver interfaces behave the same.

@blnicho
Copy link
Member

blnicho commented Mar 12, 2025

Other related issues: #1957, #2831

@Robbybp
Copy link
Contributor

Robbybp commented Mar 23, 2025

When a user has a maximization problem, I assume the only thing that changes is the sign of $f(x)$ in the Lagrangian?

@michaelbynum
Copy link
Contributor Author

michaelbynum commented Mar 24, 2025

I always forget about maximization! Good catch, @Robbybp!

@michaelbynum
Copy link
Contributor Author

So for maximization problems,

$$ \begin{align} \max & f(x) \\ \text{s.t.} \\ & c_i(x) = 0 && \forall i \in \mathcal{E} \\ & g_i(x) \leq 0 && \forall i \in \mathcal{U} \\ & h_i(x) \geq 0 && \forall i \in \mathcal{L} \end{align} $$

I suggest that the Lagrangian is the same:

$$ \begin{align} L(x, \lambda, \nu, \delta) = f(x) - \sum_{i \in \mathcal{E}} \lambda_i c_i(x) - \sum_{i \in \mathcal{U}} \nu_i g_i(x) - \sum_{i \in \mathcal{L}} \delta_i h_i(x) \end{align} $$

so that the signs of the duals flip:

$$ \begin{align} \nabla_x L(x, \lambda, \nu, \delta) = 0 \\ c(x) = 0 \\ g(x) \leq 0 \\ h(x) \geq 0 \\ \nu \geq 0 \\ \delta \leq 0 \\ \nu_i g_i(x) = 0 \\ \delta_i h_i(x) = 0 \end{align} $$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants