pyomo.doe adding more verbose output for sensitivity analysis #3525

smondal13 · 2025-03-19T14:12:30Z

Fixes # .

Summary/Motivation:

Adding more verbose is helpful to analyze the problem
Adding a new method to check the FIM can help in the general usage of the method.
Adding a new standalone function to calculate different metrics of the FIM is helpful in case the user does not want to build the get_labeled_model().

Changes proposed in this PR:

added eigenvalues, determinant of FIM, and trace of FIM for sensitivity output in compute_FIM_full_factorial()
added_check_FIM() by separating it from check_model_FIM()
added _compute_FIM_metrics()
added get_FIM_metrics()

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

I agree my contributions are submitted under the BSD license.
I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

smondal13 · 2025-03-19T14:14:16Z

@adowling2 @djlaky ready for early feedback

adowling2 · 2025-03-19T14:42:18Z

Next step:

Also add solver status

jsiirola · 2025-03-25T18:59:42Z

pyomo/contrib/doe/doe.py

            if abs(E_vals.imag[E_ind]) > 1e-8:
                self.logger.warning(
-                    "Eigenvalue has imaginary component greater than 1e-6, contact developers if this issue persists."
+                    "Eigenvalue has imaginary component greater than 1e-8, contact developers if this issue persists."


nit: this is why "embedded constants are evil". An improved solution would be to declare a module-level constant (maybe IMAG_THRESHOLD) and then use that both in the test and in the message.

adowling2 · 2025-03-25T22:12:55Z

@smondal13 See #3532 for an example of what @jsiirola is suggesting

…alue checking in compute_FIM_full_factorial()

…pute_FIM_metrics` function.

… doe object that uses the "reactor_experiment" example in doe directory. Also added the png files to gitignore that are stored when "reactor_example.py" is run.

…M_metrics.py` instead of importing idaes, only the solvers are imported.

… adding_eigen_values

…_compute_FIM_metric funciton in doe

…IM() staticmethod of DesignOfExperiments class

… in compute_FIM_full_factorial that is ready to be deleted.

smondal13 · 2025-05-01T12:41:08Z

@adowling2 Ready for review.

adowling2

@smondal13 This looks really close. I left a few minor comments.

@blnicho @jsiirola @mrmundt This is ready for a design review from the Pyomo team. I am moving it on the project board.

adowling2 · 2025-05-01T22:38:22Z

pyomo/contrib/doe/doe.py

+            "FIM provided matches expected dimensions from model and is approximately positive (semi) definite."
+        )
+
+    # TODO: Make this a static method


You can remove this TODO.

adowling2 · 2025-05-01T22:39:33Z

pyomo/contrib/doe/doe.py

+                _compute_FIM_metrics(FIM)
+            )
+            """
+            Since we are using the `_compute_FIM_metrics()` to compute the metrics in this chunk of code., This chunk of code is ready to be deleted.


Please delete if you feel it is no longer needed. It is in the Git history in case we need to find it.

adowling2 · 2025-05-01T22:40:46Z

pyomo/contrib/doe/doe.py

+    E_vals, E_vecs = np.linalg.eig(FIM)
+    E_ind = np.argmin(E_vals.real)  # index of smallest eigenvalue
+
+    # Warn the user if there is a ``large`` imaginary component (should not be)


Will you ever get to this warning? Or will _check_FIM throw an error first?

@adowling2 _check_FIM() will not trigger this warning. _check_FIM() only checks whether the FIM is square, symmetric, and positive definite. Although this method checks for the symmetry of the FIM with a tolerance, and it is unlikely that the FIM will have large imaginary eigenvalues, we have kept the warning there just in case there is a large imaginary eigenvalue.

@smondal13 Thank you.

@blnicho @mrmundt Is there a best practice for testing this warning message is properly thrown?

adowling2 · 2025-05-01T22:43:37Z

Also, @smondal13, please remove the draft status.

…_FIM_full_factorial()` which was replaced by function call `_compute_FIM_metrics()`

…ght that will be helpful for the user to understand the method.

smondal13 · 2025-05-02T13:08:44Z

@adowling2 I have implemented your suggestions. I have also added Returns in the compute_FIM_full_factorial() docstring. I thought it would be helpful for the user to know what the method is returning.

adowling2

Pending input from the Pyomo team, I think this PR is ready to merge.

adowling2 · 2025-05-14T14:12:46Z

pyomo/contrib/doe/doe.py

+    E_vals, E_vecs = np.linalg.eig(FIM)
+    E_ind = np.argmin(E_vals.real)  # index of smallest eigenvalue
+
+    # Warn the user if there is a ``large`` imaginary component (should not be)


@smondal13 Thank you.

@blnicho @mrmundt Is there a best practice for testing this warning message is properly thrown?

adowling2 · 2025-05-14T14:14:05Z

pyomo/contrib/doe/tests/test_doe_FIM_metrics.py

+
+
+class TestFIMWarning(unittest.TestCase):
+    def test_FIM_eigenvalue_warning(self):


@blnicho @mrmundt I have not tested warning messages before, so your input here is appreciated.

smondal13 added 4 commits March 18, 2025 12:08

added eigenvalues and determinant of the FIM

2f35d58

added trace as a output in compute_FIM_full_factorial()

0327879

added eigvalsh for calculating

eb78589

changed the method to calculate the minimum eigenvalues

5b2e90e

smondal13 marked this pull request as draft March 19, 2025 14:13

Merge branch 'Pyomo:main' into adding_eigen_values

f3b0f7c

jsiirola reviewed Mar 25, 2025

View reviewed changes

smondal13 added 13 commits March 26, 2025 11:56

Added img_thereshold instead of the embedded constant for imaginary v…

701947b

…alue checking in compute_FIM_full_factorial()

Alex suggested some objectives and tests

114a86e

Added IMG_THRESHOLD instead of embedded variable

cb694be

Added FIM test metrics for doe.py

0a938ac

Changed the isclose() method to ==

acf06f0

Changed back to isclose()

41a6272

Added todo list

b55786f

Merge branch 'Pyomo:main' into adding_eigen_values

46f75c1

imported _SMALL_TOLERANCE_IMG from doe.py and used it in the `com…

3cc77ef

…pute_FIM_metrics` function.

Deleted comment "Alex said..."

3ca1c4e

added 1e-6 as the _SMALL_TOLERANCE_IMG

7c2535f

Deleted the figures that was generated by runnig the example

5216672

adding unittest

708a21d

blnicho added the pyomo.DoE label Apr 29, 2025

github-project-automation bot added this to ParmEst & Pyomo.DoE Development Apr 29, 2025

blnicho moved this to Todo in ParmEst & Pyomo.DoE Development Apr 29, 2025

smondal13 added 2 commits April 29, 2025 13:21

Merge branch 'Pyomo:main' into adding_eigen_values

39f26e6

to test doe , added the "doe_test_example.py" script, which returns a…

e9e83b6

… doe object that uses the "reactor_experiment" example in doe directory. Also added the png files to gitignore that are stored when "reactor_example.py" is run.

smondal13 force-pushed the adding_eigen_values branch from 6e68c2b to e9e83b6 Compare April 30, 2025 01:17

adowling2 moved this from Todo to Development in ParmEst & Pyomo.DoE Development Apr 30, 2025

smondal13 added 7 commits April 29, 2025 23:04

The test_example file is not required. It is deleted. in `test_doe_FI…

7064966

…M_metrics.py` instead of importing idaes, only the solvers are imported.

Merge branch 'Pyomo:main' into adding_eigen_values

a475ada

Coding with Alex about compute_FIM_metrics funciton.

5011800

Merge branch 'adding_eigen_values' of github.com:smondal13/pyomo into…

b9ea35b

… adding_eigen_values

Changed the test file to test the new function and added logging for …

6d26fdf

…_compute_FIM_metric funciton in doe

added working test code for _compute_FIM_metric function and _check_F…

32079fe

…IM() staticmethod of DesignOfExperiments class

Splitted the test files into 2 separate files. Commented out the code…

e1842cf

… in compute_FIM_full_factorial that is ready to be deleted.

adowling2 reviewed May 1, 2025

View reviewed changes

adowling2 moved this from Development to Ready for design review in ParmEst & Pyomo.DoE Development May 1, 2025

smondal13 added 2 commits May 2, 2025 07:56

Deleted TODO: "make static method", and the chunk of code in `compute…

898e87f

…_FIM_full_factorial()` which was replaced by function call `_compute_FIM_metrics()`

Added Returns in the compute_FIM_full_factorial() docstring. Thou…

ec2a361

…ght that will be helpful for the user to understand the method.

smondal13 marked this pull request as ready for review May 2, 2025 13:06

Merge branch 'main' into adding_eigen_values

f0da48e

blnicho self-requested a review May 6, 2025 19:30

adowling2 approved these changes May 14, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pyomo.doe adding more verbose output for sensitivity analysis #3525

pyomo.doe adding more verbose output for sensitivity analysis #3525

smondal13 commented Mar 19, 2025 •

edited

Loading

smondal13 commented Mar 19, 2025

adowling2 commented Mar 19, 2025

jsiirola Mar 25, 2025

adowling2 commented Mar 25, 2025

smondal13 commented May 1, 2025

adowling2 left a comment

adowling2 May 1, 2025

adowling2 May 1, 2025

adowling2 May 1, 2025

smondal13 May 2, 2025 •

edited

Loading

adowling2 May 14, 2025

adowling2 commented May 1, 2025

smondal13 commented May 2, 2025

adowling2 left a comment

adowling2 May 14, 2025

adowling2 May 14, 2025



		class TestFIMWarning(unittest.TestCase):
		def test_FIM_eigenvalue_warning(self):

pyomo.doe adding more verbose output for sensitivity analysis #3525

Are you sure you want to change the base?

pyomo.doe adding more verbose output for sensitivity analysis #3525

Conversation

smondal13 commented Mar 19, 2025 • edited Loading

Fixes # .

Summary/Motivation:

Changes proposed in this PR:

Legal Acknowledgement

smondal13 commented Mar 19, 2025

adowling2 commented Mar 19, 2025

jsiirola Mar 25, 2025

Choose a reason for hiding this comment

adowling2 commented Mar 25, 2025

smondal13 commented May 1, 2025

adowling2 left a comment

Choose a reason for hiding this comment

adowling2 May 1, 2025

Choose a reason for hiding this comment

adowling2 May 1, 2025

Choose a reason for hiding this comment

adowling2 May 1, 2025

Choose a reason for hiding this comment

smondal13 May 2, 2025 • edited Loading

Choose a reason for hiding this comment

adowling2 May 14, 2025

Choose a reason for hiding this comment

adowling2 commented May 1, 2025

smondal13 commented May 2, 2025

adowling2 left a comment

Choose a reason for hiding this comment

adowling2 May 14, 2025

Choose a reason for hiding this comment

adowling2 May 14, 2025

Choose a reason for hiding this comment

smondal13 commented Mar 19, 2025 •

edited

Loading

smondal13 May 2, 2025 •

edited

Loading