Improve performance analysis tool

CMakeLists.txt

@@ -5,7 +5,7 @@ include(cmake/SpheralVersion.cmake)
 project(spheral LANGUAGES C CXX Fortran VERSION ${SPHERAL_VERSION})
 
 set(SPHERAL_ROOT_DIR ${CMAKE_CURRENT_SOURCE_DIR} CACHE PATH "Path to Spheral source directory")
-set(SPHERAL_TEST_INSTALL_PREFIX ${CMAKE_INSTALL_PREFIX}/tests)
+set(SPHERAL_TEST_INSTALL_PREFIX "")
 
 include(cmake/SetupSpheral.cmake)
 

cmake/SetupSpheral.cmake

@@ -158,7 +158,7 @@ endif()
 # Build C++ tests and install tests to install directory
 #-------------------------------------------------------------------------------
 if (ENABLE_TESTS)
-  spheral_install_python_tests(${SPHERAL_ROOT_DIR}/tests/ ${SPHERAL_TEST_INSTALL_PREFIX})
+  spheral_install_python_tests(${SPHERAL_ROOT_DIR}/tests/ ${CMAKE_INSTALL_PREFIX}/tests/${SPHERAL_TEST_INSTALL_PREFIX})
   # Always install performance.py in the top of the testing script
   install(FILES ${SPHERAL_ROOT_DIR}/tests/performance.py
     DESTINATION ${CMAKE_INSTALL_PREFIX}/tests)

docs/developer/dev/diagnostic_tools.rst

@@ -144,33 +144,39 @@ Performance Regression Testing
 When a merge to the develop branch occurs, the CI runs this regression test multiple times to accumulate benchmark timing data in a centralized directory (``/usr/WS2/sduser/Spheral/benchmark``).
 The general procedure to comparing performance regression tests is:
 
-#. Run the performance regression tests from an installation using 2 nodes (number of nodes used in benchmark run by CI):
+* Run the performance regression tests from an installation using 2 nodes (number of nodes used in benchmark run by CI):
    ::
 
-      ./spheral-ats --loc test_dir_name --numNodes 2 tests/performance.py
+      ./spheral-ats --log test_dir_name --numNodes 2 tests/performance.py
 
    There is also a ``--threads`` option to specify a given number of threads per rank.
+   The test above will create an ATS python file, ``test_dir_name/atsr.py``, as well as
 
-#. Utilize Thicket to compare the newly run times with reference times
+* In general, to compare the performance between two performance results, use:
    ::
 
-      ./spheral ./scripts/performance_analysis.py --perf-dir test_dir_name --ref /directory/of/reference/caliper/files/
+      ./spheral ./scripts/performance_analysis.py --perfdata1 /path/to/perf/data --perfdata2 /path/to/other/perf/data
+
+* To compare newly run times with reference times for regression testing purposes, use:
+   ::
+
+      ./spheral ./scripts/performance_analysis.py --perfdata test_dir_name --ref /directory/of/reference/caliper/files/
 
    The input to ``--ref`` can be also be an ATS directory created from running ``performance.py`` or just a directory of Caliper files.
    Removing the ``--ref`` input will default to comparing to benchmark timings in ``/usr/WS2/sduser/Spheral/benchmark``.
-   If comparing tests across different hardware/compiler/etc., be sure to add ``--diff-configs`` to the command line.
+   Timing trees can be displayed using the ``--display`` flag.
    The script above computes the mean (:math:`\mu`) and standard deviation (:math:`\sigma`) of the inclusive average time per rank (``Avg time/rank``) timers for each test in the reference (or benchmark) data.
    It computes a timing threshold using:
 
    .. math::
 
       \delta_{\mathrm{thresh}} = 0.08 \mu + 2 \sigma
 
-   If the :math:`t_c - \mu > \delta_{\mathrm{thresh}}` for the ``main`` region, where :math:`t_c` is the new performance time from step 1, the test is considered to have failed and the timing tree of the exclusive average time per rank (``Avg time/rank (exc)``) will be displayed.
+   If the :math:`t_c - \mu > \delta_{\mathrm{thresh}}` for the ``main`` region, where :math:`t_c` is the new performance time, the test is considered to have failed and the timing tree of the exclusive average time per rank (``Avg time/rank (exc)``) will be displayed.
    If the test configurations, like the number of time steps differed between the runs, or the hardware/install configurations did not match, it will consider the test skipped.
    Otherwise, the test is considered to have passed.
    If the :math:`t_c - \mu < -\delta_{\mathrm{thresh}}`, the performance improved significantly and the timing tree will be displayed.
 
 .. note::
 
-   If ``performance.py`` is run on a non-MPI Spheral build, it will only use 1 rank and will thread all other cores.
+   If ``performance.py`` is run on a non-MPI Spheral build, it will only use 1 rank and will thread all other available cores.