Performance studies

[plexoos]

No description provided.

outdated suggestion

outdated suggestion

outdated suggestion

outdated suggestion

outdated suggestion

outdated suggestion

outdated suggestion

outdated suggestion

outdated suggestion

[github-actions]

Cpp-linter Review

Used clang-format v18.1.3

Click here for the full clang-format patch

diff --git a/src/g4appmt.h b/src/g4appmt.h
index f0f0020..88258fc 100644
--- a/src/g4appmt.h
+++ b/src/g4appmt.h
@@ -125,2 +125 @@ struct PhotonHit : public G4VHit
-        G4cout << "Detector id: " << fid << " energy: " << fenergy << " nm"
-               << " time: " << ftime << " ns"
+        G4cout << "Detector id: " << fid << " energy: " << fenergy << " nm" << " time: " << ftime << " ns"

Have any feedback or feature suggestions? Share it here.

outdated suggestion

[ggalgoczi]

Added PATH so the script can be called from anywehere.

@plexoos do you know why I do not see all the output in simg4oxmt.log when I call the run run_performance.py?

# Run with time in bash to capture real/user/sys
                cmd = f"time simg4oxmt -g {args.gdml} -m run.mac"
                print(f"Running {threads} threads: {cmd}")
                result = subprocess.run(
                    ["bash", "-c", cmd],
                    capture_output=True, text=True 

does not capture all output for some reason?

when I call ../opt/eic-opticks/build/src/simg4oxmt -g h.gdml -m run.mac I do see things like

Opticks: NumHits: 6669 that are not in the log file. We should fix this.

[ggalgoczi]

previous problem solved in 4096e4e

[ggalgoczi]

@plexoos can you review and merge the changes into main? I am done.

[plexoos]

@ggalgoczi I am testing this PR

[plexoos]

Why was this number reduced? What small GPU RAM are you referring to? For consistency, we should be able to reproduce the published results using the same number of events.

[ggalgoczi]

In this repository, the event count is set to a conservative default because the goal of this example is to help users get a first successful run with EIC-Opticks and understand the workflow, not to serve as a paper quality reproduction test. Since users run on a wide range of hardware, a high default can fail on GPUs with limited VRAM and lead to crashes/segfaults that are frustrating to debug as a first step.

Also, because we swapped the geometry, the performance and scaling will not match the paper one-to-one anyway, the achievable speedup depends on geometry and workload characteristics.

To keep things consistent and reproducible, the README explicitly notes that users should set the event count themselves.

[plexoos]

So, what kind of speedup do you see with this geometry? Is it far from the public numbers?

If I use pfrich_min_FINAL.gdml and the patch for open cones, should I expect to reproduce the numbers in the paper?

[ggalgoczi]

1. I did not check. Can you check how much speedup you get?

2. Yes. I tested it and sent you the example result Dec 8. 3:58 PM in mattermost chat.

[github-actions]

Cpp-linter Review

Used clang-format v18.1.3

Click here for the full clang-format patch

diff --git a/CSG/csg_intersect_leaf_newcone.h b/CSG/csg_intersect_leaf_newcone.h
index 9801725..3bc042d 100644
--- a/CSG/csg_intersect_leaf_newcone.h
+++ b/CSG/csg_intersect_leaf_newcone.h
@@ -101 +101 @@ void intersect_leaf_newcone( bool& valid_isect, float4& isect, const quad& q0, c
-    const float t_cand = fminf(roots) ;
+    const float t_cand = fminf(roots);

Have any feedback or feature suggestions? Share it here.

[plexoos]

Here’s what I’m getting when I run this branch as it is right now:

c6b7d2a3be6c:/src/eic-opticks# run-performance -g ./tests/geom/pfrich_min_FINAL.gdml -o ./out_pfrich_min_FINAL/rel/

[ggalgoczi]

You need to modify the eic-opticks code by running the cone modification script , building eic-opticks and the executable again if you want to run the performance script on pfrich. Did you do all? If yes I will check this out.

[plexoos]

You need to modify the eic-opticks code by running the cone modification script , building eic-opticks and the executable again if you want to run the performance script on pfrich. Did you do all? If yes I will check this out.

A cone modification script? Do you mean there’s something in addition to applying this patch 04f4d4a that I should run?

[plexoos]

Okay, I rebuilt everything from scratch to make sure there were no leftovers from previous tests, and I’m still seeing a speed-up factor of about ~30 between single-threaded G4 and OptiX, as shown in the plot above. Please check.

[ggalgoczi]

Why did you do those modifications in 04f4d4a ? It is different from I did when I got the code working. See my modification script:
https://github.com/BNLNPPS/esi-g4ox/blob/PerformanceAnalysis/scripts/cone_opticks_intersection_modify.py

Now I am seeing the following speedup: 170x speedup wrt to single thread G4 and 10x wrt to 20 MT G4. How did I get these number:

• I fixed the cone to what I mentioned before.
• I modified the momentum of electrons to 5 MeV that was used in the paper.
• I ran optiphy/tools/run_performance.py and I checked files in timing_geant.txt and timing_optix.txt.
• Make sure to simulate 50k electrons.
• You can check if you set things up correctly with running the G4 code and seeing similar numbers when running:

time simg4oxmt -g /src/eic-opticks/tests/geom/pfrich_min_FINAL.gdml -m /src/eic-opticks/run.mac

Simulation time: 0.519579 seconds
Opticks: NumCollected:  540880
Opticks: NumCollected:  46927986
Opticks: NumHits:  8865899

Also if you modify eic-opticks or the example source code do not forget to build, eg. in my case I used:

cmake -S $OPTICKS_HOME -B $OPTICKS_BUILD -DCMAKE_INSTALL_PREFIX=$OPTICKS_PREFIX -DCMAKE_BUILD_TYPE=Release && cmake --build $OPTICKS_BUILD --parallel --target install

outdated suggestion

[github-actions]

Cpp-linter Review

Used clang-format v18.1.3

Click here for the full clang-format patch

diff --git a/CSG/csg_intersect_leaf_newcone.h b/CSG/csg_intersect_leaf_newcone.h
index 01a9845..fe8068e 100644
--- a/CSG/csg_intersect_leaf_newcone.h
+++ b/CSG/csg_intersect_leaf_newcone.h
@@ -101,2 +101,2 @@ void intersect_leaf_newcone( bool& valid_isect, float4& isect, const quad& q0, c
-    const float t_cand = fminf(roots) ; 
-    
+    const float t_cand = fminf(roots) ;
+
@@ -106,11 +106,3 @@ void intersect_leaf_newcone( bool& valid_isect, float4& isect, const quad& q0, c
-        float3 intersection_point = make_float3(
-            o.x + t_cand * d.x,
-            o.y + t_cand * d.y,
-            o.z + t_cand * d.z
-        );
-        float3 n = normalize(make_float3(
-            intersection_point.x,
-            intersection_point.y,
-            (z0 - intersection_point.z)*tth2
-        ));
-
+        float3 intersection_point = make_float3(o.x + t_cand * d.x, o.y + t_cand * d.y, o.z + t_cand * d.z);
+        float3 n =
+            normalize(make_float3(intersection_point.x, intersection_point.y, (z0 - intersection_point.z) * tth2));

Have any feedback or feature suggestions? Share it here.

outdated suggestion

[ggalgoczi]

I confirm that when moving the particlegun to 0,0,0 mm (I just pushed this change into the branch) and running
optiphy/tools/run_performance.py

I see performance data created in timing_optix.txt and timing_geant.txt files. Namely the simulation times are respecitvely 0.8 s and 11.7 s. Let me know @plexoos if we can close this PR.

[ggalgoczi]

@plexoos could you take a look?