Skip to content

Prototype new apps #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 19 commits into
base: main
Choose a base branch
from
Open

Prototype new apps #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
9f1a364
test 1: first round of apps without filestore
vsoch Apr 28, 2025
fe4a148
batch 1: size 8
vsoch Apr 30, 2025
4c0c0d4
batch 1: size 4
vsoch Apr 30, 2025
53404c7
batch 1: size 16
vsoch Apr 30, 2025
3666728
batch 1: size 32
vsoch Apr 30, 2025
fa10eb5
batch 1: size 64
vsoch Apr 30, 2025
34a5e05
update size 32 chatterbug correct params
vsoch Apr 30, 2025
9d10573
gke gpu experiments sizes 4 and 8
vsoch May 3, 2025
afbfcd5
gke gpu experiment size 16
vsoch May 3, 2025
8c291c5
gke gpu size 32
vsoch May 3, 2025
8191bea
analysis: multi-gpu-models gpu up to 32
vsoch May 3, 2025
e966816
add nccl tests analysis script
vsoch May 3, 2025
55479b9
results: fio, pytorch mnist, and multi gpu models
vsoch May 3, 2025
db38134
gke cpu size 4, 11 new apps!
vsoch May 4, 2025
7e821d9
gke cpu experiment size 8 batch 2 apps
vsoch May 4, 2025
7a9932b
google gke cpu batch 2 size 16
vsoch May 5, 2025
32fae81
gke cpu size 32 apps batch 2
vsoch May 5, 2025
0dfadd2
size 64 gke batch 2 apps
vsoch May 5, 2025
9016707
likwid results and a few more analysis scripts
vsoch May 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
28 changes: 19 additions & 9 deletions analysis/amg2023/1-run-analysis.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,14 +86,19 @@ def parse_data(indir, outdir, files):
Parse filepaths for environment, etc., and results files for data.
"""
# metrics here will be figures of merit, and seconds runtime
p = ps.ResultParser("amg2023")
p = ps.ProblemSizeParser("amg2023")

# For flux we can save jobspecs and other event data
data = {}

# It's important to just parse raw data once, and then use intermediate
for filename in files:
exp = ps.ExperimentNameParser(filename, indir)
if "compute-engine" in filename:
continue
mpi = "openmpi"
if "intel" in filename:
mpi = "intel"
if exp.prefix not in data:
data[exp.prefix] = []

Expand All @@ -112,13 +117,18 @@ def parse_data(indir, outdir, files):

jobs = ps.parse_flux_jobs(item)
for job, metadata in jobs.items():

if "log" not in metadata:
print(filename)
continue
# Parse the FOM from the item - I see three.
# This needs to throw an error if we can't find it - indicates the result file is wonky
# Figure of Merit (FOM): nnz_AP / (Setup Phase Time + 3 * Solve Phase Time) 1.148604e+09
fom_overall = get_fom_line(metadata['log'], "Figure of Merit (FOM)")
p.add_result("fom_overall", fom_overall)
p.add_result("duration", metadata['duration'])
p.add_result("fom_overall", fom_overall, mpi)
if "duration" in metadata:
p.add_result("duration", metadata['duration'], mpi)
else:
print(filename)

print("Done parsing amg2023 results!")

Expand Down Expand Up @@ -173,13 +183,13 @@ def plot_results(df, outdir, non_anon=False):
ax=axes[0],
x="nodes",
y="value",
hue="experiment",
hue="problem_size",
err_kws={"color": "darkred"},
hue_order=[
"google/gke/cpu",
#hue_order=[
# "google/gke/cpu",
# "google/compute-engine/cpu",
],
palette=cloud_colors,
# ],
# palette=cloud_colors,
order=[4, 8, 16, 32],
)
if metric == "duration":
Expand Down
126 changes: 85 additions & 41 deletions analysis/amg2023/data/amg2023-results.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,41 +1,85 @@
,experiment,cloud,env,env_type,nodes,application,metric,value,gpu_count
0,google/gke/cpu,google,gke,cpu,16,amg2023,fom_overall,480151700.0,0
1,google/gke/cpu,google,gke,cpu,16,amg2023,duration,152.01309609413147,0
2,google/gke/cpu,google,gke,cpu,16,amg2023,fom_overall,482046400.0,0
3,google/gke/cpu,google,gke,cpu,16,amg2023,duration,150.1113781929016,0
4,google/gke/cpu,google,gke,cpu,16,amg2023,fom_overall,482960600.0,0
5,google/gke/cpu,google,gke,cpu,16,amg2023,duration,149.70200538635254,0
6,google/gke/cpu,google,gke,cpu,16,amg2023,fom_overall,482267400.0,0
7,google/gke/cpu,google,gke,cpu,16,amg2023,duration,150.08514404296875,0
8,google/gke/cpu,google,gke,cpu,16,amg2023,fom_overall,481757800.0,0
9,google/gke/cpu,google,gke,cpu,16,amg2023,duration,149.86131691932678,0
10,google/gke/cpu,google,gke,cpu,32,amg2023,fom_overall,878921000.0,0
11,google/gke/cpu,google,gke,cpu,32,amg2023,duration,166.38555097579956,0
12,google/gke/cpu,google,gke,cpu,32,amg2023,fom_overall,875691900.0,0
13,google/gke/cpu,google,gke,cpu,32,amg2023,duration,166.9042501449585,0
14,google/gke/cpu,google,gke,cpu,32,amg2023,fom_overall,873602600.0,0
15,google/gke/cpu,google,gke,cpu,32,amg2023,duration,167.73378896713257,0
16,google/gke/cpu,google,gke,cpu,32,amg2023,fom_overall,876845100.0,0
17,google/gke/cpu,google,gke,cpu,32,amg2023,duration,166.47650456428528,0
18,google/gke/cpu,google,gke,cpu,32,amg2023,fom_overall,874013400.0,0
19,google/gke/cpu,google,gke,cpu,32,amg2023,duration,167.56161046028137,0
20,google/gke/cpu,google,gke,cpu,4,amg2023,fom_overall,149912000.0,0
21,google/gke/cpu,google,gke,cpu,4,amg2023,duration,120.93159031867981,0
22,google/gke/cpu,google,gke,cpu,4,amg2023,fom_overall,150533300.0,0
23,google/gke/cpu,google,gke,cpu,4,amg2023,duration,120.01596999168396,0
24,google/gke/cpu,google,gke,cpu,4,amg2023,fom_overall,150666800.0,0
25,google/gke/cpu,google,gke,cpu,4,amg2023,duration,119.97674822807312,0
26,google/gke/cpu,google,gke,cpu,4,amg2023,fom_overall,150211100.0,0
27,google/gke/cpu,google,gke,cpu,4,amg2023,duration,120.16240978240967,0
28,google/gke/cpu,google,gke,cpu,4,amg2023,fom_overall,149685600.0,0
29,google/gke/cpu,google,gke,cpu,4,amg2023,duration,120.9310314655304,0
30,google/gke/cpu,google,gke,cpu,8,amg2023,fom_overall,257540200.0,0
31,google/gke/cpu,google,gke,cpu,8,amg2023,duration,138.88910913467407,0
32,google/gke/cpu,google,gke,cpu,8,amg2023,fom_overall,256498200.0,0
33,google/gke/cpu,google,gke,cpu,8,amg2023,duration,139.47089266777039,0
34,google/gke/cpu,google,gke,cpu,8,amg2023,fom_overall,258269200.0,0
35,google/gke/cpu,google,gke,cpu,8,amg2023,duration,138.14567017555237,0
36,google/gke/cpu,google,gke,cpu,8,amg2023,fom_overall,258603700.0,0
37,google/gke/cpu,google,gke,cpu,8,amg2023,duration,138.3945276737213,0
38,google/gke/cpu,google,gke,cpu,8,amg2023,fom_overall,258313400.0,0
39,google/gke/cpu,google,gke,cpu,8,amg2023,duration,138.5294554233551,0
,experiment,cloud,env,env_type,nodes,application,problem_size,metric,value,gpu_count
0,google/gke/cpu,google,gke,cpu,16,amg2023,intel,fom_overall,479566100.0,0
1,google/gke/cpu,google,gke,cpu,16,amg2023,intel,duration,150.02887225151062,0
2,google/gke/cpu,google,gke,cpu,16,amg2023,intel,fom_overall,480194700.0,0
3,google/gke/cpu,google,gke,cpu,16,amg2023,intel,duration,149.20562863349915,0
4,google/gke/cpu,google,gke,cpu,16,amg2023,intel,fom_overall,483265400.0,0
5,google/gke/cpu,google,gke,cpu,16,amg2023,intel,duration,148.83068323135376,0
6,google/gke/cpu,google,gke,cpu,16,amg2023,intel,fom_overall,483654700.0,0
7,google/gke/cpu,google,gke,cpu,16,amg2023,intel,duration,148.3491086959839,0
8,google/gke/cpu,google,gke,cpu,16,amg2023,intel,fom_overall,482457300.0,0
9,google/gke/cpu,google,gke,cpu,16,amg2023,intel,duration,148.45357942581177,0
10,google/gke/cpu,google,gke,cpu,16,amg2023,openmpi,fom_overall,480151700.0,0
11,google/gke/cpu,google,gke,cpu,16,amg2023,openmpi,duration,152.01309609413147,0
12,google/gke/cpu,google,gke,cpu,16,amg2023,openmpi,fom_overall,482046400.0,0
13,google/gke/cpu,google,gke,cpu,16,amg2023,openmpi,duration,150.1113781929016,0
14,google/gke/cpu,google,gke,cpu,16,amg2023,openmpi,fom_overall,482960600.0,0
15,google/gke/cpu,google,gke,cpu,16,amg2023,openmpi,duration,149.70200538635254,0
16,google/gke/cpu,google,gke,cpu,16,amg2023,openmpi,fom_overall,482267400.0,0
17,google/gke/cpu,google,gke,cpu,16,amg2023,openmpi,duration,150.08514404296875,0
18,google/gke/cpu,google,gke,cpu,16,amg2023,openmpi,fom_overall,481757800.0,0
19,google/gke/cpu,google,gke,cpu,16,amg2023,openmpi,duration,149.86131691932678,0
20,google/gke/cpu,google,gke,cpu,2,amg2023,openmpi,fom_overall,63159190.0,0
21,google/gke/cpu,google,gke,cpu,2,amg2023,openmpi,fom_overall,63236600.0,0
22,google/gke/cpu,google,gke,cpu,32,amg2023,intel,fom_overall,873366200.0,0
23,google/gke/cpu,google,gke,cpu,32,amg2023,intel,duration,168.76999402046204,0
24,google/gke/cpu,google,gke,cpu,32,amg2023,intel,fom_overall,879048800.0,0
25,google/gke/cpu,google,gke,cpu,32,amg2023,intel,duration,165.79934787750244,0
26,google/gke/cpu,google,gke,cpu,32,amg2023,intel,fom_overall,885228600.0,0
27,google/gke/cpu,google,gke,cpu,32,amg2023,intel,duration,163.910582780838,0
28,google/gke/cpu,google,gke,cpu,32,amg2023,intel,fom_overall,878304300.0,0
29,google/gke/cpu,google,gke,cpu,32,amg2023,intel,duration,165.0614676475525,0
30,google/gke/cpu,google,gke,cpu,32,amg2023,intel,fom_overall,874730300.0,0
31,google/gke/cpu,google,gke,cpu,32,amg2023,intel,duration,166.17630052566528,0
32,google/gke/cpu,google,gke,cpu,32,amg2023,openmpi,fom_overall,878921000.0,0
33,google/gke/cpu,google,gke,cpu,32,amg2023,openmpi,duration,166.38555097579956,0
34,google/gke/cpu,google,gke,cpu,32,amg2023,openmpi,fom_overall,875691900.0,0
35,google/gke/cpu,google,gke,cpu,32,amg2023,openmpi,duration,166.9042501449585,0
36,google/gke/cpu,google,gke,cpu,32,amg2023,openmpi,fom_overall,873602600.0,0
37,google/gke/cpu,google,gke,cpu,32,amg2023,openmpi,duration,167.73378896713257,0
38,google/gke/cpu,google,gke,cpu,32,amg2023,openmpi,fom_overall,876845100.0,0
39,google/gke/cpu,google,gke,cpu,32,amg2023,openmpi,duration,166.47650456428528,0
40,google/gke/cpu,google,gke,cpu,32,amg2023,openmpi,fom_overall,874013400.0,0
41,google/gke/cpu,google,gke,cpu,32,amg2023,openmpi,duration,167.56161046028137,0
42,google/gke/cpu,google,gke,cpu,4,amg2023,intel,fom_overall,150054300.0,0
43,google/gke/cpu,google,gke,cpu,4,amg2023,intel,duration,119.58212518692017,0
44,google/gke/cpu,google,gke,cpu,4,amg2023,intel,fom_overall,150422500.0,0
45,google/gke/cpu,google,gke,cpu,4,amg2023,intel,duration,118.81294631958008,0
46,google/gke/cpu,google,gke,cpu,4,amg2023,intel,fom_overall,151957000.0,0
47,google/gke/cpu,google,gke,cpu,4,amg2023,intel,duration,117.66585993766785,0
48,google/gke/cpu,google,gke,cpu,4,amg2023,intel,fom_overall,150265300.0,0
49,google/gke/cpu,google,gke,cpu,4,amg2023,intel,duration,119.45544004440308,0
50,google/gke/cpu,google,gke,cpu,4,amg2023,intel,fom_overall,151190900.0,0
51,google/gke/cpu,google,gke,cpu,4,amg2023,intel,duration,118.32887387275696,0
52,google/gke/cpu,google,gke,cpu,4,amg2023,intel,fom_overall,149320300.0,0
53,google/gke/cpu,google,gke,cpu,4,amg2023,intel,duration,121.57392072677612,0
54,google/gke/cpu,google,gke,cpu,4,amg2023,openmpi,fom_overall,149912000.0,0
55,google/gke/cpu,google,gke,cpu,4,amg2023,openmpi,duration,120.93159031867981,0
56,google/gke/cpu,google,gke,cpu,4,amg2023,openmpi,fom_overall,150533300.0,0
57,google/gke/cpu,google,gke,cpu,4,amg2023,openmpi,duration,120.01596999168396,0
58,google/gke/cpu,google,gke,cpu,4,amg2023,openmpi,fom_overall,150666800.0,0
59,google/gke/cpu,google,gke,cpu,4,amg2023,openmpi,duration,119.97674822807312,0
60,google/gke/cpu,google,gke,cpu,4,amg2023,openmpi,fom_overall,150211100.0,0
61,google/gke/cpu,google,gke,cpu,4,amg2023,openmpi,duration,120.16240978240967,0
62,google/gke/cpu,google,gke,cpu,4,amg2023,openmpi,fom_overall,149685600.0,0
63,google/gke/cpu,google,gke,cpu,4,amg2023,openmpi,duration,120.9310314655304,0
64,google/gke/cpu,google,gke,cpu,8,amg2023,intel,fom_overall,255527000.0,0
65,google/gke/cpu,google,gke,cpu,8,amg2023,intel,duration,140.18634748458862,0
66,google/gke/cpu,google,gke,cpu,8,amg2023,intel,fom_overall,256846400.0,0
67,google/gke/cpu,google,gke,cpu,8,amg2023,intel,duration,138.2529284954071,0
68,google/gke/cpu,google,gke,cpu,8,amg2023,intel,fom_overall,256867400.0,0
69,google/gke/cpu,google,gke,cpu,8,amg2023,intel,duration,137.77160787582397,0
70,google/gke/cpu,google,gke,cpu,8,amg2023,intel,fom_overall,255484500.0,0
71,google/gke/cpu,google,gke,cpu,8,amg2023,intel,duration,138.67487406730652,0
72,google/gke/cpu,google,gke,cpu,8,amg2023,intel,fom_overall,256334100.0,0
73,google/gke/cpu,google,gke,cpu,8,amg2023,intel,duration,138.12138152122498,0
74,google/gke/cpu,google,gke,cpu,8,amg2023,openmpi,fom_overall,257540200.0,0
75,google/gke/cpu,google,gke,cpu,8,amg2023,openmpi,duration,138.88910913467407,0
76,google/gke/cpu,google,gke,cpu,8,amg2023,openmpi,fom_overall,256498200.0,0
77,google/gke/cpu,google,gke,cpu,8,amg2023,openmpi,duration,139.47089266777039,0
78,google/gke/cpu,google,gke,cpu,8,amg2023,openmpi,fom_overall,258269200.0,0
79,google/gke/cpu,google,gke,cpu,8,amg2023,openmpi,duration,138.14567017555237,0
80,google/gke/cpu,google,gke,cpu,8,amg2023,openmpi,fom_overall,258603700.0,0
81,google/gke/cpu,google,gke,cpu,8,amg2023,openmpi,duration,138.3945276737213,0
82,google/gke/cpu,google,gke,cpu,8,amg2023,openmpi,fom_overall,258313400.0,0
83,google/gke/cpu,google,gke,cpu,8,amg2023,openmpi,duration,138.5294554233551,0
Binary file modified analysis/amg2023/data/img/amg-duration-cpu.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading