Fix perf-simple-query email template crash and empty body issue #12235

Copilot · 2025-10-19T09:14:34Z

Fix: Email for perf microbenchmark tests (perf-simple-query) sent empty

Problem

The perf-simple-query email had two related issues:

Template Crash: Emails failed with jinja2.exceptions.UndefinedError: 'str object' has no attribute 'items'
Empty Email Body: Email body contained only {"grafana_screenshots": [], "reporter": "PerfSimpleQueryReporter"}

Root Causes

Issue 1 - Template Crash: When PerfSimpleQueryAnalyzer.check_regression() returns False (test not found in ES or missing statistics), the template crashes trying to iterate over "N/A" strings.

Issue 2 - Empty Email Body:

PerfSimpleQueryTest didn't override get_email_data() → base class overwrote file with minimal data
check_regression() returning False → no email_data.json created at all

Solutions

Fix 1 - Template Robustness (commit bff4e61): Added conditional checks before accessing data in results_perf_simple_query.html

Fix 2 - Email Data Preservation (commit 07dce1f): Added get_email_data() override to read from existing email_data.json

Fix 3 - Fallback Data Creation (commit 153bd41): Capture return value from check_regression() and create fallback email_data.json with informative content when it returns False

Fix 4 - Merged master (this commit): Merged latest changes from master branch including related performance email fixes

Result

Emails always sent with meaningful content:

With regression data: Full detailed statistics
Without regression data: Clear message explaining why data unavailable, plus test ID and job URL
Never: Empty body or crashes

Original prompt

This section details on the original issue you should resolve

<issue_title>Email for perf microbenchmark tests (perf-simple-query) is sent empty.</issue_title>
<issue_description>Email for perf microbenchmark tests (perf-simple-query) is sent empty.
Failure :

06:20:04  Failed to create email due to the following error:
06:20:04  Traceback (most recent call last):
06:20:04    File "/home/ubuntu/scylla-cluster-tests/./sct.py", line 1476, in send_email
06:20:04      reporter.send_report(test_results)
06:20:04      ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
06:20:04    File "/home/ubuntu/scylla-cluster-tests/sdcm/send_email.py", line 237, in send_report
06:20:04      report = self._generate_report(report_data)
06:20:04    File "/home/ubuntu/scylla-cluster-tests/sdcm/send_email.py", line 271, in _generate_report
06:20:04      return self.build_report(report_data)
06:20:04             ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
06:20:04    File "/home/ubuntu/scylla-cluster-tests/sdcm/send_email.py", line 282, in build_report
06:20:04      return self.render_to_html(report_data)
06:20:04             ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
06:20:04    File "/home/ubuntu/scylla-cluster-tests/sdcm/send_email.py", line 202, in render_to_html
06:20:04      html = template.render(results)
06:20:04    File "/usr/local/lib/python3.13/site-packages/jinja2/environment.py", line 1295, in render
06:20:04      self.environment.handle_exception()
06:20:04      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
06:20:04    File "/usr/local/lib/python3.13/site-packages/jinja2/environment.py", line 942, in handle_exception
06:20:04      raise rewrite_traceback_stack(source=source)
06:20:04    File "/home/ubuntu/scylla-cluster-tests/sdcm/report_templates/results_perf_simple_query.html", line 1, in top-level template code
06:20:04      {% extends 'results_base_custom.html' %}
06:20:04      ^^^^^^^^^^^^^^^^^^^^^^^^^
06:20:04    File "/home/ubuntu/scylla-cluster-tests/sdcm/report_templates/results_base_custom.html", line 140, in top-level template code
06:20:04      {% block body %}
06:20:04    File "/home/ubuntu/scylla-cluster-tests/sdcm/report_templates/results_perf_simple_query.html", line 26, in block 'body'
06:20:04      {% for key, result in test_stats.items() %}
06:20:04      ^^^^^^^^^^^^^^^^^
06:20:04    File "/usr/local/lib/python3.13/site-packages/jinja2/utils.py", line 92, in from_obj
06:20:04      if hasattr(obj, "jinja_pass_arg"):
06:20:04         ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
06:20:04  jinja2.exceptions.UndefinedError: 'str object' has no attribute 'items'

Argus

Scylla version: 2026.1.0~dev-20251017.01bcafbe2460 with build-id b2b802b8cf900a4e9e7609cdf8b205ef627c5223

Kernel Version: 6.14.0-1014-aws

Extra information

Installation details

Cluster size: 1 nodes (c7g.large)

Scylla Nodes used in this run:

- perf-microbenchmarking-amazon-ARM-p-db-node-d41f34b1-1 (3.236.164.235 | 10.12.0.222) (shards: 2)

OS / Image: ami-06e80e987d374764d (aws: N/A)

Test: scylla-enterprise-perf-simple-query-weekly-microbenchmark_arm64
Test id: d41f34b1-90f3-4293-aa72-66e42be89d9b
Test name: scylla-enterprise/perf-regression/scylla-enterprise-perf-simple-query-weekly-microbenchmark_arm64

Test method: `microbenchmarking_test.PerfSimpleQueryTest.test_perf_simple_query`

Test config file(s):

amazon_perf_simple_query_ARM.yaml

Logs:

db-cluster-d41f34b1.tar.zst
schema-logs-d41f34b1.tar.zst
sct-runner-events-d41f34b1.tar.zst
sct-d41f34b1.log.tar.zst
parallel-timelines-report-d41f34b1.tar.zst
builder-d41f34b1.log.tar.gz

Jenkins job URL

...

Fixes #12232

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: juliayakovlev <[email protected]>

juliayakovlev · 2025-10-19T10:14:14Z

@copilot the problem is also that email body is empty.
email_data.json body:

{"grafana_screenshots": [], "reporter": "PerfSimpleQueryReporter"}

juliayakovlev · 2025-10-19T10:14:53Z

@copilot can you, please, check why the email body is empty

The issue was that PerfSimpleQueryTest didn't override get_email_data(), causing the base class to overwrite the email_data.json file created by check_regression() with minimal data (only grafana_screenshots and reporter). Now get_email_data() reads from the existing email_data.json file created by check_regression(), preserving all the test statistics and results. Co-authored-by: juliayakovlev <[email protected]>

Copilot · 2025-10-19T10:19:33Z

@copilot the problem is also that email body is empty.
email_data.json body:

...

Fixed in commit f98e49e. The issue was that PerfSimpleQueryTest didn't override get_email_data(), causing the base class to overwrite the file created by check_regression(). Now get_email_data() reads from the existing email_data.json, preserving all test statistics.

Copilot · 2025-10-19T10:23:43Z

@copilot can you, please, check why the email body is empty

The email body was empty because of this flow:

Before Fix (commit 07dce1f):

check_regression() creates email_data.json with full test statistics
Later, save_email_data() calls get_email_data()
Since PerfSimpleQueryTest didn't override get_email_data(), it returned {}
Base class overwrote the file with only {"grafana_screenshots": [], "reporter": "PerfSimpleQueryReporter"}
Result: Empty email body

After Fix:

check_regression() creates email_data.json with full test statistics
get_email_data() now reads from the existing file
Returns the full data with test_stats, test_version, etc.
Base class adds grafana_screenshots to existing data
Result: Full email body with all statistics

The fix is in microbenchmarking_test.py lines 43-58, where get_email_data() now reads and preserves the data created by check_regression().

juliayakovlev · 2025-10-19T12:18:11Z

@copilot
I ran the test (Argus link but an email is empty

When check_regression() returns False (e.g., test not found in ES or missing statistics), it doesn't create email_data.json. This caused empty email bodies. Now the test captures the return value and creates a fallback email_data.json with basic information (test ID, job URL, N/A for stats) that will render with the user-friendly "data not available" message in the template. Co-authored-by: juliayakovlev <[email protected]>

Copilot · 2025-10-19T12:21:01Z

@copilot
I ran the test (Argus link but an email is empty

The issue was that when check_regression() fails (e.g., test not found in Elasticsearch), it returns False and doesn't create email_data.json at all. Fixed in commit that now captures the return value and creates fallback email_data.json with test ID, job URL, and N/A values. The template will render this with the "Test statistics are not available" message, ensuring emails are never empty.

fruch · 2025-10-19T13:16:44Z

@juliayakovlev

I think this is similar to #12234, i.e. some fix might solve both.

juliayakovlev · 2025-10-21T17:04:25Z

@copilot please, merge from master

…mail-sending

Copilot · 2025-10-21T17:09:58Z

@copilot please, merge from master

Merged master branch successfully. No conflicts - our changes to microbenchmarking_test.py and results_perf_simple_query.html template are preserved.

fruch · 2025-10-28T14:10:11Z

@juliayakovlev do we still need this one ?

Initial plan

1dc451c

Copilot AI assigned Copilot and juliayakovlev Oct 19, 2025

Copilot started work on behalf of juliayakovlev October 19, 2025 09:15 View session

Fix perf-simple-query email template to handle missing data

bff4e61

Co-authored-by: juliayakovlev <[email protected]>

Copilot AI changed the title ~~[WIP] Fix email sending for perf microbenchmark tests~~ Fix perf-simple-query email template crash when regression check fails Oct 19, 2025

Copilot AI requested a review from juliayakovlev October 19, 2025 09:32

Copilot finished work on behalf of juliayakovlev October 19, 2025 09:32

Copilot started work on behalf of juliayakovlev October 19, 2025 10:14 View session

Copilot AI changed the title ~~Fix perf-simple-query email template crash when regression check fails~~ Fix perf-simple-query email template crash and empty body issue Oct 19, 2025

Copilot finished work on behalf of juliayakovlev October 19, 2025 10:22

Copilot started work on behalf of juliayakovlev October 19, 2025 10:22 View session

Copilot finished work on behalf of juliayakovlev October 19, 2025 10:24

Copilot started work on behalf of juliayakovlev October 19, 2025 12:18 View session

Copilot finished work on behalf of juliayakovlev October 19, 2025 12:23

Copilot started work on behalf of juliayakovlev October 21, 2025 17:05 View session

Merge remote-tracking branch 'origin/master' into copilot/fix-empty-e…

35dffbd

…mail-sending

Copilot finished work on behalf of juliayakovlev October 21, 2025 17:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix perf-simple-query email template crash and empty body issue #12235

Fix perf-simple-query email template crash and empty body issue #12235

Uh oh!

Copilot AI commented Oct 19, 2025 •

edited

Loading

Uh oh!

juliayakovlev commented Oct 19, 2025

Uh oh!

juliayakovlev commented Oct 19, 2025

Uh oh!

Copilot AI commented Oct 19, 2025

Uh oh!

Copilot AI commented Oct 19, 2025

Uh oh!

juliayakovlev commented Oct 19, 2025

Uh oh!

Copilot AI commented Oct 19, 2025

Uh oh!

fruch commented Oct 19, 2025

Uh oh!

juliayakovlev commented Oct 21, 2025

Uh oh!

Copilot AI commented Oct 21, 2025

Uh oh!

fruch commented Oct 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Fix perf-simple-query email template crash and empty body issue #12235

Are you sure you want to change the base?

Fix perf-simple-query email template crash and empty body issue #12235

Uh oh!

Conversation

Copilot AI commented Oct 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Fix: Email for perf microbenchmark tests (perf-simple-query) sent empty

Problem

Root Causes

Solutions

Result

Installation details

Logs:

Uh oh!

juliayakovlev commented Oct 19, 2025

Uh oh!

juliayakovlev commented Oct 19, 2025

Uh oh!

Copilot AI commented Oct 19, 2025

Uh oh!

Copilot AI commented Oct 19, 2025

Uh oh!

juliayakovlev commented Oct 19, 2025

Uh oh!

Copilot AI commented Oct 19, 2025

Uh oh!

fruch commented Oct 19, 2025

Uh oh!

juliayakovlev commented Oct 21, 2025

Uh oh!

Copilot AI commented Oct 21, 2025

Uh oh!

fruch commented Oct 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Oct 19, 2025 •

edited

Loading