Skip to content

test(e2e): Adds Feedback Widget Capture Screenshot Maestro E2E tests #4795

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 6 commits into
base: antonis/feedback-e2e
Choose a base branch
from
Draft

test(e2e): Adds Feedback Widget Capture Screenshot Maestro E2E tests #4795

wants to merge 6 commits into from
+137 −2

Conversation

antonis
Copy link
Collaborator

@antonis antonis commented May 1, 2025 •
edited
Loading

📢 Type of change

  • Bugfix
  • New feature
  • Enhancement
  • Refactoring

Based on #4604

📜 Description

Adds Feedback Widget Capture Screenshot tests

💡 Motivation and Context

Part of #4302

💚 How did you test it?

CI (example of submitted feedback via the CI), Local:

Screen.Recording.2025-05-01.at.8.20.15.AM.mov

📝 Checklist

  • I added tests to verify changes
  • No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled
  • I updated the docs if needed.
  • I updated the wizard if needed.
  • All tests passing
  • No breaking changes

🔮 Next steps

#skip-changelog

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented May 1, 2025 •
edited
Loading

Android (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 432.92 ms 449.00 ms 16.08 ms
Size 17.75 MiB 20.15 MiB 2.40 MiB

Baseline results on branch: antonis/feedback-e2e

Startup times

Revision Plain With Sentry Diff
b12cb25 405.79 ms 404.68 ms -1.11 ms
7b594ae 456.70 ms 484.43 ms 27.73 ms
5bec854 430.17 ms 428.70 ms -1.47 ms

App size

Revision Plain With Sentry Diff
b12cb25 17.75 MiB 20.15 MiB 2.40 MiB
7b594ae 17.75 MiB 20.15 MiB 2.40 MiB
5bec854 17.75 MiB 20.15 MiB 2.40 MiB

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented May 1, 2025 •
edited
Loading

iOS (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1243.39 ms 1248.50 ms 5.11 ms
Size 3.19 MiB 4.37 MiB 1.18 MiB

Baseline results on branch: antonis/feedback-e2e

Startup times

Revision Plain With Sentry Diff
7b594ae+dirty 1217.51 ms 1222.51 ms 5.00 ms
b12cb25+dirty 1226.55 ms 1229.33 ms 2.78 ms
5bec854+dirty 1222.28 ms 1230.38 ms 8.10 ms

App size

Revision Plain With Sentry Diff
7b594ae+dirty 3.19 MiB 4.37 MiB 1.18 MiB
b12cb25+dirty 3.19 MiB 4.37 MiB 1.18 MiB
5bec854+dirty 3.19 MiB 4.37 MiB 1.18 MiB

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented May 1, 2025 •
edited
Loading

iOS (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1223.81 ms 1226.15 ms 2.33 ms
Size 2.63 MiB 3.80 MiB 1.17 MiB

Baseline results on branch: antonis/feedback-e2e

Startup times

Revision Plain With Sentry Diff
7b594ae+dirty 1217.33 ms 1216.65 ms -0.68 ms
b12cb25+dirty 1210.59 ms 1216.63 ms 6.04 ms
5bec854+dirty 1230.53 ms 1227.24 ms -3.29 ms

App size

Revision Plain With Sentry Diff
7b594ae+dirty 2.63 MiB 3.80 MiB 1.17 MiB
b12cb25+dirty 2.63 MiB 3.80 MiB 1.17 MiB
5bec854+dirty 2.63 MiB 3.80 MiB 1.17 MiB

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented May 1, 2025 •
edited
Loading

Android (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 417.00 ms 412.90 ms -4.10 ms
Size 7.15 MiB 8.42 MiB 1.26 MiB

Baseline results on branch: antonis/feedback-e2e

Startup times

Revision Plain With Sentry Diff
b12cb25+dirty 382.96 ms 391.34 ms 8.38 ms
7b594ae+dirty 400.00 ms 406.96 ms 6.96 ms
5bec854+dirty 393.61 ms 385.96 ms -7.65 ms

App size

Revision Plain With Sentry Diff
b12cb25+dirty 7.15 MiB 8.42 MiB 1.26 MiB
7b594ae+dirty 7.15 MiB 8.42 MiB 1.26 MiB
5bec854+dirty 7.15 MiB 8.42 MiB 1.26 MiB

@antonis
Copy link
Collaborator Author

antonis commented May 1, 2025

The Android app fails to open the widget on CI after the capture screenshot step. Leaving this PR in draft till the issue is investigated and fixed.

Maestro logs 

08:43:06.268 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Input text [email protected] COMPLETED
08:43:06.268 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Tap on "What's the bug? What did you expect?" RUNNING
08:43:06.398 [ INFO] maestro.Maestro.tap-BUbHBYE: Tapping on element:  UiElement(treeNode=TreeNode(attributes={text=What's the bug? What did you expect?, accessibilityText=, hintText=What's the bug? What did you expect?, ignoreBoundsFiltering=false, resource-id=, clickable=true, bounds=[20,345][300,445], enabled=true, focused=false, checked=false, scrollable=false, selected=false, class=android.widget.EditText}, children=[], clickable=true, enabled=true, focused=false, checked=false, selected=false), bounds=Bounds(x=20, y=345, width=280, height=100))
08:43:07.287 [ INFO] maestro.Maestro.tap-BUbHBYE: Refreshed element
08:43:07.287 [ INFO] maestro.Maestro.hierarchyBasedTap-hbl3e4M: Tapping at (160, 395) using hierarchy based logic for wait
08:43:07.287 [TRACE] maestro.utils.ScreenshotUtils.takeScreenshot: Taking screenshot to byte array
08:43:07.287 [TRACE] maestro.utils.ScreenshotUtils.takeScreenshot: Taking screenshot to output sink
08:43:08.157 [ INFO] maestro.Maestro.hierarchyBasedTap-hbl3e4M: Something have changed in the UI judging by view hierarchy. Proceed.
08:43:08.158 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Tap on "What's the bug? What did you expect?" COMPLETED
08:43:08.158 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Input text This is a test feedback message with a screenshot from CI e2e tests RUNNING
08:43:08.158 [ INFO] maestro.Maestro.inputText: Inputting text: This is a test feedback message with a screenshot from CI e2e tests
08:43:15.437 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Input text This is a test feedback message with a screenshot from CI e2e tests COMPLETED
08:43:15.437 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Scrolling DOWN until "Take a screenshot" is visible. RUNNING
08:43:15.608 [ INFO] maestro.orchestra.Orchestra.scrollUntilVisible: Scrolling try count: 0, DeviceWidth: 320, DeviceWidth: 640
08:43:15.608 [ INFO] maestro.orchestra.Orchestra.scrollUntilVisible: Element bounds: Bounds(x=96, y=466, width=128, height=22)
08:43:15.608 [ INFO] maestro.orchestra.Orchestra.scrollUntilVisible: Visibility Percent: 0
08:43:15.609 [ INFO] maestro.orchestra.Orchestra.scrollUntilVisible: Command centerElement: ScrollUntilVisibleCommand(selector=ElementSelector(textRegex=Take a screenshot, idRegex=null, size=null, below=null, above=null, leftOf=null, rightOf=null, containsChild=null, containsDescendants=null, traits=null, index=null, enabled=null, optional=false, selected=null, checked=null, focused=null, childOf=null), direction=DOWN, scrollDuration=601, visibilityPercentage=100, timeout=20000, waitToSettleTimeoutMs=null, centerElement=false, label=null, optional=false).centerElement
08:43:15.609 [ INFO] maestro.orchestra.Orchestra.scrollUntilVisible: visibilityPercentageNormalized: 1.0
08:43:15.609 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Scrolling DOWN until "Take a screenshot" is visible. COMPLETED
08:43:15.609 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Tap on "Take a screenshot" RUNNING
08:43:15.696 [ INFO] maestro.Maestro.tap-BUbHBYE: Tapping on element:  UiElement(treeNode=TreeNode(attributes={text=Take a screenshot, accessibilityText=, hintText=, ignoreBoundsFiltering=false, resource-id=sentry-feedback-take-screenshot-button, clickable=false, bounds=[96,466][224,488], enabled=true, focused=false, checked=false, scrollable=false, selected=false, class=android.widget.TextView}, children=[], clickable=false, enabled=true, focused=false, checked=false, selected=false), bounds=Bounds(x=96, y=466, width=128, height=22))
08:43:16.603 [ INFO] maestro.Maestro.tap-BUbHBYE: Refreshed element
08:43:16.603 [ INFO] maestro.Maestro.hierarchyBasedTap-hbl3e4M: Tapping at (160, 477) using hierarchy based logic for wait
08:43:16.603 [TRACE] maestro.utils.ScreenshotUtils.takeScreenshot: Taking screenshot to byte array
08:43:16.603 [TRACE] maestro.utils.ScreenshotUtils.takeScreenshot: Taking screenshot to output sink
08:43:17.755 [ INFO] maestro.Maestro.hierarchyBasedTap-hbl3e4M: Something have changed in the UI judging by view hierarchy. Proceed.
08:43:17.755 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Tap on "Take a screenshot" COMPLETED
08:43:17.755 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Tap on "Take Screenshot" RUNNING
08:43:17.883 [ INFO] maestro.Maestro.tap-BUbHBYE: Tapping on element:  UiElement(treeNode=TreeNode(attributes={text=Take Screenshot, accessibilityText=, hintText=, ignoreBoundsFiltering=false, resource-id=sentry-feedback-screenshot-button, clickable=false, bounds=[140,569][274,594], enabled=true, focused=false, checked=false, scrollable=false, selected=false, class=android.widget.TextView}, children=[], clickable=false, enabled=true, focused=false, checked=false, selected=false), bounds=Bounds(x=140, y=569, width=134, height=25))
08:43:18.778 [ INFO] maestro.Maestro.tap-BUbHBYE: Refreshed element
08:43:18.779 [ INFO] maestro.Maestro.hierarchyBasedTap-hbl3e4M: Tapping at (207, 581) using hierarchy based logic for wait
08:43:18.779 [TRACE] maestro.utils.ScreenshotUtils.takeScreenshot: Taking screenshot to byte array
08:43:18.779 [TRACE] maestro.utils.ScreenshotUtils.takeScreenshot: Taking screenshot to output sink
08:43:20.000 [ INFO] maestro.Maestro.hierarchyBasedTap-hbl3e4M: Something have changed in the UI judging by view hierarchy. Proceed.
08:43:20.000 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Tap on "Take Screenshot" COMPLETED
08:43:20.000 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Assert that "Report a Bug" is visible RUNNING
08:43:25.267 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Assert that "Report a Bug" is visible FAILED
08:43:25.268 [ INFO] maestro.Maestro.takeScreenshot: Taking screenshot
08:43:25.268 [TRACE] maestro.utils.ScreenshotUtils.takeScreenshot: Taking screenshot to output sink
08:43:25.327 [ERROR] maestro.orchestra.Orchestra.executeCommands: [Command execution] CommandFailed: Assertion is false: "Report a Bug" is visible
08:43:25.327 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Run feedback/captureFlow-android.yml when Platform is ANDROID FAILED

antonis
antonis commented May 2, 2025
View reviewed changes
dev-packages/e2e-tests/maestro/feedback/captureFlow-android.yml
timeout: 5_000

# Hide keyboard by tapping on a non-tappable element
- tapOn: 'Email'
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tapping on a non-tappable region is a work around to hide the keyboard since:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@krystofwoldrich krystofwoldrich Awaiting requested review from krystofwoldrich krystofwoldrich will be requested when the pull request is marked ready for review krystofwoldrich is a code owner

@lucas-zimerman lucas-zimerman Awaiting requested review from lucas-zimerman lucas-zimerman will be requested when the pull request is marked ready for review lucas-zimerman is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@antonis