[CI] Introduce Backend Integration Tests #3876
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the WalkthroughThe PR introduces comprehensive backend integration testing infrastructure by adding a new GitHub Actions workflow for CI automation, reorganizing Xcode project structure with a dedicated Backend group, creating Backend test plans, implementing backend test cases, and adding Fastlane lanes for executing tests against real and mock servers. Changes
Sequence DiagramsequenceDiagram
participant GH as GitHub Actions
participant WF as backend-checks Workflow
participant FL as Fastlane
participant Tests as Backend Tests
participant Backend as Backend Server
participant Mock as Mock Server
GH->>WF: Trigger on PR/dispatch
WF->>FL: Run test_e2e_mock lane
FL->>Mock: Start Sinatra mock server
Mock-->>FL: Server ready
FL->>Tests: Execute test suite (Backend.xctestplan)
Tests->>Backend: Send API requests
Backend-->>Tests: Respond
alt Tests Pass
Tests-->>FL: Success
FL-->>WF: Tests complete
else Tests Fail
Tests-->>FL: Failure detected
FL->>FL: Extract failed tests
FL->>Tests: Re-run failed tests only
Tests->>Backend: Retry requests
Backend-->>Tests: Respond
Tests-->>FL: Final result
end
FL-->>WF: Upload artifacts (logs, results)
WF-->>GH: Workflow complete
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes
Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (7)
.github/workflows/backend-checks.yml(1 hunks)StreamChat.xcodeproj/project.pbxproj(6 hunks)StreamChat.xcodeproj/xcshareddata/xcschemes/StreamChatUITestsApp.xcscheme(1 hunks)StreamChatUITestsAppUITests/Backend.xctestplan(1 hunks)StreamChatUITestsAppUITests/StreamChatUITestsApp.xctestplan(1 hunks)StreamChatUITestsAppUITests/Tests/Backend/Backend_Tests.swift(1 hunks)fastlane/Fastfile(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.swift
📄 CodeRabbit inference engine (AGENTS.md)
**/*.swift: Respect .swiftlint.yml rules; do not suppress SwiftLint rules broadly—scope and justify any exceptions
Adhere to the project’s zero warnings policy—fix new warnings and avoid introducing any
Files:
StreamChatUITestsAppUITests/Tests/Backend/Backend_Tests.swift
🧠 Learnings (6)
📓 Common learnings
Learnt from: CR
Repo: GetStream/stream-chat-swift PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-09-18T10:00:24.878Z
Learning: Applies to Tests/StreamChatTests/**/*.swift : Ensure tests cover core models and API surface of StreamChat
Learnt from: CR
Repo: GetStream/stream-chat-swift PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-09-18T10:00:24.878Z
Learning: Applies to Tests/StreamChatUITests/**/*.swift : Ensure tests cover view controllers and UI behaviors of StreamChatUI
Learnt from: CR
Repo: GetStream/stream-chat-swift PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-09-18T10:00:24.878Z
Learning: Applies to Tests/{StreamChatTests,StreamChatUITests}/**/*.swift : Add or extend tests in the matching module’s Tests folder
Learnt from: CR
Repo: GetStream/stream-chat-swift PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-09-18T10:00:24.878Z
Learning: Applies to Tests/{StreamChatTests,StreamChatUITests}/**/*.swift : Prefer using provided test fakes/mocks in tests when possible
📚 Learning: 2025-09-18T10:00:24.878Z
Learnt from: CR
Repo: GetStream/stream-chat-swift PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-09-18T10:00:24.878Z
Learning: Applies to Tests/{StreamChatTests,StreamChatUITests}/**/*.swift : Add or extend tests in the matching module’s Tests folder
Applied to files:
StreamChat.xcodeproj/xcshareddata/xcschemes/StreamChatUITestsApp.xcschemeStreamChatUITestsAppUITests/StreamChatUITestsApp.xctestplanStreamChatUITestsAppUITests/Tests/Backend/Backend_Tests.swiftStreamChatUITestsAppUITests/Backend.xctestplanStreamChat.xcodeproj/project.pbxproj
📚 Learning: 2025-09-18T10:00:24.878Z
Learnt from: CR
Repo: GetStream/stream-chat-swift PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-09-18T10:00:24.878Z
Learning: Applies to Tests/StreamChatTests/**/*.swift : Ensure tests cover core models and API surface of StreamChat
Applied to files:
StreamChat.xcodeproj/xcshareddata/xcschemes/StreamChatUITestsApp.xcschemeStreamChatUITestsAppUITests/StreamChatUITestsApp.xctestplanStreamChatUITestsAppUITests/Tests/Backend/Backend_Tests.swiftStreamChatUITestsAppUITests/Backend.xctestplanStreamChat.xcodeproj/project.pbxproj
📚 Learning: 2025-09-18T10:00:24.878Z
Learnt from: CR
Repo: GetStream/stream-chat-swift PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-09-18T10:00:24.878Z
Learning: Applies to Tests/StreamChatUITests/**/*.swift : Ensure tests cover view controllers and UI behaviors of StreamChatUI
Applied to files:
StreamChat.xcodeproj/xcshareddata/xcschemes/StreamChatUITestsApp.xcschemeStreamChatUITestsAppUITests/StreamChatUITestsApp.xctestplanStreamChatUITestsAppUITests/Tests/Backend/Backend_Tests.swiftStreamChatUITestsAppUITests/Backend.xctestplanStreamChat.xcodeproj/project.pbxproj
📚 Learning: 2025-09-18T10:00:24.878Z
Learnt from: CR
Repo: GetStream/stream-chat-swift PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-09-18T10:00:24.878Z
Learning: Applies to Tests/{StreamChatTests,StreamChatUITests}/**/*.swift : Prefer using provided test fakes/mocks in tests when possible
Applied to files:
StreamChat.xcodeproj/xcshareddata/xcschemes/StreamChatUITestsApp.xcschemeStreamChatUITestsAppUITests/StreamChatUITestsApp.xctestplanStreamChatUITestsAppUITests/Tests/Backend/Backend_Tests.swiftStreamChatUITestsAppUITests/Backend.xctestplanStreamChat.xcodeproj/project.pbxproj
📚 Learning: 2025-09-18T10:00:24.878Z
Learnt from: CR
Repo: GetStream/stream-chat-swift PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-09-18T10:00:24.878Z
Learning: Applies to Sources/{StreamChat,StreamChatUI}/**/*.swift : When altering public API, update inline documentation comments in source
Applied to files:
StreamChat.xcodeproj/xcshareddata/xcschemes/StreamChatUITestsApp.xcschemeStreamChatUITestsAppUITests/StreamChatUITestsApp.xctestplan
🧬 Code graph analysis (1)
StreamChatUITestsAppUITests/Tests/Backend/Backend_Tests.swift (2)
StreamChatUITestsAppUITests/Robots/UserRobot.swift (2)
login(20-24)openChannel(55-60)StreamChatUITestsAppUITests/Robots/UserRobot+Asserts.swift (4)
assertMessage(195-207)assertDeletedMessage(381-393)waitForNewReaction(993-1002)assertReaction(975-988)
🪛 actionlint (1.7.8)
.github/workflows/backend-checks.yml
28-28: property "batch" is not defined in object type {}
(expression)
🔇 Additional comments (8)
fastlane/Fastfile (2)
33-35: LGTM!The after_all hook properly ensures the Sinatra server is stopped after
test_e2e_mockexecution, preventing resource leaks.
541-565: LGTM!The
test_e2elane is well-implemented with proper error recovery that re-runs only failed tests. The configuration is appropriate for backend integration testing.StreamChat.xcodeproj/xcshareddata/xcschemes/StreamChatUITestsApp.xcscheme (1)
38-40: LGTM!The Backend test plan reference is correctly added to the scheme, consistent with the existing test plan structure.
StreamChatUITestsAppUITests/Tests/Backend/Backend_Tests.swift (4)
7-12: LGTM!The setup correctly disables the mock server and configures the API key for backend integration testing.
14-41: LGTM!The message test comprehensively covers the full lifecycle (send, edit, delete) with clear GIVEN/WHEN/THEN structure and proper assertions.
43-66: LGTM!The reaction test properly validates adding and removing reactions with appropriate wait conditions and assertions.
7-66: Update xctestplan files to reference actual test methods.The Backend.xctestplan and StreamChatUITestsApp.xctestplan files reference the non-existent test method
test_messageListUpdates_whenUserSendsMessage(). These must be updated to reference the actual test methods defined in Backend_Tests.swift:
- StreamChatUITestsAppUITests/Backend.xctestplan (line in selectedTests): replace
"Backend_Tests/test_messageListUpdates_whenUserSendsMessage()"with the correct test method names- StreamChatUITestsAppUITests/StreamChatUITestsApp.xctestplan (lines in skippedTests): remove or update the non-existent test reference
Available test methods:
test_message()andtest_reaction()⛔ Skipped due to learnings
Learnt from: CR Repo: GetStream/stream-chat-swift PR: 0 File: AGENTS.md:0-0 Timestamp: 2025-09-18T10:00:24.878Z Learning: Applies to Tests/StreamChatTests/**/*.swift : Ensure tests cover core models and API surface of StreamChatLearnt from: CR Repo: GetStream/stream-chat-swift PR: 0 File: AGENTS.md:0-0 Timestamp: 2025-09-18T10:00:24.878Z Learning: Applies to Tests/StreamChatUITests/**/*.swift : Ensure tests cover view controllers and UI behaviors of StreamChatUILearnt from: CR Repo: GetStream/stream-chat-swift PR: 0 File: AGENTS.md:0-0 Timestamp: 2025-09-18T10:00:24.878Z Learning: Applies to Tests/{StreamChatTests,StreamChatUITests}/**/*.swift : Add or extend tests in the matching module’s Tests folderLearnt from: CR Repo: GetStream/stream-chat-swift PR: 0 File: AGENTS.md:0-0 Timestamp: 2025-09-18T10:00:24.878Z Learning: Applies to Tests/{StreamChatTests,StreamChatUITests}/**/*.swift : Prefer using provided test fakes/mocks in tests when possibleLearnt from: CR Repo: GetStream/stream-chat-swift PR: 0 File: AGENTS.md:0-0 Timestamp: 2025-09-18T10:00:24.878Z Learning: Applies to Sources/{StreamChat,StreamChatUI}/**/*.swift : When altering public API, update inline documentation comments in sourceStreamChat.xcodeproj/project.pbxproj (1)
632-632: LGTM! Backend test infrastructure properly integrated.The project file changes correctly introduce the Backend test group and test plan. All file references, group hierarchy, and build phases are consistent and follow Xcode project structure conventions.
Also applies to: 3695-3696, 6318-6325, 6384-6384, 6858-6858, 12550-12550
| on: | ||
| workflow_dispatch: | ||
|
|
||
| pull_request: #TODO: DELETE AFTER TESTING |
There was a problem hiding this comment.
Remove TODO or define the trigger intention.
The TODO comment suggests this pull_request trigger is temporary for testing. Either remove it before merging or clarify the intended trigger conditions for this workflow.
🤖 Prompt for AI Agents
In .github/workflows/backend-checks.yml around line 6, the pull_request trigger
is marked with a TODO indicating it's temporary for testing; either remove the
pull_request entry if it’s not intended to run on PRs, or replace the TODO with
a clear trigger definition (e.g., specify which branches, paths, or events
should trigger this workflow) and delete the TODO comment so the workflow's
intent is explicit before merging.
.github/workflows/backend-checks.yml
Outdated
| INSTALL_YEETD: true | ||
| SKIP_SWIFT_BOOTSTRAP: true | ||
| - name: Run UI Tests (Debug) | ||
| run: bundle exec fastlane test_e2e device:"${{ env.IOS_SIMULATOR_DEVICE }}" batch:'${{ matrix.batch }}' |
There was a problem hiding this comment.
Critical: Missing matrix strategy definition.
Line 28 references matrix.batch, but the job doesn't define a matrix strategy. This will cause the workflow to fail with an undefined variable.
Add a matrix strategy to the job:
jobs:
test-backend-integration:
name: Test Backend Integration
runs-on: macos-15
+ strategy:
+ fail-fast: false
+ matrix:
+ batch: [0]
steps:Or remove the batch parameter if matrix execution isn't needed:
- run: bundle exec fastlane test_e2e device:"${{ env.IOS_SIMULATOR_DEVICE }}" batch:'${{ matrix.batch }}'
+ run: bundle exec fastlane test_e2e device:"${{ env.IOS_SIMULATOR_DEVICE }}"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| run: bundle exec fastlane test_e2e device:"${{ env.IOS_SIMULATOR_DEVICE }}" batch:'${{ matrix.batch }}' | |
| run: bundle exec fastlane test_e2e device:"${{ env.IOS_SIMULATOR_DEVICE }}" |
🧰 Tools
🪛 actionlint (1.7.8)
28-28: property "batch" is not defined in object type {}
(expression)
🤖 Prompt for AI Agents
.github/workflows/backend-checks.yml around line 28: the job references
matrix.batch but no strategy.matrix is defined, causing an undefined variable at
runtime; either add a strategy block with a matrix.batch array (e.g., strategy:
{ matrix: { batch: [1,2] } } or whatever batch values you need) to the job that
runs the Fastlane step, or remove the batch:'${{ matrix.batch }}' argument from
the run line if matrixed execution is not required; ensure the chosen fix is
applied to the same job that contains the run command so the workflow has a
valid matrix or no matrix reference.
| "selectedTests" : [ | ||
| "Backend_Tests\/test_messageListUpdates_whenUserSendsMessage()" | ||
| ], |
There was a problem hiding this comment.
Critical: Test method doesn't exist.
The test plan references Backend_Tests/test_messageListUpdates_whenUserSendsMessage(), but this test method doesn't exist in StreamChatUITestsAppUITests/Tests/Backend/Backend_Tests.swift. The actual test file only contains test_message() and test_reaction().
Apply this diff to reference an existing test:
"selectedTests" : [
- "Backend_Tests\/test_messageListUpdates_whenUserSendsMessage()"
+ "Backend_Tests\/test_message()"
],Alternatively, if you want to run all Backend_Tests, simply specify the test class:
"selectedTests" : [
- "Backend_Tests\/test_messageListUpdates_whenUserSendsMessage()"
+ "Backend_Tests"
],📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| "selectedTests" : [ | |
| "Backend_Tests\/test_messageListUpdates_whenUserSendsMessage()" | |
| ], | |
| "selectedTests" : [ | |
| "Backend_Tests\/test_message()" | |
| ], |
🤖 Prompt for AI Agents
In StreamChatUITestsAppUITests/Backend.xctestplan around lines 16-18, the
selectedTests entry references a non-existent test method
Backend_Tests/test_messageListUpdates_whenUserSendsMessage(); update the
"selectedTests" array to reference an existing test (e.g.,
"Backend_Tests/test_message()" or "Backend_Tests/test_reaction()"), or replace
the entry with the test class name "Backend_Tests" to run all tests in that
class so the test plan points to valid tests.
| "Backend_Tests", | ||
| "Backend_Tests\/test_messageListUpdates_whenUserSendsMessage()", |
There was a problem hiding this comment.
Critical: Test method doesn't exist.
Line 24 references Backend_Tests/test_messageListUpdates_whenUserSendsMessage(), but this test method doesn't exist in StreamChatUITestsAppUITests/Tests/Backend/Backend_Tests.swift. The actual test file only contains test_message() and test_reaction().
If you only want to skip the entire Backend_Tests suite (which makes sense since they run in a separate test plan), remove the specific test reference:
"skippedTests" : [
"Backend_Tests",
- "Backend_Tests\/test_messageListUpdates_whenUserSendsMessage()",
"ChannelListScrollTime",
"MessageListScrollTime"
],🤖 Prompt for AI Agents
In StreamChatUITestsAppUITests/StreamChatUITestsApp.xctestplan around lines
23-24, the plan references a non-existent test case
"Backend_Tests/test_messageListUpdates_whenUserSendsMessage()"; update the plan
to either remove the specific test reference (so the whole Backend_Tests suite
is skipped) or replace it with a valid test identifier from
StreamChatUITestsAppUITests/Tests/Backend/Backend_Tests.swift (e.g.,
test_message or test_reaction), ensuring the test path matches the actual method
name and signature.
Generated by 🚫 Danger |
SDK Size
|
|
Summary by CodeRabbit