Skip to content

Add e2e tests #354

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 29 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
1424f63
feat: QoL improvements for OAuth Callback
max-stytch Apr 7, 2025
03e89f7
Add minimal test and provider to start
olaservo Apr 12, 2025
e1911e4
Fix merger issues
olaservo Apr 12, 2025
c8130ea
Refactor tests and test utils to use proxy from sdk
olaservo Apr 12, 2025
3045b64
Leave myself notes on patterns
olaservo Apr 12, 2025
166a706
Fix unused vars
olaservo Apr 12, 2025
6a52938
Fix matcher not found
olaservo Apr 13, 2025
ae5709b
Remove unused import
olaservo Apr 13, 2025
15c651d
Update gitignore to ignore testing output files
olaservo Apr 24, 2025
732c486
Set up packages and configs
olaservo Apr 24, 2025
ae81492
Add ID to connect button
olaservo Apr 24, 2025
e4f8c0a
Remove other test files
olaservo Apr 24, 2025
c0cb1f4
Add logging for tests
olaservo Apr 24, 2025
1210f0f
Placeholder test and add clean script
olaservo Apr 25, 2025
b65d4b7
Merge branch 'main' into add-integration-tests-take-2
olaservo Apr 25, 2025
ccfd631
Update package-lock after install
olaservo Apr 25, 2025
0562df8
Merge branch 'main' into add-integration-tests-take-2
olaservo Apr 25, 2025
3adb5d1
Delete doc
olaservo Apr 25, 2025
3049942
Revert jest config change
olaservo Apr 25, 2025
ad7fa0e
Delete test utils
olaservo Apr 25, 2025
37db7a7
Add a passing sse test
olaservo Apr 26, 2025
ad629fe
Delete package-lock.json on clean
olaservo Apr 27, 2025
c3689c4
Add everything as dev dependency and update test scripts to start ser…
olaservo Apr 27, 2025
4cea627
Add comment
olaservo Apr 27, 2025
f55d5ef
Use separate projects for stdio and sse
olaservo Apr 27, 2025
b5fd3ad
Add check for initialize response info in history pane
olaservo Apr 27, 2025
d1c69c2
Add check for initialize response info in history pane for sse and re…
olaservo Apr 27, 2025
08b9412
Add some placeholder oauth and a few lines from previous attempt
olaservo Apr 28, 2025
8f0750a
Remove root scripts for now
olaservo Apr 28, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions client/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,7 @@ dist-ssr
*.njsproj
*.sln
*.sw?

# Testing output
playwright-report
test-results
115 changes: 115 additions & 0 deletions client/e2e/sse/connectServer.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
import { test, expect } from '@playwright/test';

test.describe('Connect MCP Server Using SSE In Inspector UI', () => {

test('should connect to test server using SSE and verify echo tool', async ({ page }) => {
// Load the inspector UI
await page.goto('/');

// Set server URL and transport type in localStorage
await page.evaluate(() => {
localStorage.setItem('lastSseUrl', 'http://localhost:3001/sse');
localStorage.setItem('lastTransportType', 'sse');
});
await page.reload();

// Wait for and click connect button
const connectButton = await page.waitForSelector('[data-testid="connect-button"]', { state: 'visible' });
await connectButton.click();

// Wait for connection status to show connected and verify
const connectedText = page.getByText('Connected');
await expect(connectedText).toBeVisible({ timeout: 10000 });

// Find and expand the initialize entry in History
const initializeEntry = page.getByText('1. initialize');
await expect(initializeEntry).toBeVisible();
await initializeEntry.click();

// Verify server info content in initialize response from History pane
const jsonView = page.getByTestId('history-entry-0-response')
.locator('.font-mono');
await expect(jsonView).toBeVisible();
await expect(jsonView).toContainText('capabilities');
await expect(jsonView).toContainText('serverInfo');
await expect(jsonView).toContainText('example-servers/everything');
await expect(jsonView).toContainText('version');

// Navigate to Tools tab and wait for it to be active
const toolsTab = page.getByRole('tab', { name: 'Tools' });
await toolsTab.click();
await expect(toolsTab).toHaveAttribute('aria-selected', 'true');

// Click the List Tools button to load available tools
const listToolsButton = page.getByRole('button', { name: 'List Tools' });
await expect(listToolsButton).toBeVisible();
await listToolsButton.click();

// Wait for tools list to be loaded and verify echo tool exists
const echoTool = page.getByText('echo', { exact: true });
await expect(echoTool).toBeVisible({ timeout: 5000 });
await echoTool.click();

// Wait for message input to be available
const messageInput = page.getByLabel('message');
await expect(messageInput).toBeVisible();
await messageInput.fill('Hello MCP!');

// Call the tool and wait for response
const callButton = page.getByRole('button', { name: 'Run Tool' });
await expect(callButton).toBeEnabled();
await callButton.click();

// Verify the response with increased timeout
await expect(page.getByText('Echo: Hello MCP!')).toBeVisible({ timeout: 10000 });
});

test('completes OAuth authentication flow', async ({ page }) => {
/* TODO: Add auth wrapper and reference for tests, then complete the following:
* This test will verify the e2e OAuth authentication flow:
* - Verify 401 Unauthorized triggers OAuth flow
* - Check OAuth metadata endpoint access (no CORS errors)
* - Verify redirect sequence
*
* Post-Auth Verification:
* - Verify successful connection after auth
* - Check server capabilities and tools access
*/

// Load the inspector UI
await page.goto('/');

// Set server URL and transport type in localStorage
await page.evaluate(() => {
localStorage.setItem('lastSseUrl', 'http://localhost:3001/sse');
localStorage.setItem('lastTransportType', 'sse');
});
await page.reload();

// Collect all console messages
const consoleMessages: string[] = [];
page.on('console', msg => {
consoleMessages.push(msg.text());
});

// Wait for and click connect button
const connectButton = await page.waitForSelector('[data-testid="connect-button"]', { state: 'visible' });
await connectButton.click();

// Original assertions from previous implementation attempt for reference:
// Wait for login page redirect to consent UI
// await page.waitForURL(/localhost:3001\/consent/, { timeout: 35000 }); // Increased to account for 30s delay

// Verify console messages appeared in correct order
expect(consoleMessages).toContainEqual(expect.stringContaining('401 (Unauthorized)'));
// expect(consoleMessages).toContainEqual(expect.stringContaining('Failed to connect to MCP Server via the MCP Inspector Proxy'));
// expect(consoleMessages).toContainEqual(expect.stringContaining('[Auth Flow] Got 401, starting OAuth flow'));
// expect(consoleMessages).toContainEqual(expect.stringContaining('[Auth Flow] Saved server URL:'));

// Verify OAuth metadata endpoint was accessed successfully (no CORS error)
expect(consoleMessages).not.toContainEqual(expect.stringContaining('blocked by CORS policy'));

// Verify redirect sequence
// expect(consoleMessages).toContainEqual(expect.stringContaining('[Auth Flow] Redirecting to:'));
});
});
65 changes: 65 additions & 0 deletions client/e2e/stdio/connectServer.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
import { test, expect } from '@playwright/test';

test.describe('Connect MCP Server Using stdio In Inspector UI', () => {

test('should connect to test server using stdio and verify echo tool', async ({ page }) => {
// Load the inspector UI
await page.goto('/');

// Set transport type in localStorage (stdio is default, no URL needed)
await page.evaluate(() => {
localStorage.setItem('lastTransportType', 'stdio');
});
await page.reload();

// Wait for and click connect button
const connectButton = await page.waitForSelector('[data-testid="connect-button"]', { state: 'visible' });
await connectButton.click();

// Wait for connection status to show connected and verify
const connectedText = page.getByText('Connected');
await expect(connectedText).toBeVisible({ timeout: 10000 });

// Find and expand the initialize entry in History
const initializeEntry = page.getByText('1. initialize');
await expect(initializeEntry).toBeVisible();
await initializeEntry.click();

// Verify server info content in initialize response from History pane
const jsonView = page.getByTestId('history-entry-0-response')
.locator('.font-mono');
await expect(jsonView).toBeVisible();
await expect(jsonView).toContainText('capabilities');
await expect(jsonView).toContainText('serverInfo');
await expect(jsonView).toContainText('example-servers/everything');
await expect(jsonView).toContainText('version');

// Navigate to Tools tab and wait for it to be active
const toolsTab = page.getByRole('tab', { name: 'Tools' });
await toolsTab.click();
await expect(toolsTab).toHaveAttribute('aria-selected', 'true');

// Click the List Tools button to load available tools
const listToolsButton = page.getByRole('button', { name: 'List Tools' });
await expect(listToolsButton).toBeVisible();
await listToolsButton.click();

// Wait for tools list to be loaded and verify echo tool exists
const echoTool = page.getByText('echo', { exact: true });
await expect(echoTool).toBeVisible({ timeout: 5000 });
await echoTool.click();

// Wait for message input to be available
const messageInput = page.getByLabel('message');
await expect(messageInput).toBeVisible();
await messageInput.fill('Hello MCP!');

// Call the tool and wait for response
const callButton = page.getByRole('button', { name: 'Run Tool' });
await expect(callButton).toBeEnabled();
await callButton.click();

// Verify the response with increased timeout
await expect(page.getByText('Echo: Hello MCP!')).toBeVisible({ timeout: 10000 });
});
});
13 changes: 13 additions & 0 deletions client/e2e/tsconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"extends": "../tsconfig.json",
"compilerOptions": {
"outDir": "dist",
"rootDir": ".",
"target": "ES2022",
"module": "Node16",
"moduleResolution": "Node16"
},
"include": [
"./**/*.ts"
]
}
21 changes: 19 additions & 2 deletions client/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,16 @@
"lint": "eslint .",
"preview": "vite preview --port 6274",
"test": "jest --config jest.config.cjs",
"test:watch": "jest --config jest.config.cjs --watch"
"test:watch": "jest --config jest.config.cjs --watch",
"start:test-server:sse": "cd ../node_modules/@modelcontextprotocol/server-everything && npm run start:sse",
"start:test-server:stdio": "cd ../node_modules/@modelcontextprotocol/server-everything && npm run start",
"test:e2e:sse": "concurrently --kill-others --success first \"npm run start:test-server:sse\" \"playwright test --project=sse\"",
"test:e2e:stdio": "concurrently --kill-others --success first \"npm run start:test-server:stdio\" \"playwright test --project=stdio\"",
"test:e2e:sse:ui": "concurrently --kill-others --success first \"npm run start:test-server:sse\" \"playwright test --project=sse --ui\"",
"test:e2e:stdio:ui": "concurrently --kill-others --success first \"npm run start:test-server:stdio\" \"playwright test --project=stdio --ui\"",
"test:e2e:sse:debug": "concurrently --kill-others --success first \"npm run start:test-server:sse\" \"playwright test --project=sse --debug\"",
"test:e2e:stdio:debug": "concurrently --kill-others --success first \"npm run start:test-server:stdio\" \"playwright test --project=stdio --debug\"",
"install:browsers": "playwright install chromium"
},
"dependencies": {
"@modelcontextprotocol/sdk": "^1.10.0",
Expand Down Expand Up @@ -49,21 +58,28 @@
"zod": "^3.23.8"
},
"devDependencies": {
"@modelcontextprotocol/server-everything": "2025.4.25",
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of these dependency updates are probably from a previous attempt on same branch - main ones to pay attention to are importing server-everything and adding Playwright to dev dependences.

"@eslint/js": "^9.11.1",
"@playwright/test": "^1.51.1",
"@testing-library/jest-dom": "^6.6.3",
"@testing-library/react": "^16.2.0",
"@types/cors": "^2.8.17",
"@types/express": "^5.0.1",
"@types/jest": "^29.5.14",
"@types/node": "^22.7.5",
"@types/prismjs": "^1.26.5",
"@types/react": "^18.3.10",
"@types/react-dom": "^18.3.0",
"@types/serve-handler": "^6.1.4",
"@types/testing-library__jest-dom": "^5.14.9",
"@vitejs/plugin-react": "^4.3.2",
"autoprefixer": "^10.4.20",
"co": "^4.6.0",
"cors": "^2.8.5",
"eslint": "^9.11.1",
"eslint-plugin-react-hooks": "^5.1.0-rc.0",
"eslint-plugin-react-refresh": "^0.4.12",
"express": "^5.1.0",
"globals": "^15.9.0",
"jest": "^29.7.0",
"jest-environment-jsdom": "^29.7.0",
Expand All @@ -72,6 +88,7 @@
"ts-jest": "^29.2.6",
"typescript": "^5.5.3",
"typescript-eslint": "^8.7.0",
"vite": "^6.3.0"
"vite": "^6.3.0",
"wait-on": "^8.0.3"
}
}
41 changes: 41 additions & 0 deletions client/playwright.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
testDir: './e2e',
timeout: 60 * 1000,
expect: {
timeout: 5000
},
fullyParallel: true,
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0,
workers: 1,
reporter: 'list',
use: {
baseURL: process.env.APP_URL || 'http://localhost:6274',
trace: 'on-first-retry',
},
projects: [
{
name: 'sse',
testDir: './e2e/sse',
testMatch: '**/*.spec.ts',
use: {
...devices['Desktop Chrome'],
},
},
{
name: 'stdio',
testDir: './e2e/stdio',
testMatch: '**/*.spec.ts',
use: {
...devices['Desktop Chrome'],
},
},
],
webServer: {
command: 'npm run dev',
url: 'http://localhost:6274',
reuseExistingServer: !process.env.CI,
},
});
6 changes: 5 additions & 1 deletion client/src/components/History.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -68,10 +68,14 @@ const HistoryAndNotifications = ({
<JsonView
data={request.request}
className="bg-background"
data-testid={`history-entry-${requestHistory.length - 1 - index}-request`}
/>
</div>
{request.response && (
<div className="mt-2">
<div
className="mt-2"
data-testid={`history-entry-${requestHistory.length - 1 - index}-response`}
>
<div className="flex justify-between items-center mb-1">
<span className="font-semibold text-green-600">
Response:
Expand Down
6 changes: 5 additions & 1 deletion client/src/components/Sidebar.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -467,7 +467,11 @@ const Sidebar = ({
</div>
)}
{connectionStatus !== "connected" && (
<Button className="w-full" onClick={onConnect}>
<Button
className="w-full"
onClick={onConnect}
data-testid="connect-button"
>
<Play className="w-4 h-4 mr-2" />
Connect
</Button>
Expand Down
11 changes: 11 additions & 0 deletions client/tsconfig.e2e.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"extends": "./tsconfig.app.json",
"compilerOptions": {
"moduleResolution": "node",
"allowJs": true,
"types": ["node", "@playwright/test"],
"module": "ESNext",
"target": "ESNext"
},
"include": ["e2e/**/*"]
}
3 changes: 2 additions & 1 deletion client/tsconfig.jest.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"jsx": "react-jsx",
"esModuleInterop": true,
"module": "ESNext",
"moduleResolution": "node"
"moduleResolution": "node",
"types": ["jest", "@testing-library/jest-dom", "node"]
},
"include": ["src"]
}
Loading
Loading