test: Integration test for fiori MCP using promptfoo #3705

815are · 2025-10-06T16:58:28Z

Version for Integration test for fiori MCP using promptfoo -> https://github.com/SAP/open-ux-tools/tree/feat/mcpServerPromptFooIntTest/packages/fiori-mcp-server/test/integration

Current commands/scripts:

pnpm test:integration:once - run tests single time;
pnpm test:integration:multiple - run each test 5 times;
pnpm view:integration - opens promptfoo view with test runs history;

Structure:

integration
├── assertions - contains custom assertion function to validate snapshots
├── extensions - contains extension points to handle beforeEach/beforeAll to copy fresh project before running test scenario
├── provider - custom promptfoo provider which uses @sap-ai-sdk and add tools from fiori mcp server
├── scenarios - contains prmotfoo config yaml file and tests separated into subfolders
├── snapshots - contains snapshots files which then used by assertion
├── test-data
│   ├── copy - copied project during test execution
│   └── original - contains all original apps which will be copied when test is executed(if project is listed in config of test)
└── types - separate folder for reusable types, currently only constants with folder paths

Current tests -> https://github.com/SAP/open-ux-tools/tree/feat/mcpServerPromptFooIntTest/packages/fiori-mcp-server/test/integration/scenarios

Additional change in https://github.com/SAP/open-ux-tools/blob/main/packages/fiori-mcp-server/src/types/input.ts - description for ExecuteFunctionalityInputSchema is updated as sometimes parameters was not passed within parameters object, but outside of parameters property

very first version

changeset-bot · 2025-10-06T16:58:33Z

🦋 Changeset detected

Latest commit: 8556d73

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@sap-ux/fiori-mcp-server	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

project path

dependencies resolution update

structure

add local test project and setup to use such project during test

some experiments and custom assert with snapshot

snapshot check

snapshots

text

output

snapshot segments and loose check and allow pass setup files

more tests

remove demo approach

cleanup

more scenarios

adjustments

delete obsolete files

additional test

change folder structure

lint

snapshot

cleanup

…AP/open-ux-tools into feat/mcpServerPromptFooIntTest

lint

retrigger

donal-tobin-sap

package-lock from test should not be added to source control

review comment

815are · 2025-11-10T12:12:31Z

package-lock from test should not be added to source control

@donal-tobin-sap
2 lock files from test project were removed within commit -> 8556d73

sonarqubecloud · 2025-11-10T12:17:38Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

packages/fiori-mcp-server/.gitignore

donal-tobin-sap

Changes look to be fine to me.

* origin/main: chore: apply latest changesets fix: bump specVersion to v4.0 (#3744) chore: apply latest changesets test: Integration test for fiori MCP using promptfoo (#3705)

* main: (105 commits) chore: apply latest changesets Resolve Transport Request Checks for Download Quick Fiori App Deploy (#3819) test: v4 project builder (#3682) chore: apply latest changesets fix: bump specVersion to v4.0 (#3744) chore: apply latest changesets test: Integration test for fiori MCP using promptfoo (#3705) chore: apply latest changesets fix: move message to store creds prompt, tests, changeset (#3820) chore: apply latest changesets fix: bump fallback versions (#3813) chore: apply latest changesets chore - upgrade ui5 devDeps (#3771) correct publisher to SAPOSS (#3817) chore: apply latest changesets fix: cloud system render logic, message wording, test connection logging (#3811) chore: apply latest changesets chore: bump application insights to 2.9.8 (#3812) chore: apply latest changesets fix(fiori-mcp): instrumentation key replacement (#3810) ...

* origin/main: chore: apply latest changesets feat(create): auto generate readme on build (#3760) chore: apply latest changesets Resolve Transport Request Checks for Download Quick Fiori App Deploy (#3819) test: v4 project builder (#3682) chore: apply latest changesets fix: bump specVersion to v4.0 (#3744) chore: apply latest changesets test: Integration test for fiori MCP using promptfoo (#3705) chore: apply latest changesets fix: move message to store creds prompt, tests, changeset (#3820) chore: apply latest changesets fix: bump fallback versions (#3813)

…ter/building-block-custom-filter * 'main' of github.com:SAP/open-ux-tools: (32 commits) chore: apply latest changesets feat(create): auto generate readme on build (#3760) chore: apply latest changesets Resolve Transport Request Checks for Download Quick Fiori App Deploy (#3819) test: v4 project builder (#3682) chore: apply latest changesets fix: bump specVersion to v4.0 (#3744) chore: apply latest changesets test: Integration test for fiori MCP using promptfoo (#3705) chore: apply latest changesets fix: move message to store creds prompt, tests, changeset (#3820) chore: apply latest changesets fix: bump fallback versions (#3813) chore: apply latest changesets chore - upgrade ui5 devDeps (#3771) correct publisher to SAPOSS (#3817) chore: apply latest changesets fix: cloud system render logic, message wording, test connection logging (#3811) chore: apply latest changesets chore: bump application insights to 2.9.8 (#3812) ...

…i_custom_into_fiori_tools_plugin * origin/eslint9: (52 commits) chore: apply latest changesets update building block svg (#3826) chore: apply latest changesets feat(create): auto generate readme on build (#3760) chore: apply latest changesets Resolve Transport Request Checks for Download Quick Fiori App Deploy (#3819) test: v4 project builder (#3682) chore: apply latest changesets fix: bump specVersion to v4.0 (#3744) chore: apply latest changesets test: Integration test for fiori MCP using promptfoo (#3705) chore: apply latest changesets fix: move message to store creds prompt, tests, changeset (#3820) chore: apply latest changesets fix: bump fallback versions (#3813) chore: apply latest changesets chore - upgrade ui5 devDeps (#3771) correct publisher to SAPOSS (#3817) chore: apply latest changesets fix: cloud system render logic, message wording, test connection logging (#3811) ...

…fiori_tools_plugin' into eslint9_merge_fiori_custom_into_fiori_tools_plugin_update_templates * origin/eslint9_merge_fiori_custom_into_fiori_tools_plugin: chore: apply latest changesets update building block svg (#3826) chore: apply latest changesets feat(create): auto generate readme on build (#3760) chore: apply latest changesets Resolve Transport Request Checks for Download Quick Fiori App Deploy (#3819) test: v4 project builder (#3682) chore: apply latest changesets fix: bump specVersion to v4.0 (#3744) chore: apply latest changesets test: Integration test for fiori MCP using promptfoo (#3705) chore: apply latest changesets fix: move message to store creds prompt, tests, changeset (#3820) chore: apply latest changesets fix: bump fallback versions (#3813)

* origin/main: (31 commits) chore: apply latest changesets fix: improve handling for saving systems (#3828) chore: apply latest changesets chore: Update `@sap/cf-tools` version to latest to avoid security issues (#3809) chore: apply latest changesets feat(fiori-docs-embeddings): add @sap-ux/create/README.md as resource (#3779) fix: run tests on windows (#3823) chore: apply latest changesets update building block svg (#3826) chore: apply latest changesets feat(create): auto generate readme on build (#3760) chore: apply latest changesets Resolve Transport Request Checks for Download Quick Fiori App Deploy (#3819) test: v4 project builder (#3682) chore: apply latest changesets fix: bump specVersion to v4.0 (#3744) chore: apply latest changesets test: Integration test for fiori MCP using promptfoo (#3705) chore: apply latest changesets fix: move message to store creds prompt, tests, changeset (#3820) ...

test: very first version

a3778b6

very first version

815are added the fiori-mcp @sap-ux/fiori-mcp label Oct 6, 2025

815are added 2 commits October 6, 2025 20:03

fix: project path

d0e3de6

project path

fix: dependencies resolution update

ccb9083

dependencies resolution update

815are force-pushed the feat/mcpServerPromptFooIntTest branch from 3ee8e61 to ccb9083 Compare October 8, 2025 10:38

815are and others added 24 commits October 8, 2025 13:40

feat: structure

45a94d2

structure

feat: add local test project and setup to use such project during test

029d64a

add local test project and setup to use such project during test

feat: some experiments and custom assert with snapshot

5b75239

some experiments and custom assert with snapshot

fix: snapshot check

1da2860

snapshot check

fix: snapshots

b45df53

snapshots

fix: text

ec63733

text

fix: output

38d6a83

output

feat: snapshot segments and loose check and allow pass setup files

d807f8e

snapshot segments and loose check and allow pass setup files

test: more tests

872f9dd

more tests

test: more tests

102eab8

more tests

test: remove demo approach

2518508

remove demo approach

feat: cleanup

5490baa

cleanup

test: more scenarios

1415a13

more scenarios

test: adjustments

9210c60

adjustments

feat: delete obsolete files

6954ec2

delete obsolete files

test: additional test

5e401f6

additional test

Merge branch 'main' into feat/mcpServerPromptFooIntTest

2ef2b6a

feat: change folder structure

79d723c

change folder structure

fix: lint

cb8039b

lint

fix: lint

9355a3a

lint

fix: snapshot

741f5d6

snapshot

Merge branch 'main' into feat/mcpServerPromptFooIntTest

87a2284

fix: cleanup

d9c9b91

cleanup

Merge branch 'feat/mcpServerPromptFooIntTest' of https://github.com/S…

9089898

…AP/open-ux-tools into feat/mcpServerPromptFooIntTest

815are force-pushed the feat/mcpServerPromptFooIntTest branch from 8ae5c40 to 0a849db Compare November 7, 2025 13:14

815are and others added 4 commits November 7, 2025 15:14

fix: lint

c4d0d52

lint

Merge branch 'main' into feat/mcpServerPromptFooIntTest

253a697

fix: retrigger

0793e8a

retrigger

fix: retrigger

8680493

retrigger

815are requested review from marufrasully and vinayhospete November 10, 2025 07:45

vinayhospete previously approved these changes Nov 10, 2025

View reviewed changes

marufrasully previously approved these changes Nov 10, 2025

View reviewed changes

donal-tobin-sap requested changes Nov 10, 2025

View reviewed changes

fix: review comment

8556d73

review comment

815are dismissed stale reviews from marufrasully and vinayhospete via 8556d73 November 10, 2025 12:10

815are requested review from donal-tobin-sap, marufrasully and vinayhospete November 10, 2025 12:12

marufrasully approved these changes Nov 10, 2025

View reviewed changes

donal-tobin-sap reviewed Nov 10, 2025

View reviewed changes

packages/fiori-mcp-server/.gitignore Show resolved Hide resolved

donal-tobin-sap approved these changes Nov 10, 2025

View reviewed changes

815are merged commit bfbdb77 into main Nov 10, 2025
16 checks passed

815are deleted the feat/mcpServerPromptFooIntTest branch November 10, 2025 12:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

test: Integration test for fiori MCP using promptfoo #3705

test: Integration test for fiori MCP using promptfoo #3705

Uh oh!

815are commented Oct 6, 2025 •

edited

Loading

Uh oh!

changeset-bot bot commented Oct 6, 2025 •

edited

Loading

Uh oh!

donal-tobin-sap left a comment

Uh oh!

815are commented Nov 10, 2025

Uh oh!

sonarqubecloud bot commented Nov 10, 2025

Uh oh!

Uh oh!

donal-tobin-sap left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

test: Integration test for fiori MCP using promptfoo #3705

test: Integration test for fiori MCP using promptfoo #3705

Uh oh!

Conversation

815are commented Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot bot commented Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

donal-tobin-sap left a comment

Choose a reason for hiding this comment

Uh oh!

815are commented Nov 10, 2025

Uh oh!

sonarqubecloud bot commented Nov 10, 2025

Quality Gate passed

Uh oh!

Uh oh!

donal-tobin-sap left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

815are commented Oct 6, 2025 •

edited

Loading

changeset-bot bot commented Oct 6, 2025 •

edited

Loading