feat: Add manifest validation tool #93

dimovpetar · 2025-10-21T11:49:05Z

No description provided.

coveralls · 2025-10-30T08:38:42Z

Pull Request Test Coverage Report for Build 19666892989

Details

110 of 112 (98.21%) changed or added relevant lines in 5 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.9%) to 82.266%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/utils/ui5Manifest.ts	47	49	95.92%

Totals
Change from base Build 19648105994:	0.9%
Covered Lines:	1111
Relevant Lines:	1255

💛 - Coveralls

alexandar-mitsev · 2025-11-03T10:49:02Z

Looks good to me

src/utils/ui5Manifest.ts

src/tools/run_manifest_validation/index.ts

alexandar-mitsev

now I often get "The MCP validation tool timed out". Seems that it happens mainly if I ask to validate a manifest, complete the task, then edit the manifest and ask to validate again.
Maybe it is related to the mutex log and cache. Or to the ajv tool itself
If I restart VS code, the tool can run again without a timeout. Also it runs smoothly couple of times in the same task. But fails with timeout if it runs in two different tasks in the same window.
every time when I validate manifest _version: "1.60.0" - I get an error
Error:
Failed to create UI5 manifest validate function: items value must be ["object","boolean"]
some older manifests are in the mapping, but have no tag for them, e.g. 1.50.0, 1.45.0
we should think of how to improve the tool to handle this. Maybe suggest an upgrade in that case as well?
there are many logs for format "uri" ignored and format "uri-reference" ignored in the console, but this is not an error. Can be resolved with later update

dimovpetar · 2025-11-17T13:54:55Z

now I often get "The MCP validation tool timed out". Seems that it happens mainly if I ask to validate a manifest, complete the task, then edit the manifest and ask to validate again.
Maybe it is related to the mutex log and cache. Or to the ajv tool itself
If I restart VS code, the tool can run again without a timeout. Also it runs smoothly couple of times in the same task. But fails with timeout if it runs in two different tasks in the same window.

Fixed it, it was caused by not releasing the mutex when fetching external schemas, particularly the Adaptive cards schema.

every time when I validate manifest _version: "1.60.0" - I get an error
Error:
Failed to create UI5 manifest validate function: items value must be ["object","boolean"]

The reason behind that is that UI5 Manifest schemas prior 1.68.0 are using older version of the meta schema. Our setup doesn't support it. I have added an error and hint whenever such old version of manifest is encountered, to suggest upgrade to a newer one.

some older manifests are in the mapping, but have no tag for them, e.g. 1.50.0, 1.45.0
we should think of how to improve the tool to handle this. Maybe suggest an upgrade in that case as well?

Now, upgrade is suggested if version lower than 1.68.0 is encountered.

There are many logs for format "uri" ignored and format "uri-reference" ignored in the console, but this is not an error. Can be resolved with later update

Resolved it. There is dedicated package that contains those formats, called "ajv-formats".

alexandar-mitsev

Looks good to me.

RandomByte · 2025-11-21T13:12:56Z

every time when I validate manifest _version: "1.60.0" - I get an error
Error:
Failed to create UI5 manifest validate function: items value must be ["object","boolean"]

The reason behind that is that UI5 Manifest schemas prior 1.68.0 are using older version of the meta schema. Our setup doesn't support it. I have added an error and hint whenever such old version of manifest is encountered, to suggest upgrade to a newer one.

some older manifests are in the mapping, but have no tag for them, e.g. 1.50.0, 1.45.0
we should think of how to improve the tool to handle this. Maybe suggest an upgrade in that case as well?

Now, upgrade is suggested if version lower than 1.68.0 is encountered.

@MatthiasSchmalz fyi, in case you have any comments or suggestions we would highly appreciate it. Otherwise we will go forward with this setup

CC: @codeworrior

MatthiasSchmalz · 2025-11-21T13:27:16Z

@RandomByte
I think it is ok to not support old legacy versions, but the error messages should be understandable.
"Failed to create UI5 manifest validate function: items value must be ["object","boolean"]" is not understandable.
"Can't validate manifest, because schema version < 1.68.0 is not supported" would be understandable.

However 1.67.0 is not very old, so a lot of consumers might still use it.
Do you want to suggest updating the schema but keeping an older minUI5Version? That doesn't feel so good as it can cause invalid combinations (at least Fiori Pipelines could complain about this).
Is the effort to support the old meta version that big?

RandomByte · 2025-11-21T13:14:57Z

src/utils/ui5Manifest.ts

+const fetchSchemaMutex = new Mutex();
+
+let UI5ToManifestVersionMapping: Record<string, string> | null = null;
 const MAPPING_URL = "https://raw.githubusercontent.com/SAP/ui5-manifest/main/mapping.json";


The main branch is missing versions >=2

Suggested change

const MAPPING_URL = "https://raw.githubusercontent.com/SAP/ui5-manifest/main/mapping.json";

const MAPPING_URL = "https://raw.githubusercontent.com/SAP/ui5-manifest/v2/mapping.json";

I assumed that when v2 is productively released, it will go in the main branch and then we will have it. Not sure if we need to use v2 before its stable release

@MatthiasSchmalz can you please clarify?

The colleagues also found that https://raw.githubusercontent.com/SAP/ui5-manifest/v2/mapping.json is missing the latest UI5 releases such as 1.142

@RandomByte v2 is currently in the v2 branch, while main still contains v1. We have no explicit timeline yet, when this will be switched as UI5 2 is also not released yet.
Due to this, main branch only maps to v1 schemas. You are right that we might need a better way to map this, Peter recently started a discussion here.

Yes in v2 the mapping does not contain the latest version. In the past we had only added an entry, when a new schema version is available. You can use the rule to take the highest available mapping. But yes, somehow we have entries for 1.139 and 1.138 without a new version as well.
@aleksandr-suvorov do you know why this is the case?

@MatthiasSchmalz:

My understanding is that the availability of a manifest version 2.x is not tightly coupled to a UI5 2.x release.
The manifest version is already supported since UI5 1.136 and the best practice is to adjust the manifest to this version.

In general, I'm not happy with such a complex logic to be able to validate a manifest. I would highly appreciate an API from @ui5/manifest that takes care of all of the mentioned logic and specialities. Adding the code here will not only require all other places that want to validate a manifest to add the same logic, but also makes future changes on your end very cumbersome and error prone, especially as there is no versioning (requests are just sent to the branches main / v2).

For reference, this is the UI5 Language Assistant implementation to support manifest v2: SAP/ui5-language-assistant#754

@RandomByte yes, the release of V2 is now coupled with the releases of legacy free versions, which might not be optimal, I will prepare intermediate versions also.
@matz3 ui5 manifest provides schema validation, but right now this functionality is not exposed in the API. We could reconsider it, expose and use it here, because basically we do it with the same tools and the same way.

src/tools/run_manifest_validation/runValidation.ts

test/lib/tools/run_manifest_validation/runValidation.ts

src/tools/run_manifest_validation/runValidation.ts

test/lib/tools/run_manifest_validation/runValidation.ts

matz3

The architecture document should also be updated to reflect the new changes, especially the new dependencies to external services.

src/tools/run_manifest_validation/index.ts

src/utils/ui5Manifest.ts

src/tools/run_manifest_validation/index.ts

matz3 · 2025-11-21T13:36:56Z

src/utils/ui5Manifest.ts

+	throw new Error(
+		errorMessage +
+		(supportedVersions ?
+			`\nSupported versions are: ${supportedVersions.join(", ")}.` :


This gave me the following versions:

Manifest version '1.0.0' is not supported. Please upgrade to a newer one.
Supported versions are: 1.81.0, 1.81.0, 1.80.0, 1.79.0, 1.78.0, 1.77.0, 1.76.0, 1.75.1, 1.73.1, 1.72.3, 1.72.0, 1.70.1, 1.69.0, 1.68.0.

What about 2.0.0? Isn't that version also supported? It is not listed here.

1.81.0 is listed twice

dimovpetar · 2025-11-24T11:44:28Z

@RandomByte I think it is ok to not support old legacy versions, but the error messages should be understandable. "Failed to create UI5 manifest validate function: items value must be ["object","boolean"]" is not understandable. "Can't validate manifest, because schema version < 1.68.0 is not supported" would be understandable.

However 1.67.0 is not very old, so a lot of consumers might still use it. Do you want to suggest updating the schema but keeping an older minUI5Version? That doesn't feel so good as it can cause invalid combinations (at least Fiori Pipelines could complain about this). Is the effort to support the old meta version that big?

Hello @MatthiasSchmalz ,

The error now looks like
"Manifest version '1.67.2' is not supported. Please upgrade to a newer one.
Supported versions are: 1.81.0, 1.81.0, 1.80.0, 1.79.0, 1.78.0, 1.77.0, 1.76.0, 1.75.1, 1.73.1, 1.72.3, 1.72.0, 1.70.1, 1.69.0, 1.68.0."

This tool focuses on performing validation based on concrete schema. minUI5Version value does not differ per schema. It is something that we can discuss to add on top, but not in the initial release. Same goes for supporting versions < 1.68.0.

dimovpetar marked this pull request as draft October 21, 2025 11:50

dimovpetar force-pushed the feat_run_manifest_validation branch 4 times, most recently from c0f9c6b to 85d48a7 Compare October 28, 2025 13:56

dimovpetar force-pushed the feat_run_manifest_validation branch 2 times, most recently from 9ee721e to b1ed71b Compare October 30, 2025 08:30

dimovpetar marked this pull request as ready for review October 30, 2025 14:16

dimovpetar added 12 commits November 3, 2025 14:04

feat: Add manifest validation tool

6fed175

refactor: Move manifest schema fetching to ui5Manifest.ts

278273f

test(index.ts): Add tests

dd54cd2

refactor(ui5Manifest.ts): Cache the schema

00de711

docs(README.md): List run_manifest_validation

d6970ea

refactor(ui5Manifest): Add cache

a5ddede

refactor: Improve error handling

f08c45d

test(runValidation): Add tests

a74700e

refactor: Add comment containing link to AdaptiveCards issue

3c6b99c

fix(package.json): List ajv as dependency

4c49e47

fix(runValidation): Resolve meta schemas paths using import.meta.resolve

4c87335

fix(runValidation): Throw error if manifest path is not absolute

0f4d9df

dimovpetar force-pushed the feat_run_manifest_validation branch from 6ee3a30 to 0f4d9df Compare November 3, 2025 12:11

RandomByte requested changes Nov 3, 2025

View reviewed changes

src/utils/ui5Manifest.ts Outdated Show resolved Hide resolved

src/tools/run_manifest_validation/index.ts Outdated Show resolved Hide resolved

dimovpetar added 2 commits November 5, 2025 11:13

fix(run_manifest_validation): Normalize manifest path

593f749

refactor: Fetch concrete manifest schema

7d71c5e

dimovpetar requested a review from RandomByte November 10, 2025 12:38

alexandar-mitsev suggested changes Nov 11, 2025

View reviewed changes

dimovpetar added 2 commits November 17, 2025 13:44

fix(runValidation): Properly release mutex

05113b3

fix(ui5Manifest): Throw errors for versions lt 1.68.0

be513fb

dimovpetar added 2 commits November 17, 2025 15:29

fix(runValidation): Include ajv-formats

a950b4b

refactor(ui5Manifest): Enhance more errors with supported versions info

ae0b431

dimovpetar requested a review from alexandar-mitsev November 17, 2025 13:55

alexandar-mitsev reviewed Nov 20, 2025

View reviewed changes

alexandar-mitsev approved these changes Nov 20, 2025

View reviewed changes

RandomByte reviewed Nov 21, 2025

View reviewed changes

matz3 reviewed Nov 21, 2025

View reviewed changes

dimovpetar added 6 commits November 24, 2025 15:01

test(runValidation): Increase coverage for external schemas

7d32b8f

refactor(runValidation): Add explanation why meta schemas are needed

aefa858

refactor(runValidation): Add explanation for unicodeRegExp setting

005701b

refactor(runValidation): Add coment why strict flag of ajv is off

53beef2

refactor(ui5Manifest): Add comments for lowest supported version 1.68.0

95c3527

fix(run_manifest_validation): Mark the tool as read-only

6c7eae5

dimovpetar force-pushed the feat_run_manifest_validation branch from 7673deb to 6c7eae5 Compare November 25, 2025 09:42

refactor(runValidation): Remove undefined properties from the result

286fbe7

	const MAPPING_URL = "https://raw.githubusercontent.com/SAP/ui5-manifest/main/mapping.json";
	const MAPPING_URL = "https://raw.githubusercontent.com/SAP/ui5-manifest/v2/mapping.json";

feat: Add manifest validation tool #93

Are you sure you want to change the base?

feat: Add manifest validation tool #93

Uh oh!

Conversation

dimovpetar commented Oct 21, 2025

Uh oh!

coveralls commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 19666892989

Details

💛 - Coveralls

Uh oh!

alexandar-mitsev commented Nov 3, 2025

Uh oh!

Uh oh!

Uh oh!

alexandar-mitsev left a comment

Choose a reason for hiding this comment

Uh oh!

dimovpetar commented Nov 17, 2025

Uh oh!

alexandar-mitsev left a comment

Choose a reason for hiding this comment

Uh oh!

RandomByte commented Nov 21, 2025

Uh oh!

MatthiasSchmalz commented Nov 21, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

matz3 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimovpetar commented Nov 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

coveralls commented Oct 30, 2025 •

edited

Loading