Skip to content

[Performance] Optimize local and remote specifications merging #8126

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
gonzaloriestra wants to merge 1 commit into
base: main
Choose a base branch
from
+9 −7
Draft

[Performance] Optimize local and remote specifications merging #8126

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 9 additions & 7 deletions packages/app/src/cli/services/generate/fetch-extension-specifications.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,13 @@ async function mergeLocalAndRemoteSpecs(
local: ExtensionSpecification[],
remote: RemoteSpecification[],
): Promise<RemoteAwareExtensionSpecification[]> {
// Use a Map for O(1) lookups of local specs instead of an O(N) find inside remote.map
const localSpecsMap = new Map(local.map((spec) => [spec.identifier, spec]))

// Iterate over the remote specs and merge them with the local ones
// If the local spec is missing, and the remote one has a validation schema, create a new local spec using contracts
const updated = remote.map(async (remoteSpec) => {
let localSpec = local.find((local) => local.identifier === remoteSpec.identifier)
let localSpec = localSpecsMap.get(remoteSpec.identifier)
if (!localSpec && remoteSpec.validationSchema?.jsonSchema) {
localSpec = await createRemoteOnlySpecification(remoteSpec, remoteSpec.validationSchema)
}
Expand All @@ -86,12 +89,13 @@ async function mergeLocalAndRemoteSpecs(

const result = getArrayRejectingUndefined<RemoteAwareExtensionSpecification>(await Promise.all(updated))

// Use a Set for O(1) lookups instead of nested O(N) find searches inside filters
const resultIdentifiers = new Set(result.map((spec) => spec.identifier))

// Log the specs that were defined locally but aren't in the result
// This usually means the spec is a gated one and the caller doesn't have adequate access. Or, we're in a test and
// the mocked specification set is missing something.
const presentLocalMissingRemote = local.filter(
(spec) => !result.find((result) => result.identifier === spec.identifier),
)
const presentLocalMissingRemote = local.filter((spec) => !resultIdentifiers.has(spec.identifier))
if (presentLocalMissingRemote.length > 0) {
outputDebug(
`The following extension specifications were defined locally but not found in the remote specifications: ${presentLocalMissingRemote
Expand All @@ -101,9 +105,7 @@ async function mergeLocalAndRemoteSpecs(
)
}

const presentRemoteMissingLocal = remote.filter(
(spec) => !result.find((result) => result.identifier === spec.identifier),
)
const presentRemoteMissingLocal = remote.filter((spec) => !resultIdentifiers.has(spec.identifier))
if (presentRemoteMissingLocal.length > 0) {
outputDebug(
`The following extension specifications were found in the remote specifications but not defined locally: ${presentRemoteMissingLocal
Expand Down
Loading