diff --git a/CLAUDE.md b/CLAUDE.md index 97597d798a..c2143824fa 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -137,4 +137,6 @@ The response includes a `gitBranchName` field with a suggested branch name conta Decisions live in `docs/adr/`. Run `/adr` to record one. - [A case-insensitive Decision follows Postgres's ILIKE, whatever the datasource](docs/adr/2026-09-09-a-case-insensitive-decision-follows-postgres-s-ilike-whatever-the-data.md) — 2026-09-09-a-case-insensitive-decision-follows-postgres-s-ilike-whatever-the-data [accepted] +- [The liana name the schema already carries decides a v1 agent's capabilities](docs/adr/2026-09-11-the-liana-name-the-schema-already-carries-decides-a-v1-agent-s-capabi.md) — 2026-09-11-the-liana-name-the-schema-already-carries-decides-a-v1-agent-s-capabi [accepted] +- [A schema declaring no primary key gets one derived from its `id` field](docs/adr/2026-09-11-a-schema-declaring-no-primary-key-gets-one-derived-from-its-id-field.md) — 2026-09-11-a-schema-declaring-no-primary-key-gets-one-derived-from-its-id-field [accepted] diff --git a/docs/adr/2026-09-11-a-schema-declaring-no-primary-key-gets-one-derived-from-its-id-field.md b/docs/adr/2026-09-11-a-schema-declaring-no-primary-key-gets-one-derived-from-its-id-field.md new file mode 100644 index 0000000000..90fce51835 --- /dev/null +++ b/docs/adr/2026-09-11-a-schema-declaring-no-primary-key-gets-one-derived-from-its-id-field.md @@ -0,0 +1,10 @@ +--- +status: accepted +date: 2026-09-11 +tags: [agent-bff, primary-key, v1-compatibility] +affected_components: [agent-bff] +--- + +# A schema declaring no primary key gets one derived from its `id` field + +`forest_liana` only publishes `isPrimaryKey` from 9.17.6 (2026-06-04), so every collection of an older one reaches the BFF keyless, `unpackPrimaryKey` rejects every record the agent returns and a plain list answers `500 mapping_error` — on 125 `forest-rails` environments active in the last 120 days, the twenty oldest run between 2.14.6 and 8.3.2, several in production. The read-model therefore derives a key when the schema declares none: a field literally named `id`, typed as the schema declares it, `String` when no such field exists. The trigger is the shape of the schema, not the liana version, so a collection that declares a key keeps it and every v2 agent is untouched; the fallback only fires where the alternative is a 500. Two things are guessed and both are published as guesses rather than hidden. The arity: the schema names no key, so a collection whose real one is composite packs `tenant|42` behind the same silence, and splitting that against one derived key would 500 the whole list — a derived key is flagged `derived` and `unpackPrimaryKey` passes it through whole, typed only when the declared `id` field says `Number`. The name: on the schema this was measured against, 12 of 269 collections declare no `id` field at all, so `__forest.primaryKey` carries the opaque agent id under a name that is not a column, which `ForestRecordMeta` states as the one exception to its per-column promise. Where an `id` field does exist, the context marks it `isPrimaryKey` together with `isPrimaryKeyDerived`, because nothing confirms it is the column the records are really keyed on and a filter on it can answer 200 with no row; where it does not, the context marks no field at all. The cost is bounded: every route takes the record id packed and opaque, so nothing in the request path breaks, and the invented name only misses for a consumer filtering ON the key — which a keyless collection could not do either way, since nothing in the schema names that column. Rejected: deriving only when an `id` field is declared, which leaves those 12 ordinary listable collections on the 500 this exists to remove; and reading the key from the agent's capabilities, which the v1 lianas this targets do not serve. diff --git a/docs/adr/2026-09-11-the-liana-name-the-schema-already-carries-decides-a-v1-agent-s-capabi.md b/docs/adr/2026-09-11-the-liana-name-the-schema-already-carries-decides-a-v1-agent-s-capabi.md new file mode 100644 index 0000000000..f4e4fb9058 --- /dev/null +++ b/docs/adr/2026-09-11-the-liana-name-the-schema-already-carries-decides-a-v1-agent-s-capabi.md @@ -0,0 +1,10 @@ +--- +status: accepted +date: 2026-09-11 +tags: [agent-bff, capabilities, v1-compatibility] +affected_components: [agent-bff, forestadmin-client] +--- + +# The liana name the schema already carries decides a v1 agent's capabilities + +`forest-express-sequelize`, `forest-express-mongoose` and `forest-rails` never served `POST /forest/_internal/capabilities`, so every constrained read the BFF fronts died on that 404 and the collection was unusable. On that 404 the BFF now matches the schema's `meta.liana` against those three names and synthesizes the answer from the apimap the liana already pushed: the operator set that generation implements per column type, plus the fields the schema marks filterable and sortable. Any other liana keeps throwing, so a new v2 agent is never silently downgraded to a legacy operator set — the legacy list is closed in practice while the v2 family grows. The published operators are the measured intersection of what `forest-express-sequelize 9.6.10` and `forest_liana 9.21.0` honour, not the column type's whole table: an operator only one of them implements maps to `422 unprocessable_entity`, where an operator neither advertises is the `400 invalid_filter_operator` a caller can act on — `includes_all` answers 500 with a leaking SQL fragment in Express and 422 in Rails, `i_contains` works in Rails only, so neither is published. An array column publishes no operator at all, matching v2, where `allowedOperatorsForColumnType` is keyed by primitive names only. The name is self-reported by the agent, and that is accepted: it reaches the BFF inside the schema the agent pushed and the SaaS served back, so it is exactly as trustworthy as the field list read beside it, and a wrong name can only narrow the operator set of the very agent that declared it. Rejected: an operator-set environment flag (`BFF_LEGACY_CAPABILITIES`), which asked a human to restate per deployment what the schema already carries and went stale on the next agent upgrade; and probing the agent's behaviour to infer its generation, which turns every cold capabilities read into a series of requests whose failures are indistinguishable from an agent that is merely down. diff --git a/packages/agent-bff/src/context/build-context.ts b/packages/agent-bff/src/context/build-context.ts index cc159296b5..ef091edbb9 100644 --- a/packages/agent-bff/src/context/build-context.ts +++ b/packages/agent-bff/src/context/build-context.ts @@ -38,6 +38,7 @@ export interface ContextField { inverseOf?: string; polymorphicTargets?: string[]; isPrimaryKey?: boolean; + isPrimaryKeyDerived?: boolean; isRequired?: boolean; isReadOnly?: boolean; enums?: string[]; @@ -113,6 +114,7 @@ function toContextValidations(validations: unknown[] | null | undefined): Contex function toContextField( field: FieldWithWireEnums, ambiguousKeys: ReadonlySet, + derivedPrimaryKeys: ReadonlySet, ): ContextField { const serialized: ContextField = { field: field.field, type: field.type }; @@ -126,7 +128,21 @@ function toContextField( const polymorphicTargets = toArray(field.polymorphicReferencedModels); if (polymorphicTargets.length > 0) serialized.polymorphicTargets = [...polymorphicTargets]; - if (field.isPrimaryKey) serialized.isPrimaryKey = true; + // The read-model derives a key when the schema declares none, and the BFF builds record + // identifiers from it. Publishing only the schema's flag would leave a client unable to name the + // key the BFF is actually using. + // + // A derived key is flagged as such, because it is a GUESS: the schema published nothing, so this + // field named `id` may not be the real key. Publishing it as a plain `isPrimaryKey` would send a + // client to filter on it, and a filter against a column that is not the key answers 200 with no + // row — a silence far worse than the 422 it would get on a field the collection does not expose. + if (!field.isPrimaryKey && derivedPrimaryKeys.has(field.field)) { + serialized.isPrimaryKey = true; + serialized.isPrimaryKeyDerived = true; + } else if (field.isPrimaryKey) { + serialized.isPrimaryKey = true; + } + if (field.isRequired) serialized.isRequired = true; if (field.isReadOnly) serialized.isReadOnly = true; @@ -168,10 +184,13 @@ function toContextCollection( field => typeof field === 'object' && field !== null, ); const ambiguousKeys = ambiguousRecordKeys(fields); + const derivedPrimaryKeys = new Set( + readModel.getPrimaryKeys(collection.name).map(key => key.name), + ); return { name: collection.name, - fields: fields.map(field => toContextField(field, ambiguousKeys)), + fields: fields.map(field => toContextField(field, ambiguousKeys, derivedPrimaryKeys)), actions: toArray(collection.actions) .filter(action => { const allowed = allowedActions[action?.name]; diff --git a/packages/agent-bff/src/data/agent-query.ts b/packages/agent-bff/src/data/agent-query.ts index 1899485ca8..be008cdf5d 100644 --- a/packages/agent-bff/src/data/agent-query.ts +++ b/packages/agent-bff/src/data/agent-query.ts @@ -2,6 +2,8 @@ import type { PageInput, SortClauseInput } from './request-schemas'; import type { Logger } from '../ports/logger-port'; import type { ZodType, z } from 'zod'; +import { toWireFilter } from '@forestadmin/agent-client'; + import { CountFlatInputs, ListFlatInputs, @@ -37,6 +39,7 @@ function isPlainObject(value: unknown): value is Record { const LEAF_KEYS = ['field', 'operator', 'value']; const BRANCH_KEYS = ['aggregator', 'conditions']; +const AGGREGATORS = ['And', 'Or']; function loggingRejections(logger: Logger, parse: () => T): T { try { @@ -61,6 +64,27 @@ function assertNoStrayKey(node: Record, allowed: string[]): voi } } +// An absent aggregator stays allowed and is forwarded, as the document says. A present one must name +// an aggregator that exists: anything else -- `5`, or `xor` -- passes every shape check, reaches the +// agent and comes back as a 503 agent_unavailable, where the caller can act on a 400. +// +// The comparison is case-insensitive although the document enumerates PascalCase only: +// `toWireFilter` lowercases the aggregator on the way out, so `and` already reaches the agent as the +// `and` it parses, and rejecting it here would break a caller nothing else refuses. +function assertAggregator(node: Record): void { + const { aggregator } = node; + + if (aggregator === undefined) return; + + const named = + typeof aggregator === 'string' && + AGGREGATORS.some(candidate => candidate.toLowerCase() === aggregator.toLowerCase()); + + if (!named) { + throw invalidRequest(`A filter branch aggregator must be one of: ${AGGREGATORS.join(', ')}`); + } +} + function assertFilterNode(node: unknown, depth = 0): void { if (depth > MAX_FILTER_DEPTH) throw filterTooDeep(MAX_FILTER_DEPTH); if (!isPlainObject(node)) return; @@ -73,6 +97,7 @@ function assertFilterNode(node: unknown, depth = 0): void { if (readableAsBranch) { assertNoStrayKey(node, BRANCH_KEYS); + assertAggregator(node); node.conditions.forEach(condition => assertFilterNode(condition, depth + 1)); return; @@ -181,7 +206,7 @@ export function buildListAgentQuery( ): AgentQuery { const query: AgentQuery = { timezone }; - if (body.filter !== undefined) query.filters = JSON.stringify(body.filter); + if (body.filter !== undefined) query.filters = JSON.stringify(toWireFilter(body.filter)); if (body.projection?.length) query[`fields[${collection}]`] = body.projection.join(','); if (body.sort?.length) query.sort = serializeSort(body.sort); if (body.page) Object.assign(query, serializePage(body.page)); @@ -193,7 +218,7 @@ export function buildListAgentQuery( export function buildCountAgentQuery(timezone: string, body: CountRequestBody): AgentQuery { const query: AgentQuery = { timezone }; - if (body.filter !== undefined) query.filters = JSON.stringify(body.filter); + if (body.filter !== undefined) query.filters = JSON.stringify(toWireFilter(body.filter)); applySearch(query, body); return query; diff --git a/packages/agent-bff/src/data/data-routes-middleware.ts b/packages/agent-bff/src/data/data-routes-middleware.ts index 68be41044e..507355a872 100644 --- a/packages/agent-bff/src/data/data-routes-middleware.ts +++ b/packages/agent-bff/src/data/data-routes-middleware.ts @@ -85,7 +85,12 @@ function resolveCapabilities( () => deps.store.getCapabilities( collection, - createAgentCapabilitiesFetcher({ transport: deps.transport, token: deps.token }), + createAgentCapabilitiesFetcher({ + transport: deps.transport, + token: deps.token, + store: deps.store, + logger: deps.logger, + }), ), deps.logger, ); diff --git a/packages/agent-bff/src/data/pack-id.ts b/packages/agent-bff/src/data/pack-id.ts index 3c3c460e1b..b3f311c750 100644 --- a/packages/agent-bff/src/data/pack-id.ts +++ b/packages/agent-bff/src/data/pack-id.ts @@ -7,11 +7,22 @@ export const PACKED_ID_SEPARATOR = '|'; // The only column type unpacked to a number, mirroring the agent's `IdUtils.unpackId`. const NUMBER_COLUMN_TYPE = 'Number'; +function toNumberIfWhole(value: string): string | number { + const numeric = Number(value); + + return Number.isNaN(numeric) ? value : numeric; +} + /** * Rebuild the structured primary key of a record from its opaque packed id, mirroring the agent's * `IdUtils.packId`/`unpackId` (`|`-joined values, `Number` columns cast back to numbers). Returns a * `{ pkField: value }` map for `__forest.primaryKey`. Throws a mapping error rather than emitting a * malformed key when the schema lacks key metadata or the packed id shape does not match it. + * + * A DERIVED key is the exception: the schema published none, so its arity is a guess and the real + * key may be composite. Splitting `tenant|42` against one guessed key would 500 every list of that + * collection, so a derived key takes the packed id whole — typed when the declared `id` field says + * `Number` and the id is numeric, left a string otherwise, and never a throw. */ export default function unpackPrimaryKey( packedId: string, @@ -21,6 +32,14 @@ export default function unpackPrimaryKey( throw mappingError('Cannot build primary key: the collection exposes no key metadata'); } + const [first] = primaryKeys; + + if (first.derived) { + return { + [first.name]: first.type === NUMBER_COLUMN_TYPE ? toNumberIfWhole(packedId) : packedId, + }; + } + const values = packedId.split(PACKED_ID_SEPARATOR); if (values.length !== primaryKeys.length) { diff --git a/packages/agent-bff/src/openapi/collect-unfolding.ts b/packages/agent-bff/src/openapi/collect-unfolding.ts index 52947aedae..b5a373ef0d 100644 --- a/packages/agent-bff/src/openapi/collect-unfolding.ts +++ b/packages/agent-bff/src/openapi/collect-unfolding.ts @@ -98,23 +98,35 @@ function documentedOperators( return toCanonicalOperatorSet(normalized); } +/** + * The filterable subset, plus whether a field was dropped as undocumentable rather than as genuinely + * not filterable. The caller needs the two apart: a field with no operator is an ANSWER the document + * can state, while a field dropped for operator skew leaves the filterable set incomplete, and an + * incomplete set that ends up empty must not be documented as "nothing is filterable". + */ function collectFilterableFields( collection: string, capabilities: CapabilitiesResult, logger: Logger, -): FilterableField[] { - return capabilities.fields.flatMap(field => { - if ((field.operators?.length ?? 0) === 0) return []; +): { fields: FilterableField[]; undocumentable: boolean } { + const fields: FilterableField[] = []; + let undocumentable = false; - const operators = documentedOperators( - collection, - field.name, - field.operators as string[], - logger, - ); + for (const field of capabilities.fields) { + if ((field.operators?.length ?? 0) > 0) { + const operators = documentedOperators( + collection, + field.name, + field.operators as string[], + logger, + ); - return operators === null ? [] : [{ name: field.name, operators }]; - }); + if (operators === null) undocumentable = true; + else fields.push({ name: field.name, operators }); + } + } + + return { fields, undocumentable }; } async function collectFields( @@ -132,10 +144,15 @@ async function collectFields( return UNTYPED('no_fields'); } + const filterable = collectFilterableFields(collection, capabilities, logger); + return { - projectable: capabilities.fields.map(({ name, type }) => ({ name, type })), - filterable: collectFilterableFields(collection, capabilities, logger), + projectable: capabilities.fields.map(({ name, type, sortable }) => + sortable === false ? { name, type, sortable } : { name, type }, + ), + filterable: filterable.fields, degraded: null, + ...(filterable.undocumentable ? { undocumentableFilter: true as const } : {}), }; } catch (error) { // A single unreachable collection must not cost the whole document: the collection keeps its diff --git a/packages/agent-bff/src/openapi/schemas.ts b/packages/agent-bff/src/openapi/schemas.ts index 75926e7793..d922431180 100644 --- a/packages/agent-bff/src/openapi/schemas.ts +++ b/packages/agent-bff/src/openapi/schemas.ts @@ -283,7 +283,14 @@ export const ForestRecordMetaSchema = z 'The record identity, unpacked from the agent id. A composite primary key carries one ' + 'entry per column. The values are TYPED here — a Number key column is a number — whereas ' + 'the record carries the same id as a string under `id`, so comparing the two forms ' + - 'without coercion fails.', + 'without coercion fails. One exception to the per-column promise: against a schema that ' + + 'declares no primary key at all, this carries the agent id whole under the name `id`, ' + + 'never split per column — the schema published no key, so its real shape is unknown and a ' + + 'composite one would stay packed, separator included. That `id` is a real column only when ' + + 'the schema declares a field of that name; otherwise it is not a column at all and the ' + + 'context marks no field as the key. When it does declare one, the context flags it ' + + '`isPrimaryKeyDerived`, meaning the key is a guess and a filter on it may match nothing. ' + + 'Filter on a field the context lists, never on a key name read back here.', }); export const ListResponseSchema = z @@ -297,7 +304,9 @@ export const ListResponseSchema = z description: 'Records are flat, each carrying a `__forest` envelope. A record always holds `id`, the ' + `agent id as a string — a composite key is its values joined by \`${PACKED_ID_SEPARATOR}\` — ` + - 'while `__forest.primaryKey` holds that same id typed and split per column. The list never ' + + 'while `__forest.primaryKey` holds that same id typed and split per column — with the one ' + + 'exception `ForestRecordMeta` describes, where the name it carries is not a column. ' + + 'The list never ' + 'carries a total: call the count endpoint for that, which is why `countStatus` is always ' + '`not_requested`. ' + 'It is always one page, not guaranteed to be the whole collection: a request that omitted ' + @@ -366,6 +375,17 @@ const ContextFieldSchema = z.object({ inverseOf: z.string().optional(), polymorphicTargets: z.array(z.string()).optional(), isPrimaryKey: z.boolean().optional(), + isPrimaryKeyDerived: z + .boolean() + .optional() + .openapi({ + description: + 'Set only alongside `isPrimaryKey`, when the schema declared no primary key at all and ' + + 'the BFF derived one from a field named `id`. The key is then a GUESS: it is what ' + + '`__forest.primaryKey` carries, but nothing confirms it is the column the records are ' + + 'really keyed on, so a filter on it can answer 200 with no row. Use it to read record ' + + 'identities, not to filter by identity.', + }), isRequired: z.boolean().optional(), isReadOnly: z.boolean().optional(), enums: z.array(z.string()).optional(), diff --git a/packages/agent-bff/src/openapi/unfolded-document.ts b/packages/agent-bff/src/openapi/unfolded-document.ts index 09a706ba48..d606e4772b 100644 --- a/packages/agent-bff/src/openapi/unfolded-document.ts +++ b/packages/agent-bff/src/openapi/unfolded-document.ts @@ -38,7 +38,12 @@ export default async function buildUnfoldedDocument( const unfolding = await collectUnfolding({ readModel, store: source.store, - capabilitiesFetcher: createAgentCapabilitiesFetcher({ transport: source.transport, token }), + capabilitiesFetcher: createAgentCapabilitiesFetcher({ + transport: source.transport, + token, + store: source.store, + logger: source.logger, + }), logger: source.logger, }); diff --git a/packages/agent-bff/src/openapi/unfolded-paths.ts b/packages/agent-bff/src/openapi/unfolded-paths.ts index 88a4918b60..78be1045bf 100644 --- a/packages/agent-bff/src/openapi/unfolded-paths.ts +++ b/packages/agent-bff/src/openapi/unfolded-paths.ts @@ -98,6 +98,26 @@ function fieldsEnum( return pool.add(name, { type: 'string', enum: fields, description }); } +/** + * Whether the document knows which fields the collection exposes. Only `capabilities_unavailable` + * leaves it unknown — the agent could not be asked. `no_fields` is an ANSWER: capabilities were read + * and name none, so every field is rejected, and `null` carries the enumerated set. The difference + * decides whether an empty set means "do not constrain" or "nothing is valid here". + */ +function isFieldSetKnown(fields: CollectionFields): boolean { + return fields.degraded !== 'capabilities_unavailable'; +} + +/** + * Whether an empty `filterable` is an answer. It is not enough that the field set is known: the + * collector also empties `filterable` by dropping every field whose operator set it cannot map, and + * that collection filters fine at runtime. Documenting it as filterable by nothing would refuse, at + * compile time in a generated client, a filter the agent honours. + */ +function isFilterSetKnown(fields: CollectionFields): boolean { + return isFieldSetKnown(fields) && !fields.undocumentableFilter; +} + interface OperatorGroup { fields: string[]; operators: string[]; @@ -158,9 +178,15 @@ function filterLeaves(pool: ComponentPool, plan: Pick 0 - ? ' There is one leaf alternative per operator set: a field accepts only the operators of ' + + const pairing = (() => { + if (fields.filterable.length > 0) { + return ( + ' There is one leaf alternative per operator set: a field accepts only the operators of ' + 'the alternative listing it, and an operator it does not support answers 400 ' + 'invalid_filter_operator.' - : ''; + ); + } + + if (isFilterSetKnown(fields)) { + return ( + ' No field of this collection is filterable, so there is no leaf alternative at all: ' + + 'send the empty object, or no filter.' + ); + } + + return ''; + })(); return pool.add(treeName, { description: @@ -230,6 +268,8 @@ interface FieldRefs { projectable: ReferenceObject | SchemaObject; filter: ReferenceObject; sort: ReferenceObject | SchemaObject; + anySortable: boolean; + anyProjectable: boolean; } function fieldRefs(deps: Deps, plan: Pick): FieldRefs { @@ -243,6 +283,24 @@ function fieldRefs(deps: Deps, plan: Pick) `A field of ${quoted(name)}.`, ); + // Sortable is a subset of projectable too, but only the v1 synthesis ever narrows it: a field the + // apimap marks not sortable is projectable and filterable while a sort on it answers 422 + // field_not_sortable. The enum is shared with projectable when nothing is denied, which is every + // v2 collection, so the common document gains no component. + const sortableNames = fields.projectable + .filter(field => field.sortable !== false) + .map(field => field.name); + + const sortableField = + sortableNames.length === fields.projectable.length + ? projectable + : fieldsEnum( + pool, + `SortableFields_${plan.key}`, + sortableNames, + `A sortable field of ${quoted(name)}.`, + ); + // Filterable is a strict subset of projectable: the agent reports a ManyToOne without operators, so // projecting or sorting on it works while filtering on it answers 422 field_not_filterable. Which is // why the filterable fields are not enumerated here but inside the filter leaves, where each one @@ -250,14 +308,19 @@ function fieldRefs(deps: Deps, plan: Pick) return { projectable, filter: filterSchema(deps, plan), + // Same split as the filter leaves: an UNKNOWN field set leaves sort unconstrained, because the + // runtime is the one that rejects. A known set with nothing sortable — every field denied, or + // capabilities naming no field at all — is a collection that takes no clause. + anySortable: sortableNames.length > 0 || !isFieldSetKnown(fields), + anyProjectable: fields.projectable.length > 0 || !isFieldSetKnown(fields), sort: - fields.projectable.length === 0 + sortableNames.length === 0 ? pool.reuse('SortClause', SortClauseSchema) : pool.add(`SortClause_${plan.key}`, { type: 'object', description: 'Omitting `direction` sorts ascending.', properties: { - field: projectable, + field: sortableField, direction: { type: 'string', enum: ['asc', 'desc'] }, }, required: ['field'], @@ -284,8 +347,31 @@ function requestProperties(deps: Deps, plan: Pick string); + /** Holds the schema snapshot a 404 is answered from when a legacy liana published it. */ + store: ReadModelStore; + logger: Logger; +} + +type LegacyDeps = Pick; + +/** + * The capabilities a legacy liana cannot serve, synthesized from the schema it published, or null + * when this agent is not one — a proxy blocking `/forest/_internal` answers 404 just the same, so + * the liana name decides, not the status. + */ +async function synthesizeForLegacyLiana( + collection: string, + { transport, store, logger }: LegacyDeps, +): Promise | null> { + const { collections, meta } = await store.getSchemaSnapshot(); + const schema = collections.find(entry => entry.name === collection); + + if (!schema) return null; + + if (!meta.liana || !LEGACY_LIANAS.has(meta.liana)) { + logger('Error', 'Agent serves no capabilities route, and its liana is not a legacy one', { + agentUrl: transport.url, + collection, + liana: meta.liana ?? 'absent from the published schema', + lianaVersion: meta.liana_version ?? 'unknown', + causes: 'a proxy blocking /forest/_internal, or a collection the agent no longer serves', + }); + + return null; + } + + logger('Warn', 'Legacy liana: synthesizing the capabilities from the apimap', { + agentUrl: transport.url, + collection, + liana: meta.liana, + lianaVersion: meta.liana_version ?? 'unknown', + }); + + return synthesizeCapabilities(schema, logger); } /** @@ -20,6 +65,8 @@ export interface AgentCapabilitiesFetcherOptions { export default function createAgentCapabilitiesFetcher({ transport, token, + store, + logger, }: AgentCapabilitiesFetcherOptions): CapabilitiesFetcher { const clientFor = (bearer: string) => createRemoteAgentClient({ @@ -28,11 +75,33 @@ export default function createAgentCapabilitiesFetcher({ httpRequester: transport.createRequester(bearer), }); - if (typeof token === 'string') { - const client = clientFor(token); + // A borrowed token cannot be renewed, so its client is built once; a factory mints one per fetch. + function createFetch(): CapabilitiesFetcher { + if (typeof token === 'string') { + const client = clientFor(token); + + return collection => client.collection(collection).capabilities(); + } - return collection => client.collection(collection).capabilities(); + return collection => clientFor(token()).collection(collection).capabilities(); } - return collection => clientFor(token()).collection(collection).capabilities(); + const fetch = createFetch(); + + // The liana is read from the cached snapshot, so it lags a migration by at most a schema + // generation; the synthesis is returned like a normal result, letting the cache stop the doomed + // POST from repeating on every constrained request. + return async (collection: string) => { + try { + return await fetch(collection); + } catch (error) { + if (!(error instanceof AgentHttpError) || error.status !== 404) throw error; + + const synthesized = await synthesizeForLegacyLiana(collection, { transport, store, logger }); + + if (!synthesized) throw error; + + return synthesized; + } + }; } diff --git a/packages/agent-bff/src/read-model/capabilities-cache.ts b/packages/agent-bff/src/read-model/capabilities-cache.ts index 4361bd0c37..655c62ba94 100644 --- a/packages/agent-bff/src/read-model/capabilities-cache.ts +++ b/packages/agent-bff/src/read-model/capabilities-cache.ts @@ -5,7 +5,10 @@ import { ONE_DAY_MS } from './schema-cache'; export interface CapabilitiesResult { // `type` is the agent's raw `columnType`, so it is not always a plain name: an array-of-primitive // column arrives as `['String']`, and a relation entry arrives as the marker `ManyToOne`. - fields: { name: string; type: FieldType; operators?: string[] }[]; + // `sortable` has no equivalent in a real capabilities response and is only set by the v1 synthesis + // (`synthesize-capabilities.ts`), which reads it from the apimap. It is `false` or absent, never + // `true`: the synthesis states a denial, and absent means "not stated", which sorts as before. + fields: { name: string; type: FieldType; operators?: string[]; sortable?: false }[]; } export type CapabilitiesFetcher = (collection: string) => Promise; diff --git a/packages/agent-bff/src/read-model/forest-schema-client.ts b/packages/agent-bff/src/read-model/forest-schema-client.ts index 68e95e2071..8c57a1be46 100644 --- a/packages/agent-bff/src/read-model/forest-schema-client.ts +++ b/packages/agent-bff/src/read-model/forest-schema-client.ts @@ -1,4 +1,4 @@ -import type { ForestSchemaCollection } from '@forestadmin/forestadmin-client'; +import type { ForestSchemaWithMeta } from '@forestadmin/forestadmin-client'; import { ForestHttpApi, SchemaService } from '@forestadmin/forestadmin-client'; @@ -8,7 +8,7 @@ export interface ForestSchemaClientOptions { } export interface SchemaFetcher { - fetchSchema(): Promise; + fetchSchema(): Promise; } export default class ForestSchemaClient implements SchemaFetcher { @@ -18,7 +18,7 @@ export default class ForestSchemaClient implements SchemaFetcher { this.schemaService = new SchemaService(new ForestHttpApi(), { forestServerUrl, envSecret }); } - async fetchSchema(): Promise { - return this.schemaService.getSchema(); + async fetchSchema(): Promise { + return this.schemaService.getSchemaWithMeta(); } } diff --git a/packages/agent-bff/src/read-model/read-model-store.ts b/packages/agent-bff/src/read-model/read-model-store.ts index e3814508b0..f75c808b8f 100644 --- a/packages/agent-bff/src/read-model/read-model-store.ts +++ b/packages/agent-bff/src/read-model/read-model-store.ts @@ -1,7 +1,7 @@ import type CapabilitiesCache from './capabilities-cache'; import type { CapabilitiesFetcher, CapabilitiesResult } from './capabilities-cache'; import type SchemaCache from './schema-cache'; -import type { ForestSchemaCollection } from '@forestadmin/forestadmin-client'; +import type { ForestSchemaCollection, ForestSchemaMeta } from '@forestadmin/forestadmin-client'; import ReadModel from './read-model'; @@ -9,6 +9,7 @@ const MAX_GENERATION_RETRIES = 3; export interface SchemaSnapshot { collections: ForestSchemaCollection[]; + meta: ForestSchemaMeta; readModel: ReadModel; revision: number; } @@ -47,7 +48,7 @@ export default class ReadModelStore { } async getSchemaSnapshot(): Promise { - const collections = await this.schemaCache.get(); + const { collections, meta } = await this.schemaCache.getPayload(); const { revision } = this.schemaCache; if (revision !== this.builtRevision || !this.readModel) { @@ -56,7 +57,7 @@ export default class ReadModelStore { this.capabilitiesCache.clear(); } - return { collections, readModel: this.readModel, revision }; + return { collections, meta, readModel: this.readModel, revision }; } /** diff --git a/packages/agent-bff/src/read-model/read-model.ts b/packages/agent-bff/src/read-model/read-model.ts index 6b6a275ad1..9d47226ecf 100644 --- a/packages/agent-bff/src/read-model/read-model.ts +++ b/packages/agent-bff/src/read-model/read-model.ts @@ -14,7 +14,7 @@ export type RelationTarget = | { type: RelationshipType; polymorphic: false; target: string } | { type: RelationshipType; polymorphic: true; targets: string[] }; -export type PrimaryKeyField = { name: string; type: string }; +export type PrimaryKeyField = { name: string; type: string; derived?: true }; export type ListableRelation = { name: string; foreignCollection: string }; @@ -131,6 +131,33 @@ export default class ReadModel { } } + /** + * A schema that declares no primary key at all gets one derived from its `id` field. + * + * `forest_liana` only started publishing `isPrimaryKey` in 9.17.6 (2026-06-04), and an older one + * leaves every collection without a key — which makes `unpackPrimaryKey` reject every record the + * agent returns, so a plain list answers `500 mapping_error`. The record id is there regardless: + * the agent serialises it as the JSON:API `id`, and 257 of the 269 collections in the schema this + * was measured against carry a field literally named `id`. + * + * Keyed on the shape of the schema, not on the liana: a collection that declares a key keeps it, + * and every v2 agent declares one, so this only fires where the alternative is a 500. `String` + * when no `id` field is declared, because the packed id survives a string round-trip untouched + * while a wrong numeric cast would not. + * + * The derived key is flagged, because its arity is a guess: the schema publishes no key, so a + * collection whose real one is composite packs `tenant|42` behind the same silence. Splitting + * that on the separator would find two values against one declared key and 500 the whole list, + * so `unpackPrimaryKey` passes a derived key through opaque instead of splitting it. + * + * That last case names a column the collection does not declare, which `__forest.primaryKey` + * otherwise promises is a real one. It is the deliberate trade: the 12 collections concerned are + * ordinary listable ones carrying ordinary data columns, and the alternative is a 500 on a plain + * list of them. Every route takes the id packed and opaque, so the invented name only misses for + * a consumer filtering ON the key — which a keyless collection could not do either way. + * `ForestRecordMeta` says so, and `buildContext` publishes no `isPrimaryKey` where there is no + * field to carry it. + */ private buildPrimaryKeys(collection: ForestSchemaCollection): void { const keys: PrimaryKeyField[] = []; @@ -138,6 +165,12 @@ export default class ReadModel { if (field.isPrimaryKey) keys.push({ name: field.field, type: field.type }); } + if (keys.length === 0) { + const declaredId = (collection.fields ?? []).find(field => field.field === 'id'); + + keys.push({ name: 'id', type: declaredId?.type ?? 'String', derived: true }); + } + this.primaryKeys.set(collection.name, keys); } diff --git a/packages/agent-bff/src/read-model/schema-cache.ts b/packages/agent-bff/src/read-model/schema-cache.ts index 52aa351ee7..99942367b8 100644 --- a/packages/agent-bff/src/read-model/schema-cache.ts +++ b/packages/agent-bff/src/read-model/schema-cache.ts @@ -1,7 +1,7 @@ import type { SchemaFetcher } from './forest-schema-client'; import type { Logger } from '../ports/logger-port'; import type { Metrics } from '../ports/metrics-port'; -import type { ForestSchemaCollection } from '@forestadmin/forestadmin-client'; +import type { ForestSchemaCollection, ForestSchemaMeta } from '@forestadmin/forestadmin-client'; import SchemaUnavailableError from './errors'; @@ -28,8 +28,22 @@ export interface SchemaCacheOptions { ttlMs?: number; } +/** + * The collections and the liana that published them, always read from the same source. They must + * not be fetched through separate accessors: `clear()` detaches an in-flight refresh, which still + * resolves for whoever awaited it while a newer refresh writes the entry, so a caller reading + * collections from the resolved promise and `meta` from the current entry can pair two schema + * generations -- and the legacy-capabilities fallback would then pick a liana from one and + * synthesize from the other. + */ +export interface SchemaPayload { + collections: ForestSchemaCollection[]; + meta: ForestSchemaMeta; +} + interface CacheEntry { collections: ForestSchemaCollection[]; + meta: ForestSchemaMeta; fetchedAt: number; /** * Decided when the entry is written, not when it is read: a read taken while the revalidation @@ -53,7 +67,7 @@ export default class SchemaCache { private readonly ttlMs: number; private entry: CacheEntry | null = null; - private inFlight: Promise | null = null; + private inFlight: Promise | null = null; private revisionValue = 0; private generation = 0; private revalidatingUntil = 0; @@ -73,10 +87,15 @@ export default class SchemaCache { } async get(): Promise { + return (await this.getPayload()).collections; + } + + /** The collections and their liana, guaranteed to come from the same schema generation. */ + async getPayload(): Promise { if (this.entry && this.now() < this.entry.expiresAt) { this.emitAge(); - return this.entry.collections; + return { collections: this.entry.collections, meta: this.entry.meta }; } return this.refresh(); @@ -109,11 +128,16 @@ export default class SchemaCache { return this.revisionValue; } - private async refresh(): Promise { + /** The liana that published the schema currently served, stale-serve included. */ + get meta(): ForestSchemaMeta { + return this.entry?.meta ?? {}; + } + + private async refresh(): Promise { if (!this.inFlight) { // Identity-guarded, because `clear()` detaches the in-flight fetch: a read that lands after an // invalidation must start its own, not join the one that read the invalidated schema. - const pending: Promise = this.doRefresh().finally(() => { + const pending: Promise = this.doRefresh().finally(() => { if (this.inFlight === pending) this.inFlight = null; }); @@ -123,11 +147,11 @@ export default class SchemaCache { return this.inFlight; } - private async doRefresh(): Promise { + private async doRefresh(): Promise { const { generation } = this; try { - const collections = await this.fetcher.fetchSchema(); + const { collections, meta } = await this.fetcher.fetchSchema(); // An agent always exposes collections, so an empty result is far likelier a broken response // than a valid state. Caching it would silently deny everything for 24h — treat it as a @@ -139,12 +163,17 @@ export default class SchemaCache { if (this.generation === generation) { const fetchedAt = this.now(); - this.entry = { collections, fetchedAt, expiresAt: fetchedAt + this.ttlFor(fetchedAt) }; + this.entry = { + collections, + meta, + fetchedAt, + expiresAt: fetchedAt + this.ttlFor(fetchedAt), + }; this.revisionValue += 1; this.emitAge(); } - return collections; + return { collections, meta }; } catch (error) { this.metrics.increment(SCHEMA_CACHE_REFRESH_ERROR); // The counter alone cannot tell "the SaaS returned an empty array" from "the SaaS is down" @@ -160,7 +189,7 @@ export default class SchemaCache { if (this.entry) { this.emitAge(); - return this.entry.collections; + return { collections: this.entry.collections, meta: this.entry.meta }; } // Cold cache: nothing to serve. diff --git a/packages/agent-bff/src/read-model/synthesize-capabilities.ts b/packages/agent-bff/src/read-model/synthesize-capabilities.ts new file mode 100644 index 0000000000..7e36f31f08 --- /dev/null +++ b/packages/agent-bff/src/read-model/synthesize-capabilities.ts @@ -0,0 +1,167 @@ +import type { CapabilitiesResult } from './capabilities-cache'; +import type { FieldType } from './field-type'; +import type { Logger } from '../ports/logger-port'; +import type { Operator } from '@forestadmin/datasource-toolkit'; +import type { ForestSchemaCollection, ForestSchemaField } from '@forestadmin/forestadmin-client'; + +import { toWireOperator } from '@forestadmin/agent-client'; +import { allowedOperatorsForColumnType } from '@forestadmin/datasource-toolkit'; + +import { normalizeFieldType } from './field-type'; + +/** + * The operators every supported legacy liana honours, measured against + * `forest-express-sequelize 9.6.10` and `forest_liana 9.21.0`: the scalar cases of their filter + * switch plus the date family their separate date parser handles. Names are the legacy snake_case + * wire format, which is also what a real capabilities response carries. + * + * This list exists because a legacy liana publishes a single `isFilterable` boolean per field, never + * an operator list. Advertising the column type's whole operator set instead would admit operators + * the liana rejects on its own, which the BFF maps to `422 unprocessable_entity` rather than the + * `400 invalid_filter_operator` a caller can act on. + * + * It is the intersection of the two, not the union: `includes_all` sits in the column type table for + * every array-capable type, and on a scalar column `forest-express-sequelize` answers it with a 500 + * carrying SQL while `forest_liana` rejects it outright. `i_contains` goes the other way — Rails + * honours it, Express does not — so it stays out too. + */ +export const LEGACY_LIANA_OPERATORS: ReadonlySet = new Set([ + 'after', + 'after_x_hours_ago', + 'before', + 'before_x_hours_ago', + 'blank', + 'contains', + 'ends_with', + 'equal', + 'future', + 'greater_than', + 'in', + 'less_than', + 'not_contains', + 'not_equal', + 'past', + 'present', + 'previous_month', + 'previous_month_to_date', + 'previous_quarter', + 'previous_quarter_to_date', + 'previous_week', + 'previous_week_to_date', + 'previous_x_days', + 'previous_x_days_to_date', + 'previous_year', + 'previous_year_to_date', + 'starts_with', + 'today', + 'yesterday', +]); + +/** + * The lianas that never served `/forest/_internal/capabilities`, as the Forest server itself + * enumerates them. The set is closed in practice — no new one is published — while the v2 agent + * family grows, so listing the legacy names keeps an unlisted agent on the failing path rather than + * silently downgrading it to the operator set above. + */ +export const LEGACY_LIANAS: ReadonlySet = new Set([ + 'forest-express-sequelize', + 'forest-express-mongoose', + 'forest-rails', +]); + +const MANY_TO_ONE = 'ManyToOne'; + +/** + * A `Map` and not the exported object: the type name is read from the apimap, and the table is a + * frozen object literal, so a field typed `constructor` would resolve to the Object constructor — + * truthy, and fatal on the `.map` below. A `Map` answers `undefined` for anything not a real key. + */ +const OPERATORS_BY_COLUMN_TYPE = new Map( + Object.entries(allowedOperatorsForColumnType), +); + +/** + * An array column has no name here on purpose. Unwrapping `['Number']` to `Number` would publish the + * scalar operator table on it -- `greater_than` on a Postgres array raises in the liana, which the + * BFF answers as 503 agent_unavailable. The v2 agent publishes no operator for such a column either + * (`allowedOperatorsForColumnType` is keyed by primitive names only), so the two generations agree. + */ +function primitiveNameOf(type: FieldType): string | undefined { + const normalized = normalizeFieldType(type); + + return typeof normalized === 'string' ? normalized : undefined; +} + +function operatorsFor(field: ForestSchemaField, collection: string, logger: Logger): string[] { + if (field.isFilterable === false) return []; + + const primitive = primitiveNameOf(field.type); + const allowed = primitive === undefined ? undefined : OPERATORS_BY_COLUMN_TYPE.get(primitive); + + if (!allowed) { + logger( + 'Warn', + 'No operator table for a synthesized column type; field reads as not filterable', + { + collection, + field: field.field, + type: JSON.stringify(field.type), + }, + ); + + return []; + } + + return allowed.map(toWireOperator).filter(operator => LEGACY_LIANA_OPERATORS.has(operator)); +} + +/** + * The one, none or zero capability entries a single apimap field becomes: a scalar carries its + * operators, a to-one relation is present without any (so a direct filter on it is + * `field_not_filterable`), and every other relation is dropped (so a filter on it is + * `unknown_field`). + */ +function toCapabilityFields( + field: ForestSchemaField, + collection: string, + logger: Logger, +): CapabilitiesResult['fields'] { + if (field.relationship) { + if (field.relationship !== 'BelongsTo') return []; + + // A relation carries no operators, but it is still sortable through its target, so a sort the + // liana denies has to be published as denied here too -- otherwise the BFF accepts the sort + // and forwards it instead of answering field_not_sortable. + const relation = { name: field.field, type: MANY_TO_ONE }; + + return [field.isSortable === false ? { ...relation, sortable: false } : relation]; + } + + const entry: CapabilitiesResult['fields'][number] = { + name: field.field, + type: field.type, + operators: operatorsFor(field, collection, logger), + }; + + return [field.isSortable === false ? { ...entry, sortable: false } : entry]; +} + +/** + * Build the capabilities a v1 liana would have answered, from the apimap it already pushed. + * + * The field classes mirror what the v2 agent's capabilities route emits, so the validator produces + * the same error for the same input on both generations. + */ +export default function synthesizeCapabilities( + collection: ForestSchemaCollection, + logger: Logger, +): CapabilitiesResult { + const declared = (collection.fields ?? []).filter( + (field): field is ForestSchemaField => + typeof field === 'object' && field !== null && typeof field.field === 'string', + ); + + return { + fields: declared.flatMap(field => toCapabilityFields(field, collection.name, logger)), + }; +} diff --git a/packages/agent-bff/src/validation/capabilities-validator.ts b/packages/agent-bff/src/validation/capabilities-validator.ts index 6d4d9237d1..f595cb7708 100644 --- a/packages/agent-bff/src/validation/capabilities-validator.ts +++ b/packages/agent-bff/src/validation/capabilities-validator.ts @@ -4,6 +4,7 @@ import type { CapabilitiesResult } from '../read-model/capabilities-cache'; import { normalizeOperator } from './operator-normalizer'; import { fieldNotFilterable, + fieldNotSortable, filterTooDeep, invalidFilterOperator, unknownField, @@ -16,12 +17,18 @@ export interface ValidateParams { projectionFields?: string[]; } -interface FilterLeaf { +export interface FilterLeaf { field: string; operator?: string; + value?: unknown; } -export function isBranch(node: unknown): node is { conditions: unknown[] } { +export interface FilterBranch { + aggregator?: string; + conditions: unknown[]; +} + +export function isBranch(node: unknown): node is FilterBranch { return ( typeof node === 'object' && node !== null && @@ -100,6 +107,20 @@ function validateExistence(fields: string[], index: Map): BffH return fields.filter(field => !index.has(field)).map(unknownField); } +/** + * Only the v1 synthesis states sortability, from the apimap's `isSortable`; a real capabilities + * response says nothing about it, so an absent flag means "not stated" and skips the check. Without + * this, a sort on a legacy computed field reaches the liana and raises a database error naming the + * missing column. + */ +function validateSortable(fields: string[], capabilities: CapabilitiesResult): BffHttpError[] { + const notSortable = new Set( + capabilities.fields.filter(field => field.sortable === false).map(field => field.name), + ); + + return fields.filter(field => notSortable.has(field)).map(fieldNotSortable); +} + function dedupe(errors: BffHttpError[]): BffHttpError[] { const seen = new Set(); const result: BffHttpError[] = []; @@ -143,6 +164,7 @@ export function validateAgainstCapabilities( return dedupe([ ...validateFilter(params.filter, index), ...validateExistence(params.sortFields ?? [], index), + ...validateSortable(params.sortFields ?? [], capabilities), ...validateExistence(params.projectionFields ?? [], index), ]); } diff --git a/packages/agent-bff/src/validation/operator-normalizer.ts b/packages/agent-bff/src/validation/operator-normalizer.ts index 86107a5367..ccd7760d7e 100644 --- a/packages/agent-bff/src/validation/operator-normalizer.ts +++ b/packages/agent-bff/src/validation/operator-normalizer.ts @@ -1,21 +1,13 @@ import type { Operator } from '@forestadmin/datasource-toolkit'; +import { toWireOperator } from '@forestadmin/agent-client'; import { allOperators } from '@forestadmin/datasource-toolkit'; -/** - * Mirrors the agent's capabilities serialization (`packages/agent/src/routes/capabilities.ts`), - * which converts each PascalCase operator to snake_case before returning it. Kept identical so the - * inverse map below round-trips every operator the agent can emit. - */ -export function toSnakeCaseOperator(operator: string): string { - return operator - .split(/\.?(?=[A-Z])/) - .join('_') - .toLowerCase(); -} - +// `toWireOperator` is the one PascalCase -> snake_case mapping in the monorepo, and it is the same +// spelling the agent's capabilities route emits, so the inverse map round-trips every operator the +// agent can announce. const SNAKE_TO_PASCAL = new Map( - allOperators.map(operator => [toSnakeCaseOperator(operator), operator]), + allOperators.map(operator => [toWireOperator(operator), operator]), ); /** diff --git a/packages/agent-bff/src/validation/validation-errors.ts b/packages/agent-bff/src/validation/validation-errors.ts index 43f0997adc..efdfed3184 100644 --- a/packages/agent-bff/src/validation/validation-errors.ts +++ b/packages/agent-bff/src/validation/validation-errors.ts @@ -12,6 +12,12 @@ export function fieldNotFilterable(field: string): BffHttpError { }); } +export function fieldNotSortable(field: string): BffHttpError { + return new BffHttpError(422, 'field_not_sortable', `Field is not sortable: ${field}`, { + details: { field }, + }); +} + export function filterTooDeep(maxDepth: number): BffHttpError { return new BffHttpError( 400, diff --git a/packages/agent-bff/test/build-bff-dispatcher.test.ts b/packages/agent-bff/test/build-bff-dispatcher.test.ts index 24b31e6842..ef41033cb7 100644 --- a/packages/agent-bff/test/build-bff-dispatcher.test.ts +++ b/packages/agent-bff/test/build-bff-dispatcher.test.ts @@ -95,7 +95,10 @@ describe('buildBff with an in-process dispatcher', () => { beforeEach(() => { stubEnvironmentIdFetch(); fetchSchema.mockReset(); - fetchSchema.mockResolvedValue([collection('books', [column('id'), column('title')])]); + fetchSchema.mockResolvedValue({ + collections: [collection('books', [column('id'), column('title')])], + meta: {}, + }); }); it('should serve the records the dispatcher returns, without opening a socket', async () => { diff --git a/packages/agent-bff/test/build-bff.test.ts b/packages/agent-bff/test/build-bff.test.ts index 70755d8b14..daecaf3c16 100644 --- a/packages/agent-bff/test/build-bff.test.ts +++ b/packages/agent-bff/test/build-bff.test.ts @@ -4,9 +4,9 @@ import request from 'supertest'; import { createHttpTransport } from '../src/agent/agent-transport'; import buildBff from '../src/build-bff'; +import { restoreFetchAfterEach, stubEnvironmentIdFetch } from './helpers/fetch-stub'; import { parseConfig } from '../src/config/env-config'; import version from '../src/version'; -import { restoreFetchAfterEach, stubEnvironmentIdFetch } from './helpers/fetch-stub'; jest.mock('../src/agent/agent-transport', () => { const actual = jest.requireActual('../src/agent/agent-transport'); diff --git a/packages/agent-bff/test/context/build-context.test.ts b/packages/agent-bff/test/context/build-context.test.ts index 3bbc04c3e4..10e6787a84 100644 --- a/packages/agent-bff/test/context/build-context.test.ts +++ b/packages/agent-bff/test/context/build-context.test.ts @@ -449,6 +449,78 @@ describe('buildContext', () => { }); }); + describe('when the schema declares no primary key at all', () => { + const keylessSchema = [ + { + name: 'people', + fields: [ + { field: 'id', type: 'Number' }, + { field: 'email', type: 'String' }, + ], + actions: [], + }, + ] as unknown as Parameters[0]; + + it('should flag the derived key, which a caller needs to build recordIds', () => { + const [{ fields }] = buildContext(keylessSchema, new ReadModel(keylessSchema), { + schemaRevision: 1, + }).collections; + + expect(fields.find(entry => entry.field === 'id')?.isPrimaryKey).toBe(true); + expect(fields.find(entry => entry.field === 'email')).not.toHaveProperty('isPrimaryKey'); + }); + + it('should mark the derived key as derived, so a caller does not filter on a guess', () => { + const [{ fields }] = buildContext(keylessSchema, new ReadModel(keylessSchema), { + schemaRevision: 1, + }).collections; + + expect(fields.find(entry => entry.field === 'id')?.isPrimaryKeyDerived).toBe(true); + }); + + it('should leave a declared key unflagged, since it is not a guess', () => { + const keyedSchema = [ + { + name: 'people', + fields: [ + { field: 'reference', type: 'String', isPrimaryKey: true }, + { field: 'email', type: 'String' }, + ], + actions: [], + }, + ] as unknown as Parameters[0]; + + const [{ fields }] = buildContext(keyedSchema, new ReadModel(keyedSchema), { + schemaRevision: 1, + }).collections; + + expect(fields.find(entry => entry.field === 'reference')?.isPrimaryKey).toBe(true); + expect(fields.find(entry => entry.field === 'reference')).not.toHaveProperty( + 'isPrimaryKeyDerived', + ); + }); + + it('should mark no field at all when the collection declares no `id` field either', () => { + const noIdSchema = [ + { + name: 'people', + fields: [ + { field: 'birthdate', type: 'Date' }, + { field: 'email', type: 'String' }, + ], + actions: [], + }, + ] as unknown as Parameters[0]; + + const [{ fields }] = buildContext(noIdSchema, new ReadModel(noIdSchema), { + schemaRevision: 1, + }).collections; + + expect(fields.some(entry => entry.isPrimaryKey)).toBe(false); + expect(fields.some(entry => entry.isPrimaryKeyDerived)).toBe(false); + }); + }); + describe('when the built context is validated against the published OpenAPI schema', () => { it('should round-trip through ContextResponseSchema for every shape the fixture covers', () => { const context = buildContext(schema, readModel, { schemaRevision: 3, environmentId: 42 }); diff --git a/packages/agent-bff/test/context/context-routes-middleware.test.ts b/packages/agent-bff/test/context/context-routes-middleware.test.ts index 36f0b66f76..2ee3bda46c 100644 --- a/packages/agent-bff/test/context/context-routes-middleware.test.ts +++ b/packages/agent-bff/test/context/context-routes-middleware.test.ts @@ -15,7 +15,7 @@ import { tolerateEnvironmentIdFailure } from '../../src/oauth/environment-id'; import CapabilitiesCache from '../../src/read-model/capabilities-cache'; import ReadModelStore from '../../src/read-model/read-model-store'; import SchemaCache, { ONE_DAY_MS } from '../../src/read-model/schema-cache'; -import { makeMetrics } from '../read-model/fixtures'; +import { makeMetrics, published } from '../read-model/fixtures'; const ROUTE = '/agent/v1/context'; const AUTH_SECRET = 'context-secret'; @@ -90,7 +90,7 @@ describe('contextRoutesMiddleware', () => { describe('when the route serves the contract', () => { it('should serve the contract with its collections and schema revision', async () => { - const fetchSchema = jest.fn().mockResolvedValue(schema); + const fetchSchema = jest.fn().mockResolvedValue(published(schema)); const { app } = makeRouteOnlyApp(fetchSchema); const response = await request(app.callback()).get(ROUTE); @@ -106,7 +106,7 @@ describe('contextRoutesMiddleware', () => { }); it('should carry the environment id the resolver returns', async () => { - const { app } = makeRouteOnlyApp(jest.fn().mockResolvedValue(schema), 42); + const { app } = makeRouteOnlyApp(jest.fn().mockResolvedValue(published(schema)), 42); const response = await request(app.callback()).get(ROUTE); @@ -114,7 +114,7 @@ describe('contextRoutesMiddleware', () => { }); it('should omit the environment id when the resolver returns none', async () => { - const { app } = makeRouteOnlyApp(jest.fn().mockResolvedValue(schema)); + const { app } = makeRouteOnlyApp(jest.fn().mockResolvedValue(published(schema))); const response = await request(app.callback()).get(ROUTE); @@ -123,7 +123,7 @@ describe('contextRoutesMiddleware', () => { it('should still answer the contract when the environment id cannot be resolved', async () => { const logs: { level: string; message: string; context?: unknown }[] = []; - const { store } = makeStore(jest.fn().mockResolvedValue(schema)); + const { store } = makeStore(jest.fn().mockResolvedValue(published(schema))); const app = new Koa(); app.use(createErrorMiddleware({ logger: () => {} })); @@ -154,7 +154,7 @@ describe('contextRoutesMiddleware', () => { }); it('should fetch the schema once on a cold cache and never again while it stays warm', async () => { - const fetchSchema = jest.fn().mockResolvedValue(schema); + const fetchSchema = jest.fn().mockResolvedValue(published(schema)); const { app } = makeRouteOnlyApp(fetchSchema); const coldResponse = await request(app.callback()).get(ROUTE); @@ -166,7 +166,7 @@ describe('contextRoutesMiddleware', () => { }); it('should fetch the schema again once the cached one has outlived its ttl', async () => { - const fetchSchema = jest.fn().mockResolvedValue(schema); + const fetchSchema = jest.fn().mockResolvedValue(published(schema)); let clock = 1_000_000; const { app } = makeRouteOnlyApp(fetchSchema, undefined, () => clock); @@ -199,10 +199,14 @@ describe('contextRoutesMiddleware', () => { { algorithm: 'HS256', expiresIn: '15m' } as jsonwebtoken.SignOptions, ); - const keyResponse = await request(makeFullAgentEdge(jest.fn().mockResolvedValue(schema))) + const keyResponse = await request( + makeFullAgentEdge(jest.fn().mockResolvedValue(published(schema))), + ) .get(ROUTE) .set(BFF_KEY_HEADER, RAW_KEY); - const sessionResponse = await request(makeFullAgentEdge(jest.fn().mockResolvedValue(schema))) + const sessionResponse = await request( + makeFullAgentEdge(jest.fn().mockResolvedValue(published(schema))), + ) .get(ROUTE) .set('Authorization', `Bearer ${sessionToken}`); @@ -213,7 +217,7 @@ describe('contextRoutesMiddleware', () => { it('should refuse with 403 origin_not_allowed when the key does not allow the request origin', async () => { const response = await request( - makeFullAgentEdge(jest.fn().mockResolvedValue(schema), ['https://ok.com']), + makeFullAgentEdge(jest.fn().mockResolvedValue(published(schema)), ['https://ok.com']), ) .get(ROUTE) .set(BFF_KEY_HEADER, RAW_KEY) @@ -226,7 +230,7 @@ describe('contextRoutesMiddleware', () => { describe('when the path or method does not match', () => { it('should pass through to the next middleware', async () => { - const fetchSchema = jest.fn().mockResolvedValue(schema); + const fetchSchema = jest.fn().mockResolvedValue(published(schema)); const { app } = makeRouteOnlyApp(fetchSchema); app.use(async ctx => { ctx.status = 418; diff --git a/packages/agent-bff/test/data/agent-query.test.ts b/packages/agent-bff/test/data/agent-query.test.ts index fdbe715892..23f850975d 100644 --- a/packages/agent-bff/test/data/agent-query.test.ts +++ b/packages/agent-bff/test/data/agent-query.test.ts @@ -43,7 +43,7 @@ describe('buildListAgentQuery', () => { expect(query).toEqual({ timezone: 'Europe/Paris', - filters: JSON.stringify({ field: 'email', operator: 'present' }), + filters: JSON.stringify({ field: 'email', operator: 'present', value: null }), 'fields[users]': 'id,email', sort: '-createdAt', 'page[size]': 20, @@ -80,7 +80,7 @@ describe('buildCountAgentQuery', () => { buildCountAgentQuery('Europe/Paris', { filter: { field: 'active', operator: 'equal' } }), ).toEqual({ timezone: 'Europe/Paris', - filters: JSON.stringify({ field: 'active', operator: 'equal' }), + filters: JSON.stringify({ field: 'active', operator: 'equal', value: null }), }); }); @@ -89,6 +89,129 @@ describe('buildCountAgentQuery', () => { }); }); +// Every HTTP agent reads the snake_case spelling: a v1 liana answers NoMatchingOperatorError for +// the canonical one, and the v2 agent PascalCases whatever it receives. So the rewrite is +// unconditional, and `agent-client` documents the same invariant for its own calls. +describe('the outgoing filter', () => { + it('should rewrite the canonical operator for every agent, not only a legacy one', () => { + const query = buildListAgentQuery('users', 'UTC', { + filter: { field: 'status', operator: 'Equal', value: 'published' }, + }); + + expect(JSON.parse(query.filters as string)).toEqual({ + field: 'status', + operator: 'equal', + value: 'published', + }); + }); + + it('should rewrite a leaf operator to snake_case', () => { + const query = buildListAgentQuery('users', 'UTC', { + filter: { field: 'title', operator: 'StartsWith', value: 'A' }, + }); + + expect(JSON.parse(query.filters as string)).toEqual({ + field: 'title', + operator: 'starts_with', + value: 'A', + }); + }); + + it('should rewrite the aggregator and every nested leaf of a branch', () => { + const query = buildListAgentQuery('users', 'UTC', { + filter: { + aggregator: 'And', + conditions: [ + { field: 'status', operator: 'Equal', value: 'published' }, + { + aggregator: 'Or', + conditions: [{ field: 'createdAt', operator: 'PreviousWeek' }], + }, + ], + }, + }); + + expect(JSON.parse(query.filters as string)).toEqual({ + aggregator: 'and', + conditions: [ + { field: 'status', operator: 'equal', value: 'published' }, + { + aggregator: 'or', + conditions: [{ field: 'createdAt', operator: 'previous_week', value: null }], + }, + ], + }); + }); + + it('should rewrite a count filter too, since count shares the operator contract', () => { + const query = buildCountAgentQuery('UTC', { + filter: { field: 'title', operator: 'NotContains', value: 'x' }, + }); + + expect(JSON.parse(query.filters as string)).toEqual({ + field: 'title', + operator: 'not_contains', + value: 'x', + }); + }); + + it('should add a null value to an operand-less leaf, which the liana rejects without one', () => { + const query = buildListAgentQuery('users', 'UTC', { + filter: { field: 'title', operator: 'Present' }, + }); + + expect(JSON.parse(query.filters as string)).toEqual({ + field: 'title', + operator: 'present', + value: null, + }); + }); + + it('should add the null value inside a branch too, where the liana checks every leaf', () => { + const query = buildListAgentQuery('users', 'UTC', { + filter: { + aggregator: 'And', + conditions: [ + { field: 'title', operator: 'Present' }, + { field: 'id', operator: 'Present' }, + ], + }, + }); + + expect(JSON.parse(query.filters as string)).toEqual({ + aggregator: 'and', + conditions: [ + { field: 'title', operator: 'present', value: null }, + { field: 'id', operator: 'present', value: null }, + ], + }); + }); + + it('should keep an explicit value, including a falsy one', () => { + const query = buildListAgentQuery('users', 'UTC', { + filter: { field: 'count', operator: 'Equal', value: 0 }, + }); + + expect(JSON.parse(query.filters as string).value).toBe(0); + }); + + it('should not add a value to a branch, which carries none', () => { + const query = buildListAgentQuery('users', 'UTC', { + filter: { aggregator: 'Or', conditions: [{ field: 'id', operator: 'Present' }] }, + }); + + expect(JSON.parse(query.filters as string)).not.toHaveProperty('value'); + }); + + it('should not touch a value that happens to look like an operator', () => { + const query = buildListAgentQuery('users', 'UTC', { + filter: { field: 'label', operator: 'Equal', value: 'StartsWith' }, + }); + + expect(JSON.parse(query.filters as string).value).toBe('StartsWith'); + }); +}); + describe('search in the outgoing agent query', () => { it('should send the search term under the wire name the agent reads', () => { expect(buildListAgentQuery('users', 'Europe/Paris', { search: 'ada' })).toEqual({ @@ -354,6 +477,44 @@ describe('a filter node readable as both a leaf and a branch', () => { }); }); +describe('a branch carrying an aggregator the document does not enumerate', () => { + it.each([ + ['a number', 5], + ['an unknown word', 'xor'], + ['null', null], + ])('should reject %s with 400 invalid_request', (_label, aggregator) => { + expect(() => parseCountRequest({ filter: { aggregator, conditions: [] } }, logger)).toThrow( + expect.objectContaining({ + type: 'invalid_request', + status: 400, + message: 'A filter branch aggregator must be one of: And, Or', + }), + ); + }); + + it.each([['And'], ['Or'], ['and'], ['or']])( + 'should accept %s, which the agent parses once toWireFilter lowercases it', + aggregator => { + expect(() => + parseCountRequest({ filter: { aggregator, conditions: [] } }, logger), + ).not.toThrow(); + }, + ); + + it('should reject it nested inside a legitimate branch', () => { + expect(() => + parseListRequest( + { filter: { aggregator: 'And', conditions: [{ aggregator: 5, conditions: [] }] } }, + logger, + ), + ).toThrow(expect.objectContaining({ type: 'invalid_request', status: 400 })); + }); + + it('should still accept a branch without any aggregator, which the document allows', () => { + expect(() => parseCountRequest({ filter: { conditions: [] } }, logger)).not.toThrow(); + }); +}); + describe('a filter node carrying an unknown key', () => { it.each(FLAT_PARSERS)('should reject a misspelled leaf value in %s', (_label, parse) => { expect(() => parse({ filter: { field: 'title', operator: 'Equal', valu: 'x' } })).toThrow( diff --git a/packages/agent-bff/test/data/data-routes-middleware.test.ts b/packages/agent-bff/test/data/data-routes-middleware.test.ts index a339c69211..0974f66b78 100644 --- a/packages/agent-bff/test/data/data-routes-middleware.test.ts +++ b/packages/agent-bff/test/data/data-routes-middleware.test.ts @@ -273,7 +273,7 @@ describe('data routes middleware', () => { expect(list).toHaveBeenCalledWith('users', { timezone: TIMEZONE, - filters: JSON.stringify({ field: 'email', operator: 'Present' }), + filters: JSON.stringify({ field: 'email', operator: 'present', value: null }), search: 'ada', }); }); @@ -996,7 +996,7 @@ describe('data routes middleware', () => { 'posts', expect.objectContaining({ 'fields[posts]': 'id,title', - filters: JSON.stringify({ field: 'title', operator: 'Present' }), + filters: JSON.stringify({ field: 'title', operator: 'present', value: null }), sort: '-title', }), ); @@ -1276,7 +1276,9 @@ describe('data routes middleware', () => { 'users', '7', 'posts', - expect.objectContaining({ filters: JSON.stringify(filter) }), + expect.objectContaining({ + filters: JSON.stringify({ field: 'title', operator: 'present', value: null }), + }), ); }); diff --git a/packages/agent-bff/test/data/fixtures/legacy-agent-harness.ts b/packages/agent-bff/test/data/fixtures/legacy-agent-harness.ts new file mode 100644 index 0000000000..bc5d88d888 --- /dev/null +++ b/packages/agent-bff/test/data/fixtures/legacy-agent-harness.ts @@ -0,0 +1,143 @@ +import type { Logger } from '../../../src/ports/logger-port'; +import type { SchemaFetcher } from '../../../src/read-model/forest-schema-client'; +import type { ForestSchemaCollection } from '@forestadmin/forestadmin-client'; +import type { Server } from 'http'; + +import { bodyParser } from '@koa/bodyparser'; +import http from 'http'; +import jsonwebtoken from 'jsonwebtoken'; +import Koa from 'koa'; +import net from 'net'; + +import { createHttpTransport } from '../../../src/agent/agent-transport'; +import createDataRoutesMiddleware from '../../../src/data/data-routes-middleware'; +import createErrorMiddleware from '../../../src/http/error-middleware'; +import CapabilitiesCache from '../../../src/read-model/capabilities-cache'; +import ReadModelStore from '../../../src/read-model/read-model-store'; +import SchemaCache from '../../../src/read-model/schema-cache'; + +export const AUTH_SECRET = 'b0bdf0a639c16bae8851dd24ee3d79ef0a352e957c5b86cb'; + +export async function findFreePort(): Promise { + return new Promise((resolve, reject) => { + const server = net.createServer(); + + server.on('error', reject); + server.listen(0, () => { + const { port } = server.address() as net.AddressInfo; + + server.close(() => resolve(port)); + }); + }); +} + +const TIMEZONE = 'Europe/Paris'; + +/** The apimap a forest-express-sequelize agent pushes: per-field flags, no capabilities route. */ +export const LEGACY_SCHEMA = [ + { + name: 'Article', + fields: [ + { field: 'id', type: 'Number', isPrimaryKey: true, isFilterable: true, isSortable: true }, + { field: 'title', type: 'String', isFilterable: true, isSortable: true }, + { field: 'createdAt', type: 'Date', isFilterable: true, isSortable: true }, + { field: 'computed', type: 'String', isFilterable: false, isSortable: false }, + { field: 'author', type: 'Number', relationship: 'BelongsTo', reference: 'User.id' }, + { field: 'comments', type: ['Number'], relationship: 'HasMany', reference: 'Comment.id' }, + ], + }, +] as unknown as ForestSchemaCollection[]; + +export interface LegacyAgent { + url: string; + stop: () => Promise; + /** Every request the agent saw, so a test can assert what actually went on the wire. */ + seen: { method: string; path: string; query: URLSearchParams }[]; + capabilitiesCalls: () => number; +} + +/** + * A stand-in for a v1 liana: it answers the capabilities route the way Express answers an unknown + * route -- 404 with an HTML body -- and serves records on the JSON:API list route. + */ +export async function startLegacyAgent(): Promise { + const port = await findFreePort(); + const seen: LegacyAgent['seen'] = []; + + const server: Server = http.createServer((request, response) => { + const url = new URL(request.url ?? '/', 'http://localhost'); + seen.push({ method: request.method ?? 'GET', path: url.pathname, query: url.searchParams }); + + if (url.pathname === '/forest/_internal/capabilities') { + response.writeHead(404, { 'content-type': 'text/html' }); + response.end( + '\n\n
Cannot POST /forest/_internal/capabilities
\n\n', + ); + + return; + } + + response.writeHead(200, { 'content-type': 'application/json' }); + response.end( + JSON.stringify({ + data: [{ type: 'Article', id: '1', attributes: { id: 1, title: 'Article 1' } }], + }), + ); + }); + + await new Promise(resolve => { + server.listen(port, resolve); + }); + + return { + url: `http://localhost:${port}`, + stop: () => + new Promise(resolve => { + server.close(() => resolve()); + }), + seen, + capabilitiesCalls: () => + seen.filter(entry => entry.path === '/forest/_internal/capabilities').length, + }; +} + +export function buildLegacyApp(agentUrl: string, { liana }: { liana?: string } = {}): Koa { + const token = jsonwebtoken.sign( + { id: 1, email: 'forest@forest.com', renderingId: 1, team: 'admin' }, + AUTH_SECRET, + { expiresIn: '1 hour' }, + ); + const fetcher: SchemaFetcher = { + fetchSchema: async () => ({ + collections: LEGACY_SCHEMA, + meta: liana === undefined ? {} : { liana, liana_version: '9.21.0' }, + }), + }; + const schemaCache = new SchemaCache({ + fetcher, + metrics: { increment: () => {}, gauge: () => {} }, + }); + const store = new ReadModelStore(schemaCache, new CapabilitiesCache()); + + const noopLogger: Logger = () => {}; + + const app = new Koa(); + + app.silent = true; + app.use(createErrorMiddleware({ logger: noopLogger })); + app.use(bodyParser()); + app.use(async (ctx, next) => { + ctx.state.timezone = TIMEZONE; + ctx.state.agentToken = token; + await next(); + }); + app.use( + createDataRoutesMiddleware({ + store, + transport: createHttpTransport({ agentUrl }), + logger: noopLogger, + }), + ); + + return app; +} diff --git a/packages/agent-bff/test/data/legacy-capabilities.integration.test.ts b/packages/agent-bff/test/data/legacy-capabilities.integration.test.ts new file mode 100644 index 0000000000..1d55d99587 --- /dev/null +++ b/packages/agent-bff/test/data/legacy-capabilities.integration.test.ts @@ -0,0 +1,160 @@ +import type { LegacyAgent } from './fixtures/legacy-agent-harness'; +import type Koa from 'koa'; + +import request from 'supertest'; + +import { buildLegacyApp, startLegacyAgent } from './fixtures/legacy-agent-harness'; + +// Every assertion here goes through the real synthesizer: the agent serves a 404 on the capabilities +// route, so the fields come from the apimap the harness publishes, not from a hand-written object. +describe('constrained reads in front of an agent with no capabilities route', () => { + let agent: LegacyAgent; + let app: Koa; + + beforeAll(async () => { + agent = await startLegacyAgent(); + app = buildLegacyApp(agent.url, { liana: 'forest-rails' }); + }); + + afterAll(async () => { + await agent?.stop(); + }); + + it('should serve a sort on a sortable column, and pass the ordering to the agent', async () => { + const response = await request(app.callback()) + .post('/agent/v1/Article/list') + .send({ sort: [{ field: 'title', direction: 'desc' }] }); + + expect(response.status).toBe(200); + + const listCall = agent.seen.filter(entry => entry.path === '/forest/Article').pop(); + expect(listCall?.query.get('sort')).toBe('-title'); + }); + + it('should serve a projection, and pass the requested fields to the agent', async () => { + const response = await request(app.callback()) + .post('/agent/v1/Article/list') + .send({ projection: ['id', 'title'] }); + + expect(response.status).toBe(200); + + const listCall = agent.seen.filter(entry => entry.path === '/forest/Article').pop(); + expect(listCall?.query.get('fields[Article]')).toBe('id,title'); + }); + + it('should serve a filter and send the operator in the legacy snake_case the liana parses', async () => { + const response = await request(app.callback()) + .post('/agent/v1/Article/list') + .send({ filter: { field: 'title', operator: 'StartsWith', value: 'A' } }); + + expect(response.status).toBe(200); + + const listCall = agent.seen.filter(entry => entry.path === '/forest/Article').pop(); + expect(JSON.parse(listCall?.query.get('filters') ?? '{}')).toEqual({ + field: 'title', + operator: 'starts_with', + value: 'A', + }); + }); + + it('should reject a filter on a field the apimap marks not filterable', async () => { + const response = await request(app.callback()) + .post('/agent/v1/Article/list') + .send({ filter: { field: 'computed', operator: 'Equal', value: 'x' } }); + + expect(response.status).toBe(422); + expect(response.body.error).toMatchObject({ + type: 'field_not_filterable', + details: { field: 'computed' }, + }); + }); + + it('should reject an operator the liana cannot honour, naming the ones it can', async () => { + const response = await request(app.callback()) + .post('/agent/v1/Article/list') + .send({ filter: { field: 'title', operator: 'Like', value: 'A%' } }); + + expect(response.status).toBe(400); + expect(response.body.error.type).toBe('invalid_filter_operator'); + expect(response.body.error.details.validOperators).toContain('StartsWith'); + expect(response.body.error.details.validOperators).not.toContain('Like'); + }); + + it('should reject a sort on a field the apimap marks not sortable, instead of letting SQL fail', async () => { + const response = await request(app.callback()) + .post('/agent/v1/Article/list') + .send({ sort: [{ field: 'computed' }] }); + + expect(response.status).toBe(422); + expect(response.body.error).toMatchObject({ + type: 'field_not_sortable', + details: { field: 'computed' }, + }); + }); + + it('should classify the two relation kinds as v2 does', async () => { + const toOne = await request(app.callback()) + .post('/agent/v1/Article/list') + .send({ filter: { field: 'author', operator: 'Equal', value: 1 } }); + const toMany = await request(app.callback()) + .post('/agent/v1/Article/list') + .send({ filter: { field: 'comments', operator: 'Equal', value: 1 } }); + + expect(toOne.body.error).toMatchObject({ type: 'field_not_filterable' }); + expect(toMany.body.error).toMatchObject({ type: 'unknown_field' }); + }); + + // Its own app, so the count is a delta over a cold capabilities cache: asserting an absolute + // would only hold when the tests above have already warmed the shared one. + it('should ask the agent for capabilities once per cache, not on every constrained request', async () => { + const coldApp = buildLegacyApp(agent.url, { liana: 'forest-rails' }); + const before = agent.capabilitiesCalls(); + + await request(coldApp.callback()) + .post('/agent/v1/Article/list') + .send({ sort: [{ field: 'id' }] }); + await request(coldApp.callback()) + .post('/agent/v1/Article/list') + .send({ sort: [{ field: 'id' }] }); + + expect(agent.capabilitiesCalls() - before).toBe(1); + }); + + it('should never put the agent HTML error page in a response body', async () => { + const response = await request(app.callback()) + .post('/agent/v1/Article/list') + .send({ sort: [{ field: 'title' }] }); + + expect(JSON.stringify(response.body)).not.toContain(' { + let agent: LegacyAgent; + let app: Koa; + + beforeAll(async () => { + agent = await startLegacyAgent(); + app = buildLegacyApp(agent.url, { liana: 'forest-nodejs-agent' }); + }); + + afterAll(async () => { + await agent?.stop(); + }); + + it('should fail the constrained read, but without leaking the agent HTML', async () => { + const response = await request(app.callback()) + .post('/agent/v1/Article/list') + .send({ sort: [{ field: 'title' }] }); + + expect(response.status).toBe(404); + expect(response.body.error.type).toBe('not_found'); + expect(JSON.stringify(response.body)).not.toContain(' { + const response = await request(app.callback()).post('/agent/v1/Article/list').send({}); + + expect(response.status).toBe(200); + }); +}); diff --git a/packages/agent-bff/test/data/pack-id.test.ts b/packages/agent-bff/test/data/pack-id.test.ts index e074e8b82c..e7f4cd18bb 100644 --- a/packages/agent-bff/test/data/pack-id.test.ts +++ b/packages/agent-bff/test/data/pack-id.test.ts @@ -38,4 +38,24 @@ describe('unpackPrimaryKey', () => { ]), ).toThrow(expect.objectContaining({ type: 'mapping_error', status: 500 })); }); + + describe('when the key was derived, so its arity is a guess', () => { + it('should keep a packed composite id whole rather than 500 on the segment count', () => { + expect( + unpackPrimaryKey('tenant|42', [{ name: 'id', type: 'String', derived: true }]), + ).toEqual({ id: 'tenant|42' }); + }); + + it('should still type a numeric id, which the declared id field says is a Number', () => { + expect(unpackPrimaryKey('42', [{ name: 'id', type: 'Number', derived: true }])).toEqual({ + id: 42, + }); + }); + + it('should leave a non-numeric id a string rather than throw on a Number field', () => { + expect(unpackPrimaryKey('7|ab', [{ name: 'id', type: 'Number', derived: true }])).toEqual({ + id: '7|ab', + }); + }); + }); }); diff --git a/packages/agent-bff/test/openapi/cold-start.test.ts b/packages/agent-bff/test/openapi/cold-start.test.ts new file mode 100644 index 0000000000..2d0439d9ee --- /dev/null +++ b/packages/agent-bff/test/openapi/cold-start.test.ts @@ -0,0 +1,145 @@ +import type { CapabilitiesFetcher } from '../../src/read-model/capabilities-cache'; +import type ReadModelStore from '../../src/read-model/read-model-store'; +import type { ForestSchemaCollection } from '@forestadmin/forestadmin-client'; + +import { AgentHttpError, HttpRequester, createRemoteAgentClient } from '@forestadmin/agent-client'; + +import { createHttpTransport } from '../../src/agent/agent-transport'; +import collectUnfolding, { CAPABILITIES_CONCURRENCY } from '../../src/openapi/collect-unfolding'; +import createAgentCapabilitiesFetcher from '../../src/read-model/agent-capabilities-fetcher'; +import ReadModel from '../../src/read-model/read-model'; + +// AgentHttpError stays real: the synthesis branches on `instanceof` and on the status. +jest.mock('@forestadmin/agent-client', () => ({ + ...jest.requireActual('@forestadmin/agent-client'), + createRemoteAgentClient: jest.fn(), + HttpRequester: jest.fn(), +})); + +const createRemoteAgentClientMock = createRemoteAgentClient as jest.Mock; +const mockedHttpRequester = jest.mocked(HttpRequester); + +/** + * What the first OpenAPI unfold costs in front of a legacy liana, at the size the fleet actually + * reaches: the largest schema measured carries 269 collections and 6274 fields, and on a legacy + * agent each collection costs one doomed capabilities POST plus one synthesis. + * + * The invariant that matters is not wall time — that only says how loaded the machine is. It is the + * request count (one POST per collection, no retry) and the fan-out (bounded, but not serial). The + * latency below is controlled so a serialised implementation cannot pass: at 10 collections in + * flight, 269 requests of 10 ms cost about a tenth of what they would one after another. + */ +const COLLECTIONS = 269; +const LATENCY_MS = 10; +const LEGACY_META = { liana: 'forest-rails', liana_version: '9.15.8' }; + +function legacySchema(): ForestSchemaCollection[] { + return Array.from({ length: COLLECTIONS }, (_, index) => ({ + name: `Api__Collection${index}`, + fields: [ + { field: 'id', type: 'Number', isFilterable: true, isSortable: true }, + { field: 'label', type: 'String', isFilterable: true, isSortable: true }, + { field: 'createdAt', type: 'Date', isFilterable: true, isSortable: true }, + ], + })) as unknown as ForestSchemaCollection[]; +} + +describe('the first OpenAPI unfold in front of a legacy liana', () => { + const collections = legacySchema(); + const readModel = new ReadModel(collections); + + let inFlight = 0; + let peakInFlight = 0; + let capabilityCalls = 0; + + function coldStore(): ReadModelStore { + return { + getReadModel: async () => readModel, + getSchemaSnapshot: async () => ({ collections, meta: LEGACY_META, revision: 1 }), + getCapabilities: async (name: string, fetcher: CapabilitiesFetcher) => ({ + capabilities: await fetcher(name), + readModel, + }), + } as unknown as ReadModelStore; + } + + beforeEach(() => { + inFlight = 0; + peakInFlight = 0; + capabilityCalls = 0; + + mockedHttpRequester.mockReset(); + mockedHttpRequester.mockImplementation( + () => ({ query: jest.fn(), stream: jest.fn() } as unknown as HttpRequester), + ); + + // A legacy agent answers the capabilities route with a 404, after a controlled delay so the + // fan-out is observable. + createRemoteAgentClientMock.mockReset(); + createRemoteAgentClientMock.mockReturnValue({ + collection: () => ({ + capabilities: async () => { + capabilityCalls += 1; + inFlight += 1; + peakInFlight = Math.max(peakInFlight, inFlight); + + await new Promise(resolve => { + setTimeout(resolve, LATENCY_MS); + }); + + inFlight -= 1; + + throw new AgentHttpError(404, {}, 'Cannot POST /forest/_internal/capabilities'); + }, + }), + }); + }); + + async function unfold() { + const store = coldStore(); + const started = Date.now(); + + const unfolding = await collectUnfolding({ + readModel, + store, + capabilitiesFetcher: createAgentCapabilitiesFetcher({ + transport: createHttpTransport({ agentUrl: 'https://agent' }), + token: 'tok', + store, + logger: () => undefined, + }), + logger: () => undefined, + }); + + return { unfolding, elapsed: Date.now() - started }; + } + + it('should ask the agent once per collection, with no retry behind it', async () => { + await unfold(); + + expect(capabilityCalls).toBe(COLLECTIONS); + }); + + it('should type every collection from the apimap rather than degrade it', async () => { + const { unfolding } = await unfold(); + + expect(unfolding.collections).toHaveLength(COLLECTIONS); + expect(unfolding.collections.filter(entry => entry.fields.degraded !== null)).toEqual([]); + }); + + it('should fan the doomed requests out to the configured bound, not run them one by one', async () => { + await unfold(); + + expect(peakInFlight).toBeGreaterThan(1); + expect(peakInFlight).toBeLessThanOrEqual(CAPABILITIES_CONCURRENCY); + }); + + // The assertion a serialised fan-out fails: 269 requests of 10 ms cost 2.7 s in sequence. Half of + // that is loose enough to survive a busy machine and tight enough to catch the regression. + it('should cost a fraction of what the same requests would cost in sequence', async () => { + const { elapsed } = await unfold(); + const sequential = COLLECTIONS * LATENCY_MS; + + expect(elapsed).toBeLessThan(sequential / 2); + }); +}); diff --git a/packages/agent-bff/test/openapi/collect-unfolding.test.ts b/packages/agent-bff/test/openapi/collect-unfolding.test.ts index 51bac4fcf1..7e0f8b47fa 100644 --- a/packages/agent-bff/test/openapi/collect-unfolding.test.ts +++ b/packages/agent-bff/test/openapi/collect-unfolding.test.ts @@ -155,6 +155,22 @@ describe('collectUnfolding', () => { ); }); + it('should carry the sortable denial the v1 synthesis states, so the sort enum can drop it', async () => { + const { collections } = await collect(readModel, { + capabilities: async () => ({ + fields: [ + { name: 'id', type: 'String', operators: ['equal'] }, + { name: 'fullName', type: 'String', operators: [], sortable: false as const }, + ], + }), + }); + + expect(collections[0].fields.projectable).toEqual([ + { name: 'id', type: 'String' }, + { name: 'fullName', type: 'String', sortable: false }, + ]); + }); + it('should keep a skewed field projectable, since only filtering on it fails', async () => { const { collections } = await collect(readModel, { capabilities: async () => ({ @@ -178,6 +194,28 @@ describe('collectUnfolding', () => { expect(collections[0].fields.degraded).toBeNull(); }); + // An empty `filterable` otherwise reads as "this collection filters on nothing", and the document + // then refuses, in a generated client, every filter the agent honours. + it('should mark the filterable set undocumentable when a field was dropped for skew', async () => { + const { collections } = await collect(readModel, { + capabilities: async () => ({ + fields: [{ name: 'id', type: 'String', operators: ['equal', 'teleports_to'] }], + }), + }); + + expect(collections[0].fields.undocumentableFilter).toBe(true); + }); + + it('should leave the flag off when every field maps, even one carrying no operator', async () => { + const { collections } = await collect(readModel, { + capabilities: async () => ({ + fields: [{ name: 'id', type: 'String', operators: [] }], + }), + }); + + expect(collections[0].fields).not.toHaveProperty('undocumentableFilter'); + }); + it('should degrade a collection whose capabilities call fails, and say so in the log', async () => { const logger = jest.fn(); const { collections } = await collect(readModel, { diff --git a/packages/agent-bff/test/openapi/openapi-cli.test.ts b/packages/agent-bff/test/openapi/openapi-cli.test.ts index dd462d15c2..f6f4fe0a07 100644 --- a/packages/agent-bff/test/openapi/openapi-cli.test.ts +++ b/packages/agent-bff/test/openapi/openapi-cli.test.ts @@ -30,7 +30,7 @@ const SCHEMA = [ const CAPABILITIES = { fields: [{ name: 'id', type: 'Number', operators: ['equal'] }] }; -const fetchSchema = jest.fn().mockResolvedValue(SCHEMA); +const fetchSchema = jest.fn().mockResolvedValue({ collections: SCHEMA, meta: {} }); const fetchCapabilities = jest.fn().mockResolvedValue(CAPABILITIES); const mintedTokens: string[] = []; @@ -75,7 +75,7 @@ const noopLogger: Logger = () => undefined; describe('renderOpenApi', () => { beforeEach(() => { - fetchSchema.mockReset().mockResolvedValue(SCHEMA); + fetchSchema.mockReset().mockResolvedValue({ collections: SCHEMA, meta: {} }); fetchCapabilities.mockReset().mockResolvedValue(CAPABILITIES); mintedTokens.length = 0; }); diff --git a/packages/agent-bff/test/openapi/openapi-generated-client.test.ts b/packages/agent-bff/test/openapi/openapi-generated-client.test.ts index fbf3a237aa..44c72ecff0 100644 --- a/packages/agent-bff/test/openapi/openapi-generated-client.test.ts +++ b/packages/agent-bff/test/openapi/openapi-generated-client.test.ts @@ -48,7 +48,7 @@ const CAPABILITIES = { fields: [{ name: 'id', type: 'Number', operators: ['equal const DOCUMENTED_OPERATOR = 'Equal'; const UNDOCUMENTED_OPERATOR = 'GreaterThan'; -const fetchSchema = jest.fn().mockResolvedValue(SCHEMA); +const fetchSchema = jest.fn().mockResolvedValue({ collections: SCHEMA, meta: {} }); const fetchCapabilities = jest.fn().mockResolvedValue(CAPABILITIES); jest.mock('../../src/read-model/forest-schema-client', () => ({ diff --git a/packages/agent-bff/test/openapi/openapi-routes.test.ts b/packages/agent-bff/test/openapi/openapi-routes.test.ts index 5706034aae..e191d3c9f9 100644 --- a/packages/agent-bff/test/openapi/openapi-routes.test.ts +++ b/packages/agent-bff/test/openapi/openapi-routes.test.ts @@ -28,7 +28,7 @@ const CAPABILITIES = { ], }; -const fetchSchema = jest.fn().mockResolvedValue(SCHEMA); +const fetchSchema = jest.fn().mockResolvedValue({ collections: SCHEMA, meta: {} }); const fetchCapabilities = jest.fn().mockResolvedValue(CAPABILITIES); jest.mock('../../src/read-model/forest-schema-client', () => ({ @@ -155,7 +155,7 @@ function routesFor(store: ReadModelStore, basePath?: string): Middleware { describe('GET /agent/openapi.json', () => { beforeEach(() => { - fetchSchema.mockClear().mockResolvedValue(SCHEMA); + fetchSchema.mockClear().mockResolvedValue({ collections: SCHEMA, meta: {} }); fetchCapabilities.mockClear().mockResolvedValue(CAPABILITIES); }); diff --git a/packages/agent-bff/test/openapi/openapi-unfolded.test.ts b/packages/agent-bff/test/openapi/openapi-unfolded.test.ts index 8ea62b9a64..84fea615d5 100644 --- a/packages/agent-bff/test/openapi/openapi-unfolded.test.ts +++ b/packages/agent-bff/test/openapi/openapi-unfolded.test.ts @@ -254,6 +254,15 @@ describe('the unfolded document', () => { expect(request.properties.sort.items?.$ref).toBe('#/components/schemas/SortClause_My_Coll'); }); + it('should sort on the projectable enum itself while no field denies it', () => { + const sort = schemas.SortClause_My_Coll as unknown as { + properties: { field: { $ref?: string } }; + }; + + expect(sort.properties.field.$ref).toBe('#/components/schemas/Fields_My_Coll'); + expect(Object.keys(schemas).filter(name => name.startsWith('SortableFields'))).toEqual([]); + }); + it('should leave page optional here too, pointing at the shared Page component', () => { const request = requestSchema('My%20Coll/list') as unknown as { required?: string[]; @@ -783,22 +792,369 @@ describe('a collection whose key ends in an index-like suffix', () => { }); describe('an unfolding carrying a filterable field with no operator', () => { - it('should leave it out rather than emit an enum no value satisfies', () => { + it('should register no leaf rather than emit an enum no value satisfies', () => { // An empty enum forbids every value, so the leaf would be unsatisfiable. `collectFilterableFields` - // cannot produce one, but the generator takes hand-constructible plain data. + // cannot produce one, but the generator takes hand-constructible plain data. The field set is + // known here, so dropping the only field leaves no valid leaf — not a free-form one, which would + // offer a filter every field answers 422 on. const empty = unfoldedDocument({ collections: [collectionOf('E', [{ name: 'x', operators: [] }])], }); const emptySchemas = empty.components?.schemas as Record>; - const leaf = emptySchemas.FilterLeaf_E as unknown as { - properties: { field: { enum?: string[] }; operator: { enum: string[] } }; + + expect(Object.keys(emptySchemas).filter(name => name.startsWith('FilterLeaf'))).toEqual([]); + }); +}); + +describe('an unfolding whose apimap denies a sort', () => { + // Only the v1 synthesis states sortability. The document must not offer a sort the runtime then + // answers 422 field_not_sortable on. + function documentWithDeniedSort() { + return unfoldedDocument({ + collections: [ + { + name: 'users', + fields: { + projectable: [ + { name: 'id', type: 'Number' }, + { name: 'fullName', type: 'String', sortable: false }, + ], + filterable: [{ name: 'id', operators: ['Equal'] }], + degraded: null, + }, + primaryKeys: [{ name: 'id', type: 'Number' }], + relations: [], + actions: [], + }, + ], + }); + } + + it('should leave the denied field out of the sort enum while keeping it projectable', () => { + const denied = documentWithDeniedSort(); + const deniedSchemas = denied.components?.schemas as Record>; + const sortable = deniedSchemas.SortableFields_users as unknown as { enum: string[] }; + const projectable = deniedSchemas.Fields_users as unknown as { enum: string[] }; + const sort = deniedSchemas.SortClause_users as unknown as { + properties: { field: { $ref?: string } }; }; - expect(Object.keys(emptySchemas).filter(name => name.startsWith('FilterLeaf'))).toEqual([ - 'FilterLeaf_E', - ]); - expect(leaf.properties.field.enum).toBeUndefined(); - expect(leaf.properties.operator.enum).toEqual([...allOperators]); + expect(projectable.enum).toEqual(['id', 'fullName']); + expect(sortable.enum).toEqual(['id']); + expect(sort.properties.field.$ref).toBe('#/components/schemas/SortableFields_users'); + }); + + function documentWithNoSortableField() { + return unfoldedDocument({ + collections: [ + { + name: 'users', + fields: { + projectable: [{ name: 'fullName', type: 'String', sortable: false }], + filterable: [], + degraded: null, + }, + primaryKeys: [{ name: 'id', type: 'Number' }], + relations: [], + actions: [], + }, + ], + }); + } + + function listSortOf(built: ReturnType) { + const builtSchemas = built.components?.schemas as Record>; + const list = builtSchemas.ListRequest_users as unknown as { + properties: { sort: { maxItems?: number } }; + }; + + return list.properties.sort; + } + + it('should register no sort enum when every field denies it', () => { + const noneSchemas = documentWithNoSortableField().components?.schemas as Record< + string, + Record + >; + + expect(noneSchemas.SortClause_users).toBeUndefined(); + expect(noneSchemas.SortableFields_users).toBeUndefined(); + }); + + // The shared SortClause leaves `field` an unrestricted string, so reusing it alone would advertise + // every field as sortable while each request answers 422 field_not_sortable. + it('should cap the sort array at zero when every field denies it', () => { + expect(listSortOf(documentWithNoSortableField()).maxItems).toBe(0); + }); + + it('should leave the sort array uncapped when a field allows it', () => { + expect(listSortOf(documentWithDeniedSort())).not.toHaveProperty('maxItems'); + }); + + function documentDegradedAs(degraded: 'capabilities_unavailable' | 'no_fields') { + return unfoldedDocument({ + collections: [ + { + name: 'users', + fields: { projectable: [], filterable: [], degraded }, + primaryKeys: [{ name: 'id', type: 'Number' }], + relations: [], + actions: [], + }, + ], + }); + } + + // Capabilities that could not be read leave the field set unknown: the runtime rejects, not the + // document, so capping the array would forbid a sort the agent accepts. + it('should leave the sort array uncapped when capabilities could not be read', () => { + expect(listSortOf(documentDegradedAs('capabilities_unavailable'))).not.toHaveProperty( + 'maxItems', + ); + }); + + // Capabilities naming no field is an answer, not a gap: every field is rejected, sort included. + it('should cap the sort array when capabilities name no field at all', () => { + expect(listSortOf(documentDegradedAs('no_fields')).maxItems).toBe(0); + }); +}); + +describe('an unfolding whose collection has no filterable field', () => { + function filterOf(built: ReturnType) { + const builtSchemas = built.components?.schemas as Record>; + + return { + tree: builtSchemas.Filter_users as unknown as { anyOf: { $ref?: string }[] }, + leaf: builtSchemas.FilterLeaf_users, + }; + } + + function documentWith( + fields: Parameters[0]['collections'][number]['fields'], + ) { + return unfoldedDocument({ + collections: [ + { + name: 'users', + fields, + primaryKeys: [{ name: 'id', type: 'Number' }], + relations: [], + actions: [], + }, + ], + }); + } + + // A free-form leaf here would offer a filter the validator answers 422 field_not_filterable on, + // whatever field it carries. + it('should register no leaf when the field set is known and nothing is filterable', () => { + const { tree, leaf } = filterOf( + documentWith({ + projectable: [{ name: 'fullName', type: 'String' }], + filterable: [], + degraded: null, + }), + ); + + expect(leaf).toBeUndefined(); + expect(tree.anyOf).toHaveLength(2); + }); + + it('should register no leaf when capabilities name no field at all', () => { + expect( + filterOf(documentWith({ projectable: [], filterable: [], degraded: 'no_fields' })).leaf, + ).toBeUndefined(); + }); + + // Unknown is not empty: the collection still accepts whatever it really exposes. + it('should keep the free-form leaf when capabilities could not be read', () => { + expect( + filterOf( + documentWith({ projectable: [], filterable: [], degraded: 'capabilities_unavailable' }), + ).leaf, + ).toBeDefined(); + }); +}); + +describe('an unfolding whose filterable set is incomplete rather than empty', () => { + // `collectFilterableFields` drops a field whose operator set it cannot map to canonical names, and + // leaves `degraded` null while doing it. Dropping every field that way empties `filterable` on a + // collection the agent filters fine, so the document must not state that nothing is filterable. + function documentWithSkew() { + return unfoldedDocument({ + collections: [ + { + name: 'users', + fields: { + projectable: [{ name: 'fullName', type: 'String' }], + filterable: [], + degraded: null, + undocumentableFilter: true, + }, + primaryKeys: [{ name: 'id', type: 'Number' }], + relations: [], + actions: [], + }, + ], + }); + } + + function schemasOf(built: ReturnType) { + return built.components?.schemas as Record>; + } + + it('should keep the free-form leaf when every field was dropped as undocumentable', () => { + expect(schemasOf(documentWithSkew()).FilterLeaf_users).toBeDefined(); + }); + + it('should not claim no field is filterable when the set is merely undocumentable', () => { + const tree = schemasOf(documentWithSkew()).Filter_users as unknown as { description: string }; + + expect(tree.description).not.toContain('No field of this collection is filterable'); + }); +}); + +describe('an unfolding whose capabilities name no projectable field', () => { + function documentWith(degraded: DegradedReason) { + return unfoldedDocument({ + collections: [ + { + name: 'users', + fields: { projectable: [], filterable: [], degraded }, + primaryKeys: [{ name: 'id', type: 'Number' }], + relations: [], + actions: [], + }, + ], + }); + } + + function listProjectionOf(built: ReturnType) { + const builtSchemas = built.components?.schemas as Record>; + const list = builtSchemas.ListRequest_users as unknown as { + properties: { projection: { maxItems?: number; description?: string } }; + }; + + return list.properties.projection; + } + + // `fieldsEnum` falls back to an unrestricted string for an empty list, so the document would offer + // every field name while each request answers 422 unknown_field. + it('should cap the projection array when capabilities name no field at all', () => { + expect(listProjectionOf(documentWith('no_fields')).maxItems).toBe(0); + }); + + it('should leave the projection array uncapped when capabilities could not be read', () => { + expect(listProjectionOf(documentWith('capabilities_unavailable'))).not.toHaveProperty( + 'maxItems', + ); + }); +}); + +describe('the prose that states what a known-empty field set forbids', () => { + // Inverting `isFieldSetKnown` must fail a test rather than only contradict the schema in prose. + function documentWith(degraded: DegradedReason) { + return unfoldedDocument({ + collections: [ + { + name: 'users', + fields: { projectable: [], filterable: [], degraded }, + primaryKeys: [{ name: 'id', type: 'Number' }], + relations: [], + actions: [], + }, + ], + }); + } + + function listOf(built: ReturnType) { + const builtSchemas = built.components?.schemas as Record>; + + return { + request: builtSchemas.ListRequest_users as unknown as { + properties: { + sort: { description?: string }; + projection: { description?: string }; + }; + }, + tree: builtSchemas.Filter_users as unknown as { description: string }, + }; + } + + it('should say the sort and projection arrays take nothing on a known-empty field set', () => { + const { request } = listOf(documentWith('no_fields')); + + expect(request.properties.sort.description).toContain( + 'No field of this collection is sortable', + ); + expect(request.properties.projection.description).toContain('This collection exposes no field'); + }); + + it('should say the filter tree carries no leaf on a known-empty field set', () => { + expect(listOf(documentWith('no_fields')).tree.description).toContain( + 'No field of this collection is filterable, so there is no leaf alternative at all', + ); + }); + + it('should state none of that when capabilities could not be read', () => { + const { request, tree } = listOf(documentWith('capabilities_unavailable')); + + expect(request.properties.sort).not.toHaveProperty('description'); + expect(request.properties.projection).not.toHaveProperty('description'); + expect(tree.description).not.toContain('No field of this collection is filterable'); + }); +}); + +describe('an unfolding whose parent key was derived rather than declared', () => { + // The read-model flags a key it guessed from an `id` field. Naming it `(id, String)` the way a + // declared key is named would contradict `ForestRecordMeta` in the same document. + function documentWith(primaryKeys: { name: string; type: string; derived?: true }[]) { + return unfoldedDocument({ + collections: [ + { + name: 'users', + fields: { + projectable: [{ name: 'id', type: 'String' }], + filterable: [{ name: 'id', operators: ['Equal'] }], + degraded: null, + }, + primaryKeys, + relations: [{ name: 'orders', foreignCollection: 'orders' }], + actions: [], + }, + { + name: 'orders', + fields: { + projectable: [{ name: 'id', type: 'Number' }], + filterable: [{ name: 'id', operators: ['Equal'] }], + degraded: null, + }, + primaryKeys: [{ name: 'id', type: 'Number' }], + relations: [], + actions: [], + }, + ], + } as unknown as Unfolding); + } + + function parentIdOf(built: ReturnType) { + const builtSchemas = built.components?.schemas as Record>; + const request = builtSchemas.RelationListRequest_users_orders as unknown as { + properties: { parentId: { description: string } }; + }; + + return request.properties.parentId.description; + } + + it('should hedge the description instead of naming a column the schema never published', () => { + const description = parentIdOf(documentWith([{ name: 'id', type: 'String', derived: true }])); + + expect(description).toContain('not published by the schema'); + expect(description).not.toContain('(id, String)'); + }); + + it('should name the column outright when the key was declared', () => { + expect(parentIdOf(documentWith([{ name: 'id', type: 'String' }]))).toContain('(id, String)'); }); }); diff --git a/packages/agent-bff/test/read-model/agent-capabilities-fetcher.test.ts b/packages/agent-bff/test/read-model/agent-capabilities-fetcher.test.ts index acdb5c237c..f295c93eb3 100644 --- a/packages/agent-bff/test/read-model/agent-capabilities-fetcher.test.ts +++ b/packages/agent-bff/test/read-model/agent-capabilities-fetcher.test.ts @@ -1,23 +1,72 @@ -import { HttpRequester, createRemoteAgentClient } from '@forestadmin/agent-client'; +import type { CapabilitiesFetcher } from '../../src/read-model/capabilities-cache'; +import type ReadModelStore from '../../src/read-model/read-model-store'; +import type { ForestSchemaCollection, ForestSchemaMeta } from '@forestadmin/forestadmin-client'; +import { AgentHttpError, HttpRequester, createRemoteAgentClient } from '@forestadmin/agent-client'; + +import { collection as collectionFixture, column } from './fixtures'; import { createHttpTransport } from '../../src/agent/agent-transport'; +import collectUnfolding from '../../src/openapi/collect-unfolding'; import createAgentCapabilitiesFetcher from '../../src/read-model/agent-capabilities-fetcher'; +import ReadModel from '../../src/read-model/read-model'; function transportTo(agentUrl: string, timeoutMs?: number) { return createHttpTransport({ agentUrl, timeoutMs }); } -jest.mock('@forestadmin/agent-client'); +// AgentHttpError stays real: the synthesis branches on `instanceof` and on the status, so an +// automocked constructor would make every 404 test vacuous. +jest.mock('@forestadmin/agent-client', () => ({ + ...jest.requireActual('@forestadmin/agent-client'), + createRemoteAgentClient: jest.fn(), + HttpRequester: jest.fn(), +})); const createRemoteAgentClientMock = createRemoteAgentClient as jest.Mock; const mockedHttpRequester = jest.mocked(HttpRequester); +const LEGACY_META = { liana: 'forest-express-sequelize', liana_version: '9.6.10' }; + describe('createAgentCapabilitiesFetcher', () => { const query = jest.fn(); const stream = jest.fn(); + const logger = jest.fn(); + + const usersApimap = [ + { + name: 'users', + fields: [{ field: 'name', type: 'String', isFilterable: true, isSortable: true }], + }, + ] as unknown as ForestSchemaCollection[]; + + function storeServing( + collections: ForestSchemaCollection[], + meta: ForestSchemaMeta = LEGACY_META, + ): ReadModelStore { + return { + getSchemaSnapshot: jest.fn().mockResolvedValue({ collections, meta, revision: 1 }), + } as unknown as ReadModelStore; + } + + function synthesisFrom(collections: ForestSchemaCollection[], meta?: ForestSchemaMeta) { + return { store: storeServing(collections, meta), logger }; + } + + // A store whose snapshot changes between calls, so a test can show which one the decision reads. + function storeServingInTurn( + ...snapshots: { collections: ForestSchemaCollection[]; meta: ForestSchemaMeta }[] + ): ReadModelStore { + const getSchemaSnapshot = jest.fn(); + snapshots.forEach(snapshot => + getSchemaSnapshot.mockResolvedValueOnce({ ...snapshot, revision: 1 }), + ); + + return { getSchemaSnapshot } as unknown as ReadModelStore; + } beforeEach(() => { query.mockReset(); + logger.mockReset(); createRemoteAgentClientMock.mockReset(); mockedHttpRequester.mockReset(); mockedHttpRequester.mockImplementation(() => ({ query, stream } as unknown as HttpRequester)); @@ -31,6 +80,7 @@ describe('createAgentCapabilitiesFetcher', () => { const fetcher = createAgentCapabilitiesFetcher({ transport: transportTo('https://agent'), token: 'tok', + ...synthesisFrom([]), }); const result = await fetcher('users'); @@ -54,6 +104,7 @@ describe('createAgentCapabilitiesFetcher', () => { const fetcher = createAgentCapabilitiesFetcher({ transport: transportTo('https://agent'), token: 'tok', + ...synthesisFrom([]), }); await fetcher('users'); await fetcher('orders'); @@ -75,6 +126,7 @@ describe('createAgentCapabilitiesFetcher', () => { return `tok-${minted}`; }, + ...synthesisFrom([]), }); await fetcher('users'); await fetcher('orders'); @@ -96,6 +148,7 @@ describe('createAgentCapabilitiesFetcher', () => { await createAgentCapabilitiesFetcher({ transport: transportTo('https://agent', 2500), token: 'tok', + ...synthesisFrom([]), })('users'); const httpRequester = createRemoteAgentClientMock.mock.calls[0][0] @@ -108,4 +161,227 @@ describe('createAgentCapabilitiesFetcher', () => { maxTimeAllowed: 2500, }); }); + + describe('when the agent answers the capabilities route with a 404', () => { + function fetcherRejectingWith( + error: unknown, + collections: ForestSchemaCollection[], + meta?: ForestSchemaMeta, + ) { + createRemoteAgentClientMock.mockReturnValue({ + collection: jest.fn().mockReturnValue({ + capabilities: jest.fn().mockRejectedValue(error), + }), + }); + + return createAgentCapabilitiesFetcher({ + transport: transportTo('https://agent'), + token: 'tok', + ...synthesisFrom(collections, meta), + }); + } + + const notFound = () => + new AgentHttpError(404, {}, 'Cannot POST /forest/_internal/capabilities'); + + it('should synthesize from the apimap when a legacy liana published the schema', async () => { + const fetcher = fetcherRejectingWith(notFound(), usersApimap); + + const result = await fetcher('users'); + + expect(result).toEqual({ + fields: [{ name: 'name', type: 'String', operators: expect.arrayContaining(['equal']) }], + }); + }); + + it('should name the liana and its version in the warning, so a stale agent is diagnosable', async () => { + const fetcher = fetcherRejectingWith(notFound(), usersApimap, { + liana: 'forest-rails', + liana_version: '9.21.0', + }); + + await fetcher('users'); + + expect(logger).toHaveBeenCalledWith( + 'Warn', + expect.any(String), + expect.objectContaining({ + agentUrl: 'https://agent', + collection: 'users', + liana: 'forest-rails', + lianaVersion: '9.21.0', + }), + ); + }); + + it.each(['forest-nodejs-agent', 'agent-ruby', 'agent-python', 'agent-php'])( + 'should rethrow rather than synthesize for %s, which does serve that route', + async liana => { + const fetcher = fetcherRejectingWith(notFound(), usersApimap, { liana }); + + await expect(fetcher('users')).rejects.toMatchObject({ status: 404 }); + expect(logger).toHaveBeenCalledWith( + 'Error', + expect.any(String), + expect.objectContaining({ liana }), + ); + }, + ); + + it('should rethrow for a liana name nobody has classified', async () => { + const fetcher = fetcherRejectingWith(notFound(), usersApimap, { liana: 'forest-symfony' }); + + await expect(fetcher('users')).rejects.toMatchObject({ status: 404 }); + }); + + it('should rethrow when the published schema carries no liana at all, and say so', async () => { + const fetcher = fetcherRejectingWith(notFound(), usersApimap, {}); + + await expect(fetcher('users')).rejects.toMatchObject({ status: 404 }); + expect(logger).toHaveBeenCalledWith( + 'Error', + expect.any(String), + expect.objectContaining({ liana: 'absent from the published schema' }), + ); + }); + + it('should rethrow when the schema does not know the collection, rather than invent a shape', async () => { + const fetcher = fetcherRejectingWith(notFound(), []); + + await expect(fetcher('users')).rejects.toMatchObject({ status: 404 }); + expect(logger).not.toHaveBeenCalled(); + }); + + // Criterion 5. The OpenAPI document never sees the 404: `collect-unfolding` catches a failing + // capabilities lookup and marks the collection degraded. So the liana decision shows up there as + // a different document, not a different status — typed fields for a legacy liana, the degraded + // marker for anything else. + describe('the OpenAPI document built from the same fetcher', () => { + function unfoldWith(meta: ForestSchemaMeta) { + createRemoteAgentClientMock.mockReturnValue({ + collection: jest.fn().mockReturnValue({ + capabilities: jest.fn().mockRejectedValue(notFound()), + }), + }); + + const readModel = new ReadModel([ + collectionFixture('users', [column('id'), column('name')]), + ]); + const store = storeServing(usersApimap, meta); + + return collectUnfolding({ + readModel, + store: { + ...store, + getReadModel: async () => readModel, + getCapabilities: async (name: string, fetcher: CapabilitiesFetcher) => ({ + capabilities: await fetcher(name), + readModel, + }), + } as unknown as ReadModelStore, + capabilitiesFetcher: createAgentCapabilitiesFetcher({ + transport: transportTo('https://agent'), + token: 'tok', + store, + logger, + }), + logger, + }); + } + + it('should type the fields when a legacy liana published the schema', async () => { + const { collections } = await unfoldWith(LEGACY_META); + + expect(collections[0].fields.degraded).toBeNull(); + expect(collections[0].fields.projectable.map(field => field.name)).toEqual(['name']); + }); + + it('should keep the degraded marker for a liana that should have served the route', async () => { + const { collections } = await unfoldWith({ + liana: 'forest-nodejs-agent', + liana_version: '1.98.1', + }); + + expect(collections[0].fields.degraded).toBe('capabilities_unavailable'); + }); + }); + + // Criterion 10. The rethrow path reads the snapshot before deciding, which the flag-gated + // version did not: on a healthy cache that costs nothing, but on an expired entry whose refresh + // keeps failing it is one extra attempt — bounded to one per call, never a retry loop. + describe('what the rethrow path costs the schema cache', () => { + function fetcherOver(store: ReadModelStore) { + createRemoteAgentClientMock.mockReturnValue({ + collection: jest.fn().mockReturnValue({ + capabilities: jest.fn().mockRejectedValue(notFound()), + }), + }); + + return createAgentCapabilitiesFetcher({ + transport: transportTo('https://agent'), + token: 'tok', + store, + logger, + }); + } + + it('should read the snapshot once per rethrown call', async () => { + const getSchemaSnapshot = jest.fn().mockResolvedValue({ + collections: usersApimap, + meta: { liana: 'forest-nodejs-agent' }, + revision: 1, + }); + const fetcher = fetcherOver({ getSchemaSnapshot } as unknown as ReadModelStore); + + await expect(fetcher('users')).rejects.toMatchObject({ status: 404 }); + + expect(getSchemaSnapshot).toHaveBeenCalledTimes(1); + }); + + // The snapshot read sits inside the catch, so its own failure replaces the agent's 404. Both + // are failures and the caller retries either way; what matters is that it stays one attempt. + it('should surface the schema failure when the snapshot cannot be read on the 404 path', async () => { + const getSchemaSnapshot = jest.fn().mockRejectedValue(new Error('schema unavailable')); + const fetcher = fetcherOver({ getSchemaSnapshot } as unknown as ReadModelStore); + + await expect(fetcher('users')).rejects.toThrow('schema unavailable'); + expect(getSchemaSnapshot).toHaveBeenCalledTimes(1); + }); + }); + + it('should rethrow a non-404, so a real agent failure is never read as a legacy agent', async () => { + const fetcher = fetcherRejectingWith(new AgentHttpError(500, {}, 'boom'), usersApimap); + + await expect(fetcher('users')).rejects.toMatchObject({ status: 500 }); + }); + + // The decision is read from the snapshot on every call, not captured once. That is what bounds + // a v1 -> v2 migration: the synthesis keeps engaging while the cached schema still names the + // legacy liana, and stops on the first snapshot that names the new one. + it('should follow the snapshot per call, synthesizing then rethrowing once the liana changes', async () => { + createRemoteAgentClientMock.mockReturnValue({ + collection: jest.fn().mockReturnValue({ + capabilities: jest.fn().mockRejectedValue(notFound()), + }), + }); + + const fetcher = createAgentCapabilitiesFetcher({ + transport: transportTo('https://agent'), + token: 'tok', + store: storeServingInTurn( + { collections: usersApimap, meta: { liana: 'forest-rails', liana_version: '9.21.0' } }, + { + collections: usersApimap, + meta: { liana: 'forest-nodejs-agent', liana_version: '1.98.1' }, + }, + ), + logger, + }); + + await expect(fetcher('users')).resolves.toEqual({ + fields: [{ name: 'name', type: 'String', operators: expect.arrayContaining(['equal']) }], + }); + await expect(fetcher('users')).rejects.toMatchObject({ status: 404 }); + }); + }); }); diff --git a/packages/agent-bff/test/read-model/create-read-model.test.ts b/packages/agent-bff/test/read-model/create-read-model.test.ts index 0687b070c2..1e408b8358 100644 --- a/packages/agent-bff/test/read-model/create-read-model.test.ts +++ b/packages/agent-bff/test/read-model/create-read-model.test.ts @@ -7,14 +7,17 @@ import createReadModel from '../../src/read-model/create-read-model'; jest.mock('@forestadmin/forestadmin-client'); describe('createReadModel', () => { - const getSchema = jest.fn(); + const getSchemaWithMeta = jest.fn(); beforeEach(() => { jest.clearAllMocks(); - (SchemaService as unknown as jest.Mock).mockImplementation(() => ({ getSchema })); - getSchema.mockResolvedValue([ - collection('users', [column('id')], [action('ban', '/forest/users/actions/ban')]), - ]); + (SchemaService as unknown as jest.Mock).mockImplementation(() => ({ getSchemaWithMeta })); + getSchemaWithMeta.mockResolvedValue({ + collections: [ + collection('users', [column('id')], [action('ban', '/forest/users/actions/ban')]), + ], + meta: {}, + }); }); it('should wire a store whose read-model reflects the fetched schema', async () => { diff --git a/packages/agent-bff/test/read-model/fixtures.ts b/packages/agent-bff/test/read-model/fixtures.ts index 9abbc63d99..c97ae16764 100644 --- a/packages/agent-bff/test/read-model/fixtures.ts +++ b/packages/agent-bff/test/read-model/fixtures.ts @@ -3,6 +3,8 @@ import type { ForestSchemaAction, ForestSchemaCollection, ForestSchemaField, + ForestSchemaMeta, + ForestSchemaWithMeta, } from '@forestadmin/forestadmin-client'; export function makeMetrics(): jest.Mocked { @@ -56,3 +58,10 @@ export function collection( export function makeSchema(name: string): ForestSchemaCollection[] { return [collection(name, [])]; } + +export function published( + collections: ForestSchemaCollection[], + meta: ForestSchemaMeta = {}, +): ForestSchemaWithMeta { + return { collections, meta }; +} diff --git a/packages/agent-bff/test/read-model/fixtures/legacy-rails-schema.json b/packages/agent-bff/test/read-model/fixtures/legacy-rails-schema.json new file mode 100644 index 0000000000..714d4074c8 --- /dev/null +++ b/packages/agent-bff/test/read-model/fixtures/legacy-rails-schema.json @@ -0,0 +1,1907 @@ +{ + "meta": { + "liana": "forest-rails", + "liana_version": "9.15.8", + "stack": { + "database_type": "postgresql", + "orm_version": "7.2.2.2" + } + }, + "note": "Sampled from a schema published by a forest_liana 9.15.8 backoffice: 269 collections, none declaring isPrimaryKey, which that liana never emits. Kept one or two collections per shape, field lists trimmed. It stands for the forest-rails installs still below 9.17.6, not for any particular customer \u2014 the one this came from has upgraded since.", + "collections": [ + { + "name": "Api__AccountRemuneration", + "why": "HasOne", + "fields": [ + { + "field": "created_at", + "type": "Date", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "description", + "type": "String", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "id", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "parent_transaction", + "type": "Number", + "enums": null, + "integration": null, + "reference": "Api__Transaction.id", + "relationship": "HasOne", + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "account_remuneration" + }, + { + "field": "updated_at", + "type": "Date", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + } + ], + "actions": [ + { + "name": "Report a bug to #cft-financing-backend", + "type": "global", + "base_url": null, + "endpoint": "/forest/actions/report-bug/cft_financing", + "http_method": "POST", + "redirect": null, + "download": false, + "fields": [], + "hooks": { + "load": false, + "change": [] + }, + "description": null, + "submit_button_label": null + } + ], + "segments": [] + }, + { + "name": "Api__Attachment", + "why": "BelongsTo and HasMany together", + "fields": [ + { + "field": "attached_by_qonto", + "type": "Boolean", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": false, + "isReadOnly": true, + "isRequired": false, + "isVirtual": true, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "attached_to_transaction", + "type": "Boolean", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": false, + "isReadOnly": true, + "isRequired": false, + "isVirtual": true, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "attachment_type", + "type": "Enum", + "enums": [ + "attachment", + "invoice", + "pagopa", + "receivable_invoice", + "nrc", + "mileage_receipt", + "card_acquirer_payout", + "riba" + ], + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": "0", + "inverseOf": null + }, + { + "field": "content_type", + "type": "String", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "id", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "organization", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": "Api__Organization.id", + "relationship": "BelongsTo", + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "transaction_attachments", + "type": [ + "Number" + ], + "enums": null, + "integration": null, + "reference": "Api__TransactionAttachment.id", + "relationship": "HasMany", + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "attachment" + } + ], + "actions": [ + { + "name": "Report a bug to #cft-cash-flow-management", + "type": "global", + "base_url": null, + "endpoint": "/forest/actions/report-bug/cft_cashflow_management", + "http_method": "POST", + "redirect": null, + "download": false, + "fields": [], + "hooks": { + "load": false, + "change": [] + }, + "description": null, + "submit_button_label": null + } + ], + "segments": [] + }, + { + "name": "Api__BankAccount", + "why": "BelongsTo and HasMany together", + "fields": [ + { + "field": "account_number", + "type": "String", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "account_type", + "type": "Enum", + "enums": [ + "current", + "deposit", + "card", + "closure", + "processor", + "seizure", + "release", + "remunerated", + "wealth", + "other", + "shadow" + ], + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": "0", + "inverseOf": null + }, + { + "field": "authorized_balance_cents", + "type": "Number", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": "0", + "inverseOf": null + }, + { + "field": "authorized_balance_updated_at", + "type": "Date", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "beneficiaries", + "type": [ + "Number" + ], + "enums": null, + "integration": null, + "reference": "Api__Beneficiary.id", + "relationship": "HasMany", + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "bank_account" + }, + { + "field": "id", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "legal_entity", + "type": "Number", + "enums": null, + "integration": null, + "reference": "Api__LegalEntity.id", + "relationship": "BelongsTo", + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + } + ], + "actions": [ + { + "name": "Report a bug to #cft-transfers", + "type": "global", + "base_url": null, + "endpoint": "/forest/actions/report-bug/cft_transfers", + "http_method": "POST", + "redirect": null, + "download": false, + "fields": [], + "hooks": { + "load": false, + "change": [] + }, + "description": null, + "submit_button_label": null + } + ], + "segments": [ + { + "name": "Treasury transfers" + } + ] + }, + { + "name": "Api__Beneficiary", + "why": "HasOne", + "fields": [ + { + "field": "account_number", + "type": "String", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "account_type", + "type": "Enum", + "enums": [ + "iban", + "aba", + "kantox", + "bank_code", + "bic_swift" + ], + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": "0", + "inverseOf": null + }, + { + "field": "activity_tag_code", + "type": "String", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "address_id", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "bank_account", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": "Api__BankAccount.id", + "relationship": "BelongsTo", + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "beneficiaries" + }, + { + "field": "id", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "organization", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": "Api__Organization.id", + "relationship": "HasOne", + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "beneficiaries" + }, + { + "field": "pending_transfers", + "type": [ + "Number" + ], + "enums": null, + "integration": null, + "reference": "Api__Transfer.id", + "relationship": "HasMany", + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "beneficiary" + } + ], + "actions": [ + { + "name": "Report a bug to #cft-transfers", + "type": "global", + "base_url": null, + "endpoint": "/forest/actions/report-bug/cft_transfers", + "http_method": "POST", + "redirect": null, + "download": false, + "fields": [], + "hooks": { + "load": false, + "change": [] + }, + "description": null, + "submit_button_label": null + } + ], + "segments": [] + }, + { + "name": "Api__Card", + "why": "a column type the operator table does not know", + "fields": [ + { + "field": "abusive_at", + "type": "Date", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "activated", + "type": "Boolean", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": "false", + "inverseOf": null + }, + { + "field": "activated_at", + "type": "Date", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "active_days", + "type": [ + "Number" + ], + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": "{0,1,2,3,4,5,6}", + "inverseOf": null + }, + { + "field": "address", + "type": "Number", + "enums": null, + "integration": null, + "reference": "CardLifecycle__CardAddress.id", + "relationship": "HasOne", + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "card" + }, + { + "field": "bank_account", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": "Api__BankAccount.id", + "relationship": "BelongsTo", + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "cards" + }, + { + "field": "card_shipments", + "type": [ + "Number" + ], + "enums": null, + "integration": null, + "reference": "CardLifecycle__CardShipment.id", + "relationship": "HasMany", + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "card" + }, + { + "field": "id", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + } + ], + "actions": [ + { + "name": "Report a bug to #cft-cards-issuing", + "type": "global", + "base_url": null, + "endpoint": "/forest/actions/report-bug/cft_cards_issuing", + "http_method": "POST", + "redirect": null, + "download": false, + "fields": [], + "hooks": { + "load": false, + "change": [] + }, + "description": null, + "submit_button_label": null + } + ], + "segments": [] + }, + { + "name": "Api__CardTransaction", + "why": "a column type the operator table does not know", + "fields": [ + { + "field": "amount_cents", + "type": "Number", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": "0", + "inverseOf": null + }, + { + "field": "amount_currency", + "type": "String", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": "EUR", + "inverseOf": null + }, + { + "field": "asked_amount_cents", + "type": "Number", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "atm_daily_usage_cents", + "type": "Number", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "card", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": "CardLifecycle__Card.id", + "relationship": "BelongsTo", + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "card_transactions" + }, + { + "field": "id", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "organization", + "type": "Number", + "enums": null, + "integration": null, + "reference": "Api__Organization.id", + "relationship": "HasOne", + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "risk_report_items", + "type": [ + "Number" + ], + "enums": null, + "integration": null, + "reference": "Fraud__RiskReportItem.id", + "relationship": "HasMany", + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + } + ], + "actions": [ + { + "name": "Report a bug to #cft-cards-issuing", + "type": "global", + "base_url": null, + "endpoint": "/forest/actions/report-bug/cft_cards_issuing", + "http_method": "POST", + "redirect": null, + "download": false, + "fields": [], + "hooks": { + "load": false, + "change": [] + }, + "description": null, + "submit_button_label": null + } + ], + "segments": [] + }, + { + "name": "Api__DocumentCollectionProcess", + "why": "carries actions and segments", + "fields": [ + { + "field": "created_at", + "type": "Date", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "document_collection_required_documents", + "type": [ + "Number" + ], + "enums": null, + "integration": null, + "reference": "Api__DocumentCollectionRequiredDocument.id", + "relationship": "HasMany", + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "document_collection_process" + }, + { + "field": "follow_up_email_sent_at", + "type": "Date", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "id", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "reason", + "type": "String", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "status", + "type": "Enum", + "enums": [ + "created", + "pending_upload", + "pending_review", + "waiting_doc", + "accepted", + "rejected", + "cancelled" + ], + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": "created", + "inverseOf": null + }, + { + "field": "subject", + "type": "Number", + "enums": null, + "integration": null, + "reference": "subject.id", + "relationship": "BelongsTo", + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "document_collection_process", + "polymorphicReferencedModels": [ + "DepositInterest::Remuneration", + "CompanyCreation::Document", + "Api::WalletToWallet", + "Api::SwiftIncome", + "Api::OrganizationChangeRequest", + "Api::RibaPayment", + "Api::PagopaPayment", + "Api::NrcPayment", + "Api::MembershipChangeRequest", + "Api::KycKybUpdateProcess", + "Api::Income", + "Api::F24Payment", + "Api::FinancingInstallment", + "Api::FinancingIncome", + "Api::DirectDebitCollection", + "Api::DirectDebitHold", + "Api::Check", + "Api::Card", + "Api::BillingTransfer", + "Api::AccountRemuneration", + "Api::Transfer", + "Api::DirectDebit" + ] + } + ], + "actions": [ + { + "name": "Report a bug to #cft-fraud-compliance-kycb-backend", + "type": "global", + "base_url": null, + "endpoint": "/forest/actions/report-bug/cft_kycb", + "http_method": "POST", + "redirect": null, + "download": false, + "fields": [], + "hooks": { + "load": false, + "change": [] + }, + "description": null, + "submit_button_label": null + } + ], + "segments": [ + { + "name": "FR - Pending Review Processes" + } + ] + }, + { + "name": "Api__KybProviderReport", + "why": "plain scalars only", + "fields": [ + { + "field": "created_at", + "type": "Date", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "external_id", + "type": "String", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "formatted", + "type": "String", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": false, + "isReadOnly": true, + "isRequired": false, + "isVirtual": true, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "id", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "provider_identifier", + "type": "String", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + } + ], + "actions": [ + { + "name": "Report a bug to #cft-ops-tooling", + "type": "global", + "base_url": null, + "endpoint": "/forest/actions/report-bug/cft_ops_tooling", + "http_method": "POST", + "redirect": null, + "download": false, + "fields": [], + "hooks": { + "load": false, + "change": [] + }, + "description": null, + "submit_button_label": null + } + ], + "segments": [] + }, + { + "name": "Api__KycCheckReport", + "why": "HasAndBelongsToMany", + "fields": [ + { + "field": "breakdown", + "type": "Json", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "created_at", + "type": "Date", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "deleted_at", + "type": "Date", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "documents", + "type": [ + "Number" + ], + "enums": null, + "integration": null, + "reference": "Api__Document.id", + "relationship": "HasAndBelongsToMany", + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "id", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "kyc_check", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": "Api__KycCheck.id", + "relationship": "BelongsTo", + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "kyc_check_reports" + }, + { + "field": "membership", + "type": "Number", + "enums": null, + "integration": null, + "reference": "Api__Membership.id", + "relationship": "HasOne", + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "properties", + "type": "Json", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + } + ], + "actions": [ + { + "name": "Report a bug to #cft-fraud-compliance-kycb-backend", + "type": "global", + "base_url": null, + "endpoint": "/forest/actions/report-bug/cft_kycb", + "http_method": "POST", + "redirect": null, + "download": false, + "fields": [], + "hooks": { + "load": false, + "change": [] + }, + "description": null, + "submit_button_label": null + } + ], + "segments": [] + }, + { + "name": "Api__KycKybUpdateValidationResults", + "why": "no id field at all: the fallback must invent one", + "fields": [ + { + "field": "organization_change_request", + "type": "String", + "enums": null, + "integration": null, + "reference": "Api__OrganizationChangeRequest.id", + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": false, + "isReadOnly": true, + "isRequired": false, + "isVirtual": true, + "defaultValue": null, + "inverseOf": null + } + ], + "actions": [ + { + "name": "Report a bug to #cft-fraud-compliance-kycb-backend", + "type": "global", + "base_url": null, + "endpoint": "/forest/actions/report-bug/cft_kycb", + "http_method": "POST", + "redirect": null, + "download": false, + "fields": [], + "hooks": { + "load": false, + "change": [] + }, + "description": null, + "submit_button_label": null + } + ], + "segments": [] + }, + { + "name": "Api__Membership", + "why": "carries actions and segments", + "fields": [ + { + "field": "activated_at", + "type": "Date", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "address", + "type": "Number", + "enums": null, + "integration": null, + "reference": "Api__Address.id", + "relationship": "HasOne", + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "subject" + }, + { + "field": "adverse_media_screening_clear", + "type": "Boolean", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": false, + "isReadOnly": true, + "isRequired": false, + "isVirtual": true, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "adverse_media_screenings", + "type": [ + "String" + ], + "enums": null, + "integration": null, + "reference": "Screening__AdverseMediaScreening.external_id", + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": false, + "isReadOnly": true, + "isRequired": false, + "isVirtual": true, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "age", + "type": "Number", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": false, + "isReadOnly": true, + "isRequired": false, + "isVirtual": true, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "cards", + "type": [ + "Number" + ], + "enums": null, + "integration": null, + "reference": "CardLifecycle__Card.id", + "relationship": "HasMany", + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "id", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "organization", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": "Api__Organization.id", + "relationship": "BelongsTo", + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "memberships" + } + ], + "actions": [ + { + "name": "Report a bug to #cft-spend-management-be", + "type": "global", + "base_url": null, + "endpoint": "/forest/actions/report-bug/cft_spend_management", + "http_method": "POST", + "redirect": null, + "download": false, + "fields": [], + "hooks": { + "load": false, + "change": [] + }, + "description": null, + "submit_button_label": null + } + ], + "segments": [ + { + "name": "FR-L1-Onfido-KYC To Do" + } + ] + }, + { + "name": "Api__Person", + "why": "no id field at all: the fallback must invent one", + "fields": [ + { + "field": "birth_city", + "type": "String", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "birth_country", + "type": "String", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "birth_zipcode", + "type": "String", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "birthdate", + "type": "Dateonly", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "person_address", + "type": "Number", + "enums": null, + "integration": null, + "reference": "Api__PersonAddress.id", + "relationship": "HasOne", + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "person" + } + ], + "actions": [ + { + "name": "Report a bug to #cft-fraud-compliance-kycb-backend", + "type": "global", + "base_url": null, + "endpoint": "/forest/actions/report-bug/cft_kycb", + "http_method": "POST", + "redirect": null, + "download": false, + "fields": [], + "hooks": { + "load": false, + "change": [] + }, + "description": null, + "submit_button_label": null + } + ], + "segments": [] + }, + { + "name": "Api__PersonFiscalCode", + "why": "plain scalars only", + "fields": [ + { + "field": "country", + "type": "String", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "created_at", + "type": "Date", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "fiscal_code", + "type": "String", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "id", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "person_id", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + } + ], + "actions": [ + { + "name": "Report a bug to #cft-fraud-compliance-kycb-backend", + "type": "global", + "base_url": null, + "endpoint": "/forest/actions/report-bug/cft_kycb", + "http_method": "POST", + "redirect": null, + "download": false, + "fields": [], + "hooks": { + "load": false, + "change": [] + }, + "description": null, + "submit_button_label": null + } + ], + "segments": [] + }, + { + "name": "Biller__Invoice", + "why": "HasAndBelongsToMany", + "fields": [ + { + "field": "amount_cents", + "type": "Number", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": "0", + "inverseOf": null + }, + { + "field": "created_at", + "type": "Date", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "credit_notes", + "type": [ + "Number" + ], + "enums": null, + "integration": null, + "reference": "Biller__CreditNote.id", + "relationship": "HasMany", + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "invoice" + }, + { + "field": "currency", + "type": "String", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": "EUR", + "inverseOf": null + }, + { + "field": "deleted_at", + "type": "Date", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "id", + "type": "Uuid", + "enums": null, + "integration": null, + "reference": null, + "widget": null, + "validations": [], + "isFilterable": true, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": null + }, + { + "field": "subscriptions", + "type": [ + "Number" + ], + "enums": null, + "integration": null, + "reference": "Biller__Subscription.id", + "relationship": "HasAndBelongsToMany", + "widget": null, + "validations": [], + "isFilterable": false, + "isSortable": true, + "isReadOnly": false, + "isRequired": false, + "isVirtual": false, + "defaultValue": null, + "inverseOf": "invoices" + } + ], + "actions": [ + { + "name": "Report a bug to #cft-pricing", + "type": "global", + "base_url": null, + "endpoint": "/forest/actions/report-bug/cft_pricing", + "http_method": "POST", + "redirect": null, + "download": false, + "fields": [], + "hooks": { + "load": false, + "change": [] + }, + "description": null, + "submit_button_label": null + } + ], + "segments": [] + } + ] +} \ No newline at end of file diff --git a/packages/agent-bff/test/read-model/forest-schema-client.test.ts b/packages/agent-bff/test/read-model/forest-schema-client.test.ts index 00fa10701d..ed2213a5a9 100644 --- a/packages/agent-bff/test/read-model/forest-schema-client.test.ts +++ b/packages/agent-bff/test/read-model/forest-schema-client.test.ts @@ -5,11 +5,11 @@ import ForestSchemaClient from '../../src/read-model/forest-schema-client'; jest.mock('@forestadmin/forestadmin-client'); describe('ForestSchemaClient', () => { - const getSchema = jest.fn(); + const getSchemaWithMeta = jest.fn(); beforeEach(() => { jest.clearAllMocks(); - (SchemaService as unknown as jest.Mock).mockImplementation(() => ({ getSchema })); + (SchemaService as unknown as jest.Mock).mockImplementation(() => ({ getSchemaWithMeta })); }); it('should construct a SchemaService with a ForestHttpApi and the server options', () => { @@ -25,9 +25,12 @@ describe('ForestSchemaClient', () => { }); }); - it('should delegate fetchSchema to SchemaService.getSchema', async () => { - const collections = [{ name: 'users', fields: [], actions: [] }]; - getSchema.mockResolvedValue(collections); + it('should delegate fetchSchema to SchemaService.getSchemaWithMeta, keeping the liana', async () => { + const published = { + collections: [{ name: 'users', fields: [], actions: [] }], + meta: { liana: 'forest-rails', liana_version: '9.21.0' }, + }; + getSchemaWithMeta.mockResolvedValue(published); const client = new ForestSchemaClient({ forestServerUrl: 'https://api.test', envSecret: 'secret', @@ -35,7 +38,7 @@ describe('ForestSchemaClient', () => { const result = await client.fetchSchema(); - expect(getSchema).toHaveBeenCalledTimes(1); - expect(result).toBe(collections); + expect(getSchemaWithMeta).toHaveBeenCalledTimes(1); + expect(result).toBe(published); }); }); diff --git a/packages/agent-bff/test/read-model/legacy-rails-schema.test.ts b/packages/agent-bff/test/read-model/legacy-rails-schema.test.ts new file mode 100644 index 0000000000..8e9472c71b --- /dev/null +++ b/packages/agent-bff/test/read-model/legacy-rails-schema.test.ts @@ -0,0 +1,183 @@ +import type { ForestSchemaCollection } from '@forestadmin/forestadmin-client'; + +import fs from 'fs'; +import path from 'path'; + +import unpackPrimaryKey from '../../src/data/pack-id'; +import ReadModel from '../../src/read-model/read-model'; +import synthesizeCapabilities from '../../src/read-model/synthesize-capabilities'; + +/** + * A schema a real `forest_liana 9.15.8` backoffice published: 269 collections, 6274 fields, and not + * one `isPrimaryKey` — that flag only appears in 9.17.6. Every collection there would answer + * `500 mapping_error` on a plain list without the primary-key fallback, so this pins the fallback + * against shapes a production Rails app actually contains rather than against invented ones. + * + * Whose schema it was does not matter and is not the point: that customer has since upgraded past + * 9.17.6. What the fixture stands for is the tail that has not — of the 125 `forest-rails` + * production and development environments active in the last 120 days, the twenty oldest by version + * run between 2.14.6 and 8.3.2, all of them below the line. + * + * The committed fixture is a sample: one or two collections per shape, field lists trimmed. Point + * `LEGACY_RAILS_SCHEMA` at a full schema file to run the same assertions over all of it. + */ +function loadSchema(): { collections: (ForestSchemaCollection & { why?: string })[] } { + const override = process.env.LEGACY_RAILS_SCHEMA; + const file = override ?? path.join(__dirname, 'fixtures/legacy-rails-schema.json'); + + return JSON.parse(fs.readFileSync(file, 'utf8')); +} + +describe('the schema a pre-9.17.6 forest_liana publishes', () => { + const { collections } = loadSchema(); + const readModel = new ReadModel(collections); + const noopLogger = () => undefined; + + it('should carry no isPrimaryKey at all, which is what makes the fallback load-bearing', () => { + const declared = collections.filter(collection => + (collection.fields ?? []).some(field => field.isPrimaryKey), + ); + + expect(declared).toEqual([]); + }); + + it('should still give every collection exactly one primary key', () => { + const without = readModel + .getAllowedCollections() + .filter(name => readModel.getPrimaryKeys(name).length !== 1); + + expect(without).toEqual([]); + }); + + it('should let a record id round-trip for every collection, which is the 500 this prevents', () => { + const failures = readModel.getAllowedCollections().filter(name => { + try { + unpackPrimaryKey('42', readModel.getPrimaryKeys(name)); + + return false; + } catch { + return true; + } + }); + + expect(failures).toEqual([]); + }); + + it('should take the declared type of the id field where the schema has one', () => { + const named = collections.find(collection => + (collection.fields ?? []).some(field => field.field === 'id'), + ); + const declared = (named?.fields ?? []).find(field => field.field === 'id'); + + expect(readModel.getPrimaryKeys(named!.name)).toEqual([ + { name: 'id', type: declared?.type, derived: true }, + ]); + }); + + it('should invent a string key for a collection that declares no id field', () => { + const anonymous = collections.find( + collection => !(collection.fields ?? []).some(field => field.field === 'id'), + ); + + expect(anonymous).toBeDefined(); + expect(readModel.getPrimaryKeys(anonymous!.name)).toEqual([ + { name: 'id', type: 'String', derived: true }, + ]); + }); + + it('should synthesize capabilities for every collection without throwing', () => { + const results = collections.map(collection => synthesizeCapabilities(collection, noopLogger)); + + expect(results).toHaveLength(collections.length); + expect(results.every(result => Array.isArray(result.fields))).toBe(true); + }); + + it('should publish a to-one relation without operators and omit a to-many one, as on v2', () => { + const withRelations = collections.find(collection => + (collection.fields ?? []).some(field => field.relationship === 'BelongsTo'), + ); + const capabilities = synthesizeCapabilities(withRelations!, noopLogger); + const toOne = (withRelations?.fields ?? []).find( + field => field.relationship === 'BelongsTo', + )?.field; + const toMany = (withRelations?.fields ?? []) + .filter(field => field.relationship === 'HasMany') + .map(field => field.field); + const published = capabilities.fields.map(field => field.name); + + expect(capabilities.fields.find(field => field.name === toOne)).toEqual({ + name: toOne, + type: 'ManyToOne', + }); + expect(toMany.filter(name => published.includes(name))).toEqual([]); + }); + + // The fixture is what the SaaS serves, not the raw apimap the gem pushes: the keys arrive + // camelCased. Getting that wrong once made every assertion below vacuous, so both tests start by + // proving the schema actually carries denials to find. + it('should publish no operator for a field the liana denies filtering', () => { + const denied = collections.flatMap(collection => + (collection.fields ?? []) + .filter(field => field.isFilterable === false) + .map(field => ({ collection, name: field.field })), + ); + + expect(denied.length).toBeGreaterThan(0); + + const leaked = denied.filter(({ collection, name }) => { + const published = synthesizeCapabilities(collection, noopLogger).fields.find( + entry => entry.name === name, + ); + + return (published?.operators ?? []).length > 0; + }); + + expect(leaked).toEqual([]); + }); + + it('should publish a field the liana denies sorting as not sortable', () => { + const denied = collections.flatMap(collection => + (collection.fields ?? []) + .filter(field => field.isSortable === false) + .map(field => ({ collection, name: field.field })), + ); + + expect(denied.length).toBeGreaterThan(0); + + const sortable = denied.filter(({ collection, name }) => { + const published = synthesizeCapabilities(collection, noopLogger).fields.find( + entry => entry.name === name, + ); + + return published !== undefined && published.sortable !== false; + }); + + expect(sortable).toEqual([]); + }); + + // Every sort denial in the fixture sits on a scalar column, so a relation that denies sorting has + // to be built here. Published without `sortable: false`, the BFF would accept the sort and forward + // it to a liana that rejects it. + it('should carry a sort denial onto a to-one relation', () => { + const collection = { + name: 'WithDeniedRelationSort', + fields: [ + { field: 'id', type: 'Number', isFilterable: true, isSortable: true }, + { + field: 'author', + type: 'Number', + relationship: 'BelongsTo', + reference: 'users.id', + isFilterable: true, + isSortable: false, + }, + ], + } as unknown as ForestSchemaCollection; + + const published = synthesizeCapabilities(collection, noopLogger).fields.find( + entry => entry.name === 'author', + ); + + expect(published).toEqual({ name: 'author', type: 'ManyToOne', sortable: false }); + }); +}); diff --git a/packages/agent-bff/test/read-model/read-model-store.test.ts b/packages/agent-bff/test/read-model/read-model-store.test.ts index 4b3da75769..1c26e9b6ae 100644 --- a/packages/agent-bff/test/read-model/read-model-store.test.ts +++ b/packages/agent-bff/test/read-model/read-model-store.test.ts @@ -1,6 +1,6 @@ import type { SchemaFetcher } from '../../src/read-model/forest-schema-client'; -import { makeMetrics, makeSchema } from './fixtures'; +import { makeMetrics, makeSchema, published } from './fixtures'; import CapabilitiesCache from '../../src/read-model/capabilities-cache'; import ReadModelStore from '../../src/read-model/read-model-store'; import SchemaCache, { ONE_DAY_MS } from '../../src/read-model/schema-cache'; @@ -27,7 +27,7 @@ describe('ReadModelStore', () => { describe('getSchemaSnapshot', () => { it('should return a read-model derived from the very collections it returns', async () => { - const store = build(jest.fn().mockResolvedValue(makeSchema('users'))); + const store = build(jest.fn().mockResolvedValue(published(makeSchema('users')))); const { collections, readModel, revision } = await store.getSchemaSnapshot(); @@ -42,7 +42,7 @@ describe('ReadModelStore', () => { const fetchSchema = jest.fn().mockImplementation(async () => { generation += 1; - return makeSchema(`generation-${generation}`); + return published(makeSchema(`generation-${generation}`)); }); const alwaysExpiredClock = () => { @@ -62,7 +62,7 @@ describe('ReadModelStore', () => { }); it('should read the cache revision once, so the triple cannot straddle two generations', async () => { - const store = build(jest.fn().mockResolvedValue(makeSchema('users'))); + const store = build(jest.fn().mockResolvedValue(published(makeSchema('users')))); const cache = Reflect.get(store, 'schemaCache') as SchemaCache; let reads = 0; let bumped = 0; @@ -83,8 +83,8 @@ describe('ReadModelStore', () => { it('should not label one generation of collections with another generation revision', async () => { const fetchSchema = jest .fn() - .mockResolvedValueOnce(makeSchema('first')) - .mockResolvedValueOnce(makeSchema('second')); + .mockResolvedValueOnce(published(makeSchema('first'))) + .mockResolvedValueOnce(published(makeSchema('second'))); const store = build(fetchSchema); const first = await store.getSchemaSnapshot(); @@ -101,7 +101,7 @@ describe('ReadModelStore', () => { describe('getReadModel', () => { it('should build the read-model from the fetched schema', async () => { - const store = build(jest.fn().mockResolvedValue(makeSchema('users'))); + const store = build(jest.fn().mockResolvedValue(published(makeSchema('users')))); const model = await store.getReadModel(); @@ -109,7 +109,7 @@ describe('ReadModelStore', () => { }); it('should reuse the same read-model instance on a cache hit', async () => { - const store = build(jest.fn().mockResolvedValue(makeSchema('users'))); + const store = build(jest.fn().mockResolvedValue(published(makeSchema('users')))); const a = await store.getReadModel(); const b = await store.getReadModel(); @@ -120,8 +120,8 @@ describe('ReadModelStore', () => { it('should rebuild the read-model after a schema refresh', async () => { const fetchSchema = jest .fn() - .mockResolvedValueOnce(makeSchema('users')) - .mockResolvedValueOnce(makeSchema('orders')); + .mockResolvedValueOnce(published(makeSchema('users'))) + .mockResolvedValueOnce(published(makeSchema('orders'))); const store = build(fetchSchema); const before = await store.getReadModel(); @@ -136,7 +136,7 @@ describe('ReadModelStore', () => { describe('capabilities coupling', () => { it('should fetch capabilities and cache them', async () => { - const store = build(jest.fn().mockResolvedValue(makeSchema('users'))); + const store = build(jest.fn().mockResolvedValue(published(makeSchema('users')))); const capsFetcher = jest.fn().mockResolvedValue({ fields: [] }); await store.getCapabilities('users', capsFetcher); @@ -148,8 +148,8 @@ describe('ReadModelStore', () => { it('should invalidate capabilities when the schema refreshes', async () => { const fetchSchema = jest .fn() - .mockResolvedValueOnce(makeSchema('users')) - .mockResolvedValueOnce(makeSchema('users')); + .mockResolvedValueOnce(published(makeSchema('users'))) + .mockResolvedValueOnce(published(makeSchema('users'))); const store = build(fetchSchema); const capsFetcher = jest.fn().mockResolvedValue({ fields: [] }); @@ -171,13 +171,13 @@ describe('ReadModelStore', () => { }); const fetchSchema = jest .fn() - .mockResolvedValueOnce(makeSchema('users')) + .mockResolvedValueOnce(published(makeSchema('users'))) .mockImplementationOnce(async () => { await blocked; - return makeSchema('orders'); + return published(makeSchema('orders')); }) - .mockResolvedValue(makeSchema('orders')); + .mockResolvedValue(published(makeSchema('orders'))); const store = build(fetchSchema); await store.getReadModel(); @@ -208,9 +208,9 @@ describe('ReadModelStore', () => { it('should pair capabilities and read-model from one generation when a refresh lands mid-fetch', async () => { const fetchSchema = jest .fn() - .mockResolvedValueOnce(makeSchema('users')) - .mockResolvedValueOnce(makeSchema('orders')) - .mockResolvedValue(makeSchema('orders')); + .mockResolvedValueOnce(published(makeSchema('users'))) + .mockResolvedValueOnce(published(makeSchema('orders'))) + .mockResolvedValue(published(makeSchema('orders'))); const store = build(fetchSchema); const stale = await store.getReadModel(); @@ -239,7 +239,7 @@ describe('ReadModelStore', () => { it('should not rebuild the read-model or clear capabilities when a refresh fails', async () => { const fetchSchema = jest .fn() - .mockResolvedValueOnce(makeSchema('users')) + .mockResolvedValueOnce(published(makeSchema('users'))) .mockRejectedValueOnce(new Error('boom')); // Capabilities TTL kept longer than the schema TTL so this asserts the *clear* (not a // capabilities TTL expiry) does not happen on a warm schema-refresh failure. @@ -266,7 +266,7 @@ describe('ReadModelStore', () => { describe('ageSeconds', () => { it('should reflect the schema cache age of the last good schema', async () => { - const store = build(jest.fn().mockResolvedValue(makeSchema('users'))); + const store = build(jest.fn().mockResolvedValue(published(makeSchema('users')))); await store.getReadModel(); clock += 7_000; @@ -277,7 +277,7 @@ describe('ReadModelStore', () => { describe('invalidate', () => { it('should re-read the schema on the next snapshot', async () => { - const fetchSchema = jest.fn().mockResolvedValue(makeSchema('users')); + const fetchSchema = jest.fn().mockResolvedValue(published(makeSchema('users'))); const store = build(fetchSchema); await store.getSchemaSnapshot(); @@ -288,7 +288,7 @@ describe('ReadModelStore', () => { }); it('should drop the capabilities with it, since they belong to the schema generation', async () => { - const fetchSchema = jest.fn().mockResolvedValue(makeSchema('users')); + const fetchSchema = jest.fn().mockResolvedValue(published(makeSchema('users'))); const store = build(fetchSchema); const capabilities = jest.fn().mockResolvedValue({ fields: [] }); await store.getCapabilities('users', capabilities); @@ -302,7 +302,7 @@ describe('ReadModelStore', () => { it('should drop the capabilities even when the revision does not move', async () => { const fetchSchema = jest .fn() - .mockResolvedValueOnce(makeSchema('users')) + .mockResolvedValueOnce(published(makeSchema('users'))) .mockRejectedValue(new Error('boom')); const store = build(fetchSchema); const capabilities = jest.fn().mockResolvedValue({ fields: [] }); diff --git a/packages/agent-bff/test/read-model/read-model.test.ts b/packages/agent-bff/test/read-model/read-model.test.ts index d610ce8cfc..55fcb50714 100644 --- a/packages/agent-bff/test/read-model/read-model.test.ts +++ b/packages/agent-bff/test/read-model/read-model.test.ts @@ -313,5 +313,47 @@ describe('ReadModel', () => { expect(model.getPrimaryKeys('ghost')).toEqual([]); }); + + // A forest_liana older than 9.17.6 publishes no isPrimaryKey anywhere, which used to leave the + // collection keyless and make every list answer 500 mapping_error. + describe('when the schema declares no key', () => { + it('should fall back to the id field with the type the schema gives it', () => { + const model = new ReadModel([ + collection('users', [ + { ...column('id'), type: 'Number', isPrimaryKey: false }, + column('email'), + ]), + ]); + + expect(model.getPrimaryKeys('users')).toEqual([ + { name: 'id', type: 'Number', derived: true }, + ]); + }); + + it('should fall back to a string id when the schema declares no id field either', () => { + const model = new ReadModel([collection('audits', [column('label')])]); + + expect(model.getPrimaryKeys('audits')).toEqual([ + { name: 'id', type: 'String', derived: true }, + ]); + }); + + it('should flag the key as derived, so unpacking does not split a packed composite id', () => { + const model = new ReadModel([collection('audits', [column('label')])]); + + expect(model.getPrimaryKeys('audits')[0].derived).toBe(true); + }); + + it('should leave a declared key alone, so a v2 agent is untouched', () => { + const model = new ReadModel([ + collection('users', [ + { ...column('reference'), type: 'String', isPrimaryKey: true }, + { ...column('id'), type: 'Number', isPrimaryKey: false }, + ]), + ]); + + expect(model.getPrimaryKeys('users')).toEqual([{ name: 'reference', type: 'String' }]); + }); + }); }); }); diff --git a/packages/agent-bff/test/read-model/schema-cache.test.ts b/packages/agent-bff/test/read-model/schema-cache.test.ts index ca8b900217..58779f28e9 100644 --- a/packages/agent-bff/test/read-model/schema-cache.test.ts +++ b/packages/agent-bff/test/read-model/schema-cache.test.ts @@ -1,9 +1,9 @@ import type { Logger } from '../../src/ports/logger-port'; import type { Metrics } from '../../src/ports/metrics-port'; import type { SchemaFetcher } from '../../src/read-model/forest-schema-client'; -import type { ForestSchemaCollection } from '@forestadmin/forestadmin-client'; +import type { ForestSchemaWithMeta } from '@forestadmin/forestadmin-client'; -import { makeMetrics, makeSchema } from './fixtures'; +import { makeMetrics, makeSchema, published } from './fixtures'; import SchemaUnavailableError from '../../src/read-model/errors'; import SchemaCache, { ONE_DAY_MS, @@ -14,7 +14,7 @@ import SchemaCache, { } from '../../src/read-model/schema-cache'; describe('SchemaCache', () => { - let fetcher: { fetchSchema: jest.Mock, []> }; + let fetcher: { fetchSchema: jest.Mock, []> }; let metrics: jest.Mocked; let clock: number; const now = () => clock; @@ -32,7 +32,7 @@ describe('SchemaCache', () => { describe('cold cache', () => { it('should fetch on first read and return the collections', async () => { const schema = makeSchema('users'); - fetcher.fetchSchema.mockResolvedValue(schema); + fetcher.fetchSchema.mockResolvedValue(published(schema)); const result = await build().get(); @@ -50,7 +50,9 @@ describe('SchemaCache', () => { it('should re-attempt the fetch on the next read after a cold failure (no poisoning)', async () => { const schema = makeSchema('users'); - fetcher.fetchSchema.mockRejectedValueOnce(new Error('boom')).mockResolvedValueOnce(schema); + fetcher.fetchSchema + .mockRejectedValueOnce(new Error('boom')) + .mockResolvedValueOnce(published(schema)); const cache = build(); await expect(cache.get()).rejects.toBeInstanceOf(SchemaUnavailableError); @@ -73,7 +75,7 @@ describe('SchemaCache', () => { describe('warm cache within TTL', () => { it('should serve from cache without re-fetching before 24h', async () => { const schema = makeSchema('users'); - fetcher.fetchSchema.mockResolvedValue(schema); + fetcher.fetchSchema.mockResolvedValue(published(schema)); const cache = build(); await cache.get(); @@ -87,7 +89,9 @@ describe('SchemaCache', () => { it('should re-fetch after 24h', async () => { const first = makeSchema('users'); const second = makeSchema('users-v2'); - fetcher.fetchSchema.mockResolvedValueOnce(first).mockResolvedValueOnce(second); + fetcher.fetchSchema + .mockResolvedValueOnce(published(first)) + .mockResolvedValueOnce(published(second)); const cache = build(); await cache.get(); @@ -100,7 +104,7 @@ describe('SchemaCache', () => { it('should return the same array reference on a cache hit', async () => { const schema = makeSchema('users'); - fetcher.fetchSchema.mockResolvedValue(schema); + fetcher.fetchSchema.mockResolvedValue(published(schema)); const cache = build(); const a = await cache.get(); @@ -113,7 +117,9 @@ describe('SchemaCache', () => { describe('warm cache refresh failure', () => { it('should keep serving the last good schema and emit the error counter', async () => { const good = makeSchema('users'); - fetcher.fetchSchema.mockResolvedValueOnce(good).mockRejectedValueOnce(new Error('boom')); + fetcher.fetchSchema + .mockResolvedValueOnce(published(good)) + .mockRejectedValueOnce(new Error('boom')); const cache = build(); await cache.get(); @@ -127,7 +133,7 @@ describe('SchemaCache', () => { it('should log the cause and that the stale schema was served', async () => { const logger = jest.fn(); fetcher.fetchSchema - .mockResolvedValueOnce(makeSchema('users')) + .mockResolvedValueOnce(published(makeSchema('users'))) .mockRejectedValueOnce(new Error('boom')); const cache = build(logger); @@ -145,9 +151,9 @@ describe('SchemaCache', () => { const good = makeSchema('users'); const fresh = makeSchema('users-v2'); fetcher.fetchSchema - .mockResolvedValueOnce(good) + .mockResolvedValueOnce(published(good)) .mockRejectedValueOnce(new Error('boom')) - .mockResolvedValueOnce(fresh); + .mockResolvedValueOnce(published(fresh)); const cache = build(); await cache.get(); @@ -163,9 +169,9 @@ describe('SchemaCache', () => { describe('concurrent reads', () => { it('should dedupe an in-flight fetch so concurrent cold reads fetch once', async () => { const schema = makeSchema('users'); - let resolveFetch!: (value: ForestSchemaCollection[]) => void; + let resolveFetch!: (value: ForestSchemaWithMeta) => void; fetcher.fetchSchema.mockReturnValue( - new Promise(resolve => { + new Promise(resolve => { resolveFetch = resolve; }), ); @@ -173,7 +179,7 @@ describe('SchemaCache', () => { const a = cache.get(); const b = cache.get(); - resolveFetch(schema); + resolveFetch(published(schema)); expect(await a).toBe(schema); expect(await b).toBe(schema); @@ -181,10 +187,40 @@ describe('SchemaCache', () => { }); }); + describe('the publishing liana', () => { + it('should expose the meta of the schema currently served', async () => { + const meta = { liana: 'forest-rails', liana_version: '9.21.0' }; + fetcher.fetchSchema.mockResolvedValue(published(makeSchema('users'), meta)); + const cache = build(); + + await cache.get(); + + expect(cache.meta).toEqual(meta); + }); + + it('should report an empty meta before any schema has been fetched', () => { + expect(build().meta).toEqual({}); + }); + + it('should keep the last good meta while a refresh keeps failing, like the collections', async () => { + const meta = { liana: 'forest-express-sequelize', liana_version: '9.6.10' }; + fetcher.fetchSchema + .mockResolvedValueOnce(published(makeSchema('users'), meta)) + .mockRejectedValue(new Error('boom')); + const cache = build(); + + await cache.get(); + clock += ONE_DAY_MS + 1; + await cache.get(); + + expect(cache.meta).toEqual(meta); + }); + }); + describe('age gauge', () => { it('should emit schema_cache_age_seconds reflecting the last good age on read', async () => { const schema = makeSchema('users'); - fetcher.fetchSchema.mockResolvedValue(schema); + fetcher.fetchSchema.mockResolvedValue(published(schema)); const cache = build(); await cache.get(); @@ -198,7 +234,7 @@ describe('SchemaCache', () => { describe('empty schema', () => { it('should treat an empty schema as a failed fetch on a cold cache', async () => { - fetcher.fetchSchema.mockResolvedValue([]); + fetcher.fetchSchema.mockResolvedValue(published([])); const cache = build(); await expect(cache.get()).rejects.toBeInstanceOf(SchemaUnavailableError); @@ -208,7 +244,7 @@ describe('SchemaCache', () => { it('should log the empty schema as the cause, since the counter cannot tell it from an outage', async () => { const logger = jest.fn(); - fetcher.fetchSchema.mockResolvedValue([]); + fetcher.fetchSchema.mockResolvedValue(published([])); await expect(build(logger).get()).rejects.toBeInstanceOf(SchemaUnavailableError); @@ -220,7 +256,9 @@ describe('SchemaCache', () => { it('should keep serving the last good schema when a refresh returns empty', async () => { const good = makeSchema('users'); - fetcher.fetchSchema.mockResolvedValueOnce(good).mockResolvedValueOnce([]); + fetcher.fetchSchema + .mockResolvedValueOnce(published(good)) + .mockResolvedValueOnce(published([])); const cache = build(); await cache.get(); @@ -239,8 +277,8 @@ describe('SchemaCache', () => { it('should increment on each successful refresh', async () => { fetcher.fetchSchema - .mockResolvedValueOnce(makeSchema('users')) - .mockResolvedValueOnce(makeSchema('users-v2')); + .mockResolvedValueOnce(published(makeSchema('users'))) + .mockResolvedValueOnce(published(makeSchema('users-v2'))); const cache = build(); await cache.get(); @@ -252,7 +290,7 @@ describe('SchemaCache', () => { }); it('should not increment on a cache hit', async () => { - fetcher.fetchSchema.mockResolvedValue(makeSchema('users')); + fetcher.fetchSchema.mockResolvedValue(published(makeSchema('users'))); const cache = build(); await cache.get(); @@ -263,7 +301,7 @@ describe('SchemaCache', () => { it('should not increment on a warm refresh failure', async () => { fetcher.fetchSchema - .mockResolvedValueOnce(makeSchema('users')) + .mockResolvedValueOnce(published(makeSchema('users'))) .mockRejectedValueOnce(new Error('boom')); const cache = build(); @@ -278,7 +316,7 @@ describe('SchemaCache', () => { describe('clear', () => { it('should re-read the schema on the next get', async () => { const cache = build(); - fetcher.fetchSchema.mockResolvedValue(makeSchema('users')); + fetcher.fetchSchema.mockResolvedValue(published(makeSchema('users'))); await cache.get(); cache.clear(); @@ -289,7 +327,7 @@ describe('SchemaCache', () => { it('should keep re-reading during the revalidation window, since the SaaS may still be catching up', async () => { const cache = build(); - fetcher.fetchSchema.mockResolvedValue(makeSchema('users')); + fetcher.fetchSchema.mockResolvedValue(published(makeSchema('users'))); await cache.get(); cache.clear(); await cache.get(); @@ -302,7 +340,7 @@ describe('SchemaCache', () => { it('should go back to the long TTL once the window is over', async () => { const cache = build(); - fetcher.fetchSchema.mockResolvedValue(makeSchema('users')); + fetcher.fetchSchema.mockResolvedValue(published(makeSchema('users'))); await cache.get(); cache.clear(); await cache.get(); @@ -319,7 +357,7 @@ describe('SchemaCache', () => { it('should not let a fetch started before the clear repopulate the cache', async () => { const cache = build(); const stale = makeSchema('stale'); - let releaseStale: (collections: ForestSchemaCollection[]) => void = () => undefined; + let releaseStale: (schema: ForestSchemaWithMeta) => void = () => undefined; fetcher.fetchSchema.mockReturnValueOnce( new Promise(resolve => { releaseStale = resolve; @@ -328,11 +366,11 @@ describe('SchemaCache', () => { const pending = cache.get(); cache.clear(); - releaseStale(stale); + releaseStale(published(stale)); await expect(pending).resolves.toEqual(stale); - fetcher.fetchSchema.mockResolvedValue(makeSchema('fresh')); + fetcher.fetchSchema.mockResolvedValue(published(makeSchema('fresh'))); const result = await cache.get(); expect(result).toEqual(makeSchema('fresh')); @@ -342,7 +380,7 @@ describe('SchemaCache', () => { it('should start its own fetch for a read that lands after the clear, not join the invalidated one', async () => { const cache = build(); const stale = makeSchema('stale'); - let releaseStale: (collections: ForestSchemaCollection[]) => void = () => undefined; + let releaseStale: (schema: ForestSchemaWithMeta) => void = () => undefined; fetcher.fetchSchema.mockReturnValueOnce( new Promise(resolve => { releaseStale = resolve; @@ -351,9 +389,9 @@ describe('SchemaCache', () => { const beforeClear = cache.get(); cache.clear(); - fetcher.fetchSchema.mockResolvedValue(makeSchema('fresh')); + fetcher.fetchSchema.mockResolvedValue(published(makeSchema('fresh'))); const afterClear = cache.get(); - releaseStale(stale); + releaseStale(published(stale)); await expect(beforeClear).resolves.toEqual(stale); await expect(afterClear).resolves.toEqual(makeSchema('fresh')); @@ -362,7 +400,7 @@ describe('SchemaCache', () => { it('should not promote a read taken inside the window to the long TTL once the window closes', async () => { const cache = build(); - fetcher.fetchSchema.mockResolvedValue(makeSchema('users')); + fetcher.fetchSchema.mockResolvedValue(published(makeSchema('users'))); await cache.get(); cache.clear(); @@ -376,7 +414,9 @@ describe('SchemaCache', () => { it('should keep the last good schema as a fallback when the refresh after a clear fails', async () => { const good = makeSchema('users'); - fetcher.fetchSchema.mockResolvedValueOnce(good).mockRejectedValue(new Error('boom')); + fetcher.fetchSchema + .mockResolvedValueOnce(published(good)) + .mockRejectedValue(new Error('boom')); const cache = build(); await cache.get(); @@ -391,9 +431,9 @@ describe('SchemaCache', () => { const good = makeSchema('users'); const fresh = makeSchema('users-v2'); fetcher.fetchSchema - .mockResolvedValueOnce(good) + .mockResolvedValueOnce(published(good)) .mockRejectedValueOnce(new Error('boom')) - .mockResolvedValueOnce(fresh); + .mockResolvedValueOnce(published(fresh)); const cache = build(); await cache.get(); @@ -407,7 +447,7 @@ describe('SchemaCache', () => { it('should not bump the revision for a fetch the clear invalidated', async () => { const cache = build(); - let releaseStale: (collections: ForestSchemaCollection[]) => void = () => undefined; + let releaseStale: (schema: ForestSchemaWithMeta) => void = () => undefined; fetcher.fetchSchema.mockReturnValueOnce( new Promise(resolve => { releaseStale = resolve; @@ -416,10 +456,60 @@ describe('SchemaCache', () => { const pending = cache.get(); cache.clear(); - releaseStale(makeSchema('stale')); + releaseStale(published(makeSchema('stale'))); await pending; expect(cache.revision).toBe(0); }); }); + + describe('when a clear lands while a refresh is in flight', () => { + it("should hand the detached refresh its own meta, not the newer entry's", async () => { + let releaseFirst: (() => void) | undefined; + const first = new Promise(resolve => { + releaseFirst = resolve; + }); + + let call = 0; + const generations = { + fetchSchema: async () => { + call += 1; + + if (call === 1) { + await first; + + return { + collections: [{ name: 'FromGenerationOne', fields: [] }], + meta: { liana: 'forest-rails' }, + }; + } + + return { + collections: [{ name: 'FromGenerationTwo', fields: [] }], + meta: { liana: 'forest-express-sequelize' }, + }; + }, + } as never; + + const cache = new SchemaCache({ fetcher: generations, metrics: makeMetrics() }); + + const detached = cache.getPayload(); + + cache.clear(); + + const rewritten = await cache.getPayload(); + releaseFirst?.(); + + const resolved = await detached; + + // The pair has to be internally consistent: reading the collections from the detached refresh + // and the meta from the entry a newer refresh wrote would name a liana that never published + // those collections, and the legacy synthesis branches on exactly that name. + expect(resolved.collections.map(collection => collection.name)).toEqual([ + 'FromGenerationOne', + ]); + expect(resolved.meta).toEqual({ liana: 'forest-rails' }); + expect(rewritten.meta).toEqual({ liana: 'forest-express-sequelize' }); + }); + }); }); diff --git a/packages/agent-bff/test/read-model/synthesize-capabilities.test.ts b/packages/agent-bff/test/read-model/synthesize-capabilities.test.ts new file mode 100644 index 0000000000..b30ef21f5e --- /dev/null +++ b/packages/agent-bff/test/read-model/synthesize-capabilities.test.ts @@ -0,0 +1,144 @@ +import type { Logger } from '../../src/ports/logger-port'; +import type { ForestSchemaCollection } from '@forestadmin/forestadmin-client'; + +import synthesizeCapabilities, { + LEGACY_LIANA_OPERATORS, +} from '../../src/read-model/synthesize-capabilities'; +import { normalizeOperator } from '../../src/validation/operator-normalizer'; + +// Copied from the apimap a real forest-express-sequelize 9.3.8 agent pushed, trimmed to the shapes +// that matter: a plain scalar, a computed field the liana marks unusable, a date, a to-one relation +// and a to-many one. +function v1Apimap(): ForestSchemaCollection { + return { + name: 'User', + fields: [ + { field: 'id', type: 'Number', isPrimaryKey: true, isFilterable: true, isSortable: true }, + { field: 'name', type: 'String', isFilterable: true, isSortable: true }, + { field: 'birthDate', type: 'Dateonly', isFilterable: true, isSortable: true }, + { field: 'fullName', type: 'String', isFilterable: false, isSortable: false }, + { field: 'team', type: 'Number', relationship: 'BelongsTo', reference: 'teams.id' }, + { field: 'articles', type: ['Number'], relationship: 'HasMany', reference: 'Article.userId' }, + ], + } as unknown as ForestSchemaCollection; +} + +describe('synthesizeCapabilities', () => { + let logger: Logger; + + beforeEach(() => { + logger = jest.fn(); + }); + + function fieldNamed(name: string) { + return synthesizeCapabilities(v1Apimap(), logger).fields.find(field => field.name === name); + } + + describe('the operator set', () => { + it('should only contain operators that map back to a canonical one, since an unmapped one throws at request time', () => { + const unmapped = [...LEGACY_LIANA_OPERATORS].filter(operator => !normalizeOperator(operator)); + + expect(unmapped).toEqual([]); + }); + + it('should exclude the operators a legacy liana rejects', () => { + const unsupported = [ + 'missing', + 'not_in', + 'longer_than', + 'shorter_than', + 'like', + 'i_contains', + 'i_starts_with', + 'includes_none', + ]; + + expect(unsupported.filter(operator => LEGACY_LIANA_OPERATORS.has(operator))).toEqual([]); + }); + + it('should exclude includes_all, which one liana rejects and the other answers with a 500', () => { + expect(LEGACY_LIANA_OPERATORS.has('includes_all')).toBe(false); + }); + }); + + describe('when the field is a scalar', () => { + it('should publish operators in snake_case, as a real capabilities response does', () => { + expect(fieldNamed('name')?.operators).toContain('starts_with'); + expect(fieldNamed('name')?.operators).not.toContain('StartsWith'); + }); + + it('should intersect the column type table with what the liana supports', () => { + const operators = fieldNamed('name')?.operators ?? []; + + expect(operators).toContain('contains'); + expect(operators).not.toContain('like'); + expect(operators).not.toContain('longer_than'); + }); + + it('should give a date field its date operators, which a separate liana parser handles', () => { + const operators = fieldNamed('birthDate')?.operators ?? []; + + expect(operators).toContain('today'); + expect(operators).toContain('previous_week'); + expect(operators).not.toContain('missing'); + }); + + it('should leave sortable absent when the apimap does not deny it', () => { + expect(fieldNamed('name')).not.toHaveProperty('sortable'); + }); + }); + + describe('when the apimap denies a capability', () => { + it('should publish no operator, so a filter on it is field_not_filterable and not a 500', () => { + expect(fieldNamed('fullName')?.operators).toEqual([]); + }); + + it('should state sortable false, which a real capabilities response never carries', () => { + expect(fieldNamed('fullName')?.sortable).toBe(false); + }); + }); + + describe('when the field is a relation', () => { + it('should publish a to-one relation as ManyToOne with no operators, matching v2', () => { + expect(fieldNamed('team')).toEqual({ name: 'team', type: 'ManyToOne' }); + }); + + it('should omit a to-many relation entirely, so a filter on it is unknown_field as in v2', () => { + expect(fieldNamed('articles')).toBeUndefined(); + }); + }); + + describe('when the column holds an array', () => { + it.each([['NumberList'], [['Number']]])( + 'should publish no operator for %p, since the scalar table would promise a filter that raises', + type => { + const collection = { + name: 'Odd', + fields: [{ field: 'tags', type, isFilterable: true }], + } as unknown as ForestSchemaCollection; + + const result = synthesizeCapabilities(collection, logger); + + expect(result.fields[0].operators).toEqual([]); + }, + ); + }); + + describe('when a column type has no operator table', () => { + it('should read as not filterable and log it, rather than forward a filter that would fail in SQL', () => { + const collection = { + name: 'Odd', + fields: [{ field: 'weird', type: 'SomethingElse', isFilterable: true }], + } as unknown as ForestSchemaCollection; + + const result = synthesizeCapabilities(collection, logger); + + expect(result.fields[0].operators).toEqual([]); + expect(logger).toHaveBeenCalledWith( + 'Warn', + expect.any(String), + expect.objectContaining({ collection: 'Odd', field: 'weird' }), + ); + }); + }); +}); diff --git a/packages/agent-bff/test/validation/capabilities-validator.test.ts b/packages/agent-bff/test/validation/capabilities-validator.test.ts index b990ff9f19..2f91ea56e3 100644 --- a/packages/agent-bff/test/validation/capabilities-validator.test.ts +++ b/packages/agent-bff/test/validation/capabilities-validator.test.ts @@ -222,6 +222,43 @@ describe('validateAgainstCapabilities', () => { }); }); + // Only the v1 synthesis states sortability; a real capabilities response omits it, so a field + // without the flag must keep sorting exactly as before. + describe('sortability', () => { + it('rejects a sort on a field the capabilities mark not sortable', () => { + const errors = validateAgainstCapabilities( + { sortFields: ['computed'] }, + { fields: [{ name: 'computed', type: 'String', operators: [], sortable: false }] }, + ); + + expect(errors[0]).toEqual( + expect.objectContaining({ + type: 'field_not_sortable', + status: 422, + details: { field: 'computed' }, + }), + ); + }); + + it('allows a sort when the flag is absent, which is every real capabilities response', () => { + expect( + validateAgainstCapabilities( + { sortFields: ['title'] }, + { fields: [{ name: 'title', type: 'String', operators: [] }] }, + ), + ).toEqual([]); + }); + + it('does not constrain a projection, which needs no ordering', () => { + expect( + validateAgainstCapabilities( + { projectionFields: ['computed'] }, + { fields: [{ name: 'computed', type: 'String', operators: [], sortable: false }] }, + ), + ).toEqual([]); + }); + }); + it('passes a fully valid filter, sort, and projection', () => { expect( validateAgainstCapabilities( diff --git a/packages/agent-bff/test/validation/operator-normalizer.test.ts b/packages/agent-bff/test/validation/operator-normalizer.test.ts index 7d23463c90..6f504af90b 100644 --- a/packages/agent-bff/test/validation/operator-normalizer.test.ts +++ b/packages/agent-bff/test/validation/operator-normalizer.test.ts @@ -1,9 +1,9 @@ +import { toWireOperator } from '@forestadmin/agent-client'; import { allOperators } from '@forestadmin/datasource-toolkit'; import { normalizeOperator, toCanonicalOperatorSet, - toSnakeCaseOperator, } from '../../src/validation/operator-normalizer'; describe('operator-normalizer', () => { @@ -29,7 +29,7 @@ describe('operator-normalizer', () => { it('round-trips every canonical operator through snake_case', () => { allOperators.forEach(operator => { - expect(normalizeOperator(toSnakeCaseOperator(operator))).toBe(operator); + expect(normalizeOperator(toWireOperator(operator))).toBe(operator); }); }); diff --git a/packages/agent-client/src/filter-wire-format.ts b/packages/agent-client/src/filter-wire-format.ts new file mode 100644 index 0000000000..17c04a7818 --- /dev/null +++ b/packages/agent-client/src/filter-wire-format.ts @@ -0,0 +1,70 @@ +/** + * Convert PascalCase to the snake_case spelling an HTTP agent parses. + * + * Two passes handle the two boundaries: lowercase→uppercase (`greaterThan` → `greater_than`) and a + * capital followed by a capitalised word (`IContains` → `i_contains`, `PreviousXDays` → + * `previous_x_days`). + */ +export function toWireOperator(operator: string): string { + return operator + .replace(/([a-z])([A-Z])/g, '$1_$2') + .replace(/([A-Z])([A-Z][a-z])/g, '$1_$2') + .toLowerCase(); +} + +function isBranch(node: unknown): node is { aggregator?: string; conditions: unknown[] } { + if (typeof node !== 'object' || node === null) return false; + + const { aggregator, conditions } = node as { aggregator?: unknown; conditions?: unknown }; + + // The aggregator is lowercased below, so a non-string one would throw a TypeError from inside the + // walk instead of being reported as the malformed filter it is. + return Array.isArray(conditions) && (aggregator === undefined || typeof aggregator === 'string'); +} + +function isLeaf(node: unknown): node is { field: string; operator?: string; value?: unknown } { + return ( + typeof node === 'object' && + node !== null && + typeof (node as { field?: unknown }).field === 'string' + ); +} + +/** + * Rewrite a plain condition tree into the wire format every HTTP Forest agent parses, whatever its + * generation. Three differences from the canonical in-memory shape, each measured against + * `forest-express-sequelize 9.6.10` and `forest_liana 9.21.0`: + * + * - operators are snake_case: a v1 liana answers `NoMatchingOperatorError` on `Equal`, and the v2 + * agent accepts either spelling because it PascalCases what it receives + * (`agent/src/utils/condition-tree-parser.ts`) + * - aggregators likewise: `and`, not `And` + * - every leaf carries a `value` key even when its operator takes no operand, or a v1 liana answers + * `InvalidFiltersFormat`. The toolkit's validator allows `null` for exactly those operators + * (`MAP_ALLOWED_TYPES_FOR_OPERATOR_CONDITION_TREE`), so the key is safe on both generations. + * + * A branch is recognised before a leaf: a node carrying both `conditions` and `field` is ambiguous, + * and reading it as a leaf would forward its conditions untouched. + */ +export default function toWireFilter(node: unknown): unknown { + if (isBranch(node)) { + const { aggregator, conditions } = node; + + return { + ...(aggregator === undefined ? {} : { aggregator: aggregator.toLowerCase() }), + conditions: conditions.map(toWireFilter), + }; + } + + if (isLeaf(node)) { + const { field, operator, value } = node; + + return { + field, + operator: operator === undefined ? operator : toWireOperator(operator), + value: value ?? null, + }; + } + + return node; +} diff --git a/packages/agent-client/src/index.ts b/packages/agent-client/src/index.ts index 8cbcdb87f7..f1ae682b88 100644 --- a/packages/agent-client/src/index.ts +++ b/packages/agent-client/src/index.ts @@ -69,5 +69,6 @@ export function createRemoteAgentClient(params: { }); } +export { default as toWireFilter, toWireOperator } from './filter-wire-format'; export type { RecordId, SelectOptions } from './types'; export type { File } from '@forestadmin/datasource-toolkit'; diff --git a/packages/agent-client/src/query-serializer.ts b/packages/agent-client/src/query-serializer.ts index 55eb5b16ae..48f7b6bfec 100644 --- a/packages/agent-client/src/query-serializer.ts +++ b/packages/agent-client/src/query-serializer.ts @@ -1,6 +1,7 @@ import type { SelectOptions } from './types'; import type { PlainFilter, PlainSortClause } from '@forestadmin/datasource-toolkit'; +import toWireFilter from './filter-wire-format'; import HttpRequester from './http-requester'; export default class QuerySerializer { @@ -37,58 +38,10 @@ export default class QuerySerializer { return sort.ascending ? sort.field : `-${sort.field}`; } - /** - * Serialize filters to JSON with snake_case operators and aggregators. - * - * Internally, operators use PascalCase (e.g. `Equal`, `GreaterThan`) to match - * the datasource-toolkit convention. However, HTTP backends expect snake_case: - * - Ruby (forest_liana): requires `equal`, `greater_than`, etc. - * - Node (@forestadmin/agent): accepts snake_case via ConditionTreeParser.toPascalCase() - * - * Converting to snake_case here ensures compatibility with both backends. - */ private static formatFilters(filters: PlainFilter['conditionTree']): string { if (!filters) return undefined; - return JSON.stringify(QuerySerializer.toSnakeCaseOperators(filters)); - } - - /** - * Recursively walk the condition tree and convert operators/aggregators to snake_case. - */ - private static toSnakeCaseOperators(node: unknown): unknown { - if (!node || typeof node !== 'object') return node; - - const obj = node as Record; - - if ('operator' in obj) { - return { - ...obj, - operator: QuerySerializer.toSnakeCase(obj.operator as string), - }; - } - - if ('aggregator' in obj && Array.isArray(obj.conditions)) { - return { - aggregator: (obj.aggregator as string).toLowerCase(), - conditions: obj.conditions.map(c => QuerySerializer.toSnakeCaseOperators(c)), - }; - } - - return obj; - } - - /** - * Convert PascalCase to snake_case. - * Two passes handle different patterns: - * - Pass 1: lowercase→uppercase boundaries (e.g. `greaterThan` → `greater_Than`) - * - Pass 2: uppercase sequences (e.g. `IContains` → `I_Contains`, `PreviousXDays` → `PreviousX_Days`) - */ - private static toSnakeCase(value: string): string { - return value - .replace(/([a-z])([A-Z])/g, '$1_$2') - .replace(/([A-Z])([A-Z][a-z])/g, '$1_$2') - .toLowerCase(); + return JSON.stringify(toWireFilter(filters)); } private static formatFields(collectionName: string, fields: string[]): Record { diff --git a/packages/forestadmin-client/src/index.ts b/packages/forestadmin-client/src/index.ts index 6229b4cba2..1abe966025 100644 --- a/packages/forestadmin-client/src/index.ts +++ b/packages/forestadmin-client/src/index.ts @@ -23,6 +23,8 @@ export { ForestSchemaField, ForestSchemaAction, ForestSchemaCollection, + ForestSchemaMeta, + ForestSchemaWithMeta, ActivityLogResponse, ActivityLogAction, ActivityLogType, diff --git a/packages/forestadmin-client/src/permissions/forest-http-api.ts b/packages/forestadmin-client/src/permissions/forest-http-api.ts index 4193e4a788..91c10698f8 100644 --- a/packages/forestadmin-client/src/permissions/forest-http-api.ts +++ b/packages/forestadmin-client/src/permissions/forest-http-api.ts @@ -8,6 +8,8 @@ import type { ForestAdminClientOptions, ForestAdminServerInterface, ForestSchemaCollection, + ForestSchemaMeta, + ForestSchemaWithMeta, HydratedWorkflowRun, IpWhitelistRulesResponse, McpWorkflow, @@ -123,9 +125,14 @@ export default class ForestHttpApi implements ForestAdminServerInterface { } async getSchema(options: HttpOptions): Promise { + return (await this.getSchemaWithMeta(options)).collections; + } + + async getSchemaWithMeta(options: HttpOptions): Promise { const response = await ServerUtils.query<{ data: Array<{ id: string; type: string; attributes: Record }>; included?: Array<{ id: string; type: string; attributes: Record }>; + meta?: ForestSchemaMeta & Record; }>(options, 'get', '/liana/forest-schema'); const serializer = new JSONAPISerializer(); @@ -144,7 +151,10 @@ export default class ForestHttpApi implements ForestAdminServerInterface { }); serializer.register('segments', {}); - return serializer.deserialize('collections', response) as ForestSchemaCollection[]; + return { + collections: serializer.deserialize('collections', response) as ForestSchemaCollection[], + meta: response.meta ?? {}, + }; } async postSchema(options: HttpOptions, schema: object): Promise { diff --git a/packages/forestadmin-client/src/schema/index.ts b/packages/forestadmin-client/src/schema/index.ts index 1645c5f63e..373b5348d4 100644 --- a/packages/forestadmin-client/src/schema/index.ts +++ b/packages/forestadmin-client/src/schema/index.ts @@ -1,4 +1,8 @@ -import type { ForestAdminServerInterface, ForestSchemaCollection } from '../types'; +import type { + ForestAdminServerInterface, + ForestSchemaCollection, + ForestSchemaWithMeta, +} from '../types'; import type { ForestSchema } from './types'; import crypto from 'crypto'; @@ -45,6 +49,16 @@ export default class SchemaService { return this.forestAdminServerInterface.getSchema(toHttpOptions(this.options)); } + async getSchemaWithMeta(): Promise { + const { getSchemaWithMeta } = this.forestAdminServerInterface; + + if (!getSchemaWithMeta) { + throw new Error('The configured Forest server transport does not support getSchemaWithMeta.'); + } + + return getSchemaWithMeta.call(this.forestAdminServerInterface, toHttpOptions(this.options)); + } + static serialize(schema: ForestSchema): SerializedSchema { const data = schema.collections.map(c => ({ id: c.name, ...c })); const schemaFileHash = crypto.createHash('sha1').update(JSON.stringify(schema)).digest('hex'); diff --git a/packages/forestadmin-client/src/types.ts b/packages/forestadmin-client/src/types.ts index a3a118cc4c..d5de7d86e6 100644 --- a/packages/forestadmin-client/src/types.ts +++ b/packages/forestadmin-client/src/types.ts @@ -234,6 +234,21 @@ export interface ForestSchemaCollection { actions?: ForestSchemaAction[]; } +/** + * Top-level metadata of a published schema: which agent produced it, and in which version. Both are + * optional because no server-side validation enforces them, so a consumer must handle their absence + * rather than assume the shape. + */ +export interface ForestSchemaMeta { + liana?: string; + liana_version?: string; +} + +export interface ForestSchemaWithMeta { + collections: ForestSchemaCollection[]; + meta: ForestSchemaMeta; +} + /** * Activity log response from the Forest Admin server. */ @@ -475,6 +490,7 @@ export interface ForestAdminServerInterface { // Schema operations getSchema?: (options: HttpOptions) => Promise; + getSchemaWithMeta?: (options: HttpOptions) => Promise; postSchema?: (options: HttpOptions, schema: object) => Promise; checkSchemaHash?: (options: HttpOptions, hash: string) => Promise<{ sendSchema: boolean }>; diff --git a/packages/forestadmin-client/test/__factories__/forest-admin-server-interface.ts b/packages/forestadmin-client/test/__factories__/forest-admin-server-interface.ts index 1924091734..5a4c3ce70a 100644 --- a/packages/forestadmin-client/test/__factories__/forest-admin-server-interface.ts +++ b/packages/forestadmin-client/test/__factories__/forest-admin-server-interface.ts @@ -10,6 +10,7 @@ const forestAdminServerInterface = { makeAuthService: jest.fn(), // Schema operations getSchema: jest.fn(), + getSchemaWithMeta: jest.fn(), postSchema: jest.fn(), checkSchemaHash: jest.fn(), // IP whitelist operations diff --git a/packages/forestadmin-client/test/permissions/forest-http-api.test.ts b/packages/forestadmin-client/test/permissions/forest-http-api.test.ts index 1f01172d4c..252b54239e 100644 --- a/packages/forestadmin-client/test/permissions/forest-http-api.test.ts +++ b/packages/forestadmin-client/test/permissions/forest-http-api.test.ts @@ -90,6 +90,34 @@ describe('ForestHttpApi', () => { }); }); + describe('getSchemaWithMeta', () => { + it('should return the collections together with the liana that published them', async () => { + (ServerUtils.query as jest.Mock).mockResolvedValue({ + data: [{ id: 'users', type: 'collections', attributes: { name: 'users' } }], + included: [], + meta: { liana: 'forest-rails', liana_version: '9.21.0' }, + }); + + const result = await new ForestHttpApi().getSchemaWithMeta(options); + + expect(ServerUtils.query).toHaveBeenCalledWith(options, 'get', '/liana/forest-schema'); + expect(result.collections).toHaveLength(1); + expect(result.meta).toEqual({ liana: 'forest-rails', liana_version: '9.21.0' }); + }); + + it('should answer an empty meta when the published schema carries none', async () => { + (ServerUtils.query as jest.Mock).mockResolvedValue({ + data: [{ id: 'users', type: 'collections', attributes: { name: 'users' } }], + included: [], + }); + + const result = await new ForestHttpApi().getSchemaWithMeta(options); + + expect(result.meta).toEqual({}); + expect(result.collections[0].name).toBe('users'); + }); + }); + describe('postSchema', () => { it('should call the right endpoint with schema and a 30s timeout', async () => { const schema = { data: [], meta: { schemaFileHash: 'abc123' } }; diff --git a/packages/forestadmin-client/test/schema/index.test.ts b/packages/forestadmin-client/test/schema/index.test.ts index e7507a8300..7c57dec47e 100644 --- a/packages/forestadmin-client/test/schema/index.test.ts +++ b/packages/forestadmin-client/test/schema/index.test.ts @@ -120,6 +120,37 @@ describe('SchemaService', () => { expect(result).toStrictEqual(mockCollections); }); + test('should fetch the schema and its liana metadata together', async () => { + const withMeta = { + collections: [{ name: 'users', fields: [] }], + meta: { liana: 'forest-express-sequelize', liana_version: '9.6.10' }, + }; + mockForestAdminServerInterface.getSchemaWithMeta.mockResolvedValue(withMeta); + + const options = factories.forestAdminClientOptions.build(); + const schemaService = new SchemaService(mockForestAdminServerInterface, options); + const result = await schemaService.getSchemaWithMeta(); + + expect(mockForestAdminServerInterface.getSchemaWithMeta).toHaveBeenCalledWith({ + envSecret: options.envSecret, + forestServerUrl: options.forestServerUrl, + }); + expect(result).toStrictEqual(withMeta); + }); + + test('should throw when the transport does not implement getSchemaWithMeta', async () => { + const options = factories.forestAdminClientOptions.build(); + const transport = { + ...mockForestAdminServerInterface, + getSchemaWithMeta: undefined, + }; + const schemaService = new SchemaService(transport, options); + + await expect(schemaService.getSchemaWithMeta()).rejects.toThrow( + 'The configured Forest server transport does not support getSchemaWithMeta.', + ); + }); + test('should propagate errors from the server', async () => { const networkError = new Error('Network error: connection refused'); mockForestAdminServerInterface.getSchema.mockRejectedValue(networkError); diff --git a/packages/mcp-server/test/server.test.ts b/packages/mcp-server/test/server.test.ts index d84bc98b79..93f5c123c2 100644 --- a/packages/mcp-server/test/server.test.ts +++ b/packages/mcp-server/test/server.test.ts @@ -2128,7 +2128,7 @@ describe('ForestMCPServer Instance', () => { const filters = JSON.parse(capturedQueryParams.filters as string); expect(filters).toEqual({ aggregator: 'and', - conditions: [{ field: 'email', operator: 'present' }], + conditions: [{ field: 'email', operator: 'present', value: null }], }); }); });