You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Nothing checks that their loaders still run, that their queries still return what they claim, or that they survive an engine change.
Why this is more than a hygiene gap
scripts/verify-sweep.sh and the nightly cover this repository only — tests, examples, benches, and the 11 in-repo case studies. Everything in the list above is outside it.
Meanwhile the engine changed materially in the last two days: the Cypher grammar (#454, #462, #469), the executor's null and arithmetic semantics (#460), FOREACH write behaviour (#468), snapshot import (#507, #508), and the cost model (#498, #511). Several of those are exactly the kind of change that would surface in a loader or a demo query and nowhere else.
The case-study lane already demonstrates the value: it caught a snapshot URL that was never filled in (#442) and it is the only reason we know those 11 still work. The 26 repos have no equivalent.
The obstacle worth naming
These repos depend on data, and most of it is not in the repo — snapshots from releases, S3 corpora, licensed sources. A CI lane that needs a 10 GB download is not going to run per-push, and one that silently skips when the data is absent tells you nothing, which is the #449 failure mode.
So the design question is what can be checked without the data:
loader scripts parse and their CLI contracts hold
query files are syntactically valid Cypher against the current grammar — this alone would have caught the grammar changes
DATASET_CARD.md and README.md exist and carry the required frontmatter
declared engine/SDK versions resolve
That is a real gate and it needs no corpus.
Definition of done
A reusable workflow — one file, referenced by each repo rather than copied 26 times, so a fix lands once
Tier 1, no data required: shell/Python lint on loaders, Cypher parse-check on every query file, dataset-card presence and frontmatter, dependency resolution
Rolled out to the 19 KG repos first; demos second, since several are customer-facing and may have their own constraints
A cross-repo status page, or at minimum a nightly that opens an issue on failure rather than emailing into a void
Suggested first step
The Cypher parse-check alone. It needs no data, it is a few lines against the engine's parser, and it directly protects against the class of change we have been making all week. Everything else can follow.
Observation
0 of 26 KG and demo repositories have any
.github/workflows:Nothing checks that their loaders still run, that their queries still return what they claim, or that they survive an engine change.
Why this is more than a hygiene gap
scripts/verify-sweep.shand the nightly cover this repository only — tests, examples, benches, and the 11 in-repo case studies. Everything in the list above is outside it.Meanwhile the engine changed materially in the last two days: the Cypher grammar (#454, #462, #469), the executor's null and arithmetic semantics (#460),
FOREACHwrite behaviour (#468), snapshot import (#507, #508), and the cost model (#498, #511). Several of those are exactly the kind of change that would surface in a loader or a demo query and nowhere else.The case-study lane already demonstrates the value: it caught a snapshot URL that was never filled in (#442) and it is the only reason we know those 11 still work. The 26 repos have no equivalent.
The obstacle worth naming
These repos depend on data, and most of it is not in the repo — snapshots from releases, S3 corpora, licensed sources. A CI lane that needs a 10 GB download is not going to run per-push, and one that silently skips when the data is absent tells you nothing, which is the #449 failure mode.
So the design question is what can be checked without the data:
DATASET_CARD.mdandREADME.mdexist and carry the required frontmatterThat is a real gate and it needs no corpus.
Definition of done
SKIPexplicitly with the reason when it is not — never a silent pass (LDBC benchmark reports 21/21 passed while every query returns 0 rows — default params match no data #449, bench(ldbc): a read returning 0 rows is not a pass #450)Suggested first step
The Cypher parse-check alone. It needs no data, it is a few lines against the engine's parser, and it directly protects against the class of change we have been making all week. Everything else can follow.