Skip to content

shortestPath anchored by WHERE id(a) = … is 1000x slower than the same query anchored inline #538

Description

@sandeepkunkunuru

Measured

4,000 nodes, out-degree 30, endpoints three hops apart. Same graph, same answer, same session:

query time rows
shortestPath((a:N {seq: 1})-[:KNOWS*]-(b:N {seq: 3500})) 4.2 ms 1
allShortestPaths((a:N {seq: 1})-[:KNOWS*]-(b:N {seq: 3500})) 4.8 ms 48
shortestPath((a:N)-[:KNOWS*]-(b:N)) WHERE id(a) = 1 AND id(b) = 3500 4,219 ms 1

~1000× for what is logically the same question.

The plan says why

EXPLAIN MATCH p = shortestPath((a:N)-[:KNOWS*]-(b:N))
        WHERE id(a) = 1 AND id(b) = 3500 RETURN length(p) AS len

Project (length(p) AS len)
+- Filter (id(b) = Integer(300))
   +- Unknown

Two things:

  1. id(a) = 1 has vanished and id(b) = 3500 sits above the path search. So the search is not anchored on either endpoint by the predicate — it runs and is filtered afterwards, which for a pairwise operator means running it across a cross product of candidate endpoints.
  2. The operator renders as Unknown, because ShortestPathOperator did not implement describe(). Fixed in the same PR as IC14 allShortestPaths times out at SF1 once the endpoints are 3 hops apart #516; noting it here because it is why the plan could not be read before.

The inline form lowers each endpoint to an anchored scan, so the operator receives exactly one (source, target) pair.

Why it matters

  • id() is the most natural way to anchor a path query when you already hold node ids, and it is what an application does. Inline property syntax requires a property that happens to be indexed.
  • The two forms are equivalent in openCypher. A user has no way to know that one of them is 1000× the other, and nothing in the plan told them until describe() existed.
  • It is the same class as Optimize Multi-Hop Relationship Expansion with Predicate Pushdown #328: a highly selective predicate that the planner does not use to choose an anchor.

Not the cause of #516

Worth separating, because they look alike. #516 was the algorithmallShortestPaths disabled its visited set and enumerated walks. That is fixed, and LDBC IC14 uses the inline form, so IC14's timeout was genuinely #516's. This issue is what remains once the algorithm is right.

Suggested fix

id(x) = <literal> should lower to a direct node lookup wherever x is a pattern variable, the same way an inline {prop: value} on an indexed property lowers to an IndexScan. That is narrower than general predicate-driven anchor selection (#328) and does not need the cost model: id() is unique by construction, so the estimate is 1 and there is no decision to make.

The scope worth checking alongside it: the same rewrite should apply to MATCH (n) WHERE id(n) = 5 RETURN n, which is the single most common form of a point lookup.

Definition of done

  • WHERE id(x) = <literal> lowers to a node lookup rather than a scan-plus-filter
  • The three queries above land within a small factor of each other
  • A plan-shape test asserting no NodeScan under an id() equality
  • Check id(x) IN [...] too — the same argument applies and it is how a batch lookup is written

Found while fixing #516.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingcomp:plannerQuery planning, cardinality estimation, join ordering

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions