Add incremental invalidation engine

[st0012]

Replaces the old remove_definitions_for_document + invalidate_ancestor_chains approach with a targeted invalidation engine. When a file is updated or deleted, the engine traces through the name_dependents reverse index to invalidate only the affected declarations, names, and references — instead of requiring a full graph rebuild.

How it works

consume_document_changes() (renamed from update()) and delete_document() run a three-step pipeline:

1. invalidate — read-only scan that seeds a worklist from old/new definitions and references, building a pending_detachments side table for definitions that need to be detached from their declarations
2. remove_document_data — cleans up phase 1 data (definitions, references, names, strings) from maps
3. extend — merges new content and queues work items for the resolver

The resolver still does clear_declarations + full rebuild. Wiring it to drain pending_work incrementally is a follow-up.

Invalidation worklist

The worklist processes three item types:

• Declaration — two modes:
  • Remove: no definitions remain or owner was already removed (orphaned). Cascades to members, singleton class, and descendants. Orphaned definitions are re-queued for re-resolution (e.g. class Foo::Bar survives even if Foo changes from a module to an alias).
  • Update: declaration survives but its ancestor chain may have changed (mixin added/removed, superclass changed). Clears ancestors/descendants and re-queues ancestor resolution.
• Name — structural dependency broken (name's nesting or parent scope removed). Unresolves the name and cascades to all dependents.
• References — ancestor context changed, but the name itself is still valid. Needed for mixin-related invalidation:

class Foo < Bar
  CONST
end

# Another file adds:
class Foo
  include Baz # Foo's ancestors changed, so references like CONST need re-evaluation
end

Cascade differentiation

The name_dependents reverse index distinguishes ChildName (compact syntax Foo::Bar) from NestedName (nested syntax module Foo; class Bar; end; end):

• Structural cascade (name removed): both ChildName and NestedName → Name
• Ancestor-triggered cascade (mixin changed): ChildName → Name (resolves through parent), NestedName → References (only references need rechecking)

[vinistock]

It might be worth adding a fast path for the initial indexing on boot (which can trigger no invalidation and no removal of data).

Maybe a boolean flag for skipping invalidation.

[vinistock]

Can you clarify that document change this is accounting for? I'm having a hard time understanding it.

A constant reference was changed and we're enqueuing invalidation for the reference's nesting declaration.

[st0012]

This is for

class Foo
  include Bar # constant reference, when added we invalidate Foo entirely for now
end

I've added comments for it.

[vinistock]

What does this mean?

[st0012]

Added more comments. Basically, it's triggered in

class Foo
  include Bar # this is added/removed
end

So we simply change ancestors/descendents update in this branch.

[vinistock]

This is also doing data removal. Maybe this is fine, but I'm calling it out because the method documentation mentions a separation between invalidation and removal.

[st0012]

I currently treat declaration removal as invalidation, kinda similar to unresolving a name. We remove (unresolve) the declaration here if we found it has no underlying definitions anymore.

The underlying materials (definitions, constant references, names...etc.) are only removed in remove_document_data.

[vinistock]

I could be misunderstanding and maybe this was an existing bug, but removing self is not enough. The entire ancestor chain of self must be removed from descendants. However, we should absolutely not try to perform this removal because there are module deduping rules that you cannot possibly account for with a removal.

Whenever a declaration gets invalidated, we always need to invalidate the ancestors of all descendants. In both branches of this method.

[st0012]

Both paths will update the ancestors. I've updated the document to make it more clear: invalidate_declaration either "remove/rebuild" or "update" a declaration. In both paths we update ancestors.

This also means there are optimization opportunities in both paths we can do later, which I also included in comments.

[st0012]

If we adopt #654, we should carry the memoized name_depth caching into the incremental path here. The depth sort in prepare_units is a correctness requirement — without it, 13 resolution tests fail because the resolution loop's made_progress check gives up when children are processed before parents. This means the incremental prepare_units must also sort by depth, and for large invalidation cascades (e.g., a change to a widely-used module rippling through name_dependents), the non-memoized recursive name_depth in the sort comparator would hit the same bottleneck we fixed in #654. The fix is straightforward: call compute_name_depths before sorting the pending units, same as the full resolution path.