feat: add SQL vector_search table-valued function

[summaryzb]

Summary

This PR adds a Spark SQL vector_search table-valued function for querying Lance vector columns through the Lance Spark SQL extension. The implementation resolves catalog tables or direct Lance paths, executes Lance nearest-neighbor scans per fragment and applies a Spark-side global TopK by Lance's internal _distance value while keeping _distance out of user-visible results.

Problem

Resolve #65
Lance Spark already supports nearest-neighbor reads through read options, but SQL users did not have a first-class query form for vector search. Users needed a SQL extension entry point that works with both Lance catalog tables and direct dataset paths, validates query arguments consistently, preserves fragment-parallel scans, and returns globally ordered TopK results without exposing connector-internal distance metadata.

Approach

The PR registers Lance table-valued functions through LanceSparkSessionExtensions and adds a resolution rule that rewrites unresolved Lance TVF logical nodes once their arguments are resolved. vector_search(table, column, query_vector, limit) validates foldable literal arguments, resolves the target as either a catalog LanceDataset or path-based Lance dataset, builds a Lance nearest-neighbor Query, and passes it through LanceSparkReadOptions.CONFIG_NEAREST into a DataSourceV2Relation.

To support final ranking, the relation schema is widened with a reserved internal _distance column. Fragment scanners request _distance from Lance only for nearest-neighbor scans, while normal column projection continues to filter internal columns. The logical plan then sorts globally by _distance, applies local and global limits, and projects _distance away so the public result schema matches the source table. This explicitly adds the cross-fragment TopK merge that the earlier DataFrame nearest API did not perform. TopN pushdown is disabled when a nearest-neighbor query is active so Spark keeps control of the final global ordering step.

Notice

PathOnly table pattern changes from "file://" to "file:/"

Test Coverage

• Catalog-table vector_search returns the expected global TopK result ordering.
• vector_search keeps multiple fragment scan partitions and leaves final TopK ordering to Spark.
• Internal _distance is not exposed in SELECT * results.
• Direct-path vector_search works against a Lance dataset URI and applies the requested global limit.
• Invalid non-positive limits are rejected with an IllegalArgumentException.

[LuciferYang]

cc @hamersaw
also cc @Xuanwo