Skip to content

Commit 2d4e0ed

Browse files
committed
[ty] Add evaluation test for auto-import including symbols in current module
This shouldn't happen. And indeed, currently, this results in a sub-optimal ranking.
1 parent 9ce3fa3 commit 2d4e0ed

File tree

6 files changed

+36
-0
lines changed

6 files changed

+36
-0
lines changed

crates/ty_completion_eval/completion-evaluation-tasks.csv

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
name,file,index,rank
2+
auto-import-skips-current-module,main.py,0,4
23
fstring-completions,main.py,0,1
34
higher-level-symbols-preferred,main.py,0,
45
higher-level-symbols-preferred,main.py,1,1
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
[settings]
2+
auto-import = true
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
Kadabra = 1
2+
3+
# This is meant to reflect that auto-import
4+
# does *not* include completions for `Kadabra`.
5+
# That is, before a bug was fixed, completions
6+
# would offer two variants for `Kadabra`: one
7+
# for the current module (correct) and another
8+
# from auto-import that would insert
9+
# `from main import Kadabra` into this module
10+
# (incorrect).
11+
#
12+
# Since the incorrect one wasn't ranked above
13+
# the correct one, this task unfortunately
14+
# doesn't change the evaluation results. But
15+
# I've added it anyway in case it does in the
16+
# future (or if we change our evaluation metric
17+
# to something that incorporates suggestions
18+
# after the correct one).
19+
Kada<CURSOR: Kadabra>
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
[project]
2+
name = "test"
3+
version = "0.1.0"
4+
requires-python = ">=3.13"
5+
dependencies = []
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
AbraKadabra = 1

crates/ty_completion_eval/truth/auto-import-skips-current-module/uv.lock

Lines changed: 8 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)