Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MEVD] Consider renaming IsFilterable to IsIndexed #11130

Open
roji opened this issue Mar 23, 2025 · 9 comments · May be fixed by #11167
Open

[MEVD] Consider renaming IsFilterable to IsIndexed #11130

roji opened this issue Mar 23, 2025 · 9 comments · May be fixed by #11167
Assignees
Labels
Build Features planned for next Build conference msft.ext.vectordata Related to Microsoft.Extensions.VectorData

Comments

@roji
Copy link
Member

roji commented Mar 23, 2025

First, some database allow filtering on any property, regardless of whether it has an index or not; this is the case e.g. with relational databases, where index can speed up queries but are never required. For those databases, IsFilterable doesn't express what the property actually does. We should consider renaming for this to simply indicate what it does (trigger index creation).

Second, indexes can speed up operations other than filtering, notably sorting/ordering performed by non-vector searches (#10295).

/cc @westey-m @dmytrostruk @adamsitnik

@roji roji added Build Features planned for next Build conference msft.ext.vectordata Related to Microsoft.Extensions.VectorData labels Mar 23, 2025
@roji
Copy link
Member Author

roji commented Mar 24, 2025

Interesting, I wonder what that means in terms of the actual database indexes used (presumably it has two different index types to drive each operation or something? Am not familiar with that from other databases).

Assuming we rename IsFilterable to IsIndexed in MEVD, we could make that setting trigger both IsFilterable and IsSortable on Azure AI Search; later, we could expose connector-specific extensions to allow managing this in a more fine-grained way... How does that sound?

@adamsitnik
Copy link
Member

How does that sound?

It sounds good to me, but do we have any contacts to the Azure AI Search Team? We could ensure that it's not some kind of anti-pattern.

@shethaadit
Copy link
Contributor

Hi @roji / @adamsitnik / @markwallace-microsoft, I have raised PR for this - #11167.

@roji
Copy link
Member Author

roji commented Mar 24, 2025

It sounds good to me, but do we have any contacts to the Azure AI Search Team? We could ensure that it's not some kind of anti-pattern.

I don't, but I think maybe @westey-m does - in any case we need to hear his opinion about this first.

@shethaadit thanks for the PR, but we'll need to confirm that this is the way we want to go first.

@westey-m
Copy link
Contributor

@roji Sounds like a reasonable change. The initial naming was heavily inspired by the Azure AI Search naming, but you are right in that IsIndexed definitely matches the behavior of this property much better. We may want to reconsider the IsFullTextSearchable property at the same time as well.

@roji
Copy link
Member Author

roji commented Mar 25, 2025

Good point about IsFullTextSearchable... AFAIK in at least some (most?) databases, full-text indexes are are different beast from plain indexes (since they need to take lemmatization and other language stuff into account); that may be a good reason to maintain both.

@westey-m
Copy link
Contributor

Yeah, I just meant naming wise. We should consider whether to rename it to IsFullTextIndexed. Similar to the other one it would match what we actually do a bit better.

@roji
Copy link
Member Author

roji commented Mar 25, 2025

Oh I see..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Build Features planned for next Build conference msft.ext.vectordata Related to Microsoft.Extensions.VectorData
Projects
Status: No status
Development

Successfully merging a pull request may close this issue.

5 participants