Skip to content

Conversation

baoheping
Copy link
Contributor

Description

Added Elasticsearch-based storage modules to support document chunk indexing, status tracking, and vector similarity search.

Related Issues

N/A

Changes Made

  • Implemented ESVectorStorage: vector similarity search using Elasticsearch's dense_vector
  • Implemented ESKVStorage: key-value storage for chunk metadata and task processing context
  • Implemented ESDocStatusStorage: document-level processing status tracking via Elasticsearch
  • Designed unified index mappings with dynamic meta field support
  • Added configuration options for Elasticsearch endpoints and index naming conventions

Checklist

  • [ x ] Changes tested locally
  • [ x ] Code reviewed
  • Documentation updated (if necessary)
  • Unit tests added (if applicable)

Additional Notes

These components provide full-featured Elasticsearch-based alternatives to Milvus/MongoDB storage modules, enabling pluggable backend support in the LightRAG framework.

@danielaskdd
Copy link
Collaborator

Thank you for your congratulations and efforts. As LightRAG evolves, each storage implementation necessitates corresponding upgrades. Currently, we lack the resources to consistently track and maintain the advancements and performance optimizations for every storage type. To avoid impeding the overall progress of the LightRAG system, LightRAG will temporarily discontinue the integration of new storage implementations.

@baoheping
Copy link
Contributor Author

Thanks for the clarification. I'll leave this PR open in case the project decides to resume storage integration in the future.

@danielaskdd
Copy link
Collaborator

Please resolve any conflicts and maintain synchronization between the storage interface and the main branch, enabling community users interested in utilizing ES storage to pull the code from this PR and benefit from your contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants