Indox Ecosystem

Official Website • Documentation • Discord

NEW: Subscribe to our mailing list for updates and news!

🌟 The Indox Ecosystem

The Indox Ecosystem is a comprehensive suite of tools designed to revolutionize your AI and data workflows. Our ecosystem consists of four powerful components:

1. 🔍 IndoxArcg

Advanced Retrieval-Augmented Generation (RAG) and Cache-Augmented Generation (CAG) system for intelligent information extraction and processing.

Key Features:

• Multi-format document support: Handles PDF, HTML, Markdown, LaTeX, and more.
• Intelligent clustering and chunk processing: Organizes and processes documents for efficient retrieval.
• Support for major LLM providers: Compatible with OpenAI, Google, Mistral, HuggingFace, Ollama, and others.
• Advanced RAG features:
  • Semantic caching for faster retrieval.
  • Multi-query retrieval for improved context extraction.
  • Reranking and relevance scoring for high-quality results.
• Cache-Augmented Generation (CAG):
  • Preloading and caching of documents for faster inference.
  • Smart retrieval with validation and hallucination detection.
  • Web search fallback for missing or insufficient context.
• Customizable similarity search: Supports TF-IDF, BM25, and Jaccard similarity algorithms.
• Robust error handling: Includes fallback mechanisms for retrieval failures and hallucination detection.

2. ⛏️ IndoxMiner

Powerful data extraction and mining tool leveraging LLMs.

• Schema-based structured data extraction
• Multi-format support with OCR capabilities
• Flexible validation and type safety
• Async processing for scalability
• High-resolution PDF support

3. 📊 IndoxJudge

Comprehensive LLM and RAG evaluation framework.

• Multiple evaluation metrics (Faithfulness, Toxicity, BertScore, etc.)
• Safety and bias assessment
• Multi-model comparison capabilities
• RAG-specific evaluation metrics
• Extensible framework for custom metrics

4. 🔄 IndoxGen

Advanced synthetic data generation suite with three specialized components:

• IndoxGen Core: LLM-powered synthetic data generation
• IndoxGen-Tensor: TensorFlow-based GAN data generation
• IndoxGen-Torch: PyTorch-based GAN data generation

📦 Quick Installation

Install the entire ecosystem:

pip install indoxArcg indoxminer indoxjudge indoxgen indoxgen-tensor indoxgen-torch

Or install components separately:

pip install indoxArcg       # Core RAG or Cag functionality
pip install indoxminer     # Data extraction
pip install indoxjudge     # LLM evaluation
pip install indoxgen       # Synthetic data generation

🚀 Model Support

Model Provider	indoxArcg	IndoxJudge	IndoxGen
OpenAI	✅	✅	✅
Google	✅	✅	✅
Mistral	✅	✅	✅
HuggingFace	✅	✅	✅
Ollama	✅	✅	❌
Anthropic	❌	❌	❌

💡 Getting Started

Check out our example notebooks:

• indoxArcg Pipeline
• IndoxJudge Evaluation
• IndoxMiner Extraction
• IndoxGen Data Generation

🛣️ Roadmap

• Unified web interface for all components
• Docker support across the ecosystem
• Enhanced integration between components
• Advanced privacy and security features
• Multi-language support expansion
• Additional model provider integrations

🤝 Contributing

We welcome contributions to any component of the Indox ecosystem! Please check our Contributing Guidelines for more information.

📄 License

This project is licensed under the AGPL License - see the LICENSE file for details.

---

  .----------------.  .-----------------. .----------------.  .----------------.  .----------------.
| .--------------. || .--------------. || .--------------. || .--------------. || .--------------. |
| |     _____    | || | ____  _____  | || |  ________    | || |     ____     | || |  ____  ____  | |
| |    |_   _|   | || ||_   \|_   _| | || | |_   ___ `.  | || |   .'    `.   | || | |_  _||_  _| | |
| |      | |     | || |  |   \ | |   | || |   | |   `. \ | || |  /  .--.  \  | || |   \ \  / /   | |
| |      | |     | || |  | |\ \| |   | || |   | |    | | | || |  | |    | |  | || |    > `' <    | |
| |     _| |_    | || | _| |_\   |_  | || |  _| |___.' / | || |  \  `--'  /  | || |  _/ /'`\ \_  | |
| |    |_____|   | || ||_____|\____| | || | |________.'  | || |   `.____.'   | || | |____||____| | |
| |              | || |              | || |              | || |              | || |              | |
| '--------------' || '--------------' || '--------------' || '--------------' || '--------------' |
  '----------------'  '----------------'  '----------------'  '----------------'  '----------------'

Made with ❤️ by OSLLM.ai