VisualSearch 🖼️ 🔍

A Hierarchical-Learning based Reverse Image Search Engine for humans.

Hierarchical modules:

• Classification
• Transfer Learning based feature generator

Code Requirements 🦄

You can install Conda for python which resolves all the dependencies for machine learning.

pip install -r requirements.txt

Description 🌈

Reverse image search (or as it is more technically known, instance retrieval) enables developers and researchers to build scenarios beyond simple keyword search. From discovering visually similar objects on Pinterest to recommending similar songs on Spotify to camera-based product search on Amazon, a similar class of technology under the hood is used. This is inspired by the following modules

• Google Lens
• Amazon Camera Search

Modules 💫

• Image Similarity
• Feature Extraction
• Similarity Search

Dataset 🗃️

Currently, I have used the Caltech 101 Dataset. In this dataset, we have pictures of objects belonging to 101 categories. About 40 to 800 images per category and most categories have about 50 images. However, the learnings from this can be extrapolated for domain specific datasets.

Search Engine 🕊️

Elasticsearch has been used as a search engine. Elasticsearch is a search engine based on the Lucene library. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents. Specifically, this pipeline uses dense_vactor field in elasticsearch. A dense_vector field stores dense vectors of float values.

File Organization 🗄️

├── VisualSearch (Current Directory)
    ├── data : Data folder
        ├── a
            ├── 1.png
            ├── 2.png
            .
            .
        ├── b
            .
            .
        └── z
    ├── models : Trained classifier model
        └── classifier.h5 : classifier model
    ├── resources : resources folder
    ├── src : Source code
        ├── classifier
            └── Classifier.py
        ├── elasticsearch
            └── ElasticSearch.py
        ├── pretrained
            ├── PretrainedFactory.py
            └── ResNet50.py
        └── utils
            └── utils.py
    ├── VisualSearch.py : Driving class
    ├── application.yaml : yaml properties
    ├── LICENSE
    ├── requirements.txt
    └── readme.md
        

Python Implementation 👨‍🔬

1. Classifier : Training on pretrained - ResNet50 for classifying
2. Feature Extractor : Pretrained ResNet50 for feature extraction
3. Search Engine : Elasticsearch as a search engine

If you face any problem, kindly raise an issue

Setup 🖥️

1. The pipeline is setup to be a one-click program.
2. Configure the application.yaml file
3. Run VisualSearch.py to run the application.

Execution 🐉

python3 VisualSearch.py -yml application.yaml

Results 📊

References: 🔱

• Practical Deep Learning for Cloud, Mobile, and Edge by Anirudh Koul, Siddha Ganju, Meher Kasam
• Text similarity search with vector fields