I2-CE is an efficient implementation of Convolutional Neural Networks (CNNs), targeting edge devices. It aims at achieving an efficient trade-off between lower runtime and limited accuracy degradation.
- ⚡ Fast inference of CNN models
- 🧩 Robust accuracy brought by ensambling
- 🚀 SIMD acceleration
- 📖 Open training and inference code
-
Ensembles
- multiple instances of CNNs can improve accuracy by providing slightly different perspective on the task.
- runtime suffers from repeated computations
-
Codebooks
- each layer's weights are limited to a small look-up table (the codebook). Each learner in the ensemble has its own set of codebooks. Accuracy may suffer, depending on the codebook size.
- multiple indexes to the codebooks can be packed into a single word.Small codebooks and packed indexes are cache friendly, runtime is positively impacted.
-
Shared indexes
- CNNs are trained so to have share the same set of indexes to their own codebooks. More constraints on the weights distribution can damange accuracy.
- With only one set of indexes, the ensemble require less memory. Moreover, this regularity can be highly exploited with Single-Instruction Multiple-Data (SIMD) instructions. Inference is faster
The repository is organized as follows:
I2-CE/
|-- training
|-- inference
- training : files for the training methodology used to obtain ensemble of learners with codebooks and shared indexes.
- inference : files for the inference strategy. Particularly, the generic convolution and dense layers are implemented according to our methodology. Full AlexNet and VGG16 implementations are provided.
NOTE: the I2-CE implementation targets and ARMv8 core with SVE vectorial extension for SIMD instructions.