Gemma quantize method not implemented?

[ashep29]

Are these warnings something to be concerned about?

preprocessor = keras_hub.models.GemmaCausalLMPreprocessor.from_preset(
"gemma_1.1_instruct_2b_en", sequence_length=256
)
gemma_lm = keras_hub.models.GemmaCausalLM.from_preset(
"gemma_1.1_instruct_2b_en", preprocessor=preprocessor
)
gemma_lm.quantize("int8")
gemma_lm.backbone.enable_lora(rank=4)
gemma_lm.summary()

Downloading from https://www.kaggle.com/api/v1/models/keras/gemma/keras/gemma_1.1_instruct_2b_en/4/download/config.json...

100%|███████████████████████████████████████████| 785/785 [00:00<00:00, 128kB/s]

Downloading from https://www.kaggle.com/api/v1/models/keras/gemma/keras/gemma_1.1_instruct_2b_en/4/download/tokenizer.json...

100%|██████████████████████████████████████████| 591/591 [00:00<00:00, 94.3kB/s]

Downloading from https://www.kaggle.com/api/v1/models/keras/gemma/keras/gemma_1.1_instruct_2b_en/4/download/assets/tokenizer/vocabulary.spm...

100%|██████████████████████████████████████| 4.04M/4.04M [00:01<00:00, 2.20MB/s]
normalizer.cc(51) LOG(INFO) precompiled_charsmap is empty. use identity normalization.

Downloading from https://www.kaggle.com/api/v1/models/keras/gemma/keras/gemma_1.1_instruct_2b_en/4/download/model.weights.h5...

100%|██████████████████████████████████████| 4.67G/4.67G [04:30<00:00, 18.5MB/s]
/home/asheple2/.local/lib/python3.9/site-packages/keras/src/models/model.py:391: UserWarning: Layer InputLayer does not have a quantize method implemented.
warnings.warn(str(e))
/home/asheple2/.local/lib/python3.9/site-packages/keras/src/models/model.py:391: UserWarning: Layer RMSNormalization does not have a quantize method implemented.
warnings.warn(str(e))
/home/asheple2/.local/lib/python3.9/site-packages/keras/src/models/model.py:391: UserWarning: Layer RotaryEmbedding does not have a quantize method implemented.
warnings.warn(str(e))
/home/asheple2/.local/lib/python3.9/site-packages/keras/src/models/model.py:391: UserWarning: Layer Softmax does not have a quantize method implemented.
warnings.warn(str(e))
/home/asheple2/.local/lib/python3.9/site-packages/keras/src/models/model.py:391: UserWarning: Layer Dropout does not have a quantize method implemented.
warnings.warn(str(e))

When training, I get this output after the first epoch:

W0000 00:00:1742807763.942093 62622 assert_op.cc:38] Ignoring Assert operator compile_loss/sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/assert_equal_1/Assert/Assert

[sonali-kumari1]

Hi @ashep29 -
The warnings you are seeing related to quantize method indicates that certain layers (like InputLayer, RMSNormalization, RotaryEmbedding, etc.) don't support quantization So, you can safely ignore these warnings. Assert Warning can also be ignored if the training seems fine. However, if you notice any instability in model performance or training failure, could you please provide details about your training setup ?Thanks!

[ashep29]

Thank @sonali-kumari1.

Training is no good. The loss increases, accuracy decreases.

Epoch 1/3

W0000 00:00:1742807763.942093 62622 assert_op.cc:38] Ignoring Assert operator compile_loss/sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/assert_equal_1/Assert/Assert

959/960 ━━━━━━━━━━━━━━━━━━━━ 0s 148ms/step - loss: 0.8408 - sparse_categorical_accuracy: 0.2905

W0000 00:00:1742807924.945180 62623 assert_op.cc:38] Ignoring Assert operator compile_loss/sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/assert_equal_1/Assert/Assert

960/960 ━━━━━━━━━━━━━━━━━━━━ 190s 165ms/step - loss: 0.8408 - sparse_categorical_accuracy: 0.2905
Epoch 2/3
960/960 ━━━━━━━━━━━━━━━━━━━━ 144s 149ms/step - loss: 0.8460 - sparse_categorical_accuracy: 0.2893
Epoch 3/3
960/960 ━━━━━━━━━━━━━━━━━━━━ 144s 150ms/step - loss: 0.8532 - sparse_categorical_accuracy: 0.2895

[sonali-kumari1]

@ashep29, Thanks for the update. Could you please share details of your training loop (batch size, epochs, optimizer), the dataset you’re using and your environment.