Problem with quantization of GRU model

Programming Q&A
1

hii TinyML community,

After a long search in other forums for my problem, any solution fund won’t work for me. So, I remember my membership in Tiny Machine Learning Community, I hope that you can help me to overcome this problem.

the problem is while doing post-training integer quantization of a GRU model, it gives me the following error :

ValueError: Failed to parse the model: pybind11::init(): factory function returned nullptr.

quantization problem
quantization problem1538×516 38.1 KB

I think the problem is with my representative_dataset function.

info :
the input shape of my model is (number of samples, time_steps, features) (where number of samples = 52000, time_steps = 20 , features = 3)

# Convert the model to the TensorFlow Lite format without quantization
num_calibration_steps=1

converter = tf.lite.TFLiteConverter.from_saved_model(GRUMODEL_TF)
converter.optimizations = [tf.lite.Optimize.DEFAULT]

# Save the model to disk
#open(MODEL_NO_QUANT_TFLITE, "wb").write(model_no_quant_tflite)

def representative_dataset_gen():
	for sample in XX_data:
	    sample = np.expand_dims(sample.astype(np.float32), axis=0)
	    yield [sample]


# Set the optimization flag.
converter.optimizations = [tf.lite.Optimize.DEFAULT]
# Enforce integer only quantization
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8
# Provide a representative dataset to ensure we quantize correctly.
converter.representative_dataset = repr_data_gen
model_tflite = converter.convert()

# Save the model to disk
open(GRUMODEL_TFLITE, "wb").write(model_tflite)
2

Have you checked this github issue? Someone suggested running the model without quantization and apparently it worked for the poster.

3

thanks @ yogeshwarb404 for your reply,

I have already checked it, but my goal is to compare the performance of the full integer quantized model with the Non-quantized tflite model and check the effect of quantization on the model degradation.

4

They’ve also mentioned it working on the nightly version of tensorflow, which is backed up by this StackOverflow answer. Maybe that could help?