Problem with quantization of GRU model

hii TinyML community,

After a long search in other forums for my problem, any solution fund won’t work for me. So, I remember my membership in Tiny Machine Learning Community, I hope that you can help me to overcome this problem.

the problem is while doing post-training integer quantization of a GRU model, it gives me the following error :

ValueError: Failed to parse the model: pybind11::init(): factory function returned nullptr.

I think the problem is with my representative_dataset function.

info :
the input shape of my model is (number of samples, time_steps, features) (where number of samples = 52000, time_steps = 20 , features = 3)

# Convert the model to the TensorFlow Lite format without quantization

converter = tf.lite.TFLiteConverter.from_saved_model(GRUMODEL_TF)
converter.optimizations = [tf.lite.Optimize.DEFAULT]

# Save the model to disk
#open(MODEL_NO_QUANT_TFLITE, "wb").write(model_no_quant_tflite)

def representative_dataset_gen():
	for sample in XX_data:
	    sample = np.expand_dims(sample.astype(np.float32), axis=0)
	    yield [sample]

# Set the optimization flag.
converter.optimizations = [tf.lite.Optimize.DEFAULT]
# Enforce integer only quantization
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8
# Provide a representative dataset to ensure we quantize correctly.
converter.representative_dataset = repr_data_gen
model_tflite = converter.convert()

# Save the model to disk
open(GRUMODEL_TFLITE, "wb").write(model_tflite)

Have you checked this github issue? Someone suggested running the model without quantization and apparently it worked for the poster.

thanks @ yogeshwarb404 for your reply,

I have already checked it, but my goal is to compare the performance of the full integer quantized model with the Non-quantized tflite model and check the effect of quantization on the model degradation.

They’ve also mentioned it working on the nightly version of tensorflow, which is backed up by this StackOverflow answer. Maybe that could help?