Error using tflite delegate in python

Other OS Support Other OS-related topics
1

I trying to use tflite delegate in python, but whenever my code tries to initialize the interpreter using code:

lib_path = '/usr/lib/libQnnTFLiteDelegate.so'
delegate = tf.lite.experimental.load_delegate(lib_path, options={'backend_type': 'htp'})
interpreter = tf.lite.Interpreter(model_content=tflite_model, experimental_delegates=[delegate])

it throws error as interpreter tries to get initialized. The error:

<W> Initializing HtpProvider
 <W> Specified config SOC, ignoring on real target
 <E> createUnsignedPD unsigned PD or DSPRPC_GET_DSP_INFO not supported by HTP
 <E> DspTransport.openSession qnn_open failed, 0x00000072, prio 100
 <E> DspTransport.getHandle failed, error 0x00000007
 <E> createDspTransportInstance failed to config transport object
 <E> error in creation of transport instance
 <W> Failed to create transport instance: 1002
 <E> Failed to create transport for device, error: 1002
 <E> Failed to load skel, error: 1002
 <E> Transport layer setup failed: 14001
 <E> Failed to parse default platform info: 14001
 <E> Failed to load default platform info: 14001
 <E> Failed to parse platform config: 14001
Traceback (most recent call last):
  File "/home/user/tflite delegate/test.py", line 35, in <module>
    interpreter = tf.lite.Interpreter(
  File "/home/user/env_py310/lib/python3.10/site-packages/tensorflow/lite/python/interpreter.py", line 495, in __init__
    self._interpreter.ModifyGraphWithDelegate(
RuntimeError: Restored original execution plan after delegate application failure.

I have tried write the code inspired from this in the github issue, but it didn’t work. I have tried multiple tflite model exported for qcs6490 as well as the model provided in the github issue which might worked on some other sbc having qcs6490 as mentioned in the github issue.

Please help me fix this. Or let me know if there is any proper documentation to use tflite delegate in python to run tflite models on NPU / HTP.

2

Dear customer,we are currently syncing this issue internally.

3

Dear customer,
Qualcomm supports the tflite C/C+ interface, you may consider utilizing the C/C+ interface.
You can refer to the following documentation for more information:
https://docs.qualcomm.com/bundle/publicresource/topics/80-70014-54/arch.html
https://docs.qualcomm.com/bundle/publicresource/topics/80-70017-15BY/tflite.html
https://docs.qualcomm.com/bundle/publicresource/80-50450-52_REV_AB_Qualcomm_TensorFlow_Lite_SDK_Tools_Quick_Start_Guide.pdf

4

the pdf doc you shared has command examples for android only. One of the command line argument is either only available on the android but not on the debian os that I have on my rubikpi, or it is removed in newer version so don’t work.
The command that I tried to test if it is working on rubik pi is:
benchmark_model --graph=mobilenetv2.tflite --external_delegate_path=libQnnTFLiteDelegate.so --externel_delegate_options='backend_type:htp;library_path:/usr/lib/libQnnHtp.so;skel_library_dir:/usr/lib/rfsa/adsp' and I had mobilenetv2.tflite and libQnnTFLiteDelegate.so in the working directory, but it threw error as:

INFO: STARTING!
WARN: Unconsumed cmdline flags: --externel_delegate_options=backend_type:htp;library_path:/usr/lib/libQnnHtp.so;skel_library_dir:/usr/lib/rfsa/adsp
INFO: Log parameter values verbosely: [0]
INFO: Graph: [mobilenetv2.tflite]
INFO: External delegate path: [libQnnTFLiteDelegate.so]
INFO: Loaded model mobilenetv2.tflite
INFO: EXTERNAL delegate created.
ERROR: Restored original execution plan after delegate application failure.
ERROR: Failed to apply EXTERNAL delegate.
ERROR: Benchmarking failed.

I am trying to go through multiple qualcomm docs to find something that can help me run any custom model on this sbc, but they don’t seem to have clear instruction for sbc based on qcs6490 chip. If you have some working code / program that builds and runs model from scratch, please share.

5

Dear customer,
Please refer to the following documentation, which we hope will be of assistance to you:
https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/tflite_delegate.html