I’m trying to run inference on the RUBIK Pi (Debian 13, v1.0.1) using TFLite in C++, and I would like to utilize the DSP (HTP).
I first attempted to use the Hexagon delegate, but I was unable to successfully create the delegate instance. I then tried the QNN delegate. When specifying kHtpBackend, I am able to create the delegate and attach it to the interpreter, but inference fails.
The model I’m using is a quantized ResNet50 (INT8). The same code works correctly with the XNNPACK delegate and the GPU delegate. The issue occurs only when using the HTP backend. I also verified that my QNN delegate construction code matches the example provided in the official Ubuntu documentation, but the HTP backend still fails on the RUBIK Pi.
Additionally, when I run inference using the standalone QNN runtime examples, there are no errors, so I believe the hardware and QNN libraries are installed correctly.
Has anyone successfully used the DSP (Hexagon or QNN / HTP) with TFLite on the RUBIK Pi? Any information about required configuration steps, supported model formats, or environment setup would be greatly appreciated.
First, I ran qnn-platform-validator on my current environment (Debian 13) to test QNN.
The DSP unit test failed with the following output:
PF_VALIDATOR: DEBUG: Starting calculator test
PF_VALIDATOR: DEBUG: Loading sample stub: libQnnHtpV68CalculatorStub.so
PF_VALIDATOR: DEBUG: Successfully loaded DSP library - ‘libQnnHtpV68CalculatorStub.so’. Setting up pointers.
PF_VALIDATOR: ERROR: -2 . Error while executing the sum function.
PF_VALIDATOR: ERROR: Please use testsig if using unsigned images.
PF_VALIDATOR: ERROR: Also make sure ADSP_LIBRARY_PATH points to directory containing skels.
Unit Test on the backend DSP: Failed.
QNN is NOT supported for backend DSP on the device.
*********** Results Summary ***********
Backend = DSP
{
Backend Hardware : Supported
Backend Libraries : Found
Library Version : Not Queried
Core Version : Hexagon Architecture V68
Unit Test : Failed
}
Based on this output, does it mean that the DSP backend is currently not supported on Debian 13?
Secondly, I ran qnn-platform-validator on a new RUBIK Pi board running Ubuntu, and the DSP unit test passed:
Backend = DSP
{
Backend Hardware : Supported
Backend Libraries : Found
Library Version : Not Queried
Core Version : Hexagon Architecture V68
Unit Test : Passed
}
I also tested the same setup on Ubuntu by installing the same LiteRT version (v1.4.0) that I used on Debian 13, and running the same code. However, I encountered two new issues:
QNN delegate only partially offloads operators
The code runs, but the QNN delegate with kHtpBackend option is unable to offload the CONV_2D operators in the ResNet50 INT8 model, which is unexpected. The following errors appear:
Initializing HtpProvider
/prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.39.0/release/snpe_src/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38: dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc
ERROR: [Qnn] [4294967295] has incorrect Value 1435628880, expected equal to 0.
ERROR: [Qnn] QnnBackend_validateOpConfig failed 3110
ERROR: [Qnn] Failed to validate op node_id_0_op_type_Conv2d_op_count_0 with error 0xc26
However, for operators that are successfully delegated, the DSP appears to execute them correctly. For example:
With verbose logging enabled, I can see execution traces indicating that the DSP is running, and the final inference outputs are correct.
Still, since convolution layers fail while some operations succeed, it is unclear whether the delegate is functioning partially or whether additional configuration is required.
I also tested using the kGpuBackend option, but in this case no operators were offloaded, and the following errors were observed:
Could this be an issue related to the model format or the conversion process?
It is hard to believe that these common operators are not intended to be offloaded.
Unable to build the GPU delegate on Ubuntu
On Ubuntu, I was not able to build the shared library for the GPU delegate. I will investigate further (this may be related to my build scripts), but in order to narrow it down, could you confirm which LiteRT version has been validated and confirmed working on Ubuntu?
@rosh Thank you for the information. Regarding the “qnn-tools,” are you referring to tools such as qnn-platform-validator? I found these already included in the OS image that I flashed onto the RUBIK Pi. Do I still need to install them separately?
For the QNN delegate header files, I downloaded them from Qualcomm’s website. The corresponding shared libraries also appear to be included under /usr/lib in the OS image.
@rosh Thank you very much for the guidance. It seems that most of the technical issues I encountered on Debian 13 are now resolved, and I expect the remaining ones will be as well once I apply the suggested steps.
Regarding the operator offloading issue, I will test with other models. I suspect the problem may be related to the kernel size of the convolution operators in the current model.
