Hexagon compilation and debug

Technical Sharing
1

Using RubikPI for Hexagon Development

I wanted to share the steps I followed to compile and debug some trivial code on the RubikPI targeting the Hexagon DSP.

  • Download and install the HexagonTools on the RubikPI using qpm-cli

       qpm-cli --download-only hexagon19.0
   qpm-cli --install Hexagon19.0.19.0.06.1.Linux-ARM64.qik

  • I could not get the HexagonSDK to load on the RubkiPI so I loaded on my Windows machine, you only need a few things from the SDK.
    qpm-cli --install hexagonsdk6.x

  • Make sure fastrpc is installed on the RubikPI
    sudo apt-get install qcom-fastrpc1
    * If you have to install qcom-fastrpc1 you may need to reboot after it is installed.

  • Copy run_main_on_hexagon and librun_main_on_hexagon_skel.so from the SDK install to the target.

    • <HEXAGON_SDK_ROOT>/libs/run_main_on_hexagon/ship/UbuntuARM_aarch64/run_main_on_hexagon

    • <HEXAGON_SDK_ROOT>/libs/run_main_on_hexagon/ship/hexagon_toolv19_v68/librun_main_on_hexagon_skel.so

      • To keep things simple make a directory $HOME/hexagon and copy these files to that directory.

At this point you have enough to compile and run a simple program.

#include <stdio.h>
#include <stdlib.h>

int main() {
    FILE *fd = fopen("output.txt", "w");
    if (fd == NULL) {
        perror("Failed to open file");
        return EXIT_FAILURE;
    }

    fprintf(fd, "Hello World!\n");
    fclose(fd);

    return 0;
}

Makefile:

CC=$(HOME)/Qualcomm/HEXAGON_Tools/19.0.06/Tools/bin/hexagon-clang

hello.so: hello.c
	$(CC) -g -O0 -shared -fpic -mv68 hello.c -o $@

run: hello.so
	./run_main_on_hexagon 3 hello.so stack_size=0x50000
	@echo
	@echo output.txt
	@cat output.txt

clean:
	rm -f output.txt hello.so

The directory would contain the following files:

# ls -1
Makefile
hello.c
librun_main_on_hexagon_skel.so
run_main_on_hexagon

Running the program should produce the following output:

./run_main_on_hexagon 3 hello.so stack_size=0x50000
Attempting to run on unsigned PD on domain 3
RPC to Hexagon DSP with args: "hello.so stack_size=0x50000 "
Successfully called main() on Hexagon DSP and received return value of 0.

output.txt
Hello World!

Debugging

This requires a few more tools from the SDK and manually loading a kernel module.

Load the kernel module, (make sure you match the kernel version and needs to be done before running the debug agent)

sudo insmod /lib/modules/6.8.0-1054-qcom/kernel/drivers/char/rdbg.ko.zst

Copy remote_debug_agent_cdsp from the Hexagon SDK to the RubikPI:

  • <HEXAGON_SDK_ROOT>/tools/debug/remote_debug_agent/le/remote_debug_agent_cdsp
# sudo ./remote_debug_agent_cdsp 
Calling Agent Constructor
Calling Image Constructor For Image META_PROC
Calling Session Constructor For PID 0
Completed Agent Constructor
Press q or CTRL-C to quit.
=========================================================
Created Port For Session PID:0 Name:meta Port 56283
=========================================================

Once this is running you can check it with lynx or connect to the rubikpi via a web-browser. I used lynx

lynx localhost:56283

If everything is working you should see this:

                                                              Remote Debug Agent
                   Welcome to the Remote Debug Agent Web Page

List of Remote Processes

      CPU     Subsystem PID Process Name Debug Port
   Hexagonv68 CDSP      0   _ASID0_      37131

Remote Processes List Data Files

    1. Process List XML: pslist.xml








Commands: Use arrow keys to move, '?' for help, 'q' to quit, '<-' to go back.
  Arrow keys: Up and Down to move.  Right to follow a link; Left to go back.
 H)elp O)ptions P)rint G)o M)ain screen Q)uit /=search [delete]=history list

To actually debug a program you must set an environment variable FASTRPC_PROCESS_ATTRS to 2.

export FASTRPC_PROCESS_ATTRS=2
./run_main_on_hexagon 3 hello.so stack_size=0x50000

Attempting to run on unsigned PD on domain 3
RPC to Hexagon DSP with args: "hello.so stack_size=0x50000 "

Do CTRL-R in the lynx window and you should see something like this:

                                                              Remote Debug Agent
                   Welcome to the Remote Debug Agent Web Page

List of Remote Processes

      CPU     Subsystem PID          Process Name          Debug Port
   Hexagonv68 CDSP      3   /frpc/f05adb50 run_main_on_hex 38389
   Hexagonv68 CDSP      0   _ASID0_                        37743

Remote Processes List Data Files

    1. Process List XML: pslist.xml







Commands: Use arrow keys to move, '?' for help, 'q' to quit, '<-' to go back.
  Arrow keys: Up and Down to move.  Right to follow a link; Left to go back.
 H)elp O)ptions P)rint G)o M)ain screen Q)uit /=search [delete]=history list

Now attach to the program using hexagon-lldb. It is not obvious, but the program you are going to be debugging is named fastrpc_shell_unsinged_3.
hexagon-lldb /usr/lib/dsp/cdsp/fastrpc_shell_unsigned_3

At the lldb prompt use the command “gdb-remote” to attach, and set a breakpoint in main, should look like this:

(lldb) gdb-remote 38389
Process 3465 stopped
* thread #1, stop reason = signal SIGTRAP
    frame #0: 0x00d1f13c fastrpc_shell_unsigned_3`qurt_ptrace + 4
fastrpc_shell_unsigned_3`qurt_ptrace:
->  0xd1f13c <+4>:    { 	jumpr r31 }

fastrpc_shell_unsigned_3`qurt_sysenv_get_impl:
    0xd1f140 <+0>:    { 	trap0(#0x1a) }
    0xd1f144 <+4>:    { 	jumpr r31 }

fastrpc_shell_unsigned_3`qurt_thread_secure_mode_start:
    0xd1f148 <+0>:    { 	immext(#0xa82c0)

fastrpc_shell_unsigned_3`qurt_thread_secure_mode_start:
    0xd1f14c <+4>:      	r0 = add(pc,##0xa82ec) }
(lldb) b main
Breakpoint 1: where = fastrpc_shell_unsigned_3`main, address = 0x00d9fee8
(lldb) c
Process 3465 resuming
1 location added to breakpoint 1
Process 3465 stopped
* thread #9, stop reason = breakpoint 1.2
    frame #0: 0x2000a41c hello.so`main at hello.c:5:16
   2   	#include <stdlib.h>
   3   	
   4   	int main() {
-> 5   	    FILE *fd = fopen("output.txt", "w");
   6   	    if (fd == NULL) {
   7   	        perror("Failed to open file");
   8   	        return EXIT_FAILURE;
(lldb)

Errors you might see

If you see this error double check that qcom-fastrpc1 has been installed. If you just installed it you may need to reboot.

ERROR 0x72: FastRPC Capability API failed. Falling back to signed pd.Unsigned PD is not supported on domain 3.
Error 1: Failed to call main() on DSP
Failed to close handle

If you see this error you might have an issue with DSP_LIBRARY_PATH or be missing librun_main_on_hexagon_skel.so:

Attempting to run on unsigned PD on domain 3
Domain 3 failed to open (0x80000406), run_main_on_hexagon_URI_domain: file:///librun_main_on_hexagon_skel.so?run_main_on_hexagon_skel_handle_invoke&_modver=1.0&_dom=cdsp
Error -2147482618: Failed to call main() on DSP
Failed to close handle
2

Thank you for sharing this and for the detailed write-up.

1 Like