Repository navigation
First speed results: kuna, rsleigh and Ghidra on the same three files #3858
CassieFox23
started this conversation in
Show and tell
Replies: 3 comments
|
TL;DR: I read the C from three decompilers beside the disassembly, 21 functions in all. Of those, kuna matched on 19 and was close on 2. Ghidra matched on 14 of its 16. None of rsleigh's 21 was fully right. This is one reader, so it is not a benchmark. Edit: five follow-ups are in below, as Updates 1 to 5. Update 4 corrects what I said about kuna's case 3: it was right. The tools are kuna (Apache-2.0), rsleigh (Apache-2.0) and Ghidra (Apache-2.0). This covers the "Reading the C for quality" item in my Not run list above. I did not compile any of the C here. Update 1 does. The saved outputs were gone, so I re-ran the one-function job once per tool. The kuna and rsleigh byte counts match the timed runs. The Ghidra count is within 18 of its timed run (the log adds a line). The functions:
CrashLogger function:
On these two functions, Ghidra was the most faithful. The kuna C was the most readable.
Edit: kuna's case 3 is fine (Update 4). Update 3 says why rsleigh's CrashLogger C is thin. Update 1: does the C compile?I fed 14 outputs to gcc 16.2.1 and clang 22.1.8. The flags were
Passing is a low bar. It doesn't show the code does the right thing. What I compiled, and every stub I added
kuna's two failing lines: (*(void *)swi(0x29))(5);
return (unsigned long long *)(*v4)(); // v4 is a void *Update 2: five APoseFix functions (kuna and rsleigh)I read five APoseFix functions beside the disassembly. Ghidra is not in this one. It did not finish on this file, so its column waits for the overnight rerun.
kuna dropped nothing here. The cost is clutter. The first one has about 35 stack variables. There are made-up struct names and some Each function, one row per tool
How I picked and read them
Update 3: why rsleigh's CrashLogger C is nearly emptyThe function is an MSVC
The jump, and why the finder misses itmovsx rax, byte [rdx+0x48]
lea r8, [image base]
mov eax, [r8+rax*4+0x6a650]
add rax, r8
jmp raxRead from rsleigh-decompile/src/jump_table.rs and the raw P-code at 13765bf. I did not test a patched build, so which of these is enough is not known:
Update 4: kuna's case 3 was right after allI was wrong above. kuna didn't lose lines in case 3. The lines it left out can never run.
Case 3 in the disassembly (trimmed, comments mine)18006a5a9: lea rax,[rbp-0x48] ; first temp
18006a5ad: mov [rbp-0x50],rax ; its address, at rsp+0x20
18006a5b1: movups xmm0,[rbp-0x48] ; move first temp
18006a5b5: movups [rbp-0x28],xmm0 ; into the second
18006a5c5: mov qword [rbp-0x30],0xf ; first temp's capacity = 15
18006a5e0: call 0x180092b30 ; both calls get the second temp
18006a5ea: call 0x180095bb0
18006a5f0: mov r8,[rbp-0x30]
18006a5f4: cmp r8,0xf
18006a5f8: jbe 0x18006a572 ; always taken, so the free is skippedUpdate 5: ten more functions, picked at randomI read ten more functions beside the disassembly, five from
Each function, one row per tool
I also read four more How I picked and scored them
Not runEdit: the follow-ups are done (Updates 1 to 5). What is below is still not run. The overnight Ghidra rerun is in this comment.
The C, quoted line by line
v4 = a0->field_0xa8;
if (((a1->field_0xa8 != 3) && (a1->field_0xa8 != 9)) && ((a1->field_0xac & 0xf000) != 0x4000)) {
if (v4 == 9 || v4 == 3)
return 0xffffffff;
...
return strcmp(v2,v1); // tail-call
if (iVar2 == 3) {
if (param_0->field_a8 == 9) {
...
if (iVar3 & 0xf000) != 0x4000 /* !S_ISDIR */) {
iVar1 = *(int *)(param_1 + 0x15);
if (((*(int *)(param_2 + 0x15) == 3) || (*(int *)(param_2 + 0x15) == 9)) ||
((*(uint *)((long)param_2 + 0xac) & 0xf000) == 0x4000)) {
if (iVar1 == 9 || iVar1 == 3) goto LAB_00103e85;CrashLogger, kuna. Case 3 has the same lines as case 2 (Update 4 says why that is right). The disassembly for case 3 has a second 32-byte move and a free after the two calls: void sub_18006a500(unsigned long long *a0,void *a1) // return-dupe x2
...
case 2:
...
sub_180092b30((long long)a1 + 0x18,&v1);
sub_180095bb0(&v1);
return;
case 3:
(the same lines as case 2)CrashLogger, rsleigh. This is the whole body. long func_18006a500(long param_0, long param_1, float fparam_0) {
int iVar1;
iVar1 = *(uint32_t*)(0x6a650 + DAT_180000000 + (char)param_1->field_48 * 4);
}CrashLogger, Ghidra: switch(*(undefined1 *)(param_2 + 0x48)) {
case 2:
...
FUN_180092b30(param_2 + 0x18,&local_30);
FUN_180095bb0(&local_30);
break;
case 3:
...
local_50 = (void *)(local_30 & 0xffffffffffffff00);
...
if (0xf < uStack_38) {None of the three shows the call that the jump table sends index -1 to. I did not trace that call. How I read them
Test setup
🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊 |
0 replies
|
An update: REA is adding kuna support.
🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊 |
0 replies
|
TL;DR: kuna finished all five overnight runs. rsleigh did not finish all its functions in 90 minutes. Ghidra ran out of the 12 GB memory limit on all three jobs, even with a 4 GB heap. These are the overnight runs for the Not run list in the first post. The tools are kuna, rsleigh and Ghidra, all Apache-2.0. Two tools hit a cap. A cap hit is not a speed, so rsleigh and Ghidra have no time for the 10 MB file. The two files are the ones from the first post: CrashLoggerSSE 1.25.0 (MIT) and Skyrim A-Pose Fix 1.0.0.0 (GPL-3.0). Both are 64-bit x86 Windows DLLs.
One pass each on the 10 MB file:
Three runs on the 3.2 MB file, median (min to max):
The caps in every run:
What the logs show about the two that did not finish:
Still not run
How these compare to the first postLoad is the average number of programs running or waiting for a core in the last minute. This box has 24 threads. It was 0.19 to 0.93 at the start of every overnight run. I sampled it only at the start and the end.
Memory and process counts for every runPeak memory is in MB: the largest process from
Commands
Test setup
🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊 |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
TL;DR: I timed kuna (Apache-2.0), rsleigh (Apache-2.0) and Ghidra (Apache-2.0) on the same three files and the same three jobs. On the 3.2 MB file, kuna and rsleigh decompile one function in under a second, and Ghidra needs 38 seconds. Every function to C takes kuna 24 to 86 seconds and Ghidra 100 seconds, and rsleigh did not finish in 40 minutes.
These are first results, and more tests are coming. Ghidra on the 10 MB file will be rerun with a smaller heap on a quiet box. Every Ghidra run there hit our 12 GB memory cap, and one then ran to the 40 minute limit. Those cells say "did not finish under our cap". They are not a Ghidra speed.
Some rows ran while other test jobs shared the box. Load is the average number of programs running or waiting for a core in the last minute, and this box has 24 threads. A row is "under load" when the 1-minute load went over 4. One of the three 8-worker CrashLogger runs was under load. All three took 23.4 to 23.9 s.
The files:
ls(GPL-3.0-or-later), 153 KB.The jobs are one function, list functions, and every function to C. CrashLogger, median of 3 runs, Ghidra heap 8 GB:
decompile-alltook 16 min 27 s anddecompile-projecttook 30 min 46 s. rsleigh listed 17,730 functions in 1.4 s.Not run
Edit: the overnight runs are in this comment. The two kuna runs are done. Ghidra and rsleigh ran but hit their caps, so they still have no result. The C quality read is done, in this comment.
kunaRun overnight: 26 min 57 s, exit 0.--mode faston the 10 MB file: the default mode already picks fast there.kunaTimed overnight withdecompile-project --jobs: not timed.decompile-all --jobs 8was.--jobs 8: 17 min 42 s on the 10 MB file, 26.84 s (median of 3) on the 3.2 MB file.Reading the C for quality."With C" means not empty and no error.Full tables for CrashLogger, the 3.2 MB file, with 3 runs per row
Peak memory is in MB: the largest process, then the whole run. Under load: "yes" means the load went over 4 at some point. "No" means I sampled it and it never did. "Start only" means it was 4 or less at the start, and I did not sample during the run. On one-function rows, errors 0 means the exit code was 0. I did not check the text.
time -v/ cgroupdecompile FILE ADDR --addrFILE ADDRanalyzeHeadless+BenchDecompOne(60 s limit)analyzeHeadless+BenchCountFunctionsfunctions FILE --jsonfunctions FILE --json --mode fastFILE(list)analyzeHeadless+BenchDecompAll(10 s limit)decompile-project FILE -o DIRdecompile-project FILE -o DIR --mode fastdecompile-all FILE --json --jobs 8FILE --all --jsonGhidra's own analysis took 32 to 34 s of each Ghidra row.
Full tables for ls, the 153 KB file, with 3 runs per row
time -v/ cgroupdecompile FILE ADDR --addrFILE ADDRanalyzeHeadless+BenchDecompOne(60 s limit)analyzeHeadless+BenchCountFunctionsfunctions FILE --jsonfunctions FILE --json --mode fastFILE(list)analyzeHeadless+BenchDecompAll(120 s limit)analyzeHeadless+BenchDecompAll(10 s limit)decompile-project FILE -o DIRdecompile-project FILE -o DIR --mode fastdecompile-all FILE --json --jobs 8FILE --all --jsonGhidra's own analysis took 6 s of the 10 to 11 s. The first Ghidra decompile rows used a 120 s limit per function. The second set used 10 s, to match the other files.
Full tables for APoseFix, the 10 MB file, with one pass per row
The kuna one-function and decompile commands also had
--max-fn-memory 649.time -v/ cgroupdecompile FILE ADDR --addrFILE ADDRanalyzeHeadless+BenchDecompOne(60 s limit)analyzeHeadless+BenchCountFunctionsfunctions FILE --jsonFILE(list)analyzeHeadless+BenchDecompAll(10 s limit)decompile-project FILE -o DIRdecompile-all FILE --json --jobs 8In kuna
decompile-all, 319 errors came from the memory limit. 189 came from the 10 s limit. 1 came from a stalled worker. 3 were internal errors. kuna's own warning says heavy functions can trip the 10 s limit under load. This run was under load, so some of the 189 may not repeat on a quiet box (unverified).Two Ghidra runs ended at the cap after about 75 s (exit 137, the kernel ended the Java process). The third ran to the 40 minute limit with the cap in use. None printed a count, so I do not know how far analysis got.
What each job measures, the limits and the load
functions. rsleigh with no address. For Ghidra, import and auto-analysis, then a script that counts the functions.lsrows first used 120 s.decompile-projectruns 1 worker by default.decompile-all --jobs 8runs 8. rsleigh uses--all --json.decompile-allhas a 10 s limit per function on the two Windows files. Its own warning says so. Ghidra's heap was 8 GB inside a 12 GB cap, with swap off.--mode fastruns 2 and 3 started at 4.14 and 3.55. They took 92.8 s and 89.5 s, against 92.4 s for run 1.decompile-projectrows had a mean load of 9 to 16, with a peak of 24.decompile-projecttotal differs from itsfunctionscount (14,543 against 14,873 on the 10 MB file). I have not reconciled the counts.Commands, with the exit code of every run
/usr/bin/time -v.Kis the kuna binary andRis the rsleigh binary.GRwraps Ghidra'sanalyzeHeadless, with a fresh project that is deleted at the end.Fis the file.DIRis an output folder.LABELis the run name.ADDRis the function address:0x3e10for ls,0x18006a500for CrashLogger.GADDRis0x103e10for ls, because Ghidra puts the image base at0x100000. For CrashLogger it equalsADDR.ls-A-kuna-r1..3,cl-A-kuna-r1..3$K decompile $F $ADDR --addrap-A-kuna-r1$K decompile $F 0x1800c2bd0 --addr --max-fn-memory 649*-A-rsleigh-r1..3(ap-A-rsleigh-r1)$R $F $ADDR*-A-ghidra-r1..3$GR LABEL $F one $GADDRap-A-ghidra-r1$GR LABEL $F one 0x1800c2bd0*-B-ghidra-r1..3$GR LABEL $F count $GADDRap-B-ghidra-r1$GR LABEL $F count 0x1800c2bd0*-B-kuna-r1..3,ap-B-kuna-r1$K functions $F --jsonls-B-kuna-fast-r1..3,cl-B-kuna-fast-r1..3$K functions $F --json --mode fast*-B-rsleigh-r1..3,ap-B-rsleigh-r1$R $Fls-C-ghidra-r1..3$GR LABEL $F all 120ls-C-ghidra-t10-r1..3$GR LABEL $F all 10cl-C-ghidra-r1..3$GR LABEL $F all 10ap-C-ghidra-r1$GR LABEL $F all 10*-C-kuna-proj-r1..3,ap-C-kuna-proj-r1$K decompile-project $F -o DIR(ap adds--max-fn-memory 649)ls-C-kuna-proj-fast-r1..3,cl-C-kuna-proj-fast-r1..3$K decompile-project $F -o DIR --mode fast*-C-kuna-dall-j8-r1..3,ap-C-kuna-dall-j8-r1$K decompile-all $F --json --jobs 8(ap adds--max-fn-memory 649)ls-C-rsleigh-all-r1..3$R $F --all --jsoncl-C-rsleigh-all-r1$R $F --all --jsonls-*-ghidracells, 4:19 PM$GR ...Test setup
c5ef54156f4eff3d94769dd839ca465ad5c302293d397fd452fb805f1b32812227ce4c2d8d070b4216fdef0e762c97379354adb4bf8a439d4b8b6484b02ebd8254e936d37decdad9ecaf189b1de347605a0a99037ec87bd17679b6cbde871a0c🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊
All reactions