QLiberator decompiler

Anything QL Software or Programming Related.
Post Reply
RWAP
RWAP Master
Posts: 2834
Joined: Sun Nov 28, 2010 4:51 pm
Location: Stone, United Kingdom
Contact:

Re: QLiberator decompiler

Post by RWAP »

Yes, this is some excellent progress - and the decompilers provide great tools for future preservation projects


Martin_Head
Aurora
Posts: 847
Joined: Tue Dec 17, 2013 1:17 pm

Re: QLiberator decompiler

Post by Martin_Head »

EmmBee wrote:The manual says: "The minimum size of a channel table is 3 entries, for channels 0, 1 and 2" - so its the actual number of channels that need to be given.
CMD$ is initially set up by QLiberator as a variable to read the incoming Option command string and can be used as a normal variable afterwards.

Sounds like you're making excellent progress. Are you compiling with QDOS or SMSQ/E?
In the source code I have a 'REMark $$chan=9' But after compiling, at the start of the program, see the code snippet above, the number of channels should be 9, but I get 10.

The decompiler assigned CMD$ it's own variable name. But after recompiling, QLiberator got upset about trying to read a string (the wrongly named CMD$) that should have something in it.

I wrote and compiled all my test programs for the decompiler in QDOS under Qemulator. But I have started using QPC2 to recompile QLiberator and the Patch program, due to the amount of time it takes in Qemulator. It takes minutes just to load the QLIB_BAS program, and more minutes to compile it. Whereas in QPC2 the compile time is like 1 second with the BASIC compiler compiling itself, and 0 seconds with the original Qliberator. When using a _WRK file.


EmmBee
Trump Card
Posts: 240
Joined: Fri Jan 13, 2012 5:29 pm
Location: Kent

Re: QLiberator decompiler

Post by EmmBee »

I am using QLiberator version 3.36.
When I compile, in the top right-hand corner I get the message version 3.35
I've just tried compiling a program with a REMark $$chan=9
and QLiberator reports: Highest channel : 9


Martin_Head
Aurora
Posts: 847
Joined: Tue Dec 17, 2013 1:17 pm

Re: QLiberator decompiler

Post by Martin_Head »

EmmBee wrote:I am using QLiberator version 3.36.
When I compile, in the top right-hand corner I get the message version 3.35
I've just tried compiling a program with a REMark $$chan=9
and QLiberator reports: Highest channel : 9
What does the QLIB_PATCH program report as the number of channels, or if you disassemble the start of the program and look at the stored number of channels.

I think I see the the same as you describe, QLiberator reports 9 channels, but 10 is actually stored in the program.

Also I'm pretty sure (I'm not sitting at the right computer at the moment), My Qliberator reports it's version as 3.36. And the PQLIB_PATCH program is 3.35


User avatar
pjw
QL Wafer Drive
Posts: 1286
Joined: Fri Jul 11, 2014 8:44 am
Location: Norway
Contact:

Re: QLiberator decompiler

Post by pjw »

Im using V3.36. It reports V3.36 during compilation. QLib_obj has 9 channels stored in location $32. Other compiled programs appear to have $$chan=n + 1 channels stored in the program.


Per
dont be happy. worry
- ?
User avatar
RalfR
Aurora
Posts: 870
Joined: Fri Jun 15, 2018 8:58 pm

Re: QLiberator decompiler

Post by RalfR »

Martin_Head wrote:There is one other odd thing I have noticed when comparing the original QLIB with my recompiled one's. There is a bit of data stored between the end of name list, and the start of the name/variable table
Perhaps this is something with the mouse handling. Marcel has cleaned one of the extensions in QLib for me (The extension for the mouse things) and they just work, if a bit of code is set in the heap, perhaps they store the mouse coordinate there.

Marcel may better clarify this.


4E75 7000
Martin_Head
Aurora
Posts: 847
Joined: Tue Dec 17, 2013 1:17 pm

Re: QLiberator decompiler

Post by Martin_Head »

I think I have figured out what the 'something odd' is. It's 'External Procedures and Functions'. In the QLiberator manual around page 14.9 it talks about using QLIB! to start the compiler from BASIC. I've just got to figure out exactly how it works, to get the decompiler to recognize it.

I think the description of $$chan= , in the manual. Should say something like, It's the highest channel number used by the program. That gives you the extra 1 for channel 0.
QLiberator uses channels 3,4,5,6, and 7. So if you specify $$chan=8, then you either leave one spare channel, or maybe one of the built in SuperBASIC extensions uses a channel.

Looking in the source for QLIB_BAS I find

Code: Select all

1850 var04E8$ = "02-NOLINE,03-STAT,04-NODIS,05-NONAMES,06-AUTOF,07-RUN,08-DEBUG,09-NOBEEP,10-NOWINDS,11-TRACE,20-OBJ,21-LIST,23-NAME,"
which looks like the list of instructions for CMD$.
They don't quite match my QLiberator manual. -WINDS is missing. But -NOBEEP, -NOWINDS, and -TRACE are fair enough. But what's -NODIS. No display??

Anyone fancy playing with it, to find out what it does?


User avatar
pjw
QL Wafer Drive
Posts: 1286
Joined: Fri Jul 11, 2014 8:44 am
Location: Norway
Contact:

Re: QLiberator decompiler

Post by pjw »

The first time I noticed there was a "no display" option was with some software that came from JMS. It was a Thing allowing you to compile a program directly from QD. Its a long time ago, and I didnt use it for real, so no details. JMS may have patched QLib? It seemed so at the time..


Per
dont be happy. worry
- ?
User avatar
mk79
QL Wafer Drive
Posts: 1349
Joined: Sun Feb 02, 2014 10:54 am
Location: Esslingen/Germany
Contact:

Re: QLiberator decompiler

Post by mk79 »

I have most of this stuff already figured out, but you're too fast for me ;) i'm too busy with other stuff right now, but here's the start of QLIB_obj as I currently understand it

Code: Select all

base:           bra.w   qlib_start
; ---------------------------------------------------------------------------
                dc.w 0
                dc.w $4AFB
                dc.w 9
                dc.b 'Qlib_3.35'
                dc.b '             '
                dc.l token_base-base
                dc.l name_table-base
                dc.l 0                  ; Line table
                dc.w 4
                dc.l data_tokens-base   ; First DATA token
                dc.w 9                  ; Channel count
                dc.w $80                ; Buffer size
                dc.w 512                ; Heap chunk size
                dc.l 7000               ; Heap size
                dc.w 916                ; Return stack size
                dc.w 816                ; Name table entry of CMD$ variable
                dc.l name_list-base     ; Name list
                dc.l ext_qutil-base     ; Extensions
                dc.l 20386              ; Data space size
                dc.l 5500               ; Stack size
                dc.b 0                  ; STATS
                dc.b 1                  ; AUTOF
                dc.l externals-base     ; External procedures
                dc.w 1
                dc.w $FF00              ; WINDS
                dc.l 0
                dc.l 0
                dc.l 0
                dc.l 0
                dc.l 0
                dc.l 0
                dc.l 0
                dc.l 0
                dc.l 0
                dc.l 0
                dc.w 0
; ---------------------------------------------------------------------------
job_start:
                bra.w   job_entry
; ---------------------------------------------------------------------------
external_entry:
                bra.w   external_entry_2

job_entry:
                moveq   #sms.info,d0
                trap    #1
                lea     base(pc),a1
                cmpa.l  a1,a6
                beq.s   job_do          ; Normal jobs have a1 = a6
                cmpa.l  2(a6),a1        ; Started by JMP trampoline?
                bne.s   init_externals  ; LRESPRed extension, just initialise externals
job_do:
                moveq   #0,d6
                suba.l  a4,a5
                move.l  a5,d4
                adda.l  a6,a4
                movea.l a1,a5
                bsr.w   check_embedded_rte
                beq.s   job_rte_init
                movea.l sys_sbab(a0),a0 ; 'QL SuperBASIC' Area Base
                adda.w  #sb_offs,a0     ; offset from JCB to SuperBASIC vars
                move.l  sbq_runtime(a0),d0 ; System RTE?
                beq.s   no_system_runtime
                movea.l d0,a0

job_rte_init:   jsr     8(a0)           ; Main runtime entry
job_exit:       moveq   #-1,d1
                moveq   #sms.frjb,d0    ; Forced Remove JoB
                trap    #1

no_system_runtime:
                bsr.s   write_rte_missing
                bra.s   job_exit
; End of function base


; =============== S U B R O U T I N E =======================================
write_rte_missing:
                lea     txt_runtime_missing(pc),a1 ; "\nRuntimes missing !\n"
                suba.l  a0,a0
                movea.w (ut.wtext).w,a2 ; Write TEXT
                jsr     (a2)
                rts
; End of function write_rte_missing

; =============== S U B R O U T I N E =======================================
check_embedded_rte:
                lea     embed_rte_base(pc),a1
                cmpi.w  #$6000,(a1)     ; bra.l
                bne.s   no_embedded_rte
                cmpi.w  #$6000,4(a1)    ; bra.l
                bne.s   no_embedded_rte
                movea.l a1,a0
no_embedded_rte:rts
; End of function check_embedded_rte


; =============== S U B R O U T I N E =======================================
; Token address of external in d4

externals_entry:
                lea     base(pc),a4
                bsr.s   check_embedded_rte
                beq.s   jmp_embedded_rte
                move.l  sbq_runtime(a6),d0 ; Try to get system RTE
                beq.s   rte_not_found
                movea.l d0,a0
jmp_embedded_rte:
                jmp     $10(a0)
; ---------------------------------------------------------------------------
rte_not_found:
                bsr.s   write_rte_missing
                bra.s   exit
; ---------------------------------------------------------------------------
init_externals:
                movea.l a1,a4
                move.l  $52(a4),d4      ; External proc/fn list
                bra.s   init_externals_start
; ---------------------------------------------------------------------------

init_exernals_loop:
                move.l  (a5),d4

init_externals_start:
                beq.s   exit
                lea     (a4,d4.l),a5
                movea.l a5,a1
                addq.l  #4,a1
                move.l  a4,-(sp)
                movea.w (sb.inipr).w,a2 ; INITialise PRocedure table
                jsr     (a2)
                movea.l (sp)+,a4
                bra.s   init_exernals_loop
; ---------------------------------------------------------------------------
exit:           moveq   #0,d0
                rts
; End of function externals_entry

; ---------------------------------------------------------------------------
txt_runtime_missing:
		dc.w 20
                dc.b $A
                dc.b 'Runtimes missing !',$A
I have disassembled most of the runtime, the other libraries of qlib_sys and the private QLIB_obj extensions. And with "disassembled" I mean the code is probably better commented than the original source code ever was ;) But it's still so much work to do and I'm at a point where it frankly gets hard to justify doing this just for fun. Maybe I need to start a gofundme campaign or something like that :mrgreen:

Cheers, Marcel


Martin_Head
Aurora
Posts: 847
Joined: Tue Dec 17, 2013 1:17 pm

Re: QLiberator decompiler

Post by Martin_Head »

mk79 wrote:I have most of this stuff already figured out, but you're too fast for me ;) i'm too busy with other stuff right now, but here's the start of QLIB_obj as I currently understand it

Code: Select all

                dc.l 0                  ; Line table
                dc.w 4
                dc.l data_tokens-base   ; First DATA token


                dc.w $FF00              ; WINDS
I have disassembled most of the runtime, the other libraries of qlib_sys and the private QLIB_obj extensions. And with "disassembled" I mean the code is probably better commented than the original source code ever was ;) But it's still so much work to do and I'm at a point where it frankly gets hard to justify doing this just for fun. Maybe I need to start a gofundme campaign or something like that :mrgreen:

Cheers, Marcel
You've identified a few more things in that header than I have.
The dc.w 4 just after the line number table. I don't know exactly what it does, but in the QLIB source, if it's 0,1, or greater than 4, you get a BEEP, and maybe something else, otherwise it reads some value.

The 'dc.w $FF00 ; WINDS'. If I remember rightly, the QLIB source reads it as a byte, rather than a word, and treats it as a true/false flag.

You say you have done the private QLIB_obj extensions, Do you mean the embedded SuperBASIC extensions, with all the pointer interface and GENCODE type extensions? As I was thinking about doing them. Because I thought you were just doing the stuff that came on the QLib disk.

I have not done much with QLiberator in the last few days, But I have identified the format of the 'External Proc/Functions' data blocks (at least the parts I think I need to know). And added support in the decompiler to list the Procedures and Functions that need their names changed and a 'REMark $$external' added before them.

I still want to try to get QLIB to compile itself to an _OBJ file the same size as the original, and do a byte by byte comparison.


Post Reply