QLiberator v3.38

Anything QL Software or Programming Related.
User avatar
pjw
QL Wafer Drive
Posts: 1286
Joined: Fri Jul 11, 2014 8:44 am
Location: Norway
Contact:

Re: QLiberator v3.38

Post by pjw »

EmmBee, Martin,
There is still a problem with EXT_xxx, but the symptoms have shifted a bit.
Take the following program:

Code: Select all

10 REMark $$asmb=ram1_QLTST_BIN,0,10
20 :
30 EXT_PROC 'MY_PROC'
40 EXT_FN 'MY_FUN%'
50 :
60 MY_PROC
70 QUIT MY_FUN%(99)
(MY_PROC does nothing and MY_FUN% just returns the integer it is given)

With the toolkit (QLTST_BIN) loaded when the program is QSAVEd, the program compiles and works (PRINT FEW(qltst_obj) returns 99). In 3.37 the program would compile but then complain about MY_PROC when EXECuted.

In 3.38 if the toolkit is NOT loaded when qsaved, Qlib barfs and will not compile it at all, citing "Ambiguous name" for MY_PROC.

Functions defined with EXT_FN, however, still behave like in 3.37.

The problem is that these behaviours are not entirely consistent! With different, more complex programs, sometimes they compile but then dont work, or they wont compile without first being qsaved with the toolkit present.

I hope this can help to isolate the problem!


Per
dont be happy. worry
- ?
User avatar
dilwyn
Mr QL
Posts: 2753
Joined: Wed Dec 01, 2010 10:39 pm

Re: QLiberator v3.38

Post by dilwyn »

You would be very welcome to use my SCRXLIM/SCRYLIM extension in QLib if it does what's needed of it.

I'm afraid I don't know how QDOS on uQLx handles the bigger-than-512x256 screens, hence my note of caution. I suspect that the worst that would happen if there was no PE on uQLX running QDOS is that QLib would end up stuck within 512x256, but until someone tries it I won't be able to say.

As an exercice last night I threw the biggest BASIC program I have at QLib3.38 - Launchpad. It compiled fine, including linking the extensions and ran fine after compilation. Not yet had time to compare for any object code differences between this and the original version compiled with QLib 3.36.

Question: I appreciate that the bin files included with this are for the QLib sources itself, and the patched qlib_run. Are the remainder of the files from QLib 3.36 (e.g. qlib_ext) OK for use with QLib3.38?


Martin_Head
Aurora
Posts: 847
Joined: Tue Dec 17, 2013 1:17 pm

Re: QLiberator v3.38

Post by Martin_Head »

pjw wrote:EmmBee, Martin,
There is still a problem with EXT_xxx, but the symptoms have shifted a bit.
Take the following program:

Code: Select all

10 REMark $$asmb=ram1_QLTST_BIN,0,10
20 :
30 EXT_PROC 'MY_PROC'
40 EXT_FN 'MY_FUN%'
50 :
60 MY_PROC
70 QUIT MY_FUN%(99)
(MY_PROC does nothing and MY_FUN% just returns the integer it is given)

With the toolkit (QLTST_BIN) loaded when the program is QSAVEd, the program compiles and works (PRINT FEW(qltst_obj) returns 99). In 3.37 the program would compile but then complain about MY_PROC when EXECuted.

In 3.38 if the toolkit is NOT loaded when qsaved, Qlib barfs and will not compile it at all, citing "Ambiguous name" for MY_PROC.

Functions defined with EXT_FN, however, still behave like in 3.37.

The problem is that these behaviours are not entirely consistent! With different, more complex programs, sometimes they compile but then dont work, or they wont compile without first being qsaved with the toolkit present.

I hope this can help to isolate the problem!
I do come back to this problem every now and then. If I remember correctly, when the program is compiled the EXT_PROC and EXT_FN doesn't always store the name as a Proc/Fun, but as variable. I was going to try to trace the EXT_xx commands in the compiler to try to see where it failed, but I could not get the BASIC version of QLib 3.37 to run. Probably something stupid I was doing. EmmBee, You probably have a much better idea of how QLib actually compiles than I do.

I'm still trying to get my decompiled QLib to recompile as a copy of the original. To try to find what the differences are. But I'm having trouble getting the name lists to be created in the correct order.
I'm pretty sure that the original QLib was compiled under Minerva. If I compile under Minerva, then the compiled version of QLib has the same length as the original QLib, and I don't see an odd effect of negative numbers in the program being a positive number followed by a negate instruction. I think it's got to do with Minerva having Integer tokenization.


User avatar
RalfR
Aurora
Posts: 871
Joined: Fri Jun 15, 2018 8:58 pm

Re: QLiberator v3.38

Post by RalfR »

dilwyn wrote:Question: I appreciate that the bin files included with this are for the QLib sources itself, and the patched qlib_run. Are the remainder of the files from QLib 3.36 (e.g. qlib_ext) OK for use with QLib3.38?
Yes, as far as I know, these are the extracted BIN files from Martin's first decompilation. And yes, these are the files for QLib itself (all QLibx_BIN files). EmmBee (as he has stated earlier) does not do anything with ASM, just SB.

Because (original) QLib "remembers" the last position of the panel just inside 512x256, EmmBee has used the environment variable scheme. To make it more "genuine" in QLib's way, QLIB_BIN must be modified.


4E75 7000
EmmBee
Trump Card
Posts: 240
Joined: Fri Jan 13, 2012 5:29 pm
Location: Kent

Re: QLiberator v3.38

Post by EmmBee »

dilwyn wrote:You would be very welcome to use my SCRXLIM/SCRYLIM extension in QLib if it does what's needed of it.

I'm afraid I don't know how QDOS on uQLx handles the bigger-than-512x256 screens, hence my note of caution. I suspect that the worst that would happen if there was no PE on uQLX running QDOS is that QLib would end up stuck within 512x256, but until someone tries it I won't be able to say.

As an exercice last night I threw the biggest BASIC program I have at QLib3.38 - Launchpad. It compiled fine, including linking the extensions and ran fine after compilation. Not yet had time to compare for any object code differences between this and the original version compiled with QLib 3.36.

Question: I appreciate that the bin files included with this are for the QLib sources itself, and the patched qlib_run. Are the remainder of the files from QLib 3.36 (e.g. qlib_ext) OK for use with QLib3.38?
Hi Dilwyn,

Yes, I would like to use your code. If there are any uQLx users who are not using the PE yet, then they shouldn't notice any difference!
The usual files from v3.36 including qlib_ext etc. should be used of course to start everything up. The BIN files included in the zip are just for the compiler source. We will now be using up all of the 8 allowed assembler inclusion files.

EmmBee


EmmBee
Trump Card
Posts: 240
Joined: Fri Jan 13, 2012 5:29 pm
Location: Kent

Re: QLiberator v3.38

Post by EmmBee »

pjw wrote:EmmBee, Martin,
There is still a problem with EXT_xxx, but the symptoms have shifted a bit.
I find I get different results on QDOS-Minerva and QPC2. While it works on QDOS, on QPC2 I get error messages of "Keyword must be at start of program", even when EXT_PROC / EXT_FN is already at the start. I have traced this problem to a differing RP_ADDR which does not match the start_adr. How to overcome that I do not know, yet. However, I have come across another puzzle...

At line 19080 in our code, we have ... 19080 DEFine PROCedure Gen_Ext_Proc_FN. During this process, it searches through its own name table looking for a matching name, and if found it does various things and then exits the loop. At least that is how it looks - but it doesn't. The way it's written it only checks the first entry - because of lines 19260 and 19270. I believe this is a mistake and those lines need to be reversed. I would like to ask Martin whether that could have been caused by him when doing the decompile. Alternatively, it might be a hard-coded error in the original code. This mishap could explain the differing results that Per is getting.


Martin_Head
Aurora
Posts: 847
Joined: Tue Dec 17, 2013 1:17 pm

Re: QLiberator v3.38

Post by Martin_Head »

EmmBee wrote:
pjw wrote:EmmBee, Martin,
There is still a problem with EXT_xxx, but the symptoms have shifted a bit.
I find I get different results on QDOS-Minerva and QPC2. While it works on QDOS, on QPC2 I get error messages of "Keyword must be at start of program", even when EXT_PROC / EXT_FN is already at the start. I have traced this problem to a differing RP_ADDR which does not match the start_adr. How to overcome that I do not know, yet. However, I have come across another puzzle...

At line 19080 in our code, we have ... 19080 DEFine PROCedure Gen_Ext_Proc_FN. During this process, it searches through its own name table looking for a matching name, and if found it does various things and then exits the loop. At least that is how it looks - but it doesn't. The way it's written it only checks the first entry - because of lines 19260 and 19270. I believe this is a mistake and those lines need to be reversed. I would like to ask Martin whether that could have been caused by him when doing the decompile. Alternatively, it might be a hard-coded error in the original code. This mishap could explain the differing results that Per is getting.
Started having a look at this, this afternoon, Got a bit short of time, but this is what I've found

dump from qlib_bas_337

Code: Select all

19080 DEFine PROCedure Gen_Ext_Proc_Fn (class)
19090   IF (RP_ADDR  <> start_adr) THEN
19100     program_error 1, 22
19110   ELSE
19120     REPeat loop
19130       Getsym
19140       IF End_of_Stmt% THEN
19150         EXIT loop
19160       END IF
19170       IF (sym1% = 7) THEN
19180         FOR i = 1 TO NumNames%
19190           RNAME name_adr(i), p0%, struct_base%, FOR_ranges%, ExtPFNname$
19200           IF (ExtPFNname$ == sym4$) THEN
19210             unpack
19220             p2% = class
19230             p4% = 1
19240             pack
19250             WNAME name_adr(i), p0%
19260           END IF
19270           EXIT i
19280         END FOR i
19290       ELSE
19300         IF (sym1% = 6) THEN
19310           program_error 0, 24
19320         END IF
19330       END IF
19340     END REPeat loop
19350   END IF
19360 END DEFine Gen_Ext_Proc_Fn
dump from my qlib_bas

Code: Select all

9080 DEFine PROCedure procFun18239 (var08C8)
19090 IF (RP_ADDR  <> var08B0) THEN
19100  procFun21693 1,22
19110 ELSE
19120  REPeat var05D8
19130   procFun18547
19140   IF var0630% THEN
19150    EXIT var05D8
19160   END IF
19170   IF (var0450% = 7) THEN
19180    FOR var03E8 = 1 TO var08D0%
19190     RNAME var0578(var03E8),var0580%,var0560%,var0568%,var08D8$
19200     IF (var08D8$ == var07A0$) THEN
19210      procFun18779
19220      var06B0% = var08C8
19230      var06A0% = 1
19240      procFun18833
19250      WNAME var0578(var03E8),var0580%
19260     END IF
19270     EXIT var03E8
19280    END FOR var03E8
19290   ELSE
19300    IF (var0450% = 6) THEN
19310     procFun21693 0,24
19320    END IF
19330   END IF
19340  END REPeat var05D8
19350 END IF
19360 END DEFine procFun18239
dump from my qlib6_bas raw output from the decompiler

Code: Select all

18239 DEFine PROCedure procFun18239 (var08C8)
18248 IF (RP_ADDR  <> var08B0) THEN 
18262  procFun21693 1,22
18280  ELSE 
18286  var05D8 = 0
18294  procFun18547 
18300  IF var0630% THEN 
18310    GO TO 18510 : END IF
18316  IF (var0450% = 7) THEN 
18330   FOR var03E8 = 1 TO var08D0%
18358   RNAME var0578(var03E8),var0580%,var0560%,var0568%,var08D8$
18388   IF (var08D8$ == var07A0$) THEN 
18402    procFun18779 
18408    var06B0% = var08C8
18416    var06A0% = 1
18424    procFun18833 
18430    WNAME var0578(var03E8),var0580%
18448    ELSE 
18454    END FOR var03E8 : END IF
18466   ELSE 
18472   IF (var0450% = 6) THEN 
18486    procFun21693 0,24 : END IF : END IF
18504  GO TO 18294 : END IF
18510 END DEFine procFun18239
Notice the difference around 18430.

dump from my disassembly of the original QLib

Code: Select all

0028733A  0080 08C8
          00D6   ori.l     #$08C800D6,d0               ......
00287340  06B0008A0001
          00D6           addi.l    #$008A0001,-$2A(a0,d0.w)    ........
00287348  06A0

18424     procFun18833 
          58
            CE 0000019C           bclr      d0,(a4)+                    ..

18430     WNAME var0578(var03E8),var0580%
00287350  0280 03E8
          16
            86 10 01 0578
          78 00 0580           bclr      d2,d0                       ..
0028735E  0096 0198

          00CA 00000010  else?    --> 00287374 : exit for
end if

18454                       END FOR var03E8 
00287368  00A203E8FFFF   ori.l     #$03E8FFFF,-(a2)            ......
0028736E  FF9C           ILLEGAL   INSTRUCTION                 ..
00287370  00000004       ori.b     #$04,d0                     ....

00287374  00CA 00000024   else??  --> 28739A

0028737A  00D2           ILLEGAL   INSTRUCTION                 ..
0028737C  0450008A       subi.w    #$008A,(a0)                 .P..
00287380  00064ACC       ori.b     #$CC,d6                     ..J.
00287384  00000016       ori.b     #$16,d0                     ....
00287388  588A           addq.l    #$04,a2                     X.
0028738A  00006600       ori.b     #$00,d0                     ..f.
I know it looks a bit of a mess, But I was obviously having a bit of confusion around there with the question marks.
I'm a bit short of time to hand decompile today. But I thought I would quickly post this in case something pops out at you.


Martin_Head
Aurora
Posts: 847
Joined: Tue Dec 17, 2013 1:17 pm

Re: QLiberator v3.38

Post by Martin_Head »

OK, So I've hand decompiled the procedure

Code: Select all

18239 DEFine PROCedure procFun18239 (var08C8)
          76 01 08C8

18248 IF (RP_ADDR  <> var08B0) THEN 
0028729A  58             preceed a namelist keyword or proc/fun call
            96 0250      call keyword table entry
          00D4 08B0      fetch variable
          4C             <>
            CC 0000001C  if not true --> 002872C0   line 18286

18262  procFun21693 1,22
          58             preceed a namelist keyword or proc/fun call
            8A 0001      place 1 on the stack
          66 00          no seperator
002872AE  008A 0016      place 22 on the stack
          66 00          no seperator
002872B4  00CE 00000D5E  call proc/fun


002872BA  00CA 000000E4  else?  goto --> 2873A0    end of proc


18286 var05D8 = 0     (repeat loop)
002872C0  008A 0000      place 0 on stack
          0084 05D8      assign variable
          
18294  procFun18547
002872C8  58             preceed a namelist keyword or proc/fun call
            CE 00000100  call proc/fun
            
18300  IF var0630% THEN 
002872CE  00D2 0630      fetch a variable
          00CC 0000000A  if not true  goto --> 2872DE
                                              |
    
002872D8  00CA 000000C6  goto --> 2873A0 end of proc

                                              |
                                 end if <-----
18316  IF (var0450% = 7) THEN 
002872DE  00D2 0450     fetch variable
          008A 0007     place 7 on the stack
          4A            =
            CC 00000092 if not true  goto --> 28737A  line 18472


18330 FOR var03E8 = 1 TO var08D0%
002872EC  009E 03E8     control variable
          008A 0001     start
          00D2 08D0     end
          008A 0001     step
          00A4 03E8 00000008 00000070 
          
18358 RNAME var0578(var03E8),var0580%,var0560%,var0568%,var08D8$               
00287308  02            preceed parameters of a command
            80 03E8     fetch a variable
          16            duplicate item on stack
            86 10 01 0578  get array element         var0578(var03E8)
          78 10 0580           
00287316  78 10 0560           
0028731A  78 10 0568
          78 00 08D8
          0096 0190     call keyword table entry     RNAME

18388 IF (var08D8$ == var07A0$) THEN 
00287326  0080 08D8      fetch variable
          0080 07A0      fetch variable
0028732E  56             ==
            CC 00000038  if not true --> goto 287368   -------------------->
                                                                            |
18402 procFun18779 
            58             preceed a namelist keyword or proc/fun call
              CE 0000017C  call proc/fun


18408 var06B0% = var08C8
0028733A  0080 08C8      fetch variable
          00D6 06B0      assign variable

18416 var06A0% = 1
          008A 0001      put 1 on the stack
          00D6 06A0      assign variable

18424 procFun18833 
          58             preceed a namelist keyword or proc/fun call
            CE 0000019C  call proc/fun

18430 WNAME var0578(var03E8),var0580%
00287350  02             preceed a command
            80 03E8      get variable            var03E8
          16             duplicate item on stack
            86 10 01 0578   get array element    var0578
          78 00 0580     get a name list entry   var0580%
0028735E  0096 0198      call proc/fun           WNAME


          00CA 00000010  goto -->  00287374   ------>
                                                     |  broken out of FOR loop

                                                                            |
18454 END FOR var03E8                            end if   <-----------------
00287368  00A2 03E8 FFFFFF9C 
00287370            00000004

                                                    |
                                               <----
00287374  00CA 00000024  goto -->  0028739A    ------->
                                                       |

18472 IF (var0450% = 6) THEN
0028737A  00D2 0450      fetch variable
          008A 0006      place 6 on the stack
          4A             =
            CC 00000016  if not true, goto -> 28739A ->|

18486 procFun21693 0,24
00287388  58             preceed a namelist keyword or proc/fun call
            8A 0000      place 0 on the stack
          66 00          add no seperator
0028738E  008A 0018      place 24 on the stack
          66 00          add no seperator
00287394  00CE 00000C7E  call proc/fun

                                                       |
18504  GO TO 18294 (end rep loop)              <-------
0028739A  00CA FFFFFF2C  goto --> 2872C6



18510 END DEFine procFun18239
002873A0  5E             return/end define
And tidied up the decompile. Hopefully I've sorted all the goto's out correctly.
The exit and end if have moved. See if it makes more sense

Code: Select all

18239 DEFine PROCedure procFun18239 (var08C8)
18248  IF (RP_ADDR  <> var08B0) THEN 
18262   procFun21693 1,22
18280  ELSE 
18286   REPeat loop
18294    procFun18547 
18300    IF var0630% THEN 
18310     EXIT loop
18312    END IF
18316    IF (var0450% = 7) THEN 
18330     FOR var03E8 = 1 TO var08D0%
18358      RNAME var0578(var03E8),var0580%,var0560%,var0568%,var08D8$
18388      IF (var08D8$ == var07A0$) THEN 
18402       procFun18779 
18408       var06B0% = var08C8
18416       var06A0% = 1
18424       procFun18833 
18430       WNAME var0578(var03E8),var0580%
18448       EXIT var03E8
18450      END IF
18454     END FOR var03E8 
18466    ELSE 
18472     IF (var0450% = 6) THEN 
18486      procFun21693 0,24
18488     END IF
18490    END IF
18504   END REPeat loop
18505  END IF
18510 END DEFine procFun18239


EmmBee
Trump Card
Posts: 240
Joined: Fri Jan 13, 2012 5:29 pm
Location: Kent

Re: QLiberator v3.38

Post by EmmBee »

Thanks for your work on this, Martin. It's looking good, now. This seems to cure the problems Per was talking about.
Martin_Head wrote:I'm still trying to get my decompiled QLib to recompile as a copy of the original. To try to find what the differences are. But I'm having trouble getting the name lists to be created in the correct order.
I have an idea, here. All the names used are stored in the wrk/sav file that is created. The resident names are listed first. The idea is to re-order these names to the order in the original QLIB_obj file. As far as I can see, all the names are there, but the original contains an additional name, "CLEAR". If that name is added somewhere to the QLIB_BAS file, say at the end where it will make no difference, then all the figures should add up correctly. Besides creating a new name table to overwrite the existing one, the name indexes in the program will also have to be patched. This might seem to be a fiddly job, but I have made a start on this.


Martin_Head
Aurora
Posts: 847
Joined: Tue Dec 17, 2013 1:17 pm

Re: QLiberator v3.38

Post by Martin_Head »

EmmBee wrote:I have an idea, here. All the names used are stored in the wrk/sav file that is created. The resident names are listed first. The idea is to re-order these names to the order in the original QLIB_obj file. As far as I can see, all the names are there, but the original contains an additional name, "CLEAR". If that name is added somewhere to the QLIB_BAS file, say at the end where it will make no difference, then all the figures should add up correctly. Besides creating a new name table to overwrite the existing one, the name indexes in the program will also have to be patched. This might seem to be a fiddly job, but I have made a start on this.
This is the latest path I have been trying. And I am getting closer....
For the CLEAR, I added a CLEAR at the start of the program, with a REMark $$off, and a REMark $$on around it. That got the CLEAR back.
Then there are a few keywords that get a bit jumbled up. RMODE appears before some Q_ERR and QLIB_USE etc. Where RMODE is part of the QLIB1_BIN file, which is loaded later.
I have been playing around with the order that the extensions are loaded to try to get the name list ordering right.
I have also tried patching the QLIB1_BIN into two files. One with only RMODE, and one without RMODE.

I'm getting closer to the right order, But those Q_ERR and QLIB_USE etc. still appear in the wrong order/mixed in the name list. Although in their files they are in the right order. It's as though Minerva (that I testing with) fills in gaps in the name list before adding new ones on the end.

I know I get a bit obsessed with this sometimes, and I have to walk away :D


Post Reply