Qxltool[s]

Anything QL Software or Programming Related.
User avatar
NormanDunbar
Forum Moderator
Posts: 2251
Joined: Tue Dec 14, 2010 9:04 am
Location: Leeds, West Yorkshire, UK
Contact:

Qxltool[s]

Post by NormanDunbar »

I'm looking at making this utility work, properly.

I've delved into the code and come up with some weird stuff that Richard was coding (and to be fair, he had zero documentation) but with my own docs on my Wiki, I'm making progress. However, I have noticed that on Windows running GCC as the compiler, it's almost impossible to get the QLWA header struct to be exactly 64 bytes, and also, a directory entry too seems to be bigger than 64 for most compilation options. One thing that's giving me grief is Richard's use of a 'time_t' for one of the dates in the QL Directory entry. That really does not compute - given that a time_t contains quite a few INTs and yet, it appears to be either 8 or 4 bytes depending on my compilation options. (See below.)

I've tried compiling in -m32 and -m64 to get 32 and 64 bit versions and with -fpac-struct=1 to pack to byte alignment, here's the results:

Code: Select all

On Windows with Gcc I get:

64bit (-m64):

HEADER = 66 (Should be 64)
QLDIR = 72 (Should be 64)
Time_t = 8. (Needs to be 4)


64bit (-m64 -fpack-struct=1)

HEADER = 64 (Should be 64)
QLDIR = 68 (Should be 64)
Time_t = 8. (Needs to be 4)


32bit (-m32):

HEADER = 66 (Should be 64)
QLDIR = 68 (Should be 64)
Time_t = 4. (Needs to be 4)


32bit (-m32 -fpack-struct=1)

HEADER = 64
QLDIR = 64
Time_t = 4. (Needs to be 4)
So, only the last one is correct. And only when compiled with 32bit output. Sigh.


I'm wondering if anyone has managed to compile the utility and get it to work? I'm also thinking that the compiler version I'm using is way too modern:

Code: Select all

$ gcc --version
gcc.exe (tdm64-1) 5.1.0


Why do they put lightning conductors on churches?
Author of Arduino Software Internals
Author of Arduino Interrupts

No longer on Twitter, find me on https://mastodon.scot/@NormanDunbar.
User avatar
tofro
Font of All Knowledge
Posts: 2685
Joined: Sun Feb 13, 2011 10:53 pm
Location: SW Germany

Re: Qxltool[s]

Post by tofro »

Norman,

overlaying structures over foreign binary data has more or less come from a somewhat accepted practice it used to be 30 years ago to a clear and absolute no-no. What you see is one of the reasons why.

It would seem to me that other than considering the image an array of unsigned char and writing getter and setter functions to fill internal structs with arbitrary length, packing and byte order, there is no way to save such code and make it fit for this, the next and the one after the next C compiler generation.

Tobias


ʎɐqǝ ɯoɹɟ ǝq oʇ ƃuᴉoƃ ʇou sᴉ pɹɐoqʎǝʞ ʇxǝu ʎɯ 'ɹɐǝp ɥO
User avatar
NormanDunbar
Forum Moderator
Posts: 2251
Joined: Tue Dec 14, 2010 9:04 am
Location: Leeds, West Yorkshire, UK
Contact:

Re: Qxltool[s]

Post by NormanDunbar »

Hi Tobias,

You've just confirmed what I've been trying to avoid! An array of unsigned char.

Thanks.


Cheers,
Norm.


Why do they put lightning conductors on churches?
Author of Arduino Software Internals
Author of Arduino Interrupts

No longer on Twitter, find me on https://mastodon.scot/@NormanDunbar.
User avatar
mk79
QL Wafer Drive
Posts: 1349
Joined: Sun Feb 02, 2014 10:54 am
Location: Esslingen/Germany
Contact:

Re: Qxltool[s]

Post by mk79 »

The headers should use the (newer) standard integer sizes, so u_long = uint32_t etc. Time_t is of course a no-go for a binary aligned structure as its size differs, it should be uint32_t and its value converted from QL date as needed.

Why should HEADER be 64-byte? I'm counting 66 as the "map" member is at position $40 = 64 and defined as one word. Usually one would declare it as map[0] here to make its size unspecified, then the structure would be 64.

When all that is changed the sizes look fine to me (64-bit gcc).


User avatar
NormanDunbar
Forum Moderator
Posts: 2251
Joined: Tue Dec 14, 2010 9:04 am
Location: Leeds, West Yorkshire, UK
Contact:

Re: Qxltool[s]

Post by NormanDunbar »

Hi Marcel,

yes, you are correct in that the first entry in the map is included in the header. My test script was ignoring that and only counting up to that point - so wasn't including the map itself in the header. Sorry for not making that clear.

And yes, I did also do another version with the 'uint*_t' data types, which is a far better idea in my book, and got the following:

Code: Select all

64bit (-m64):

HEADER = 66 (Should be 64)
QLDIR = 72 (Should be 64)


64bit (-m64 -fpack-struct=1)

HEADER = 64 (Should be 64)
QLDIR = 64 (Should be 64)


32bit (-m32):

HEADER = 66 (Should be 64)
QLDIR = 64 (Should be 64)


32bit (-m32 -fpack-struct=1)

HEADER = 64
QLDIR = 60 (should be 64)
Which is still off for 32bit compiles. Sigh.


Cheers,
Norm.


Why do they put lightning conductors on churches?
Author of Arduino Software Internals
Author of Arduino Interrupts

No longer on Twitter, find me on https://mastodon.scot/@NormanDunbar.
swensont
Forum Moderator
Posts: 252
Joined: Tue Dec 06, 2011 3:30 am
Location: SF Bay Area
Contact:

Re: Qxltool[s]

Post by swensont »

I've used qxltool in the past and have been bitten by a bad QXL.win file. I am looking forward to seeing this tool working and am willing to help test it. I have access to both 32- and 64-bit Linux systems to test with.

Tim


User avatar
NormanDunbar
Forum Moderator
Posts: 2251
Joined: Tue Dec 14, 2010 9:04 am
Location: Leeds, West Yorkshire, UK
Contact:

Re: Qxltool[s]

Post by NormanDunbar »

Hi Tim,

I'm on Linux 64 myself, but I have 32bit running in VirtualBox for testing, Windows 64bit also. But the more the merrier.

The fact that any qxltool hard disc worked is surprising as the header is all over the place - most likely due to field alignment in the header and a bit of weirdness in the map. But Richard had to work with zero docs, so I/we have a bit of an advantage!


Cheers,
Norm.


Why do they put lightning conductors on churches?
Author of Arduino Software Internals
Author of Arduino Interrupts

No longer on Twitter, find me on https://mastodon.scot/@NormanDunbar.
User avatar
mk79
QL Wafer Drive
Posts: 1349
Joined: Sun Feb 02, 2014 10:54 am
Location: Esslingen/Germany
Contact:

Re: Qxltool[s]

Post by mk79 »

NormanDunbar wrote: 64bit (-m64):
Testing without structure packing is useless.

I usually do it like this within the source code, then the intend is clear (doesn't rely on makefile) and the setting is only applied to the types between the pragmas.

Code: Select all

#pragma pack(push, 1)
typedef struct
[...]
} QLDIR;
#pragma pack(pop)
32bit (-m32 -fpack-struct=1)

HEADER = 64
QLDIR = 60 (should be 64)
Either you have made an error with the type-conversion or the compiler has some very serious bug. Structures can never be shorter than their members combined, so 60 should be impossible. But difficult to tell without the source that you're actually testing.

Cheers, Marcel


User avatar
NormanDunbar
Forum Moderator
Posts: 2251
Joined: Tue Dec 14, 2010 9:04 am
Location: Leeds, West Yorkshire, UK
Contact:

Re: Qxltool[s]

Post by NormanDunbar »

Morning Marcel,
mk79 wrote:Testing without structure packing is useless.
I know, but I was just testing all (well, some) possible combinations to see the difference(s).
mk79 wrote:I usually do it like this within the source code, then the intend is clear (doesn't rely on makefile) and the setting is only applied to the types between the pragmas.

Code: Select all

#pragma pack(push, 1)
typedef struct
[...]
} QLDIR;
#pragma pack(pop)
Yes I like this method too, however, the code is supposed to be compatible with C68 and I'm pretty sure it doesn't do pragmas.
mk79 wrote: HEADER = 64
QLDIR = 60 (should be 64)

Either you have made an error with the type-conversion or the compiler has some very serious bug. Structures can never be shorter than their members combined, so 60 should be impossible. But difficult to tell without the source that you're actually testing.
Alternatively, maybe I just can't type '36' as the maximum size of a QL file name! I had 32.

So, I fixed the 36, I wrapped the structs in #pragmas as above, and compiling with just -m32 or -m64 works perfectly.

So, the question is, does anyone need this utility to run on a QDOS machine compiled with C68 or are we all running on emulators on Linux etc?

As far as I remember, C68 doesn't do #pragma, nor does it do the stdint.h stuff - but I could be remembering incorrectly. The stdint stuff shouldn't be difficult to set up a header file, and probably the pragmas could be #ifdef'd I suppose.


Thanks for pointing out the errors or my ways!


Cheers,
Norm.


Why do they put lightning conductors on churches?
Author of Arduino Software Internals
Author of Arduino Interrupts

No longer on Twitter, find me on https://mastodon.scot/@NormanDunbar.
User avatar
mk79
QL Wafer Drive
Posts: 1349
Joined: Sun Feb 02, 2014 10:54 am
Location: Esslingen/Germany
Contact:

Re: Qxltool[s]

Post by mk79 »

NormanDunbar wrote:Yes I like this method too, however, the code is supposed to be compatible with C68 and I'm pretty sure it doesn't do pragmas.
Alternatively, you can use the existing ifdef'd PACKED attribute, but apply it to the whole structures and not just a few members:

Code: Select all

ypedef struct
{
    char id[4];
[...]
} PACKED HEADER;
I'm not too familiar with the gcc intricacies, but I think the effect should be the same as with the pragma. Frankly I don't understand why the PACKED attribute was only applied to a few members in the first place.
Alternatively, maybe I just can't type '36' as the maximum size of a QL file name! I had 32.
:-)
So, the question is, does anyone need this utility to run on a QDOS machine compiled with C68 or are we all running on emulators on Linux etc?
Even if that was the case, which I cannot quite imagine because QDOS does usually have native access to WIN files, the existing binary will continue to work anyway.

Cheers, Marcel


Post Reply