File Headers and loss of information

Discussion and advice about emulating the QL on other machines.
Post Reply
RWAP
RWAP Master
Posts: 2834
Joined: Sun Nov 28, 2010 4:51 pm
Location: Stone, United Kingdom
Contact:

File Headers and loss of information

Post by RWAP »

I am sure the emulator authors will put me right if I get any of this information incorrect.
[Rob is it worth making this a sticky?]

The QL uses the file header to store extra information about files, mainly the dataspace to be set aside when a program is executed (see information on SEXEC command in the user manual).

Normally, if you unzip a QL file direct onto the PC using a PC unzip program (such as WinZip), the file header information is stripped off.

However, the emulators get around this in various ways.
a) QLAY and QL2K all store a secondary file for executable programs, when you use SEXEC within the emulator to save a file to the PC's hard disk. The secondary file contains the information normally found within the header. The tools provided for copying QL floppy disks to the hard disk also do this.
b) QPC2 does nothing with the information so far as I am aware and it is lost if you save a file to DOSx_ drive. However, as QPC2 normally stores all of its files within its own hard disk container (qxl.win), this should not be an issue as the qxl.win format provides for file headers as if you were using a QL.
c) Q-emuLator adds the necessary information to the start of the file before it is stored on the PC hard disk and strips it off when the program or file is loaded into the emulator. This works whether you use SEXEC, or COPY from a QL floppy disk (or image). Q-emuLator can also use qxl.win files (as with QPC2).

So the moral is this, if you are sent a zip file containing QL programs (or download it from one of the websites), then you always MUST unzip it from within an emulator. If you use Winzip (or similar on the PC or MAC), then you will lose any header information and will need to add it back to the executable files with SEXEC.

The requisite unzip files are available from the Sinclair QL Homepage

Hope this is useful.


User avatar
vanpeebles
Commissario Pebbli
Posts: 2815
Joined: Sat Nov 20, 2010 7:13 pm
Location: North East UK

Re: File Headers and loss of information

Post by vanpeebles »

Cheers for the guide, if you write the files to floppy from within an emulator will they still work ok? When I was adding some extra dats to a QL Floppy I'm sure it stripped my zip adventure intepreter because I had to overwrite it with a working copy due to bad parameter errors.

Is it something that applies to windows or will other operating systems do the same?


RWAP
RWAP Master
Posts: 2834
Joined: Sun Nov 28, 2010 4:51 pm
Location: Stone, United Kingdom
Contact:

Re: File Headers and loss of information

Post by RWAP »

vanpeebles wrote:Cheers for the guide, if you write the files to floppy from within an emulator will they still work ok? When I was adding some extra dats to a QL Floppy I'm sure it stripped my zip adventure intepreter because I had to overwrite it with a working copy due to bad parameter errors.

Is it something that applies to windows or will other operating systems do the same?
Yes, provided that it is a QL formatted floppy, they should work in all emulators and the QL, whatever you have used to get the files onto it.

The zip adventure interpreter was missing the headers on the download file on Dilwyn's site, until a few days ago - maybe you had the old version?


User avatar
vanpeebles
Commissario Pebbli
Posts: 2815
Joined: Sat Nov 20, 2010 7:13 pm
Location: North East UK

Re: File Headers and loss of information

Post by vanpeebles »

I have got the newer version but it's possible I got my floppies mixed up :oops:


User avatar
programandala.net
Chuggy Microdrive
Posts: 74
Joined: Mon Dec 13, 2010 12:41 pm
Location: Spain
Contact:

Re: File Headers and loss of information

Post by programandala.net »

RWAP wrote: Yes, provided that it is a QL formatted floppy, they should work in all emulators and the QL, whatever you have used to get the files onto it.
I wonder why the zip format has become so popular for QL online software distribution. I think disk images are much better (or even microdrive images!). You use a disk image the same way you did with a real disk and the real machine, no problems! And you can get an actual disk from the image file whenever you want.

Images of the original media are very common in retrocomputing. I use tape and disk images a lot with ZX Spectrum, Jupiter Ace and SAM Coupé emulators. It's very comfortable. Some emulators can manage also compressed disk images directly (e.g. my_disk.dsk.gz or my_disk.dsk.zip).

Maybe the problem was the QL emulators and their evolution? Even today, not all of them can manage disk images.


Marcos Cruz (programandala.net)
User avatar
vanpeebles
Commissario Pebbli
Posts: 2815
Joined: Sat Nov 20, 2010 7:13 pm
Location: North East UK

Re: File Headers and loss of information

Post by vanpeebles »

That's a good point, I think at some point a disk image archive of QL Software would be a good idea even just for preservation. Some of the media now is getting on in age.


RWAP
RWAP Master
Posts: 2834
Joined: Sun Nov 28, 2010 4:51 pm
Location: Stone, United Kingdom
Contact:

Re: File Headers and loss of information

Post by RWAP »

programandala.net wrote: Maybe the problem was the QL emulators and their evolution? Even today, not all of them can manage disk images.
Only Q-emuLator can currently handle disk images and that was added to v3.0 at my behest, as I needed an easy means to transfer programs from my own QL Software Preservation Project onto the compact flash card I use with the HxC Floppy disk drive emulator on my QL :-)

I guess at the time of creating the QL emulators, disk images were thought not necessary, as people wanted the ability to really use hard disks for the larger files which the QL is capable of handling.

It is too late down the road now to create floppy disk images, and also you have to remember that the vast majority (99.9%) of QL software supplied on disk could always be transferred onto other media quite readily. There are only two which use floppy disk protection, that i know of - one is the Ekotek Flight Simulator (yes Jeff, I know I still have the disk on loan), the other one I have come across escapes me...

That reminds me - I need someone to make a disk image of the Ekotek Flight simulator, as it seems impossible with only a USB disk drive on my PC now. I'd ask Dilwyn, but he might just throw another PC out of the window!

Images were really more important for microdrive cartridges, where copy protection was rife. Again, only Q-emuLator v3 has this ability (another recent addition), and using a small tool written by Daniele Terdina (the author of Q-emuLator), I can make images of microdrive cartridges and save them to disk (or virtual floppy disks with HxC!).

I guess microdrive imaging never really came into the equation, as most people had moved away from this media, or abandoned the early software by the time emulators came to the fore. Plus there is still not the reverse routine to take an image and make an original microdrive cartridge from it!

Unfortunately, due to the age of microdrive cartridges, many are now badly corrupted and it can be difficult to make a full image of one - normally its a matter of getting an image of a partially working key cartridge, copying the files from another source, and hoping the microdrive image is sufficient to let the program get past its copy protection routines.

Sadly there are too many cases of some software which I have not been able to preserve and not enough time!

In the end, it is easier to try and crack the copy protection, so that a program backup can be made from several sources and merged.


User avatar
programandala.net
Chuggy Microdrive
Posts: 74
Joined: Mon Dec 13, 2010 12:41 pm
Location: Spain
Contact:

Re: File Headers and loss of information

Post by programandala.net »

RWAP wrote: b) QPC2 does nothing with the information so far as I am aware and it is lost if you save a file to DOSx_ drive.
I keep all my docs, data and program sources in DOSx_, so I can manipulate them also with external tools (even the boot file, loaded by a fake one in the WIN file); the installed programs are in a WIN file. This way everything works fine and I can manipulate my files with both SMSQ/E and Debian.


Marcos Cruz (programandala.net)
User avatar
programandala.net
Chuggy Microdrive
Posts: 74
Joined: Mon Dec 13, 2010 12:41 pm
Location: Spain
Contact:

Re: File Headers and loss of information

Post by programandala.net »

RWAP wrote: Only Q-emuLator can currently handle disk images and that was added to v3.0
Also UQXL can.
RWAP wrote: It is too late down the road now to create floppy disk images, and also you have to remember that the vast majority (99.9%) of QL software supplied on disk could always be transferred onto other media quite readily.
I have some original disks, and I already created their images. Probably I will never use those programs again (in fact I'm considering selling them, with some QL hardware), but it's good to preserve them just in case someday it will be possible to publish them for free somewhere (a kind of the huge http://worldofspectrum.org/ or the modest http://jupiter-ace.co.uk/). I think it's worthwhile to preserve as much as possible for the future. Nobody knows whom those works can be useful to: Some time ago I found in the World of Spectrum's database two little programs that could be very useful for me: a patch for Artic Forth to add microdrive support, and a whole screen editor for Abersoft Forth. Both programs are "missing in action". No copies found yet, though the programs were advertised and sold. So I had to change my retroprogramming project or reinvent the wheel once more.

But I was thinking more about distributing new software than preserving the old one. If I wanted to distribute a program written by me that includes several files (executables, sources, data, pictures, directories...) I think it would be much easier (for me and for the user) to use a floppy disk image (zipped or not) than a zip file. Of course I'm supposing the emulators could handle it. It's a bit easier to "mount" the floppy disk image than to unzip a file inside the emulator.

Anyway, WIN files have the same advantages than floppy disk images, besides they can be much larger and more emulators support them.

Also, Q-emuLator 3 has a new feature: it can "mount" zip files, but of course read only. (The new .qlpak format seems interesting for distributing certain software, especially games).


Marcos Cruz (programandala.net)
Post Reply