QDesign source code

[mk79]

This is a continuation of the thread viewtopic.php?f=11&t=3991&start=50 and the current state of my Christmas project. Not sure what it's worth in the end, but I have reverse engineered all the machine code extensions and the basic source code of QDesign itself. The QDesign basic extension is some 8000 lines of code, the Basic code over 5000 lines.

For the assembler extensions I used the IDA Pro disassembler, which makes figuring out this amount of code possible at all. It lets one easily rename labels, make cross references and give names to constants very comfortably, thus slowly creating a more and more complete picture of what's going on. I always compare it to figuring out a 10000 pieces puzzle, maybe pointless but satisfying nonetheless. It also results in interesting insights on the way, there is for example a huge bug in one of the extensions (C_STRG$) that crashes Basic instantly. Not sure how this could go unnoticed for such a long time, the code probably only ever worked when compiled. Anyway, it's fixed in the attached code.

For the Basic code I tried to create a workflow that allows me to do a similar thing, just not as interactively. I created a Perl script that cleans up the code and applies certain translations, guided by a substitution file that I gradually fill the more I understand of the code. It's a bit quick&dirty but still powerful: a line starting with a number adds a REMark at this line number. If the line exists the REMark is appended to the existing line. A line starting with "-" and a number deletes the line. All other lines define substitutions in the form "procFun71531,init_menu_sprites". After adding a few lines I execute the Perl script again and continue to work with the newly created file, slowly making the source more readable. Sometimes variable names go through multiple guesses before a clear picture forms And some might still be wrong

For QDesign this file contains the reversed names of 154 procedures and 709 variables. It also adds 227 comments. I think the result is pretty cool overall and even seems to work

Thanks to Martin Head for providing the decompiled code, couldn't have done this without it!

Apart from the most immediate bug fixes mentioned above I didn't yet improve anything in the code, that might be for a later time. But maybe this is also interesting for some other parties.

Cheers, Marcel

[pjw]

Well, that was the easy bit! Now to make it work!
(Only kidding, I hope you know. Its probably almost as hard as unscrambling eggs!)
It compiles nicely, but still wont run in my normal setup (SMSQ/E (>512x256)x16), which is all I had time for now.

[dilwyn]

Thank you Marcel and all concerned in this. Although I don't have enough time on my hands to get involved myself, I really hope someone can make an effective GD2 art and graphics program of this one day. It's one of those things we need.

[Tinyfpga]

I am in the process of starting a topic tiled, "why is programming so hard", mainly because I find it so. Clearly this does not apply to mk79, who is reverse engineering Qdesign as a Christmas project with the same difficulty I have, reverse engineering a Christmas cake by eating it.

I was mostly interested by your use of IDA Pro, so I looked it up. I assume that your IDA Pro is not a private purchase. Is a plugin required to disassemble 68000 code?

I found the following quote on the internet:- "It is very easy to make an argument that IDA Pro is so fantastically under-priced (>$3000.00) that it has killed the market for these kinds of tools by setting a bad price point."

I assume that SMSQE disassemblers are not good enough for this kind of work.

[tofro]

I am in the process of starting a topic tiled, "why is programming so hard", mainly because I find it so. Clearly this does not apply to mk79, who is reverse engineering Qdesign as a Christmas project with the same difficulty I have, reverse engineering a Christmas cake by eating it.

That's the wrong way of approaching it. Programming is actually terribly easy, not much more than eating a Christmas cake, but with less long-lasting consequences - It's just a matter of perseverance, practice and starting with baby steps. If you keep telling yourself "It's hard", it's gonna stay that way. And don't hang your aim too high at the beginning.

I found the following quote on the internet:- "It is very easy to make an argument that IDA Pro is so fantastically under-priced (>$3000.00) that it has killed the market for these kinds of tools by setting a bad price point."

I assume that SMSQE disassemblers are not good enough for this kind of work.

Well, SMSQ/E disassemblers (1) are good enough for this kind of work, but need a huge amount of tedious and pedantic note-taking and -maintaining, plus a lot more investment in terms of time (and a lot more frustration). IDA does that for you and more (and, thankfully, doesn't even know what frustration is).

There's actually a relatively new free competition product to IDA - Ghidra, developed by the NSA for their internal usage (I don't even want to know what they use it for....) that can also do 68000 code re-engineering. It's not quite as slick to use and rather intended as a de-compiler (for programs written in C), but it could very well be used for the same purposes. (But don't even think about downloading and using it - That's not the "baby steps" mentioned above)

(1) I have used DEA in the past with surprisingly good results. Maybe it's time to advertise this surprisingly good (and heavily underrated) piece of code (for a QL) a bit here.

[Tinyfpga]

I meant hard, as in playing a musical instrument is hard. As you suggest, anyone can code but one is faced with an immediate problem when doing so. One needs to quickly cross a very high barrier (especially if one is not naturally talented) before it becomes enjoyable enough to carry on learning.

That barrier can be lowered, even for the congenitally hopeless, if one has some guidance. Despite reading, as has been suggested by a number of forum members, most of the BASIC manuals on Dilwyn's site, I still stare at my screen wondering why the program I have written doesn't work. It is guidance that I seek on this forum, hence the proposed and slightly provocative title "why is programming so hard"

Something thing, I have found, is that programming a Raspberry pi (ironically developed as a learning tool) is much, much, worse than programming SMS.

[mk79]

Well, that was the easy bit! Now to make it work!
(Only kidding, I hope you know. Its probably almost as hard as unscrambling eggs!)

Not quite and I actually like to do this stuff, it's just very time intensive

It compiles nicely, but still wont run in my normal setup (SMSQ/E (>512x256)x16), which is all I had time for now.

This is the original 4.04, so most graphics commands still require mode 4.

[NormanDunbar]

Programming is not hard, on a QL or derivatives, or on a Raspberry Pi, or even in Windows -- but beginning programmers won't be daft enough to go down that route!

I program the QL, in Basic, assembly, C and recently, Pascal. But not Forth or BCPL yet.

I program the Arduino/ATmega328 in C++ and assembly.

I program the Raspberry Pi in C++, and the Pico in C++ too, but never Python.

I used to program Windows in C, C++ and Delphi (Pascal), and occasionally, Visual Basic. Once, I even used COBOL!

That's not to mention the various mainframes I used to work with.

If programming was hard, I'd not be able.


You've mentioned problems previously and I've offered to help, to explain etc, but you've never come back to me -- unless I missed it of course.

Starting simple is good advice. KISS works wonders. Don't attempt to write a FP shoot-em-up or Lemmings but how about the old favourite, the computer thinks of a number, you get limited goes to guess it. The computer days higher or lower until you win or run out of moves.

When you perfect the actual logic, then you can tart it up with graphics if appropriate, but get it working first. Try it?

Anyway, what's bugging you? What are you trying to do? What's going wrong? What's working? What advice do you need?

Cheers,
Norm.

[mk79]

I was mostly interested by your use of IDA Pro, so I looked it up. I assume that your IDA Pro is not a private purchase. Is a plugin required to disassemble 68000 code?

No, my employer pays for the license. And no plugin is required, it can do a few dozen processors out of the box (though it needs quite some post-processing to make the code compatible with QL assemblers, but I have automated most of that, too).

IDA has a python API, so I developed some SMSQ/E specific plug-ins that for example automatically decode Config Level 2 blocks or Basic command definitions.

I found the following quote on the internet:- "It is very easy to make an argument that IDA Pro is so fantastically under-priced (>$3000.00) that it has killed the market for these kinds of tools by setting a bad price point."

"under-priced" and IDA Pro in one sentence, amazing.

I assume that SMSQE disassemblers are not good enough for this kind of work.

DISA is a good tool and it's possible to do these things with it, with a few more features like renaming labels it could even be a great tool, but for me it's not worth the trouble as I already have all I need...

[mk79]

There's actually a relatively new free competition product to IDA - Ghidra, developed by the NSA for their internal usage (I don't even want to know what they use it for....) that can also do 68000 code re-engineering. It's not quite as slick to use and rather intended as a de-compiler (for programs written in C), but it could very well be used for the same purposes. (But don't even think about downloading and using it - That's not the "baby steps" mentioned above)

Yes, Ghidra is the latest hype in the scene and seriously threatens IDA. I found it pretty frustrating to use, but if I didn't have an IDA licence already I'd check it out. I actually tried to analyze the Hermes code with it but failed (as IDA does not have MCS48 support. I'm >50% done writing the CPU plug-in for it but somewhat lost interest). Definitely not for beginner's, though. I've heard great things about Hopper for Mac, alas it does not have 68k support.